1# bcc Reference Guide 2 3Intended for search (Ctrl-F) and reference. For tutorials, start with [tutorial.md](tutorial.md). 4 5This guide is incomplete. If something feels missing, check the bcc and kernel source. And if you confirm we're missing something, please send a pull request to fix it, and help out everyone. 6 7## Contents 8 9- [BPF C](#bpf-c) 10 - [Events & Arguments](#events--arguments) 11 - [1. kprobes](#1-kprobes) 12 - [2. kretprobes](#2-kretprobes) 13 - [3. Tracepoints](#3-tracepoints) 14 - [4. uprobes](#4-uprobes) 15 - [5. uretprobes](#5-uretprobes) 16 - [6. USDT probes](#6-usdt-probes) 17 - [7. Raw Tracepoints](#7-raw-tracepoints) 18 - [Data](#data) 19 - [1. bpf_probe_read()](#1-bpf_probe_read) 20 - [2. bpf_probe_read_str()](#2-bpf_probe_read_str) 21 - [3. bpf_ktime_get_ns()](#3-bpf_ktime_get_ns) 22 - [4. bpf_get_current_pid_tgid()](#4-bpf_get_current_pid_tgid) 23 - [5. bpf_get_current_uid_gid()](#5-bpf_get_current_uid_gid) 24 - [6. bpf_get_current_comm()](#6-bpf_get_current_comm) 25 - [7. bpf_get_current_task()](#7-bpf_get_current_task) 26 - [8. bpf_log2l()](#8-bpflog2l) 27 - [9. bpf_get_prandom_u32()](#9-bpf_get_prandom_u32) 28 - [Debugging](#debugging) 29 - [1. bpf_override_return()](#1-bpf_override_return) 30 - [Output](#output) 31 - [1. bpf_trace_printk()](#1-bpf_trace_printk) 32 - [2. BPF_PERF_OUTPUT](#2-bpf_perf_output) 33 - [3. perf_submit()](#3-perf_submit) 34 - [Maps](#maps) 35 - [1. BPF_TABLE](#1-bpf_table) 36 - [2. BPF_HASH](#2-bpf_hash) 37 - [3. BPF_ARRAY](#3-bpf_array) 38 - [4. BPF_HISTOGRAM](#4-bpf_histogram) 39 - [5. BPF_STACK_TRACE](#5-bpf_stack_trace) 40 - [6. BPF_PERF_ARRAY](#6-bpf_perf_array) 41 - [7. BPF_PERCPU_ARRAY](#7-bpf_percpu_array) 42 - [8. BPF_LPM_TRIE](#8-bpf_lpm_trie) 43 - [9. BPF_PROG_ARRAY](#9-bpf_prog_array) 44 - [10. BPF_DEVMAP](#10-bpf_devmap) 45 - [11. BPF_CPUMAP](#11-bpf_cpumap) 46 - [12. map.lookup()](#12-maplookup) 47 - [13. map.lookup_or_init()](#13-maplookup_or_init) 48 - [14. map.delete()](#14-mapdelete) 49 - [15. map.update()](#15-mapupdate) 50 - [16. map.insert()](#16-mapinsert) 51 - [17. map.increment()](#17-mapincrement) 52 - [18. map.get_stackid()](#18-mapget_stackid) 53 - [19. map.perf_read()](#19-mapperf_read) 54 - [20. map.call()](#20-mapcall) 55 - [21. map.redirect_map()](#21-mapredirect_map) 56 - [Licensing](#licensing) 57 58- [bcc Python](#bcc-python) 59 - [Initialization](#initialization) 60 - [1. BPF](#1-bpf) 61 - [2. USDT](#2-usdt) 62 - [Events](#events) 63 - [1. attach_kprobe()](#1-attach_kprobe) 64 - [2. attach_kretprobe()](#2-attach_kretprobe) 65 - [3. attach_tracepoint()](#3-attach_tracepoint) 66 - [4. attach_uprobe()](#4-attach_uprobe) 67 - [5. attach_uretprobe()](#5-attach_uretprobe) 68 - [6. USDT.enable_probe()](#6-usdtenable_probe) 69 - [7. attach_raw_tracepoint()](#7-attach_raw_tracepoint) 70 - [Debug Output](#debug-output) 71 - [1. trace_print()](#1-trace_print) 72 - [2. trace_fields()](#2-trace_fields) 73 - [Output](#output) 74 - [1. perf_buffer_poll()](#1-perf_buffer_poll) 75 - [Maps](#maps) 76 - [1. get_table()](#1-get_table) 77 - [2. open_perf_buffer()](#2-open_perf_buffer) 78 - [3. items()](#3-items) 79 - [4. values()](#4-values) 80 - [5. clear()](#5-clear) 81 - [6. print_log2_hist()](#6-print_log2_hist) 82 - [7. print_linear_hist()](#6-print_linear_hist) 83 - [Helpers](#helpers) 84 - [1. ksym()](#1-ksym) 85 - [2. ksymname()](#2-ksymname) 86 - [3. sym()](#3-sym) 87 - [4. num_open_kprobes()](#4-num_open_kprobes) 88 89- [BPF Errors](#bpf-errors) 90 - [1. Invalid mem access](#1-invalid-mem-access) 91 - [2. Cannot call GPL only function from proprietary program](#2-cannot-call-gpl-only-function-from-proprietary-program) 92 93- [Environment Variables](#envvars) 94 - [1. kernel source directory](#1-kernel-source-directory) 95 - [2. kernel version overriding](#2-kernel-version-overriding) 96 97# BPF C 98 99This section describes the C part of a bcc program. 100 101## Events & Arguments 102 103### 1. kprobes 104 105Syntax: kprobe__*kernel_function_name* 106 107```kprobe__``` is a special prefix that creates a kprobe (dynamic tracing of a kernel function call) for the kernel function name provided as the remainder. You can also use kprobes by declaring a normal C function, then using the Python ```BPF.attach_kprobe()``` (covered later) to associate it with a kernel function. 108 109Arguments are specified on the function declaration: kprobe__*kernel_function_name*(struct pt_regs *ctx [, *argument1* ...]) 110 111For example: 112 113```C 114int kprobe__tcp_v4_connect(struct pt_regs *ctx, struct sock *sk) 115 [...] 116} 117``` 118 119This instruments the tcp_v4_connect() kernel function using a kprobe, with the following arguments: 120 121- ```struct pt_regs *ctx```: Registers and BPF context. 122- ```struct sock *sk```: First argument to tcp_v4_connect(). 123 124The first argument is always ```struct pt_regs *```, the remainder are the arguments to the function (they don't need to be specified, if you don't intend to use them). 125 126Examples in situ: 127[code](https://github.com/iovisor/bcc/blob/4afa96a71c5dbfc4c507c3355e20baa6c184a3a8/examples/tracing/tcpv4connect.py#L28) ([output](https://github.com/iovisor/bcc/blob/5bd0eb21fd148927b078deb8ac29fff2fb044b66/examples/tracing/tcpv4connect_example.txt#L8)), 128[code](https://github.com/iovisor/bcc/commit/310ab53710cfd46095c1f6b3e44f1dbc8d1a41d8#diff-8cd1822359ffee26e7469f991ce0ef00R26) ([output](https://github.com/iovisor/bcc/blob/3b9679a3bd9b922c736f6061dc65cb56de7e0250/examples/tracing/bitehist_example.txt#L6)) 129<!--- I can't add search links here, since github currently cannot handle partial-word searches needed for "kprobe__" ---> 130 131### 2. kretprobes 132 133Syntax: kretprobe__*kernel_function_name* 134 135```kretprobe__``` is a special prefix that creates a kretprobe (dynamic tracing of a kernel function return) for the kernel function name provided as the remainder. You can also use kretprobes by declaring a normal C function, then using the Python ```BPF.attach_kretprobe()``` (covered later) to associate it with a kernel function. 136 137Return value is available as ```PT_REGS_RC(ctx)```, given a function declaration of: kretprobe__*kernel_function_name*(struct pt_regs *ctx) 138 139For example: 140 141```C 142int kretprobe__tcp_v4_connect(struct pt_regs *ctx) 143{ 144 int ret = PT_REGS_RC(ctx); 145 [...] 146} 147``` 148 149This instruments the return of the tcp_v4_connect() kernel function using a kretprobe, and stores the return value in ```ret```. 150 151Examples in situ: 152[code](https://github.com/iovisor/bcc/blob/4afa96a71c5dbfc4c507c3355e20baa6c184a3a8/examples/tracing/tcpv4connect.py#L38) ([output](https://github.com/iovisor/bcc/blob/5bd0eb21fd148927b078deb8ac29fff2fb044b66/examples/tracing/tcpv4connect_example.txt#L8)) 153 154### 3. Tracepoints 155 156Syntax: TRACEPOINT_PROBE(*category*, *event*) 157 158This is a macro that instruments the tracepoint defined by *category*:*event*. 159 160Arguments are available in an ```args``` struct, which are the tracepoint arguments. One way to list these is to cat the relevant format file under /sys/kernel/debug/tracing/events/*category*/*event*/format. 161 162The ```args``` struct can be used in place of ``ctx`` in each functions requiring a context as an argument. This includes notably [perf_submit()](#3-perf_submit). 163 164For example: 165 166```C 167TRACEPOINT_PROBE(random, urandom_read) { 168 // args is from /sys/kernel/debug/tracing/events/random/urandom_read/format 169 bpf_trace_printk("%d\\n", args->got_bits); 170 return 0; 171} 172``` 173 174This instruments the random:urandom_read tracepoint, and prints the tracepoint argument ```got_bits```. 175 176Examples in situ: 177[code](https://github.com/iovisor/bcc/blob/a4159da8c4ea8a05a3c6e402451f530d6e5a8b41/examples/tracing/urandomread.py#L19) ([output](https://github.com/iovisor/bcc/commit/e422f5e50ecefb96579b6391a2ada7f6367b83c4#diff-41e5ecfae4a3b38de5f4e0887ed160e5R10)), 178[search /examples](https://github.com/iovisor/bcc/search?q=TRACEPOINT_PROBE+path%3Aexamples&type=Code), 179[search /tools](https://github.com/iovisor/bcc/search?q=TRACEPOINT_PROBE+path%3Atools&type=Code) 180 181### 4. uprobes 182 183These are instrumented by declaring a normal function in C, then associating it as a uprobe probe in Python via ```BPF.attach_uprobe()``` (covered later). 184 185Arguments can be examined using ```PT_REGS_PARM``` macros. 186 187For example: 188 189```C 190int count(struct pt_regs *ctx) { 191 char buf[64]; 192 bpf_probe_read(&buf, sizeof(buf), (void *)PT_REGS_PARM1(ctx)); 193 bpf_trace_printk("%s %d", buf, PT_REGS_PARM2(ctx)); 194 return(0); 195} 196``` 197 198This reads the first argument as a string, and then prints it with the second argument as an integer. 199 200Examples in situ: 201[code](https://github.com/iovisor/bcc/blob/4afa96a71c5dbfc4c507c3355e20baa6c184a3a8/examples/tracing/strlen_count.py#L26) 202 203### 5. uretprobes 204 205These are instrumented by declaring a normal function in C, then associating it as a uretprobe probe in Python via ```BPF.attach_uretprobe()``` (covered later). 206 207Return value is available as ```PT_REGS_RC(ctx)```, given a function declaration of: *function_name*(struct pt_regs *ctx) 208 209For example: 210 211```C 212BPF_HISTOGRAM(dist); 213int count(struct pt_regs *ctx) { 214 dist.increment(PT_REGS_RC(ctx)); 215 return 0; 216} 217``` 218 219This increments the bucket in the ```dist``` histogram that is indexed by the return value. 220 221Examples in situ: 222[code](https://github.com/iovisor/bcc/blob/4afa96a71c5dbfc4c507c3355e20baa6c184a3a8/examples/tracing/strlen_hist.py#L39) ([output](https://github.com/iovisor/bcc/blob/4afa96a71c5dbfc4c507c3355e20baa6c184a3a8/examples/tracing/strlen_hist.py#L15)), 223[code](https://github.com/iovisor/bcc/blob/4afa96a71c5dbfc4c507c3355e20baa6c184a3a8/tools/bashreadline.py) ([output](https://github.com/iovisor/bcc/commit/aa87997d21e5c1a6a20e2c96dd25eb92adc8e85d#diff-2fd162f9e594206f789246ce97d62cf0R7)) 224 225### 6. USDT probes 226 227These are User Statically-Defined Tracing (USDT) probes, which may be placed in some applications or libraries to provide a user-level equivalent of tracepoints. The primary BPF method provided for USDT support method is ```enable_probe()```. USDT probes are instrumented by declaring a normal function in C, then associating it as a USDT probe in Python via ```USDT.enable_probe()```. 228 229Arguments can be read via: bpf_usdt_readarg(*index*, ctx, &addr) 230 231For example: 232 233```C 234int do_trace(struct pt_regs *ctx) { 235 uint64_t addr; 236 char path[128]; 237 bpf_usdt_readarg(6, ctx, &addr); 238 bpf_probe_read(&path, sizeof(path), (void *)addr); 239 bpf_trace_printk("path:%s\\n", path); 240 return 0; 241}; 242``` 243 244This reads the sixth USDT argument, and then pulls it in as a string to ```path```. 245 246Examples in situ: 247[code](https://github.com/iovisor/bcc/commit/4f88a9401357d7b75e917abd994aa6ea97dda4d3#diff-04a7cad583be5646080970344c48c1f4R24), 248[search /examples](https://github.com/iovisor/bcc/search?q=bpf_usdt_readarg+path%3Aexamples&type=Code), 249[search /tools](https://github.com/iovisor/bcc/search?q=bpf_usdt_readarg+path%3Atools&type=Code) 250 251### 7. Raw Tracepoints 252 253Syntax: RAW_TRACEPOINT_PROBE(*event*) 254 255This is a macro that instruments the raw tracepoint defined by *event*. 256 257The argument is a pointer to struct ```bpf_raw_tracepoint_args```, which is defined in [bpf.h](https://github.com/iovisor/bcc/blob/master/src/cc/compat/linux/bpf.h). The struct field ```args``` contains all parameters of the raw tracepoint where you can found at linux tree [include/trace/events](https://github.com/torvalds/linux/tree/master/include/trace/events) 258directory. 259 260For example: 261```C 262RAW_TRACEPOINT_PROBE(sched_switch) 263{ 264 // TP_PROTO(bool preempt, struct task_struct *prev, struct task_struct *next) 265 struct task_struct *prev = (struct task_struct *)ctx->args[1]; 266 struct task_struct *next= (struct task_struct *)ctx->args[2]; 267 s32 prev_tgid, next_tgid; 268 269 bpf_probe_read(&prev_tgid, sizeof(prev->tgid), &prev->tgid); 270 bpf_probe_read(&next_tgid, sizeof(next->tgid), &next->tgid); 271 bpf_trace_printk("%d -> %d\\n", prev_tgid, next_tgid); 272} 273``` 274 275This instruments the sched:sched_switch tracepoint, and prints the prev and next tgid. 276 277Examples in situ: 278[search /tools](https://github.com/iovisor/bcc/search?q=RAW_TRACEPOINT_PROBE+path%3Atools&type=Code) 279 280## Data 281 282### 1. bpf_probe_read() 283 284Syntax: ```int bpf_probe_read(void *dst, int size, const void *src)``` 285 286Return: 0 on success 287 288This copies a memory location to the BPF stack, so that BPF can later operate on it. For safety, all memory reads must pass through bpf_probe_read(). This happens automatically in some cases, such as dereferencing kernel variables, as bcc will rewrite the BPF program to include the necessary bpf_probe_reads(). 289 290Examples in situ: 291[search /examples](https://github.com/iovisor/bcc/search?q=bpf_probe_read+path%3Aexamples&type=Code), 292[search /tools](https://github.com/iovisor/bcc/search?q=bpf_probe_read+path%3Atools&type=Code) 293 294### 2. bpf_probe_read_str() 295 296Syntax: ```int bpf_probe_read_str(void *dst, int size, const void *src)``` 297 298Return: 299 - \> 0 length of the string including the trailing NULL on success 300 - \< 0 error 301 302This copies a `NULL` terminated string from memory location to BPF stack, so that BPF can later operate on it. In case the string length is smaller than size, the target is not padded with further `NULL` bytes. In case the string length is larger than size, just `size - 1` bytes are copied and the last byte is set to `NULL`. 303 304Examples in situ: 305[search /examples](https://github.com/iovisor/bcc/search?q=bpf_probe_read_str+path%3Aexamples&type=Code), 306[search /tools](https://github.com/iovisor/bcc/search?q=bpf_probe_read_str+path%3Atools&type=Code) 307 308### 3. bpf_ktime_get_ns() 309 310Syntax: ```u64 bpf_ktime_get_ns(void)``` 311 312Return: current time in nanoseconds 313 314Examples in situ: 315[search /examples](https://github.com/iovisor/bcc/search?q=bpf_ktime_get_ns+path%3Aexamples&type=Code), 316[search /tools](https://github.com/iovisor/bcc/search?q=bpf_ktime_get_ns+path%3Atools&type=Code) 317 318### 4. bpf_get_current_pid_tgid() 319 320Syntax: ```u64 bpf_get_current_pid_tgid(void)``` 321 322Return: ```current->tgid << 32 | current->pid``` 323 324Returns the process ID in the lower 32 bits (kernel's view of the PID, which in user space is usually presented as the thread ID), and the thread group ID in the upper 32 bits (what user space often thinks of as the PID). By directly setting this to a u32, we discard the upper 32 bits. 325 326Examples in situ: 327[search /examples](https://github.com/iovisor/bcc/search?q=bpf_get_current_pid_tgid+path%3Aexamples&type=Code), 328[search /tools](https://github.com/iovisor/bcc/search?q=bpf_get_current_pid_tgid+path%3Atools&type=Code) 329 330### 5. bpf_get_current_uid_gid() 331 332Syntax: ```u64 bpf_get_current_uid_gid(void)``` 333 334Return: ```current_gid << 32 | current_uid``` 335 336Returns the user ID and group IDs. 337 338Examples in situ: 339[search /examples](https://github.com/iovisor/bcc/search?q=bpf_get_current_uid_gid+path%3Aexamples&type=Code), 340[search /tools](https://github.com/iovisor/bcc/search?q=bpf_get_current_uid_gid+path%3Atools&type=Code) 341 342### 6. bpf_get_current_comm() 343 344Syntax: ```bpf_get_current_comm(char *buf, int size_of_buf)``` 345 346Return: 0 on success 347 348Populates the first argument address with the current process name. It should be a pointer to a char array of at least size TASK_COMM_LEN, which is defined in linux/sched.h. For example: 349 350```C 351#include <linux/sched.h> 352 353int do_trace(struct pt_regs *ctx) { 354 char comm[TASK_COMM_LEN]; 355 bpf_get_current_comm(&comm, sizeof(comm)); 356[...] 357``` 358 359Examples in situ: 360[search /examples](https://github.com/iovisor/bcc/search?q=bpf_get_current_comm+path%3Aexamples&type=Code), 361[search /tools](https://github.com/iovisor/bcc/search?q=bpf_get_current_comm+path%3Atools&type=Code) 362 363### 7. bpf_get_current_task() 364 365Syntax: ```bpf_get_current_task()``` 366 367Return: current task as a pointer to struct task_struct. 368 369Returns a pointer to the current task's task_struct object. This helper can be used to compute the on-CPU time for a process, identify kernel threads, get the current CPU's run queue, or retrieve many other pieces of information. 370 371With Linux 4.13, due to issues with field randomization, you may need two #define directives before the includes: 372```C 373#define randomized_struct_fields_start struct { 374#define randomized_struct_fields_end }; 375#include <linux/sched.h> 376 377int do_trace(void *ctx) { 378 struct task_struct *t = (struct task_struct *)bpf_get_current_task(); 379[...] 380``` 381 382Examples in situ: 383[search /examples](https://github.com/iovisor/bcc/search?q=bpf_get_current_task+path%3Aexamples&type=Code), 384[search /tools](https://github.com/iovisor/bcc/search?q=bpf_get_current_task+path%3Atools&type=Code) 385 386### 8. bpf_log2l() 387 388Syntax: ```unsigned int bpf_log2l(unsigned long v)``` 389 390Returns the log-2 of the provided value. This is often used to create indexes for histograms, to construct power-of-2 histograms. 391 392Examples in situ: 393[search /examples](https://github.com/iovisor/bcc/search?q=bpf_log2l+path%3Aexamples&type=Code), 394[search /tools](https://github.com/iovisor/bcc/search?q=bpf_log2l+path%3Atools&type=Code) 395 396### 9. bpf_get_prandom_u32() 397 398Syntax: ```u32 bpf_get_prandom_u32()``` 399 400Returns a pseudo-random u32. 401 402Example in situ: 403[search /examples](https://github.com/iovisor/bcc/search?q=bpf_get_prandom_u32+path%3Aexamples&type=Code), 404[search /tools](https://github.com/iovisor/bcc/search?q=bpf_get_prandom_u32+path%3Atools&type=Code) 405 406## Debugging 407 408### 1. bpf_override_return() 409 410Syntax: ```int bpf_override_return(struct pt_regs *, unsigned long rc)``` 411 412Return: 0 on success 413 414When used in a program attached to a function entry kprobe, causes the 415execution of the function to be skipped, immediately returning `rc` instead. 416This is used for targeted error injection. 417 418bpf_override_return will only work when the kprobed function is whitelisted to 419allow error injections. Whitelisting entails tagging a function with 420`BPF_ALLOW_ERROR_INJECTION()` in the kernel source tree; see `io_ctl_init` for 421an example. If the kprobed function is not whitelisted, the bpf program will 422fail to attach with ` ioctl(PERF_EVENT_IOC_SET_BPF): Invalid argument` 423 424 425```C 426int kprobe__io_ctl_init(void *ctx) { 427 bpf_override_return(ctx, -ENOMEM); 428 return 0; 429} 430``` 431 432## Output 433 434### 1. bpf_trace_printk() 435 436Syntax: ```int bpf_trace_printk(const char *fmt, int fmt_size, ...)``` 437 438Return: 0 on success 439 440A simple kernel facility for printf() to the common trace_pipe (/sys/kernel/debug/tracing/trace_pipe). This is ok for some quick examples, but has limitations: 3 args max, 1 %s only, and trace_pipe is globally shared, so concurrent programs will have clashing output. A better interface is via BPF_PERF_OUTPUT(). 441 442Examples in situ: 443[search /examples](https://github.com/iovisor/bcc/search?q=bpf_trace_printk+path%3Aexamples&type=Code), 444[search /tools](https://github.com/iovisor/bcc/search?q=bpf_trace_printk+path%3Atools&type=Code) 445 446### 2. BPF_PERF_OUTPUT 447 448Syntax: ```BPF_PERF_OUTPUT(name)``` 449 450Creates a BPF table for pushing out custom event data to user space via a perf ring buffer. This is the preferred method for pushing per-event data to user space. 451 452For example: 453 454```C 455struct data_t { 456 u32 pid; 457 u64 ts; 458 char comm[TASK_COMM_LEN]; 459}; 460BPF_PERF_OUTPUT(events); 461 462int hello(struct pt_regs *ctx) { 463 struct data_t data = {}; 464 465 data.pid = bpf_get_current_pid_tgid(); 466 data.ts = bpf_ktime_get_ns(); 467 bpf_get_current_comm(&data.comm, sizeof(data.comm)); 468 469 events.perf_submit(ctx, &data, sizeof(data)); 470 471 return 0; 472} 473``` 474 475The output table is named ```events```, and data is pushed to it via ```events.perf_submit()```. 476 477Examples in situ: 478[search /examples](https://github.com/iovisor/bcc/search?q=BPF_PERF_OUTPUT+path%3Aexamples&type=Code), 479[search /tools](https://github.com/iovisor/bcc/search?q=BPF_PERF_OUTPUT+path%3Atools&type=Code) 480 481### 3. perf_submit() 482 483Syntax: ```int perf_submit((void *)ctx, (void *)data, u32 data_size)``` 484 485Return: 0 on success 486 487A method of a BPF_PERF_OUTPUT table, for submitting custom event data to user space. See the BPF_PERF_OUTPUT entry. (This ultimately calls bpf_perf_event_output().) 488 489Examples in situ: 490[search /examples](https://github.com/iovisor/bcc/search?q=perf_submit+path%3Aexamples&type=Code), 491[search /tools](https://github.com/iovisor/bcc/search?q=perf_submit+path%3Atools&type=Code) 492 493## Maps 494 495Maps are BPF data stores, and are the basis for higher level object types including tables, hashes, and histograms. 496 497### 1. BPF_TABLE 498 499Syntax: ```BPF_TABLE(_table_type, _key_type, _leaf_type, _name, _max_entries)``` 500 501Creates a map named ```_name```. Most of the time this will be used via higher-level macros, like BPF_HASH, BPF_HIST, etc. 502 503Methods (covered later): map.lookup(), map.lookup_or_init(), map.delete(), map.update(), map.insert(), map.increment(). 504 505Examples in situ: 506[search /examples](https://github.com/iovisor/bcc/search?q=BPF_TABLE+path%3Aexamples&type=Code), 507[search /tools](https://github.com/iovisor/bcc/search?q=BPF_TABLE+path%3Atools&type=Code) 508 509### 2. BPF_HASH 510 511Syntax: ```BPF_HASH(name [, key_type [, leaf_type [, size]]])``` 512 513Creates a hash map (associative array) named ```name```, with optional parameters. 514 515Defaults: ```BPF_HASH(name, key_type=u64, leaf_type=u64, size=10240)``` 516 517For example: 518 519```C 520BPF_HASH(start, struct request *); 521``` 522 523This creates a hash named ```start``` where the key is a ```struct request *```, and the value defaults to u64. This hash is used by the disksnoop.py example for saving timestamps for each I/O request, where the key is the pointer to struct request, and the value is the timestamp. 524 525Methods (covered later): map.lookup(), map.lookup_or_init(), map.delete(), map.update(), map.insert(), map.increment(). 526 527Examples in situ: 528[search /examples](https://github.com/iovisor/bcc/search?q=BPF_HASH+path%3Aexamples&type=Code), 529[search /tools](https://github.com/iovisor/bcc/search?q=BPF_HASH+path%3Atools&type=Code) 530 531### 3. BPF_ARRAY 532 533Syntax: ```BPF_ARRAY(name [, leaf_type [, size]])``` 534 535Creates an int-indexed array which is optimized for fastest lookup and update, named ```name```, with optional parameters. 536 537Defaults: ```BPF_ARRAY(name, leaf_type=u64, size=10240)``` 538 539For example: 540 541```C 542BPF_ARRAY(counts, u64, 32); 543``` 544 545This creates an array named ```counts``` where with 32 buckets and 64-bit integer values. This array is used by the funccount.py example for saving call count of each function. 546 547Methods (covered later): map.lookup(), map.update(), map.increment(). Note that all array elements are pre-allocated with zero values and can not be deleted. 548 549Examples in situ: 550[search /examples](https://github.com/iovisor/bcc/search?q=BPF_ARRAY+path%3Aexamples&type=Code), 551[search /tools](https://github.com/iovisor/bcc/search?q=BPF_ARRAY+path%3Atools&type=Code) 552 553### 4. BPF_HISTOGRAM 554 555Syntax: ```BPF_HISTOGRAM(name [, key_type [, size ]])``` 556 557Creates a histogram map named ```name```, with optional parameters. 558 559Defaults: ```BPF_HISTOGRAM(name, key_type=int, size=64)``` 560 561For example: 562 563```C 564BPF_HISTOGRAM(dist); 565``` 566 567This creates a histogram named ```dist```, which defaults to 64 buckets indexed by keys of type int. 568 569Methods (covered later): map.increment(). 570 571Examples in situ: 572[search /examples](https://github.com/iovisor/bcc/search?q=BPF_HISTOGRAM+path%3Aexamples&type=Code), 573[search /tools](https://github.com/iovisor/bcc/search?q=BPF_HISTOGRAM+path%3Atools&type=Code) 574 575### 5. BPF_STACK_TRACE 576 577Syntax: ```BPF_STACK_TRACE(name, max_entries)``` 578 579Creates stack trace map named ```name```, with a maximum entry count provided. These maps are used to store stack traces. 580 581For example: 582 583```C 584BPF_STACK_TRACE(stack_traces, 1024); 585``` 586 587This creates stack trace map named ```stack_traces```, with a maximum number of stack trace entries of 1024. 588 589Methods (covered later): map.get_stackid(). 590 591Examples in situ: 592[search /examples](https://github.com/iovisor/bcc/search?q=BPF_STACK_TRACE+path%3Aexamples&type=Code), 593[search /tools](https://github.com/iovisor/bcc/search?q=BPF_STACK_TRACE+path%3Atools&type=Code) 594 595### 6. BPF_PERF_ARRAY 596 597Syntax: ```BPF_PERF_ARRAY(name, max_entries)``` 598 599Creates perf array named ```name```, with a maximum entry count provided, which must be equal to the number of system cpus. These maps are used to fetch hardware performance counters. 600 601For example: 602 603```C 604text=""" 605BPF_PERF_ARRAY(cpu_cycles, NUM_CPUS); 606""" 607b = bcc.BPF(text=text, cflags=["-DNUM_CPUS=%d" % multiprocessing.cpu_count()]) 608b["cpu_cycles"].open_perf_event(b["cpu_cycles"].HW_CPU_CYCLES) 609``` 610 611This creates a perf array named ```cpu_cycles```, with number of entries equal to the number of cpus/cores. The array is configured so that later calling map.perf_read() will return a hardware-calculated counter of the number of cycles elapsed from some point in the past. Only one type of hardware counter may be configured per table at a time. 612 613Methods (covered later): map.perf_read(). 614 615Examples in situ: 616[search /tests](https://github.com/iovisor/bcc/search?q=BPF_PERF_ARRAY+path%3Atests&type=Code) 617 618### 7. BPF_PERCPU_ARRAY 619 620Syntax: ```BPF_PERCPU_ARRAY(name [, leaf_type [, size]])``` 621 622Creates NUM_CPU int-indexed arrays which are optimized for fastest lookup and update, named ```name```, with optional parameters. Each CPU will have a separate copy of this array. The copies are not kept synchronized in any way. 623 624 625Defaults: ```BPF_PERCPU_ARRAY(name, leaf_type=u64, size=10240)``` 626 627For example: 628 629```C 630BPF_PERCPU_ARRAY(counts, u64, 32); 631``` 632 633This creates NUM_CPU arrays named ```counts``` where with 32 buckets and 64-bit integer values. 634 635Methods (covered later): map.lookup(), map.update(), map.increment(). Note that all array elements are pre-allocated with zero values and can not be deleted. 636 637Examples in situ: 638[search /examples](https://github.com/iovisor/bcc/search?q=BPF_PERCPU_ARRAY+path%3Aexamples&type=Code), 639[search /tools](https://github.com/iovisor/bcc/search?q=BPF_PERCPU_ARRAY+path%3Atools&type=Code) 640 641### 8. BPF_LPM_TRIE 642 643Syntax: `BPF_LPM_TRIE(name [, key_type [, leaf_type [, size]]])` 644 645Creates a longest prefix match trie map named `name`, with optional parameters. 646 647Defaults: `BPF_LPM_TRIE(name, key_type=u64, leaf_type=u64, size=10240)` 648 649For example: 650 651```c 652BPF_LPM_TRIE(trie, struct key_v6); 653``` 654 655This creates an LPM trie map named `trie` where the key is a `struct key_v6`, and the value defaults to u64. 656 657Methods (covered later): map.lookup(), map.lookup_or_init(), map.delete(), map.update(), map.insert(), map.increment(). 658 659Examples in situ: 660[search /examples](https://github.com/iovisor/bcc/search?q=BPF_LPM_TRIE+path%3Aexamples&type=Code), 661[search /tools](https://github.com/iovisor/bcc/search?q=BPF_LPM_TRIE+path%3Atools&type=Code) 662 663### 9. BPF_PROG_ARRAY 664 665Syntax: ```BPF_PROG_ARRAY(name, size)``` 666 667This creates a program array named ```name``` with ```size``` entries. Each entry of the array is either a file descriptor to a bpf program or ```NULL```. The array acts as a jump table so that bpf programs can "tail-call" other bpf programs. 668 669Methods (covered later): map.call(). 670 671Examples in situ: 672[search /examples](https://github.com/iovisor/bcc/search?q=BPF_PROG_ARRAY+path%3Aexamples&type=Code), 673[search /tests](https://github.com/iovisor/bcc/search?q=BPF_PROG_ARRAY+path%3Atests&type=Code), 674[assign fd](https://github.com/iovisor/bcc/blob/master/examples/networking/tunnel_monitor/monitor.py#L24-L26) 675 676### 10. BPF_DEVMAP 677 678Syntax: ```BPF_DEVMAP(name, size)``` 679 680This creates a device map named ```name``` with ```size``` entries. Each entry of the map is an `ifindex` to a network interface. This map is only used in XDP. 681 682For example: 683```C 684BPF_DEVMAP(devmap, 10); 685``` 686 687Methods (covered later): map.redirect_map(). 688 689Examples in situ: 690[search /examples](https://github.com/iovisor/bcc/search?q=BPF_DEVMAP+path%3Aexamples&type=Code), 691 692### 11. BPF_CPUMAP 693 694Syntax: ```BPF_CPUMAP(name, size)``` 695 696This creates a cpu map named ```name``` with ```size``` entries. The index of the map represents the CPU id and each entry is the size of the ring buffer allocated for the CPU. This map is only used in XDP. 697 698For example: 699```C 700BPF_CPUMAP(cpumap, 16); 701``` 702 703Methods (covered later): map.redirect_map(). 704 705Examples in situ: 706[search /examples](https://github.com/iovisor/bcc/search?q=BPF_CPUMAP+path%3Aexamples&type=Code), 707 708### 12. map.lookup() 709 710Syntax: ```*val map.lookup(&key)``` 711 712Lookup the key in the map, and return a pointer to its value if it exists, else NULL. We pass the key in as an address to a pointer. 713 714Examples in situ: 715[search /examples](https://github.com/iovisor/bcc/search?q=lookup+path%3Aexamples&type=Code), 716[search /tools](https://github.com/iovisor/bcc/search?q=lookup+path%3Atools&type=Code) 717 718### 13. map.lookup_or_init() 719 720Syntax: ```*val map.lookup_or_init(&key, &zero)``` 721 722Lookup the key in the map, and return a pointer to its value if it exists, else initialize the key's value to the second argument. This is often used to initialize values to zero. 723 724Examples in situ: 725[search /examples](https://github.com/iovisor/bcc/search?q=lookup_or_init+path%3Aexamples&type=Code), 726[search /tools](https://github.com/iovisor/bcc/search?q=lookup_or_init+path%3Atools&type=Code) 727 728### 14. map.delete() 729 730Syntax: ```map.delete(&key)``` 731 732Delete the key from the hash. 733 734Examples in situ: 735[search /examples](https://github.com/iovisor/bcc/search?q=delete+path%3Aexamples&type=Code), 736[search /tools](https://github.com/iovisor/bcc/search?q=delete+path%3Atools&type=Code) 737 738### 15. map.update() 739 740Syntax: ```map.update(&key, &val)``` 741 742Associate the value in the second argument to the key, overwriting any previous value. 743 744Examples in situ: 745[search /examples](https://github.com/iovisor/bcc/search?q=update+path%3Aexamples&type=Code), 746[search /tools](https://github.com/iovisor/bcc/search?q=update+path%3Atools&type=Code) 747 748### 16. map.insert() 749 750Syntax: ```map.insert(&key, &val)``` 751 752Associate the value in the second argument to the key, only if there was no previous value. 753 754Examples in situ: 755[search /examples](https://github.com/iovisor/bcc/search?q=insert+path%3Aexamples&type=Code) 756 757### 17. map.increment() 758 759Syntax: ```map.increment(key[, increment_amount])``` 760 761Increments the key's value by `increment_amount`, which defaults to 1. Used for histograms. 762 763Examples in situ: 764[search /examples](https://github.com/iovisor/bcc/search?q=increment+path%3Aexamples&type=Code), 765[search /tools](https://github.com/iovisor/bcc/search?q=increment+path%3Atools&type=Code) 766 767### 18. map.get_stackid() 768 769Syntax: ```int map.get_stackid(void *ctx, u64 flags)``` 770 771This walks the stack found via the struct pt_regs in ```ctx```, saves it in the stack trace map, and returns a unique ID for the stack trace. 772 773Examples in situ: 774[search /examples](https://github.com/iovisor/bcc/search?q=get_stackid+path%3Aexamples&type=Code), 775[search /tools](https://github.com/iovisor/bcc/search?q=get_stackid+path%3Atools&type=Code) 776 777### 19. map.perf_read() 778 779Syntax: ```u64 map.perf_read(u32 cpu)``` 780 781This returns the hardware performance counter as configured in [5. BPF_PERF_ARRAY](#5-bpf_perf_array) 782 783Examples in situ: 784[search /tests](https://github.com/iovisor/bcc/search?q=perf_read+path%3Atests&type=Code) 785 786### 20. map.call() 787 788Syntax: ```void map.call(void *ctx, int index)``` 789 790This invokes ```bpf_tail_call()``` to tail-call the bpf program which the ```index``` entry in [9. BPF_PROG_ARRAY](#9-bpf_prog_array) points to. A tail-call is different from the normal call. It reuses the current stack frame after jumping to another bpf program and never goes back. If the ```index``` entry is empty, it won't jump anywhere and the program execution continues as normal. 791 792For example: 793 794```C 795BPF_PROG_ARRAY(prog_array, 10); 796 797int tail_call(void *ctx) { 798 bpf_trace_printk("Tail-call\n"); 799 return 0; 800} 801 802int do_tail_call(void *ctx) { 803 bpf_trace_printk("Original program\n"); 804 prog_array.call(ctx, 2); 805 return 0; 806} 807``` 808 809```Python 810b = BPF(src_file="example.c") 811tail_fn = b.load_func("tail_call", BPF.KPROBE) 812prog_array = b.get_table("prog_array") 813prog_array[c_int(2)] = c_int(tail_fn.fd) 814b.attach_kprobe(event="some_kprobe_event", fn_name="do_tail_call") 815``` 816 817This assigns ```tail_call()``` to ```prog_array[2]```. In the end of ```do_tail_call()```, ```prog_array.call(ctx, 2)``` tail-calls ```tail_call()``` and executes it. 818 819**NOTE:** To prevent infinite loop, the maximum number of tail-calls is 32 ([```MAX_TAIL_CALL_CNT```](https://github.com/torvalds/linux/search?l=C&q=MAX_TAIL_CALL_CNT+path%3Ainclude%2Flinux&type=Code)). 820 821Examples in situ: 822[search /examples](https://github.com/iovisor/bcc/search?l=C&q=call+path%3Aexamples&type=Code), 823[search /tests](https://github.com/iovisor/bcc/search?l=C&q=call+path%3Atests&type=Code) 824 825### 21. map.redirect_map() 826 827Syntax: ```int map.redirect_map(int index, int flags)``` 828 829This redirects the incoming packets based on the ```index``` entry. If the map is [10. BPF_DEVMAP](#10-bpf_devmap), the packet will be sent to the transmit queue of the network interface that the entry points to. If the map is [11. BPF_CPUMAP](#11-bpf_cpumap), the packet will be sent to the ring buffer of the ```index``` CPU and be processed by the CPU later. 830 831If the packet is redirected successfully, the function will return XDP_REDIRECT. Otherwise, it will return XDP_ABORTED to discard the packet. 832 833For example: 834```C 835BPF_DEVMAP(devmap, 1); 836 837int redirect_example(struct xdp_md *ctx) { 838 return devmap.redirect_map(0, 0); 839} 840int xdp_dummy(struct xdp_md *ctx) { 841 return XDP_PASS; 842} 843``` 844 845```Python 846ip = pyroute2.IPRoute() 847idx = ip.link_lookup(ifname="eth1")[0] 848 849b = bcc.BPF(src_file="example.c") 850 851devmap = b.get_table("devmap") 852devmap[c_uint32(0)] = c_int(idx) 853 854in_fn = b.load_func("redirect_example", BPF.XDP) 855out_fn = b.load_func("xdp_dummy", BPF.XDP) 856b.attach_xdp("eth0", in_fn, 0) 857b.attach_xdp("eth1", out_fn, 0) 858``` 859 860Examples in situ: 861[search /examples](https://github.com/iovisor/bcc/search?l=C&q=redirect_map+path%3Aexamples&type=Code), 862 863## Licensing 864 865Depending on which [BPF helpers](kernel-versions.md#helpers) are used, a GPL-compatible license is required. 866 867The special BCC macro `BPF_LICENSE` specifies the license of the BPF program. You can set the license as a comment in your source code, but the kernel has a special interface to specify it programmatically. If you need to use GPL-only helpers, it is recommended to specify the macro in your C code so that the kernel can understand it: 868 869```C 870// SPDX-License-Identifier: GPL-2.0+ 871#define BPF_LICENSE GPL 872``` 873 874Otherwise, the kernel may reject loading your program (see the [error description](#2-cannot-call-gpl-only-function-from-proprietary-program) below). Note that it supports multiple words and quotes are not necessary: 875 876```C 877// SPDX-License-Identifier: GPL-2.0+ OR BSD-2-Clause 878#define BPF_LICENSE Dual BSD/GPL 879``` 880 881Check the [BPF helpers reference](kernel-versions.md#helpers) to see which helpers are GPL-only and what the kernel understands as GPL-compatible. 882 883**If the macro is not specified, BCC will automatically define the license of the program as GPL.** 884 885# bcc Python 886 887## Initialization 888 889Constructors. 890 891### 1. BPF 892 893Syntax: ```BPF({text=BPF_program | src_file=filename} [, usdt_contexts=[USDT_object, ...]] [, cflags=[arg1, ...]] [, debug=int])``` 894 895Creates a BPF object. This is the main object for defining a BPF program, and interacting with its output. 896 897Exactly one of `text` or `src_file` must be supplied (not both). 898 899The `cflags` specifies additional arguments to be passed to the compiler, for example `-DMACRO_NAME=value` or `-I/include/path`. The arguments are passed as an array, with each element being an additional argument. Note that strings are not split on whitespace, so each argument must be a different element of the array, e.g. `["-include", "header.h"]`. 900 901The `debug` flags control debug output, and can be or'ed together: 902- `DEBUG_LLVM_IR = 0x1` compiled LLVM IR 903- `DEBUG_BPF = 0x2` loaded BPF bytecode and register state on branches 904- `DEBUG_PREPROCESSOR = 0x4` pre-processor result 905- `DEBUG_SOURCE = 0x8` ASM instructions embedded with source 906- `DEBUG_BPF_REGISTER_STATE = 0x10` register state on all instructions in addition to DEBUG_BPF 907 908Examples: 909 910```Python 911# define entire BPF program in one line: 912BPF(text='int do_trace(void *ctx) { bpf_trace_printk("hit!\\n"); return 0; }'); 913 914# define program as a variable: 915prog = """ 916int hello(void *ctx) { 917 bpf_trace_printk("Hello, World!\\n"); 918 return 0; 919} 920""" 921b = BPF(text=prog) 922 923# source a file: 924b = BPF(src_file = "vfsreadlat.c") 925 926# include a USDT object: 927u = USDT(pid=int(pid)) 928[...] 929b = BPF(text=bpf_text, usdt_contexts=[u]) 930 931# add include paths: 932u = BPF(text=prog, cflags=["-I/path/to/include"]) 933``` 934 935Examples in situ: 936[search /examples](https://github.com/iovisor/bcc/search?q=BPF+path%3Aexamples+language%3Apython&type=Code), 937[search /tools](https://github.com/iovisor/bcc/search?q=BPF+path%3Atools+language%3Apython&type=Code) 938 939### 2. USDT 940 941Syntax: ```USDT({pid=pid | path=path})``` 942 943Creates an object to instrument User Statically-Defined Tracing (USDT) probes. Its primary method is ```enable_probe()```. 944 945Arguments: 946 947- pid: attach to this process ID. 948- path: instrument USDT probes from this binary path. 949 950Examples: 951 952```Python 953# include a USDT object: 954u = USDT(pid=int(pid)) 955[...] 956b = BPF(text=bpf_text, usdt_contexts=[u]) 957``` 958 959Examples in situ: 960[search /examples](https://github.com/iovisor/bcc/search?q=USDT+path%3Aexamples+language%3Apython&type=Code), 961[search /tools](https://github.com/iovisor/bcc/search?q=USDT+path%3Atools+language%3Apython&type=Code) 962 963## Events 964 965### 1. attach_kprobe() 966 967Syntax: ```BPF.attach_kprobe(event="event", fn_name="name")``` 968 969Instruments the kernel function ```event()``` using kernel dynamic tracing of the function entry, and attaches our C defined function ```name()``` to be called when the kernel function is called. 970 971For example: 972 973```Python 974b.attach_kprobe(event="sys_clone", fn_name="do_trace") 975``` 976 977This will instrument the kernel ```sys_clone()``` function, which will then run our BPF defined ```do_trace()``` function each time it is called. 978 979You can call attach_kprobe() more than once, and attach your BPF function to multiple kernel functions. 980 981See the previous kprobes section for how to instrument arguments from BPF. 982 983Examples in situ: 984[search /examples](https://github.com/iovisor/bcc/search?q=attach_kprobe+path%3Aexamples+language%3Apython&type=Code), 985[search /tools](https://github.com/iovisor/bcc/search?q=attach_kprobe+path%3Atools+language%3Apython&type=Code) 986 987### 2. attach_kretprobe() 988 989Syntax: ```BPF.attach_kretprobe(event="event", fn_name="name")``` 990 991Instruments the return of the kernel function ```event()``` using kernel dynamic tracing of the function return, and attaches our C defined function ```name()``` to be called when the kernel function returns. 992 993For example: 994 995```Python 996b.attach_kretprobe(event="vfs_read", fn_name="do_return") 997``` 998 999This will instrument the kernel ```vfs_read()``` function, which will then run our BPF defined ```do_return()``` function each time it is called. 1000 1001You can call attach_kretprobe() more than once, and attach your BPF function to multiple kernel function returns. 1002 1003See the previous kretprobes section for how to instrument the return value from BPF. 1004 1005Examples in situ: 1006[search /examples](https://github.com/iovisor/bcc/search?q=attach_kretprobe+path%3Aexamples+language%3Apython&type=Code), 1007[search /tools](https://github.com/iovisor/bcc/search?q=attach_kretprobe+path%3Atools+language%3Apython&type=Code) 1008 1009### 3. attach_tracepoint() 1010 1011Syntax: ```BPF.attach_tracepoint(tp="tracepoint", fn_name="name")``` 1012 1013Instruments the kernel tracepoint described by ```tracepoint```, and when hit, runs the BPF function ```name()```. 1014 1015This is an explicit way to instrument tracepoints. The ```TRACEPOINT_PROBE``` syntax, covered in the earlier tracepoints section, is an alternate method with the advantage of auto-declaring an ```args``` struct containing the tracepoint arguments. With ```attach_tracepoint()```, the tracepoint arguments need to be declared in the BPF program. 1016 1017For example: 1018 1019```Python 1020# define BPF program 1021bpf_text = """ 1022#include <uapi/linux/ptrace.h> 1023 1024struct urandom_read_args { 1025 // from /sys/kernel/debug/tracing/events/random/urandom_read/format 1026 u64 __unused__; 1027 u32 got_bits; 1028 u32 pool_left; 1029 u32 input_left; 1030}; 1031 1032int printarg(struct urandom_read_args *args) { 1033 bpf_trace_printk("%d\\n", args->got_bits); 1034 return 0; 1035}; 1036""" 1037 1038# load BPF program 1039b = BPF(text=bpf_text) 1040b.attach_tracepoint("random:urandom_read", "printarg") 1041``` 1042 1043Notice how the first argument to ```printarg()``` is now our defined struct. 1044 1045Examples in situ: 1046[code](https://github.com/iovisor/bcc/blob/a4159da8c4ea8a05a3c6e402451f530d6e5a8b41/examples/tracing/urandomread-explicit.py#L41) 1047 1048### 4. attach_uprobe() 1049 1050Syntax: ```BPF.attach_uprobe(name="location", sym="symbol", fn_name="name")``` 1051 1052Instruments the user-level function ```symbol()``` from either the library or binary named by ```location``` using user-level dynamic tracing of the function entry, and attach our C defined function ```name()``` to be called whenever the user-level function is called. 1053 1054Libraries can be given in the name argument without the lib prefix, or with the full path (/usr/lib/...). Binaries can be given only with the full path (/bin/sh). 1055 1056For example: 1057 1058```Python 1059b.attach_uprobe(name="c", sym="strlen", fn_name="count") 1060``` 1061 1062This will instrument ```strlen()``` function from libc, and call our BPF function ```count()``` when it is called. Note how the "lib" in "libc" is not necessary to specify. 1063 1064Other examples: 1065 1066```Python 1067b.attach_uprobe(name="c", sym="getaddrinfo", fn_name="do_entry") 1068b.attach_uprobe(name="/usr/bin/python", sym="main", fn_name="do_main") 1069``` 1070 1071You can call attach_uprobe() more than once, and attach your BPF function to multiple user-level functions. 1072 1073See the previous uprobes section for how to instrument arguments from BPF. 1074 1075Examples in situ: 1076[search /examples](https://github.com/iovisor/bcc/search?q=attach_uprobe+path%3Aexamples+language%3Apython&type=Code), 1077[search /tools](https://github.com/iovisor/bcc/search?q=attach_uprobe+path%3Atools+language%3Apython&type=Code) 1078 1079### 5. attach_uretprobe() 1080 1081Syntax: ```BPF.attach_uretprobe(name="location", sym="symbol", fn_name="name")``` 1082 1083Instruments the return of the user-level function ```symbol()``` from either the library or binary named by ```location``` using user-level dynamic tracing of the function return, and attach our C defined function ```name()``` to be called whenever the user-level function returns. 1084 1085For example: 1086 1087```Python 1088b.attach_uretprobe(name="c", sym="strlen", fn_name="count") 1089``` 1090 1091This will instrument ```strlen()``` function from libc, and call our BPF function ```count()``` when it returns. 1092 1093Other examples: 1094 1095```Python 1096b.attach_uprobe(name="c", sym="getaddrinfo", fn_name="do_entry") 1097b.attach_uprobe(name="/usr/bin/python", sym="main", fn_name="do_main") 1098``` 1099 1100You can call attach_uretprobe() more than once, and attach your BPF function to multiple user-level functions. 1101 1102See the previous uretprobes section for how to instrument the return value from BPF. 1103 1104Examples in situ: 1105[search /examples](https://github.com/iovisor/bcc/search?q=attach_uretprobe+path%3Aexamples+language%3Apython&type=Code), 1106[search /tools](https://github.com/iovisor/bcc/search?q=attach_uretprobe+path%3Atools+language%3Apython&type=Code) 1107 1108### 6. USDT.enable_probe() 1109 1110Syntax: ```USDT.enable_probe(probe=probe, fn_name=name)``` 1111 1112Attaches a BPF C function ```name``` to the USDT probe ```probe```. 1113 1114Example: 1115 1116```Python 1117# enable USDT probe from given PID 1118u = USDT(pid=int(pid)) 1119u.enable_probe(probe="http__server__request", fn_name="do_trace") 1120``` 1121 1122To check if your binary has USDT probes, and what they are, you can run ```readelf -n binary``` and check the stap debug section. 1123 1124Examples in situ: 1125[search /examples](https://github.com/iovisor/bcc/search?q=enable_probe+path%3Aexamples+language%3Apython&type=Code), 1126[search /tools](https://github.com/iovisor/bcc/search?q=enable_probe+path%3Atools+language%3Apython&type=Code) 1127 1128### 7. attach_raw_tracepoint() 1129 1130Syntax: ```BPF.attach_raw_tracepoint(tp="tracepoint", fn_name="name")``` 1131 1132Instruments the kernel raw tracepoint described by ```tracepoint``` (```event``` only, no ```category```), and when hit, runs the BPF function ```name()```. 1133 1134This is an explicit way to instrument tracepoints. The ```RAW_TRACEPOINT_PROBE``` syntax, covered in the earlier raw tracepoints section, is an alternate method. 1135 1136For example: 1137 1138```Python 1139b.attach_raw_tracepoint("sched_swtich", "do_trace") 1140``` 1141 1142Examples in situ: 1143[search /tools](https://github.com/iovisor/bcc/search?q=attach_raw_tracepoint+path%3Atools+language%3Apython&type=Code) 1144 1145## Debug Output 1146 1147### 1. trace_print() 1148 1149Syntax: ```BPF.trace_print(fmt="fields")``` 1150 1151This method continually reads the globally shared /sys/kernel/debug/tracing/trace_pipe file and prints its contents. This file can be written to via BPF and the bpf_trace_printk() function, however, that method has limitations, including a lack of concurrent tracing support. The BPF_PERF_OUTPUT mechanism, covered earlier, is preferred. 1152 1153Arguments: 1154 1155- ```fmt```: optional, and can contain a field formatting string. It defaults to ```None```. 1156 1157Examples: 1158 1159```Python 1160# print trace_pipe output as-is: 1161b.trace_print() 1162 1163# print PID and message: 1164b.trace_print(fmt="{1} {5}") 1165``` 1166 1167Examples in situ: 1168[search /examples](https://github.com/iovisor/bcc/search?q=trace_print+path%3Aexamples+language%3Apython&type=Code), 1169[search /tools](https://github.com/iovisor/bcc/search?q=trace_print+path%3Atools+language%3Apython&type=Code) 1170 1171### 2. trace_fields() 1172 1173Syntax: ```BPF.trace_fields(nonblocking=False)``` 1174 1175This method reads one line from the globally shared /sys/kernel/debug/tracing/trace_pipe file and returns it as fields. This file can be written to via BPF and the bpf_trace_printk() function, however, that method has limitations, including a lack of concurrent tracing support. The BPF_PERF_OUTPUT mechanism, covered earlier, is preferred. 1176 1177Arguments: 1178 1179- ```nonblocking```: optional, defaults to ```False```. When set to ```True```, the program will not block waiting for input. 1180 1181Examples: 1182 1183```Python 1184while 1: 1185 try: 1186 (task, pid, cpu, flags, ts, msg) = b.trace_fields() 1187 except ValueError: 1188 continue 1189 [...] 1190``` 1191 1192Examples in situ: 1193[search /examples](https://github.com/iovisor/bcc/search?q=trace_print+path%3Aexamples+language%3Apython&type=Code), 1194[search /tools](https://github.com/iovisor/bcc/search?q=trace_print+path%3Atools+language%3Apython&type=Code) 1195 1196## Output 1197 1198Normal output from a BPF program is either: 1199 1200- per-event: using PERF_EVENT_OUTPUT, open_perf_buffer(), and perf_buffer_poll(). 1201- map summary: using items(), or print_log2_hist(), covered in the Maps section. 1202 1203### 1. perf_buffer_poll() 1204 1205Syntax: ```BPF.perf_buffer_poll()``` 1206 1207This polls from all open perf ring buffers, calling the callback function that was provided when calling open_perf_buffer for each entry. 1208 1209Example: 1210 1211```Python 1212# loop with callback to print_event 1213b["events"].open_perf_buffer(print_event) 1214while 1: 1215 b.perf_buffer_poll() 1216``` 1217 1218Examples in situ: 1219[code](https://github.com/iovisor/bcc/blob/08fbceb7e828f0e3e77688497727c5b2405905fd/examples/tracing/hello_perf_output.py#L61), 1220[search /examples](https://github.com/iovisor/bcc/search?q=perf_buffer_poll+path%3Aexamples+language%3Apython&type=Code), 1221[search /tools](https://github.com/iovisor/bcc/search?q=perf_buffer_poll+path%3Atools+language%3Apython&type=Code) 1222 1223## Maps 1224 1225Maps are BPF data stores, and are used in bcc to implement a table, and then higher level objects on top of tables, including hashes and histograms. 1226 1227### 1. get_table() 1228 1229Syntax: ```BPF.get_table(name)``` 1230 1231Returns a table object. This is no longer used, as tables can now be read as items from BPF. Eg: ```BPF[name]```. 1232 1233Examples: 1234 1235```Python 1236counts = b.get_table("counts") 1237 1238counts = b["counts"] 1239``` 1240 1241These are equivalent. 1242 1243### 2. open_perf_buffer() 1244 1245Syntax: ```table.open_perf_buffers(callback, page_cnt=N, lost_cb=None)``` 1246 1247This operates on a table as defined in BPF as BPF_PERF_OUTPUT(), and associates the callback Python function ```callback``` to be called when data is available in the perf ring buffer. This is part of the recommended mechanism for transferring per-event data from kernel to user space. The size of the perf ring buffer can be specified via the ```page_cnt``` parameter, which must be a power of two number of pages and defaults to 8. If the callback is not processing data fast enough, some submitted data may be lost. ```lost_cb``` will be called to log / monitor the lost count. If ```lost_cb``` is the default ```None``` value, it will just print a line of message to ```stderr```. 1248 1249Example: 1250 1251```Python 1252# process event 1253def print_event(cpu, data, size): 1254 event = ct.cast(data, ct.POINTER(Data)).contents 1255 [...] 1256 1257# loop with callback to print_event 1258b["events"].open_perf_buffer(print_event) 1259while 1: 1260 b.perf_buffer_poll() 1261``` 1262 1263Note that the data structure transferred will need to be declared in C in the BPF program, and in Python. For example: 1264 1265```C 1266// define output data structure in C 1267struct data_t { 1268 u32 pid; 1269 u64 ts; 1270 char comm[TASK_COMM_LEN]; 1271}; 1272``` 1273 1274```Python 1275# define output data structure in Python 1276TASK_COMM_LEN = 16 # linux/sched.h 1277class Data(ct.Structure): 1278 _fields_ = [("pid", ct.c_ulonglong), 1279 ("ts", ct.c_ulonglong), 1280 ("comm", ct.c_char * TASK_COMM_LEN)] 1281``` 1282 1283Perhaps in a future bcc version, the Python data structure will be automatically generated from the C declaration. 1284 1285Examples in situ: 1286[code](https://github.com/iovisor/bcc/blob/08fbceb7e828f0e3e77688497727c5b2405905fd/examples/tracing/hello_perf_output.py#L59), 1287[search /examples](https://github.com/iovisor/bcc/search?q=open_perf_buffer+path%3Aexamples+language%3Apython&type=Code), 1288[search /tools](https://github.com/iovisor/bcc/search?q=open_perf_buffer+path%3Atools+language%3Apython&type=Code) 1289 1290### 3. items() 1291 1292Syntax: ```table.items()``` 1293 1294Returns an array of the keys in a table. This can be used with BPF_HASH maps to fetch, and iterate, over the keys. 1295 1296Example: 1297 1298```Python 1299# print output 1300print("%10s %s" % ("COUNT", "STRING")) 1301counts = b.get_table("counts") 1302for k, v in sorted(counts.items(), key=lambda counts: counts[1].value): 1303 print("%10d \"%s\"" % (v.value, k.c.encode('string-escape'))) 1304``` 1305 1306This example also uses the ```sorted()``` method to sort by value. 1307 1308Examples in situ: 1309[search /examples](https://github.com/iovisor/bcc/search?q=clear+items%3Aexamples+language%3Apython&type=Code), 1310[search /tools](https://github.com/iovisor/bcc/search?q=clear+items%3Atools+language%3Apython&type=Code) 1311 1312### 4. values() 1313 1314Syntax: ```table.values()``` 1315 1316Returns an array of the values in a table. 1317 1318### 5. clear() 1319 1320Syntax: ```table.clear()``` 1321 1322Clears the table: deletes all entries. 1323 1324Example: 1325 1326```Python 1327# print map summary every second: 1328while True: 1329 time.sleep(1) 1330 print("%-8s\n" % time.strftime("%H:%M:%S"), end="") 1331 dist.print_log2_hist(sym + " return:") 1332 dist.clear() 1333``` 1334 1335Examples in situ: 1336[search /examples](https://github.com/iovisor/bcc/search?q=clear+path%3Aexamples+language%3Apython&type=Code), 1337[search /tools](https://github.com/iovisor/bcc/search?q=clear+path%3Atools+language%3Apython&type=Code) 1338 1339### 6. print_log2_hist() 1340 1341Syntax: ```table.print_log2_hist(val_type="value", section_header="Bucket ptr", section_print_fn=None)``` 1342 1343Prints a table as a log2 histogram in ASCII. The table must be stored as log2, which can be done using the BPF function ```bpf_log2l()```. 1344 1345Arguments: 1346 1347- val_type: optional, column header. 1348- section_header: if the histogram has a secondary key, multiple tables will print and section_header can be used as a header description for each. 1349- section_print_fn: if section_print_fn is not None, it will be passed the bucket value. 1350 1351Example: 1352 1353```Python 1354b = BPF(text=""" 1355BPF_HISTOGRAM(dist); 1356 1357int kprobe__blk_account_io_completion(struct pt_regs *ctx, struct request *req) 1358{ 1359 dist.increment(bpf_log2l(req->__data_len / 1024)); 1360 return 0; 1361} 1362""") 1363[...] 1364 1365b["dist"].print_log2_hist("kbytes") 1366``` 1367 1368Output: 1369 1370``` 1371 kbytes : count distribution 1372 0 -> 1 : 3 | | 1373 2 -> 3 : 0 | | 1374 4 -> 7 : 211 |********** | 1375 8 -> 15 : 0 | | 1376 16 -> 31 : 0 | | 1377 32 -> 63 : 0 | | 1378 64 -> 127 : 1 | | 1379 128 -> 255 : 800 |**************************************| 1380``` 1381 1382This output shows a multi-modal distribution, with the largest mode of 128->255 kbytes and a count of 800. 1383 1384This is an efficient way to summarize data, as the summarization is performed in-kernel, and only the count column is passed to user space. 1385 1386Examples in situ: 1387[search /examples](https://github.com/iovisor/bcc/search?q=print_log2_hist+path%3Aexamples+language%3Apython&type=Code), 1388[search /tools](https://github.com/iovisor/bcc/search?q=print_log2_hist+path%3Atools+language%3Apython&type=Code) 1389 1390### 6. print_linear_hist() 1391 1392Syntax: ```table.print_linear_hist(val_type="value", section_header="Bucket ptr", section_print_fn=None)``` 1393 1394Prints a table as a linear histogram in ASCII. This is intended to visualize small integer ranges, eg, 0 to 100. 1395 1396Arguments: 1397 1398- val_type: optional, column header. 1399- section_header: if the histogram has a secondary key, multiple tables will print and section_header can be used as a header description for each. 1400- section_print_fn: if section_print_fn is not None, it will be passed the bucket value. 1401 1402Example: 1403 1404```Python 1405b = BPF(text=""" 1406BPF_HISTOGRAM(dist); 1407 1408int kprobe__blk_account_io_completion(struct pt_regs *ctx, struct request *req) 1409{ 1410 dist.increment(req->__data_len / 1024); 1411 return 0; 1412} 1413""") 1414[...] 1415 1416b["dist"].print_linear_hist("kbytes") 1417``` 1418 1419Output: 1420 1421``` 1422 kbytes : count distribution 1423 0 : 3 |****** | 1424 1 : 0 | | 1425 2 : 0 | | 1426 3 : 0 | | 1427 4 : 19 |****************************************| 1428 5 : 0 | | 1429 6 : 0 | | 1430 7 : 0 | | 1431 8 : 4 |******** | 1432 9 : 0 | | 1433 10 : 0 | | 1434 11 : 0 | | 1435 12 : 0 | | 1436 13 : 0 | | 1437 14 : 0 | | 1438 15 : 0 | | 1439 16 : 2 |**** | 1440[...] 1441``` 1442 1443This is an efficient way to summarize data, as the summarization is performed in-kernel, and only the values in the count column are passed to user space. 1444 1445Examples in situ: 1446[search /examples](https://github.com/iovisor/bcc/search?q=print_linear_hist+path%3Aexamples+language%3Apython&type=Code), 1447[search /tools](https://github.com/iovisor/bcc/search?q=print_linear_hist+path%3Atools+language%3Apython&type=Code) 1448 1449## Helpers 1450 1451Some helper methods provided by bcc. Note that since we're in Python, we can import any Python library and their methods, including, for example, the libraries: argparse, collections, ctypes, datetime, re, socket, struct, subprocess, sys, and time. 1452 1453### 1. ksym() 1454 1455Syntax: ```BPF.ksym(addr)``` 1456 1457Translate a kernel memory address into a kernel function name, which is returned. 1458 1459Example: 1460 1461```Python 1462print("kernel function: " + b.ksym(addr)) 1463``` 1464 1465Examples in situ: 1466[search /examples](https://github.com/iovisor/bcc/search?q=ksym+path%3Aexamples+language%3Apython&type=Code), 1467[search /tools](https://github.com/iovisor/bcc/search?q=ksym+path%3Atools+language%3Apython&type=Code) 1468 1469### 2. ksymname() 1470 1471Syntax: ```BPF.ksymname(name)``` 1472 1473Translate a kernel name into an address. This is the reverse of ksym. Returns -1 when the function name is unknown. 1474 1475Example: 1476 1477```Python 1478print("kernel address: %x" % b.ksymname("vfs_read")) 1479``` 1480 1481Examples in situ: 1482[search /examples](https://github.com/iovisor/bcc/search?q=ksymname+path%3Aexamples+language%3Apython&type=Code), 1483[search /tools](https://github.com/iovisor/bcc/search?q=ksymname+path%3Atools+language%3Apython&type=Code) 1484 1485### 3. sym() 1486 1487Syntax: ```BPF.sym(addr, pid, show_module=False, show_offset=False)``` 1488 1489Translate a memory address into a function name for a pid, which is returned. A pid of less than zero will access the kernel symbol cache. The `show_module` and `show_offset` parameters control whether the module in which the symbol lies should be displayed, and whether the instruction offset from the beginning of the symbol should be displayed. These extra parameters default to `False`. 1490 1491Example: 1492 1493```Python 1494print("function: " + b.sym(addr, pid)) 1495``` 1496 1497Examples in situ: 1498[search /examples](https://github.com/iovisor/bcc/search?q=sym+path%3Aexamples+language%3Apython&type=Code), 1499[search /tools](https://github.com/iovisor/bcc/search?q=sym+path%3Atools+language%3Apython&type=Code) 1500 1501### 4. num_open_kprobes() 1502 1503Syntax: ```BPF.num_open_kprobes()``` 1504 1505Returns the number of open k[ret]probes. Can be useful for scenarios where event_re is used while attaching and detaching probes. Excludes perf_events readers. 1506 1507Example: 1508 1509```Python 1510b.attach_kprobe(event_re=pattern, fn_name="trace_count") 1511matched = b.num_open_kprobes() 1512if matched == 0: 1513 print("0 functions matched by \"%s\". Exiting." % args.pattern) 1514 exit() 1515``` 1516 1517Examples in situ: 1518[search /examples](https://github.com/iovisor/bcc/search?q=num_open_kprobes+path%3Aexamples+language%3Apython&type=Code), 1519[search /tools](https://github.com/iovisor/bcc/search?q=num_open_kprobes+path%3Atools+language%3Apython&type=Code) 1520 1521# BPF Errors 1522 1523See the "Understanding eBPF verifier messages" section in the kernel source under Documentation/networking/filter.txt. 1524 1525## 1. Invalid mem access 1526 1527This can be due to trying to read memory directly, instead of operating on memory on the BPF stack. All memory reads must be passed via bpf_probe_read() to copy memory into the BPF stack, which can be automatic by the bcc rewriter in some cases of simple dereferencing. bpf_probe_read() does all the required checks. 1528 1529Example: 1530 1531``` 1532bpf: Permission denied 15330: (bf) r6 = r1 15341: (79) r7 = *(u64 *)(r6 +80) 15352: (85) call 14 15363: (bf) r8 = r0 1537[...] 153823: (69) r1 = *(u16 *)(r7 +16) 1539R7 invalid mem access 'inv' 1540 1541Traceback (most recent call last): 1542 File "./tcpaccept", line 179, in <module> 1543 b = BPF(text=bpf_text) 1544 File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 172, in __init__ 1545 self._trace_autoload() 1546 File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 612, in _trace_autoload 1547 fn = self.load_func(func_name, BPF.KPROBE) 1548 File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 212, in load_func 1549 raise Exception("Failed to load BPF program %s" % func_name) 1550Exception: Failed to load BPF program kretprobe__inet_csk_accept 1551``` 1552 1553## 2. Cannot call GPL only function from proprietary program 1554 1555This error happens when a GPL-only helper is called from a non-GPL BPF program. To fix this error, do not use GPL-only helpers from a proprietary BPF program, or relicense the BPF program under a GPL-compatible license. Check which [BPF helpers](https://github.com/iovisor/bcc/blob/master/docs/kernel-versions.md#helpers) are GPL-only, and what licenses are considered GPL-compatible. 1556 1557Example calling `bpf_get_stackid()`, a GPL-only BPF helper, from a proprietary program (`#define BPF_LICENSE Proprietary`): 1558 1559``` 1560bpf: Failed to load program: Invalid argument 1561[...] 15628: (85) call bpf_get_stackid#27 1563cannot call GPL only function from proprietary program 1564``` 1565 1566# Environment Variables 1567 1568## 1. Kernel source directory 1569 1570eBPF program compilation needs kernel sources or kernel headers with headers 1571compiled. In case your kernel sources are at a non-standard location where BCC 1572cannot find then, its possible to provide BCC the absolute path of the location 1573by setting `BCC_KERNEL_SOURCE` to it. 1574 1575## 2. Kernel version overriding 1576 1577By default, BCC stores the `LINUX_VERSION_CODE` in the generated eBPF object 1578which is then passed along to the kernel when the eBPF program is loaded. 1579Sometimes this is quite inconvenient especially when the kernel is slightly 1580updated such as an LTS kernel release. Its extremely unlikely the slight 1581mismatch would cause any issues with the loaded eBPF program. By setting 1582`BCC_LINUX_VERSION_CODE` to the version of the kernel that's running, the check 1583for verifying the kernel version can be bypassed. This is needed for programs 1584that use kprobes. This needs to be encoded in the format: `(VERSION * 65536) + 1585(PATCHLEVEL * 256) + SUBLEVEL`. For example, if the running kernel is `4.9.10`, 1586then can set `export BCC_LINUX_VERSION_CODE=264458` to override the kernel 1587version check successfully. 1588