failed to create BPF link for perf_event FD X: -EACCES

[gmarler]

When attempting to attach a program to the syscalls/sys_exit_munmap tracepoint via libbpf-rs, I get this error and panic:

2025-10-22T19:22:49.127031Z  INFO lightswitch::profiler: munmap and process exit tracing BPF programs loaded
libbpf: prog 'tracer_exit_munmap': failed to create BPF link for perf_event FD 186: -EACCES
libbpf: prog 'tracer_exit_munmap': failed to attach to tracepoint 'syscalls/sys_exit_munmap': -EACCES
libbpf: prog 'tracer_exit_munmap': failed to auto-attach: -EACCES

And I'm running this as the root user on a physical machine.

Not seeing any issues in the libbpf debug log or via bpftool log messaging, so it doesn't look like a validation error either.

The BPF program in question resides here.

I'm not clear on if this is something I'm doing wrong, or an actual bug in libbpf or libbpf-rs.

[gmarler]

Here is the full libbpf debug output for trying to run the program:

lightswitch-munmap-bpf-errors.txt

[danielocfb]

Perhaps try out using retsnoop to understand where the error is coming from in the kernel, exactly.

[gmarler]

Great idea! A basic look for EACCES seems to indicate a problem in SElinux, even though I've confirmed it's completely disabled - or possibly security_file_alloc():

# src/retsnoop -e '*sys_bpf' -a ':kernel/bpf/*.c' -x -EACCES -T -k /usr/lib/debug/usr/lib/modules/5.14.0-570.24.1.el9_6.x86_64/vmlinux
Receiving data...
15:27:33.126185 -> 15:27:33.126401 TID/PID 139078/139078 (lightswitch/lightswitch):

FUNCTION CALLS                                      RESULT                 DURATION
-------------------------------------------------   --------------------  ---------
→ __x64_sys_bpf
    → __sys_bpf
        ↔ bpf_check_uarg_tail_zero                  [0]                     4.376us
        ↔ bpf_lsm_bpf                               [0]                    15.146us
        → link_create
            ↔ __bpf_prog_get                        [0xffffb4f599621000]   19.622us
            ↔ bpf_prog_attach_check_attach_type     [0]                     2.678us
            → bpf_perf_link_attach
                → bpf_link_prime
                    ↔ bpf_lsm_file_alloc_security   [0]                    11.190us
                ← bpf_link_prime                    [0]                    30.334us
            ← bpf_perf_link_attach                  [-EACCES]              51.162us
            ↔ __bpf_prog_put                        [void]                 10.655us
        ← link_create                               [-EACCES]             142.812us
    ← __sys_bpf                                     [-EACCES]             201.345us
← __x64_sys_bpf                                     [-EACCES]             209.492us

Need to figure out how to drill more deeply into bpf_perf_link_attach() to see where EACCES is originating from.

[gmarler]

Ok, drilled down to perf_event_set_bpf_prog():

# retsnoop -e '*sys_bpf' -a 'bpf_*' -a 'perf_event*' -a '*alloc*' -a '*link*' -a '*lsm*' -a '*selinux*' -a ':kernel/bpf/*.c' -x -EACCES -T -k /usr/lib/debug/usr/lib/modules/5.   ↳14.0-570.24.1.el9_6.x86_64/vmlinux
Receiving data...

16:28:09.967692 -> 16:28:09.970909 TID/PID 158361/158361 (lightswitch/lightswitch):                                                                        12:28:09 [16/679673]

FUNCTION CALLS                                                      RESULT                  DURATION
-----------------------------------------------------------------   --------------------  ----------
→ __x64_sys_bpf
    → __sys_bpf
        ↔ bpf_check_uarg_tail_zero                                  [0]                     15.307us                                                                                   ↔ selinux_bpf                                               [0]                      9.267us                                                                                   ↔ bpf_lsm_bpf                                               [0]                     12.982us                                                                                   → link_create
            ↔ __bpf_prog_get                                        [0xffffb4f599182000]    18.989us                                                                                       ↔ bpf_prog_attach_check_attach_type                     [0]                     15.351us
            → bpf_perf_link_attach
                ↔ perf_event_get                                    [0xffff9fa45bf16e00]    21.206us
                ↔ kmalloc_trace                                     [0xffff9fa464753000]    83.938us
                → bpf_link_prime
                    ↔ alloc_fd                                      [179]                   61.022us
                    → alloc_file_pseudo
                        → d_alloc_pseudo
                            → __d_alloc
                                → kmem_cache_alloc_lru
                                    ↔ memcg_list_lru_alloc          [0]                     33.206us
                                ← kmem_cache_alloc_lru              [0xffff9fa45a7ad080]   113.326us
                            ← __d_alloc                             [0xffff9fa45a7ad080]   172.360us
                        ← d_alloc_pseudo                            [0xffff9fa45a7ad080]   262.705us
                        → alloc_empty_file
                            → __alloc_file
                                ↔ kmem_cache_alloc                  [0xffff9fa45bf17600]    10.250us
                                → security_file_alloc
                                    ↔ kmem_cache_alloc              [0xffff9fa46a8221e0]    14.349us
                                    ↔ hook_file_alloc_security      [0]                    367.762us
                                    ↔ selinux_file_alloc_security   [0]                     83.986us
                                    ↔ bpf_lsm_file_alloc_security   [0]                     73.653us
                                ← security_file_alloc               [0]                    768.696us
                            ← __alloc_file                          [0xffff9fa45bf17600]   862.626us
                        ← alloc_empty_file                          [0xffff9fa45bf17600]   927.942us
                    ← alloc_file_pseudo                             [0xffff9fa45bf17600]  1291.241us
                ← bpf_link_prime                                    [0]                   1458.026us
                ↔ perf_event_set_bpf_prog                           [-EACCES]               11.329us
            ← bpf_perf_link_attach                                  [-EACCES]             1983.027us
            ↔ __bpf_prog_put                                        [void]                  13.764us
        ← link_create                                               [-EACCES]             2114.736us
    ← __sys_bpf                                                     [-EACCES]             2221.251us
    ↔ perf_event_task_tick                                          [void]                 286.350us
← __x64_sys_bpf                                                     [-EACCES]             3191.767us

This is one of Redhat's RHEL 9.x kernels, so it looks like this is either a configuration issue, or a bug they'll have to drill into, rather than anything to do with libbpf-rs or even libbpf - we'll see.

[gmarler]

Well, maybe not - this seems to be a verifier check against prog->aux->max_ctx_offset being exceeded here.

So this could still be my code, or a libbpf/libbpf-rs problem.

I'll note that this is the last BPF program to get attached, where all previous ones in the Rust/libbpf-rs program startup attach/link successfully, so perhaps an off by one issue?

[anakryiko]

@gmarler can you share source code of your tracepoint program? if you are trying to access some string content or something like that, you might need to use bpf_probe_read_kernel() to get that part.

[gmarler]

Of course, it's here, repeated from above.

Not trying to access any string, rather a map entry initialized by the matching tracepoint/syscalls/sys_enter_munmap program, which is here.

Then a tracer_event_t is constructed and dropped off in a ring buffer.

[yonghong-song]

Okay, I agree that the -EACCES is triggered by the following code in kernel/events/core.c:

        if (is_tracepoint || is_syscall_tp) {
                int off = trace_event_get_offsets(event->tp_event);

                if (prog->aux->max_ctx_offset > off)
                        return -EACCES;
        }

Which kernel you are actually using? One of above kernel reference is
https://elixir.bootlin.com/linux/v5.19.17/source/kernel/events/core.c#L10097
which is kernel 5.19.17.

Maybe the kernel is too old or incomplete backport?

[gmarler]

It's a Redhat RHEL 9.x kernel, so the baseline is Linux 5.14.0, but it's got lots of backports from later Linux 5.x and even 6.x.

Note that it can't be because the kernel is too old in general, as I've loaded several other BPF programs with my libbpf-rs program - this program just happens to be the very last one I load, which makes me suspicious. And even their RHEL 8.x kernels had enough backports from Linux 5.x and 6.x for this program to work without issues for over a year now.

I've opened a case with Redhat to get the exact info on the kernel in question and to drill into from their end, but it could be any of:

• Incomplete backport, as you noted
• The size of the last program my program tries to load isn't being calculated correctly on the Rust or the kernel side - I posted the libbpf debug log earlier above, so that may give clues
• The offset(s) for the entry and exit from that munmap() tracepoint that I've instrumented has been borked by me in some way I don't realize I've done, so the offset comparison being done isn't right.