Skip to main content
UltraInstinct
Back to latest articles
Cybersecurity13 min read

How Linux eBPF Rootkits Evade Detection in Production

Learn how Linux eBPF rootkits use bpf_probe_write_user, modify syscall buffers, hook tracepoints, and bypass runtime security agents like Falco and Tetragon.

Featured visual representing How Linux eBPF Rootkits Evade Detection in Production

Traditional Linux rootkits modified the kernel by inserting custom loadable kernel modules (.ko) or patching the system call table (sys_call_table) directly in memory. Modern Linux kernel security mitigations, including strict module signature verification (CONFIG_MODULE_SIG_FORCE), kernel page table isolation, and lockdown modes, have made out-of-tree kernel text modification largely impractical on locked-down production systems. In response, attacker techniques have adapted to the kernel's native programmable execution environment. Linux eBPF rootkits use the extended Berkeley Packet Filter virtual machine to execute hostile logic directly inside supervisor context without inserting out-of-tree modules or altering compiled kernel instructions.

Because eBPF runs within an in-kernel JIT-compiled engine verified for memory safety prior to loading, endpoint detection and response (EDR) agents and observability daemons frequently treat eBPF objects as trustworthy telemetry sources. Defensive tools hook the same tracepoints, kprobes, and Linux Security Module (LSM) hooks that an adversary can target. When an attacker possesses elevated capabilities (CAP_BPF, CAP_PERFMON, or CAP_SYS_ADMIN), rogue eBPF programs sit alongside defensive monitors. From this vantage point, offensive bytecode alters execution context, masks processes, and intercepts network packets before user-space logging daemons receive an event.

Understanding how these stealth techniques work requires examining the verifier's operational boundary, how specific in-kernel helpers alter user memory, how execution ordering affects event-driven monitoring tools, and what concrete forensic controls can expose them.

The Verifier Boundary and How Linux eBPF Rootkits Pass Validation

Every eBPF program submitted via the sys_bpf system call must pass through the kernel verifier before the Just-In-Time (JIT) compiler emits native machine instructions. The verifier parses the bytecode into a directed acyclic graph (DAG) to evaluate all possible control-flow paths. Its job is strictly bounded: it guarantees that the program cannot dereference invalid pointers, read uninitialized stack slots, cause null-pointer dereferences, access arbitrary kernel memory outside sanctioned helper abstractions, or enter infinite execution loops.

The verifier tracks register states across six primary register types: NOT_INIT, SCALAR_VALUE, PTR_TO_CTX, PTR_TO_STACK, PTR_TO_MAP_VALUE, and PTR_TO_MEM. For any pointer arithmetic, the verifier computes minimum and maximum variable bounds (umin_value, umax_value, smin_value, smax_value). If a memory access falls outside a known buffer size, the verifier rejects the program with -EACCES.

Critically, the verifier does not evaluate program intent. It validates safety invariants, not operational semantics. An eBPF program that monitors a process for debugging purposes looks architecturally identical to a program designed to silence security alerts. If an offensive program obeys the instruction count limits (up to 1,000,000 instructions for root programs), accesses only valid stack offsets, and restricts itself to the helper functions allowed for its program type (BPF_PROG_TYPE_KPROBE, BPF_PROG_TYPE_TRACEPOINT, or BPF_PROG_TYPE_LSM), the verifier marks the bytecode safe and passes it to the JIT compiler.

The kernel includes several built-in helper functions that offensive developers repurpose:

  1. bpf_probe_write_user: Writes arbitrary data from kernel buffers into user-space virtual addresses belonging to the current task.
  2. bpf_override_return: Injects synthetic return values into kernel functions to short-circuit system call execution, requiring kernels built with CONFIG_BPF_KPROBE_OVERRIDE.
  3. bpf_map_lookup_elem and bpf_map_update_elem: Facilitates cross-probe state tracking, command-and-control communication, and user-space synchronization via BPF maps.

Because these helpers are official parts of the kernel API, using them does not trigger an execution fault or an anomalous memory violation inside the kernel.

In-Flight Syscall Tampering with bpf_probe_write_user

The primary mechanism for altering application behavior without touching kernel text is bpf_probe_write_user. The helper signature is straightforward:

long bpf_probe_write_user(void *dst, const void *src, u32 len);

When invoked, the helper verifies that the target address dst falls within the user-space address space bounds (TASK_SIZE) of the calling task using access_ok(). It checks that the current context is not an interrupt handler (in_interrupt()). If those conditions pass, it calls the internal architecture-specific memory copy routine (such as copy_to_user_nofault). The kernel emits a one-time dmesg warning (bpf_probe_write_user: ...) on the very first invocation across the entire boot lifetime, but subsequent calls execute silently.

Attackers hook kprobes on sys_enter architectures to rewrite parameters in flight. Consider process execution tampering: an administrator runs a routine binary, or an automation tool calls /bin/ps. The rootkit hooks the entrypoint to the execve system call (__x64_sys_execve), inspects the target path in user memory using bpf_probe_read_user_str, and overwrites the pathname buffer with an alternative path before the kernel's do_execveat_common reads the string from user memory into kernel space.

The following simplified eBPF snippet demonstrates this parameter mutation on an x86_64 architecture:

#include 
#include 
#include 

char LICENSE[] SEC("license") = "GPL";

SEC("kprobe/__x64_sys_execve")
int BPF_KPROBE(tamper_execve, struct pt_regs *regs) {
    char target_binary[] = "/bin/dmesg";
    char replacement_binary[] = "/bin/true";
    char path_buf[16] = {0};

    // Extract the pointer to the filename argument (first syscall argument in rdi)
    void *user_filename_ptr = (void *)PT_REGS_PARM1(regs);
    if (!user_filename_ptr)
        return 0;

    // Read the user-supplied string
    long ret = bpf_probe_read_user_str(path_buf, sizeof(path_buf), user_filename_ptr);
    if (ret < 0)
        return 0;

    // Compare against the target binary
    for (int i = 0; i < 10; i++) {
        if (path_buf[i] != target_binary[i])
            return 0;
    }

    // Overwrite the user buffer in place prior to kernel consumption
    bpf_probe_write_user(user_filename_ptr, replacement_binary, sizeof(replacement_binary));
    return 0;
}

Because bpf_probe_write_user modifies the memory in the context of the running thread before the kernel copies the path into its internal struct filename, the kernel executes the altered binary path. The original string requested by the caller never enters the kernel's execution pipeline.

Blind Spots in Event-Driven Detection Engines

Security platforms such as Falco, Tetragon, and Tracee rely on eBPF to monitor Linux systems. They place probes on raw tracepoints (sys_enter_*, sys_exit_*) or LSM hooks (security_bprm_check, security_file_open) and push event structures into an asynchronous queue, such as a BPF_MAP_TYPE_PERF_EVENT_ARRAY or a BPF_MAP_TYPE_RINGBUF. User-space daemons consume this queue, evaluate runtime detection rules, and alert on malicious actions.

This architecture introduces three operational blind spots:

1. Probe Attachment Ordering and Race Conditions

When multiple eBPF programs hook the same kernel event or tracepoint, the kernel does not guarantee priority based on security posture. If an attacker attaches a kprobe to __x64_sys_execve before the security agent attaches its monitoring probe, the rootkit modifies the user-space argument buffer before the monitor can read the original value. The monitoring agent's probe reads the already-mutated string.

Conversely, if the monitoring agent hooks sys_enter while the rootkit hooks an internal function called immediately after (such as do_sys_openat2), the monitoring agent records the clean path requested at syscall entry, while the rootkit mutates the buffer in memory before the file descriptor is allocated. This time-of-check to time-of-use (TOCTOU) discrepancy allows the attack to bypass static syscall validation.

2. Telemetry Queue Starvation

Security monitoring tools rely on low-latency consumer loops in user space to pull events from kernel ring buffers. This architecture shares mechanical similarities with high-throughput kernel-to-user queues analyzed in io_uring vs pread: How Batched NVMe I/O Actually Scales, where consumer processing delays allow kernel ring buffers to fill rapidly.

If an attacker floods the system with monitored events (such as high-frequency read or fstat cycles), the kernel ring buffer drops telemetry records due to queue exhaustion. While the defensive agent struggles with buffer overflows, the malicious eBPF program executes targeted operations unrecorded.

3. File Descriptor Manipulation

eBPF objects are identified and accessed via file descriptors. When a defensive agent initializes, it creates maps to store runtime configurations and filter lists. These file descriptors reside in the monitoring daemon's file descriptor table located in /proc/[pid]/fd/.

An attacker running with system privileges can inspect the target process, locate the BPF map file descriptors, call bpf(BPF_MAP_UPDATE_ELEM, ...), and overwrite the defensive agent's rules directly in kernel memory. If the rootkit suppresses alerting rules within the agent's own map, the defensive agent goes blind without crashing or restarting.

Process Hiding and Memory Redirection Internals

One of the most persistent operational goals of a rootkit is hiding processes from auditing tools like ps, pstree, and top. In Linux, these utilities do not query an internal kernel IPC bus; they parse entries within the pseudo-filesystem mounted at /proc. When an administrator runs ps, the tool calls the getdents64 system call on /proc to list all active process IDs represented as directory names.

A rootkit can hook the exit probe of getdents64 (sys_exit_getdents64) to edit the returned directory record array directly in user memory before the calling utility parses it.

The getdents64 system call fills a user buffer with a sequence of variable-length linux_dirent64 structures:

struct linux_dirent64 {
    u64        d_ino;     /* 64-bit inode number */
    s64        d_off;     /* 64-bit offset to next structure */
    unsigned short d_reclen;  /* Size of this dirent */
    unsigned char  d_type;    /* File type */
    char           d_name[];  /* Filename (null-terminated) */
};

When iterating over the buffer, user space reads the record length d_reclen and adds it to the current pointer to find the start of the next linux_dirent64 struct.

To hide a process, the eBPF rootkit executes the following steps inside sys_exit_getdents64:

  1. Retrieves the user buffer address passed during sys_enter_getdents64 and stored in a BPF hash map keyed by thread ID (bpf_get_current_pid_tgid()).
  2. Iterates over the entries in the user buffer using bounded loops (#pragma unroll).
  3. Reads d_name using bpf_probe_read_user_str and checks if the directory name matches a hidden PID stored in a control map.
  4. If a match occurs, the rootkit locates the preceding linux_dirent64 entry and increases its d_reclen field by the length of the hidden entry.
  5. Writes the updated d_reclen back into user memory using bpf_probe_write_user.

When ps parses the directory buffer, its loop increments the pointer by the expanded d_reclen of the preceding record, jumping completely over the hidden process record. The target PID remains running, scheduled by the kernel, but it vanishes from standard user-space accounting.

Buffer layout without rootkit:
[ Dirent A (reclen=32) ] -> [ Dirent B (PID 9999, reclen=32) ] -> [ Dirent C (reclen=32) ]

Buffer layout after rootkit modification:
[ Dirent A (reclen=64)                              ] ----------> [ Dirent C (reclen=32) ]
                             (Dirent B skipped entirely)

Managing this shared state across multiple CPU cores requires maintaining maps of hidden PIDs and network sockets. When multiple CPUs process high rates of getdents64 traversals or socket operations while updating global maps, cross-CPU synchronization can introduce severe latency.

The microarchitectural costs of unaligned data and contended multi-core synchronization mirrors the hardware bottlenecks detailed in How Cache Line False Sharing Degrades App Performance. High-performance rootkits avoid global BPF array contention by utilizing per-CPU hash maps (BPF_MAP_TYPE_PERCPU_HASH), preventing cache coherence stalls that could expose the rootkit via performance anomaly monitoring.

Network evasion uses the same paradigm at the traffic layer. By attaching eBPF programs to the eXpress Data Path (XDP) or Traffic Control (TC) subsystems, a rootkit intercepts inbound and outbound raw Ethernet frames. The program inspects packet headers for specific magic values, drops or modifies packets containing reverse-shell payloads, and prevents packet sniffers using raw AF_PACKET sockets (such as tcpdump or libpcap) from seeing the traffic.

Kernel Hardening and Forensic Detection Mechanisms

Detecting eBPF rootkits requires shifting visibility from user-space syscall monitoring to kernel-level BPF object inspection. Standard system utilities fail because the adversary operates inside the supervisor layer beneath them.

Auditing Loaded Programs with bpftool

The primary utility for inspecting active in-kernel programs is bpftool. It queries the kernel directly via BPF_PROG_GET_NEXT_ID and BPF_MAP_GET_NEXT_ID. Running:

bpftool prog show

outputs all loaded programs, their JIT-compiled sizes, their loaded type, and whether they are attached to kprobes or tracepoints:

142: kprobe  name tamper_execve  tag 3b4a2e6f1c890a21  gpl
    loaded_at 2026-09-18T14:22:01+0000  uid 0
    xlated 368B  jited 244B  memlock 4096B  map_ids 45

If an eBPF rootkit also hooks the bpf() system call itself to filter the output of BPF_PROG_GET_NEXT_ID, running bpftool on the infected host will return false data. Forensic analysis must rely on physical memory acquisition (via LiME or crash dumps) to walk the kernel's bpf_prog_aux structures directly from kernel RAM.

Administrators can extract the JIT-compiled x86_64 assembly instructions to verify the exact bytecode running in supervisor mode:

bpftool prog dump jited id 142

Inspecting the disassembled output reveals invocations of bpf_probe_write_user directly in the instruction stream:

  48:   mov    %rdi,%rax
  4b:   mov    -0x18(%rbp),%rsi
  4f:   mov    $0xa,%edx
  54:   callq  0xffffffff8121f0a0   # 

System Hardening Configurations

Production servers must enforce tight boundaries on the eBPF subsystem. Several kernel controls neutralize rogue program insertion:

First, disable unprivileged eBPF access entirely. When disabled, processes lacking CAP_SYS_ADMIN or CAP_BPF cannot issue calls to sys_bpf:

sysctl -w kernel.unprivileged_bpf_disabled=2

The value 2 locks the configuration permanently; it cannot be reverted to 0 without a full system reboot.

Second, restrict JIT spraying attacks and harden compiled machine instructions:

sysctl -w net.core.bpf_jit_harden=2

A value of 2 enables constant blinding for all programs, inserting randomizing masks over constants to mitigate JIT exploitation techniques.

Third, restrict or disable kprobes entirely if tracing is not required for production services:

sysctl -w debug.kprobes-optimization=0

LSM-Based BPF Program Verification

Modern kernels (5.7+) support BPF LSM (CONFIG_BPF_LSM), allowing security engineers to write eBPF programs that enforce access controls over eBPF itself. The security_bpf and security_bpf_prog LSM hooks intercept program load requests before verification:

#include 
#include 
#include 
#include 

SEC("lsm/bpf_prog")
int BPF_PROG(restrict_ebpf_types, struct bpf_prog *prog, union bpf_attr *attr) {
    // Prohibit attaching kprobes to executive syscall entrypoints
    if (prog->type == BPF_PROG_TYPE_KPROBE) {
        return -EPERM;
    }
    return 0;
}

Combining signed eBPF modules (verifying cryptographic signatures before loading) with mandatory access control policies limits the kernel attack surface, preventing unauthorized administrators or compromised privileged containers from abusing kernel helpers for rootkit execution.

Conclusion

Linux eBPF rootkits do not depend on classical memory safety vulnerabilities or memory corruption bugs. They function by repurposing legitimate, in-kernel observability and tracing features against defensive monitoring infrastructure. By using kernel helpers like bpf_probe_write_user and attaching to system call tracepoints, a privileged adversary can intercept and rewrite arguments, mask running processes within /proc, and suppress network traffic before security instrumentation can record the changes.

Mitigating this vector requires moving beyond passive, user-space event collection. Defending production clusters requires enforcing strict capability boundaries, permanently locking down unprivileged eBPF interfaces via sysctl, deploying LSM policies that gate program load operations, and conducting out-of-band audits of loaded BPF bytecode against known-good integrity manifests.

Measured on our own hardware: What 64 Bytes of Padding Are Worth

How much throughput does false sharing cost when several threads increment counters that share one cache line, compared with the same counters padded onto separate lines?

We ran it. The numbers below come from a program executed on the server hosting this site on 2026-09-20 — an AMD EPYC 9354P 32-Core Processor with 8 cores visible, 31.3 GB of memory, Linux 6.8.0-139-generic.

Metric Value
slowdown factor 12.2
iterations per thread 10000000
padded best seconds 0.0318
padded mean seconds 0.0403
padded million ops per sec 2514.92
repeats 5
shared line best seconds 0.3881
shared line mean seconds 0.42
shared line million ops per sec 206.15
threads 8

This is a shared virtual server, not an isolated test rig, so treat the absolute figures as indicative and the ratio between the two cases as the finding. The full method, the machine specification, and the complete source code are on the What 64 Bytes of Padding Are Worth benchmark page, so you can check the method or run it yourself.

Measured on our own hardware: io_uring vs pread: Random Reads at Queue Depth 32

How many random 4 KiB reads per second can one thread sustain with serial pread() syscalls, compared with io_uring submitting 32 at a time, when the page cache is taken out of the picture?

We ran it. The numbers below come from a program executed on the server hosting this site on 2026-09-20 — an AMD EPYC 9354P 32-Core Processor with 8 cores visible, 31.3 GB of memory, Linux 6.8.0-139-generic.

Metric Value
io uring speedup factor 7.85
block bytes 4096
blocks read 20000
io uring best seconds 0.4712
io uring iops 42449
o direct true
pread best seconds 3.6977
pread iops 5409
pread mean latency us 184.88
queue depth 32
repeats 3

This is a shared virtual server, not an isolated test rig, so treat the absolute figures as indicative and the ratio between the two cases as the finding. The full method, the machine specification, and the complete source code are on the io_uring vs pread: Random Reads at Queue Depth 32 benchmark page, so you can check the method or run it yourself.

References