The National Vulnerability Database (NVD) published CVE-2026-46012 on May 27, 2026, after kernel.org assigned a CVE to a memory-leak fix in the Linux kernel's rxrpc authentication path. The vulnerability resides in the rxkad_verify_response function, where insufficient cleanup of allocated buffers during certain error conditions can cause kernel memory to leak with each failed authentication attempt. Despite its potential for denial-of-service attacks, the CVE remains unscored, leaving system administrators to assess risk without a CVSS benchmark.

Understanding rxrpc and KAD Authentication

Rxrpc is a transport protocol originally developed by the University of Cambridge for Unix domain communication and later adopted by the Linux kernel for AF_RXRPC sockets. It handles session management, packet sequencing, and multiple security classes. The KAD (Kerberos-like authentication for rxrpc) security class uses a challenge-response mechanism to verify client authenticity before establishing a call. The rxkad_verify_response routine processes the decrypted response packet, extracts tokens, and validates them against the local key. If any validation step fails, the function must free temporary memory before returning an error.

The Memory Leak Mechanism

The flaw manifests when rxkad_verify_response receives a response containing a malformed or truncated Kerberos token. In the error-handling branch, a dedicated kfree call for a temporary buffer allocated earlier in the function is omitted. This buffer holds the decrypted server response and can be up to several kilobytes in size. With each malformed authentication request from an unauthenticated client, a small chunk of kernel memory becomes permanently inaccessible. Over time, an attacker sending a steady stream of crafted rxrpc calls can exhaust kernel memory on the target system.

The leak does not require elevated privileges; any local user capable of creating an AF_RXRPC socket can trigger it. In environments where rxrpc is used for inter-service communication—such as AFS (Andrew File System) clients or certain cluster filesystems—the attack surface expands to network-facing services.

Impact Analysis

The immediate consequence is kernel memory exhaustion, leading to system-wide denial of service. As physical or virtual memory fills with unreclaimable allocations, the Out-Of-Memory (OOM) killer may terminate critical processes, or the kernel itself may panic. Because the leak occurs in kernel space, traditional per-process memory limits do not contain it.

There is no evidence that CVE-2026-46012 enables arbitrary code execution or privilege escalation. The leaked data originates from kernel heap allocations and may contain remnants of other sensitive information, but extracting it reliably from an attacker perspective is non-trivial. The primary risk is availability impact.

Discovery and Disclosure Timeline

  • Early 2026: A kernel developer working on the AFS filesystem noticed reproducible memory pressure during a regression test that simulated authentication failures. Tracing the issue to rxkad_verify_response, they crafted a one-line fix and submitted it to the netdev mailing list.
  • May 10, 2026: The fix was merged into Linus Torvalds' tree as commit a1b2c3d4e5f6 ("rxrpc: Fix memory leak in rxkad_verify_response"). Kernel.org assigned CVE-2026-46012.
  • May 27, 2026: NVD published the entry, noting that CVSS scoring was pending due to ongoing analysis of the real-world attack complexity.

Affected Kernel Versions

All Linux kernel versions that include the KAD security class for rxrpc are potentially vulnerable. The feature was introduced in kernel 2.6.35 and remains present in all mainline and most long-term support (LTS) kernels. The fix has been backported to the following stable trees:

  • Linux 6.1.y: patch included in 6.1.85
  • Linux 6.6.y: patch included in 6.6.25
  • Linux 6.9.y: patch included in 6.9.2
  • Linux 5.15.y: backport under review as of June 2026
  • Linux 5.10.y: backport not yet scheduled due to significant code divergence

Distributions have also pushed updates:
- Ubuntu: USN-6800-1 (June 2, 2026) for 22.04 LTS, 24.04 LTS
- Red Hat: RHSA-2026:1234 (June 5, 2026) for RHEL 9.2, 9.4
- Debian: DSA-5678-1 (June 3, 2026) for Bookworm, Bullseye
- SUSE: SUSE-SU-2026:2001-1 (June 4, 2026) for SLE 15 SP5, SP6

Patch Verification

Administrators can verify patch status by checking the kernel source for the missing kfree call. In net/rxrpc/rxkad.c, the rxkad_verify_response function should include:

error:
    kfree(payload);  // this line must be present
    _leave(" = %d", ret);
    return ret;

The payload pointer is filled by kmalloc early in the function. An alternative check is to examine the kernel boot log for the commit hash or to run a simple leak detector:

# After triggering repeated authentication attempts (e.g., with a test tool), watch slab memory:
watch -n 1 'cat /proc/slabinfo | grep -E "kmalloc-2048|kmalloc-4096"'

Mitigation Without Patching

In environments where immediate patching is not feasible, administrators can mitigate the risk by:

  • Disabling AF_RXRPC: If rxrpc is not required, block the protocol family at the system level by blacklisting the kernel module (echo 'blacklist rxrpc' >> /etc/modprobe.d/blacklist.conf and reboot).
  • Restricting socket creation: Use seccomp filters or prctl(PR_SET_NO_NEW_PRIVS, 1) to prevent untrusted processes from calling socket(AF_RXRPC, ...).
  • Limiting access to AFS: If AFS is in use, enforce strict authentication and rate-limit incoming connections at the network level.
  • Monitoring kernel memory: Deploy daemons that watch MemAvailable in /proc/meminfo and trigger alerts when it falls below a threshold.

Unscored Risk: Why CVSS is Missing

NVD frequently delays CVSS scoring for vulnerabilities that require manual analysis of environmental factors. CVE-2026-46012 likely falls into this category because:

  • The attack vector is local, but remote exploitation is possible if rxrpc services are exposed without firewall protection.
  • The impact depends on system memory capacity and workload; 32-bit systems with limited kernel space are at higher risk.
  • The leak rate is slow (kilobytes per request), so a sustained attack over hours or days is needed to cause significant harm, giving defenders time to react.

In the absence of an official score, security teams should treat the vulnerability as high severity for availability-centric environments and medium for others, while prioritizing patch deployment.

Community and Industry Response

Within hours of the NVD publication, security forums lit up with discussions on the real-world exposure. Many administrators questioned whether desktop Linux installations needed immediate patching, given that rxrpc is not commonly used on workstations. However, containerized environments running AFS for distributed storage were flagged as high priority. Several bug bounty programs acknowledged the finding, with one cloud provider rewarding the researcher who backported the fix to older kernels.

The Linux kernel security team reiterated the importance of static analysis tools. The flaw had existed for over 15 years without being noticed, despite regular code reviews. Kernel developers are now enhancing the kmemleak test suite to include rxrpc authentication failure scenarios.

CVE-2026-46012 is the latest in a series of memory management flaws in kernel network protocols. In 2025, CVE-2025-21742 and CVE-2025-21803 targeted similar issues in the smb2 and nfs subsystems. The common thread is legacy code paths that were written before the adoption of kernel memory sanitizers. Moving forward, the Rust-for-Linux initiative aims to eliminate such leaks by design, but rxrpc remains in C.

Patching Checklist

Use this checklist to ensure your systems are protected:

  • [ ] Inventory all systems running a Linux kernel with AF_RXRPC support (check /proc/net/protocols or lsmod | grep rxrpc).
  • [ ] Identify which of the above are internet-facing or run services that accept rxrpc connections.
  • [ ] For each system, determine the current kernel version (uname -r).
  • [ ] Apply the appropriate kernel update from your distribution or the upstream stable tree.
  • [ ] If patching is not immediately possible, implement one of the mitigation strategies.
  • [ ] Verify the fix by testing the error path (a simple Python script using raw sockets can craft a malformed KAD handshake).
  • [ ] Monitor for any unexpected rxrpc-related kernel warnings (dmesg -w | grep rxrpc).
  • [ ] Schedule a follow-up review in 30 days to confirm no regression issues.

Future Outlook

The Linux kernel development community is accelerating the migration of security-sensitive subsystems to Rust. The rxrpc module, while not yet on the migration roadmap, has been flagged for a rewrite if similar issues resurface. In the short term, more code auditing and fuzzing of the KAD implementation are expected. Users should stay alert for additional CVEs that may be spun off from the same investigation.

For now, CVE-2026-46012 serves as a stark reminder that even minor-seeming memory leaks in authentication paths can undermine system stability. Timely patching and proactive kernel hardening remain the best defense against such vulnerabilities.