OpenSSL Silently Patches "HollowByte" Memory-Exhaustion DoS Vulnerability

OpenSSL Silently Patches "HollowByte" Memory-Exhaustion DoS (TL-2026-1554), also tracked as HollowByte, is a medium-severity software vulnerability, first published 2026-07-20. It has no confirmed attribution, affects OpenSSL Project OpenSSL, maps to 15 MITRE ATT&CK techniques (T1046, T1071.001, T1190), and is covered by 9 detection rules and 22 indicators of compromise.

Key facts for TL-2026-1554

Threat ID
TL-2026-1554
Also known as
HollowByte
Severity
MEDIUM
Status
PATCHED
Category
VULNERABILITY
First published
2026-07-20
Last reviewed
2026-07-20
Attribution confidence
LOW
Motivation
UNKNOWN
Target sectors
technology, government administration, finance, health, telecoms, cloud, retail, education, manufacturing, critical-infrastructure
Target regions
Global
Detection rules
9
Indicators of compromise
22

Malware and tooling in OpenSSL Silently Patches "HollowByte" Memory-Exhaustion DoS

Malware and tooling: grow_init_buf incremental-allocation patch (PR #30792/#30793/#30794)

Okta's Red Team discovered "HollowByte," an unauthenticated pre-handshake denial-of-service flaw in OpenSSL where an 11-byte TLS message declares a body of up to 131 KB, forcing the server to pre-allocate the full claimed size before any data validation. OpenSSL silently fixed the issue in 4.0.1, 3.6.3, 3.5.7, 3.4.6, and 3.0.21 (released 2026-06-09) with no CVE, advisory, or changelog entry.

How OpenSSL Silently Patches "HollowByte" Memory-Exhaustion DoS works

HollowByte exploits a legacy behavior in OpenSSL's TLS handshake state machine: the ClientHello (and other handshake) message header carries a 4-byte record with a 3-byte length field declaring the size of the incoming message body. Vulnerable OpenSSL versions call grow_init_buf() in ssl/statem/statem_lib.c to pre-allocate a buffer sized to the attacker-declared length via OPENSSL_clear_realloc()/malloc() immediately upon reading the header — before any of the claimed body bytes arrive or are validated. An attacker can therefore send just 11 bytes that declare a message body up to 131 KB, causing the server to allocate the full amount and then block waiting on a worker thread for data that never arrives.

The vulnerability's severity is compounded by glibc's memory allocator (ptmalloc) behavior: when the attacker drops the connection and OpenSSL frees the oversized buffer, glibc retains small-to-medium freed chunks for potential reuse rather than returning them to the kernel. By varying the claimed size on every connection, an attacker prevents the allocator from ever reusing the freed chunks, causing persistent heap fragmentation. Resident Set Size (RSS) climbs continuously and remains elevated even after the malicious connections are dropped, requiring a full process restart to reclaim the fragmented memory.

Okta's testing on NGINX demonstrated the practical impact: a 1 GB memory-constrained host was OOM-killed after only 547 MB of memory became locked in fragments, and a 16 GB host had 25% of total memory permanently locked while the attack traffic stayed low-bandwidth and low-connection-count enough to evade standard rate-limiting and connection-ceiling defenses. Because OpenSSL underpins the TLS stack of Apache, NGINX, Node.js, Python, Ruby, PHP, MySQL, and PostgreSQL, the exposure is effectively supply-chain-wide across any TLS-terminating service linking a vulnerable OpenSSL build.

OpenSSL's security team classified the report as a "bug or hardening" fix rather than a formal security vulnerability, so it received no CVE, no security advisory, and no CHANGES.md/vulnerabilities-page entry — the fix shipped silently inside routine point releases 4.0.1, 3.6.3, 3.5.7, 3.4.6, and 3.0.21 on 2026-06-09. The remediation (upstream PR #30792 for master/4.0, backported in #30793 for 3.6/3.5/3.4 and #30794 for 3.0) changes grow_init_buf() to grow the receive buffer incrementally as bytes actually arrive on the wire, rather than trusting the header's declared length — capped at roughly eight reallocations per message to avoid a quadratic growth cost. Reviewers explicitly scoped the fix to TLS only; DTLS uses a structurally similar pre-allocation pattern but was left unpatched because a DTLS-side fix was judged "too complicated and invasive" for this round, leaving datagram-based TLS deployments (VPNs, VoIP, some IoT/telemetry stacks) exposed to the same class of attack. As of 2026-07-18 no public proof-of-concept exploit code had been observed, and no in-the-wild exploitation has been reported.

MITRE ATT&CK techniques used in TL-2026-1554

Discovery

T1046 Network Service Discovery

Command and Control

T1071.001 Web Protocols

Initial Access

T1190 Exploit Public-Facing Application

Impact

T1499 Endpoint Denial of Service; T1499.001 OS Exhaustion Flood; T1499.002 Service Exhaustion Flood; T1499.003 Application Exhaustion Flood; T1499.004 Application or System Exploitation

Resource Development

T1583.005 Botnet; T1584.005 Botnet; T1588.006 Vulnerabilities

Reconnaissance

T1592.002 Software; T1592.004 Client Configurations; T1595.001 Scanning IP Blocks; T1595.002 Vulnerability Scanning

Affected products and versions in OpenSSL Silently Patches "HollowByte" Memory-Exhaustion DoS

  • OpenSSL Project — OpenSSL
    Vulnerable versions: 3.0.0-3.0.20; 3.4.0-3.4.5; 3.5.0-3.5.6; 3.6.0-3.6.2; 4.0.0
    Fixed in: 3.0.21; 3.4.6; 3.5.7; 3.6.3; 4.0.1
  • OpenSSL Project — OpenSSL (DTLS)
    Vulnerable versions: all currently shipping DTLS implementations
  • Multiple (downstream) — Any application or service statically/dynamically linking a vulnerable OpenSSL build (Apache HTTP Server, NGINX, Node.js, Python, Ruby, PHP, MySQL, PostgreSQL)
    Vulnerable versions: builds linking OpenSSL prior to 3.0.21/3.4.6/3.5.7/3.6.3/4.0.1
    Fixed in: builds relinked/upgraded to patched OpenSSL

Remediation for OpenSSL Silently Patches "HollowByte" Memory-Exhaustion DoS

Patches

  • OpenSSL 4.0.1 (master/4.0 branch, PR #30792)
  • OpenSSL 3.6.3 / 3.5.7 / 3.4.6 (backport, PR #30793)
  • OpenSSL 3.0.21 (backport, PR #30794)

Immediate actions

  • Upgrade OpenSSL to 4.0.1, 3.6.3, 3.5.7, 3.4.6, or 3.0.21 (or later) on any host that terminates TLS
  • Inventory all applications and containers statically or dynamically linking OpenSSL (Apache, NGINX, Node.js, Python, Ruby, PHP, MySQL, PostgreSQL) and prioritize internet-facing TLS listeners
  • Monitor server RSS/heap fragmentation growth on TLS-terminating processes as a leading indicator of exploitation attempts, since connection-count and bandwidth-based alerting can be evaded
  • Restart TLS-terminating worker processes on a rolling basis where memory fragmentation is suspected, since freed HollowByte buffers are not reclaimed by glibc without a process restart

Workarounds

  • Where upgrading is not immediately possible, front TLS-terminating services with a reverse proxy/load balancer that enforces early record-size sanity limits independent of OpenSSL's own buffer allocation
  • Constrain per-worker memory via cgroups/ulimits so a single fragmented worker cannot exhaust host-wide memory, and configure automatic worker recycling on memory-growth thresholds

Longer-term hardening

  • Track DTLS deployments (VPN, VoIP, IoT telemetry) separately, since the incremental-buffer-growth fix was scoped to TLS only and DTLS remains structurally vulnerable to the same pre-allocation pattern
  • Add heap-fragmentation / RSS-growth-under-low-bandwidth detections to DoS monitoring baselines, since HollowByte is designed to stay under connection-rate and bandwidth alerting thresholds
  • Establish a process for tracking silent "bug or hardening" fixes in critical dependencies like OpenSSL that ship without CVEs or advisories

Weaknesses (CWE) in OpenSSL Silently Patches "HollowByte" Memory-Exhaustion DoS

CWE-400, CWE-770, CWE-789

Timeline of OpenSSL Silently Patches "HollowByte" Memory-Exhaustion DoS

  • OpenSSL maintainer Matt Caswell (mattcaswell) opens PR #30792 to grow the TLS handshake init buffer incrementally (chunk + SSL3_HM_HEADER_LENGTH per read) instead of pre-allocating the full attacker-declared message length in grow_init_buf().
  • PR #30792 is reviewed and approved by OpenSSL maintainers npajkovsky, Sashan, and t8m; reviewer paulidale raises a quadratic-growth performance concern that t8m addresses by confirming the incremental approach caps growth at roughly eight reallocations per message.
  • OpenSSL maintainer Matt Caswell merges PR #30792 changing grow_init_buf() to grow the TLS handshake receive buffer incrementally instead of pre-allocating the full attacker-declared length; backports #30793 (3.6/3.5/3.4) and #30794 (3.0) follow, with Caswell explicitly scoping the fix to TLS only and noting DTLS 'would be much more complicated and invasive' to fix in the same pass.
  • Okta Security publishes the technical write-up "OpenSSL HollowByte: A DoS Hiding in 11 Bytes" detailing the grow_init_buf() allocation flow and memory-fragmentation impact.
  • OpenSSL ships the HollowByte fix silently inside routine point releases 4.0.1, 3.6.3, 3.5.7, 3.4.6, and 3.0.21, with no CVE, security advisory, or changelog entry describing the fix.
  • Okta's Red Team publicly attributes discovery of the HollowByte flaw and describes the 11-byte trigger, glibc fragmentation mechanics, and 1GB/16GB test results.
  • The Hacker News, BleepingComputer, SecurityAffairs, CybersecurityNews, and GBHackers publish coverage of the silent HollowByte fix, amplifying awareness of the undisclosed vulnerability.
  • No public proof-of-concept exploit code or in-the-wild exploitation of HollowByte has been observed as of this date.
  • SecurityWeek publishes coverage of the silent HollowByte fix; TL-Intel Harness ingests the article via the SecurityWeek RSS feed and opens threat TL-2026-1554.

Sources cited for OpenSSL Silently Patches "HollowByte" Memory-Exhaustion DoS

Threats related to OpenSSL Silently Patches "HollowByte" Memory-Exhaustion DoS

Detection coverage for TL-2026-1554

As of 2026-07-20, Threadlinqs Intelligence publishes 9 detection rule(s) for TL-2026-1554 across Splunk SPL, Microsoft KQL and Sigma, covering 22 indicator(s) of compromise. The whole corpus is readable without an account; a free account unlocks full detection query text in Splunk SPL, Microsoft KQL and Sigma; paid tiers add raw indicator values, correlation and the MCP server. Threadlinqs MCP server · View plans.

Threadlinqs Intelligence — Real-Time Threat Detection Platform

[ 0 threats ] [ 0 det ] [ CRIT: 0 ] [ HIGH: 0 ]
// threat_feed
$ sort --newest
Showing all threats

Latest Threats