# OpenSSL Silently Patches "HollowByte" Memory-Exhaustion DoS Vulnerability

> Okta's Red Team discovered "HollowByte," an unauthenticated pre-handshake denial-of-service flaw in OpenSSL where an 11-byte TLS message declares a body of up to 131 KB, forcing the server to pre-allocate the full claimed size before any data validation. OpenSSL silently fixed the issue in 4.0.1, 3.6.3, 3.5.7, 3.4.6, and 3.0.21 (released 2026-06-09) with no CVE, advisory, or changelog entry.

- **Published:** 2026-07-20T00:00:00Z
- **Last reviewed:** 2026-07-20T00:00:00Z
- **Canonical:** https://intel.threadlinqs.com/threat/TL-2026-1554
- **ID:** TL-2026-1554
- **Severity:** MEDIUM
- **Category:** VULNERABILITY
- **Status:** PATCHED
- **Detections:** 9 · **IOCs:** 22 (full data via the Threadlinqs MCP server — Purple tier)

## Description

HollowByte exploits a legacy behavior in OpenSSL's TLS handshake state machine: the ClientHello (and other handshake) message header carries a 4-byte record with a 3-byte length field declaring the size of the incoming message body. Vulnerable OpenSSL versions call grow_init_buf() in ssl/statem/statem_lib.c to pre-allocate a buffer sized to the attacker-declared length via OPENSSL_clear_realloc()/malloc() immediately upon reading the header — before any of the claimed body bytes arrive or are validated. An attacker can therefore send just 11 bytes that declare a message body up to 131 KB, causing the server to allocate the full amount and then block waiting on a worker thread for data that never arrives.

The vulnerability's severity is compounded by glibc's memory allocator (ptmalloc) behavior: when the attacker drops the connection and OpenSSL frees the oversized buffer, glibc retains small-to-medium freed chunks for potential reuse rather than returning them to the kernel. By varying the claimed size on every connection, an attacker prevents the allocator from ever reusing the freed chunks, causing persistent heap fragmentation. Resident Set Size (RSS) climbs continuously and remains elevated even after the malicious connections are dropped, requiring a full process restart to reclaim the fragmented memory.

Okta's testing on NGINX demonstrated the practical impact: a 1 GB memory-constrained host was OOM-killed after only 547 MB of memory became locked in fragments, and a 16 GB host had 25% of total memory permanently locked while the attack traffic stayed low-bandwidth and low-connection-count enough to evade standard rate-limiting and connection-ceiling defenses. Because OpenSSL underpins the TLS stack of Apache, NGINX, Node.js, Python, Ruby, PHP, MySQL, and PostgreSQL, the exposure is effectively supply-chain-wide across any TLS-terminating service linking a vulnerable OpenSSL build.

OpenSSL's security team classified the report as a "bug or hardening" fix rather than a formal security vulnerability, so it received no CVE, no security advisory, and no CHANGES.md/vulnerabilities-page entry — the fix shipped silently inside routine point releases 4.0.1, 3.6.3, 3.5.7, 3.4.6, and 3.0.21 on 2026-06-09. The remediation (upstream PR #30792 for master/4.0, backported in #30793 for 3.6/3.5/3.4 and #30794 for 3.0) changes grow_init_buf() to grow the receive buffer incrementally as bytes actually arrive on the wire, rather than trusting the header's declared length — capped at roughly eight reallocations per message to avoid a quadratic growth cost. Reviewers explicitly scoped the fix to TLS only; DTLS uses a structurally similar pre-allocation pattern but was left unpatched because a DTLS-side fix was judged "too complicated and invasive" for this round, leaving datagram-based TLS deployments (VPNs, VoIP, some IoT/telemetry stacks) exposed to the same class of attack. As of 2026-07-18 no public proof-of-concept exploit code had been observed, and no in-the-wild exploitation has been reported.

## MITRE ATT&CK

- T1595.002 Vulnerability Scanning
- T1592.002 Software
- T1583.005 Botnet
- T1588.006 Vulnerabilities
- T1190 Exploit Public-Facing Application
- T1046 Network Service Discovery
- T1499 Endpoint Denial of Service
- T1499.001 OS Exhaustion Flood
- T1499.002 Service Exhaustion Flood
- T1499.004 Application or System Exploitation
- T1499.003 Application Exhaustion Flood
- T1592.004 Client Configurations
- T1595.001 Scanning IP Blocks
- T1584.005 Botnet
- T1071.001 Web Protocols

## Sources

- [OpenSSL Silently Fixes 'HollowByte' DoS Vulnerability](https://www.securityweek.com/openssl-silently-fixes-hollowbyte-dos-vulnerability/)
- [OpenSSL HollowByte: A DoS Hiding in 11 Bytes](https://sec.okta.com/articles/2026/06/openssl-hollowbtye-a-dos-hiding-in-11-bytes/)
- [OpenSSL HollowByte Flaw Could Freeze Server Memory with 11-Byte TLS Requests](https://thehackernews.com/2026/07/openssl-hollowbyte-flaw-could-freeze.html)
- [HollowByte DDoS flaw bloats OpenSSL server memory with 11-byte payload](https://www.bleepingcomputer.com/news/security/hollowbyte-ddos-flaw-bloats-openssl-server-memory-with-11-byte-payload/)
- [OpenSSL Fixes HollowByte Memory Exhaustion Bug](https://securityaffairs.com/195588/hacking/openssl-fixes-hollowbyte-memory-exhaustion-bug.html)
- [OpenSSL "HollowByte" Vulnerability Lets Hackers Crash Servers With Just 11 Bytes](https://cybersecuritynews.com/openssl-hollowbyte-vulnerability/)
- [OpenSSL DoS Vulnerability Lets Remote Attackers Exhaust Server Memory With an 11-Byte Payload](https://gbhackers.com/openssl-dos-vulnerability/)
- [Grow the init_buf incrementally as we receive data (master/4.0)](https://github.com/openssl/openssl/pull/30792)
- [Grow the init_buf incrementally as we receive data (3.6/3.5/3.4 backport)](https://github.com/openssl/openssl/pull/30793)
- [Grow the init_buf incrementally as we receive data (3.0 backport)](https://github.com/openssl/openssl/pull/30794)

## Full data

Detection queries (Splunk SPL / Microsoft KQL / Sigma) and IOC values require the Threadlinqs MCP server (Purple tier): https://intel.threadlinqs.com/mcp

Canonical: https://intel.threadlinqs.com/threat/TL-2026-1554