# HollowByte: OpenSSL Pre-Authentication TLS DoS Flaw Bloats Server Memory With 11-Byte Payload

> Okta's Red Team disclosed HollowByte, a pre-authentication denial-of-service weakness in OpenSSL where an unauthenticated attacker sends a malicious 11-byte TLS handshake header declaring a much larger message body, causing OpenSSL to pre-allocate up to 131KB of memory before any payload data arrives or is validated. OpenSSL treated the issue as a silent hardening fix rather than a CVE-tracked vulnerability, patching it in the same June 9, 2026 release train (4.0.1, 3.6.3, 3.5.7, 3.4.6, 3.0.21) that also fixed 17 formally-tracked CVEs, including the critical PKCS7_verify() heap use-after-free (CVE-2026-45447) and the QUIC PATH_CHALLENGE unbounded memory growth flaw (CVE-2026-34183).

- **Published:** 2026-07-17T00:00:00Z
- **Last reviewed:** 2026-07-17T00:00:00Z
- **Canonical:** https://intel.threadlinqs.com/threat/TL-2026-1457
- **ID:** TL-2026-1457
- **Severity:** MEDIUM
- **Category:** VULNERABILITY
- **Status:** ACTIVE
- **Detections:** 9 · **IOCs:** 21 (full data via the Threadlinqs MCP server — Purple tier)
- **CVEs:** CVE-2026-45447, CVE-2026-34183, CVE-2026-34182, CVE-2026-35188, CVE-2026-42764, CVE-2026-45445

## Description

HollowByte is a memory-exhaustion denial-of-service weakness in OpenSSL's TLS handshake-message parsing, publicly detailed by Okta's Red Team on July 16, 2026 (published via BleepingComputer July 17, 2026) and independently by Okta's own security advisory (sec.okta.com). The root cause is a classic allocate-before-validate pattern: during the ClientHello / handshake message phase, OpenSSL reads a 4-byte handshake message header containing a 3-byte length field that declares how much payload data will follow. Vulnerable versions call grow_init_buf(), which in turn calls OPENSSL_clear_realloc() -> malloc(), to pre-allocate a receive buffer sized to the attacker-declared length BEFORE any of the actual payload bytes have arrived on the wire or been validated. An attacker who sends just 11 bytes total -- a TCP/TLS connection open plus a malformed 4-byte header claiming an oversized body -- can force the server to allocate up to 131KB per connection while the worker thread blocks indefinitely waiting for payload data that never arrives, an amplification ratio of roughly 11,900:1.

Because each individual allocation (131KB) and each individual connection may fall under naive connection-count or per-request size limits, standard rate-limiting and connection-ceiling defenses are insufficient to stop the attack -- an attacker opens many low-and-slow connections, each looking unremarkable in isolation. Okta's testing showed the effect compounds via glibc's memory allocator behavior: glibc does not immediately return small-to-medium freed allocations to the OS, retaining them for reuse. Waves of connections using randomized claimed sizes prevent heap consolidation and fragment the process heap, so memory remains bloated even after the offending connections are torn down. Okta tested unpatched and patched OpenSSL instances running NGINX under load; in a 1GB-constrained test environment, unpatched OpenSSL servers were driven to an OOM-kill after allocating roughly 547MB; in a higher-spec 16GB environment, the technique consumed approximately 25% of total system memory.

OpenSSL's maintainers characterized the fix as a hardening improvement rather than a security vulnerability warranting a CVE, and it shipped silently as three merged pull requests (#30792, #30793, #30794) bundled into the same June 9, 2026 release train that formally patched 17 numbered CVEs across the actively-maintained branches (4.0.1, 3.6.3, 3.5.7, 3.4.6, 3.0.21). The release train's headline issue was CVE-2026-45447 (CVSS 9.8 Critical), a heap use-after-free in PKCS7_verify(): when a PKCS#7 or S/MIME SignedData structure's digestAlgorithms field is present as an empty ASN.1 SET, OpenSSL may incorrectly free a caller-owned BIO during verification, so a subsequent BIO_free() by the calling application triggers a use-after-free that can crash the process, corrupt the heap, or potentially enable remote code execution; it affects OpenSSL from the 1.0.2/1.1.1 branches through 4.0, with fixes also back-ported to premium-support branches 1.1.1zh and 1.0.2zq. The same release also fixed CVE-2026-34183 (Moderate), unbounded memory growth in the QUIC PATH_CHALLENGE handler -- a remote peer can flood a QUIC client or server with PATH_CHALLENGE frames, forcing allocation of a PATH_RESPONSE frame per challenge that is only freed once the (malicious, non-cooperating) peer acknowledges it, exhausting heap memory and causing abnormal termination -- a resource-exhaustion bug in the same allocate-on-untrusted-input family as HollowByte itself, affecting OpenSSL 4.0, 3.6, 3.5, 3.4, and 3.0 (not 1.1.1/1.0.2). Other issues in the same train included CVE-2026-34182 (CMS AuthEnvelopedData forgery), CVE-2026-35188 (double-free in OCSP staple verification, 4.0.1/3.6.3), CVE-2026-42764 (NULL pointer dereference in QUIC server packet handling, 4.0.1/3.6.3/3.5.7), CVE-2026-45445 (AES-OCB IV bypass enabling universal forgery, all versions), plus roughly ten Low-severity issues spanning ASN.1 parsing buffer over-reads, PKCS#12 validation weaknesses, CMS decryption NULL dereferences, certificate-verification crashes, and email-address validation errors. FIPS-validated modules were unaffected by most of the release's issues since the vulnerable code paths sit outside the FIPS module boundary.

The fix for HollowByte itself replaces pre-allocation-from-declared-length with incremental/deferred buffer growth: OpenSSL now grows the receive buffer only as data actually arrives on the socket, ignoring the attacker's claimed length until it is backed by real bytes.

Because OpenSSL underlies the TLS stack of an enormous swath of internet-facing infrastructure -- either linked directly or bundled -- the blast radius extends far beyond OpenSSL itself to downstream software including NGINX, Apache HTTP Server, Node.js, Python (via the ssl module), Ruby, PHP, MySQL, PostgreSQL, and most mainstream Linux distributions that ship OpenSSL as the default TLS provider. No CVE has been assigned to HollowByte as of publication, it is not listed in the CISA Known Exploited Vulnerabilities (KEV) catalog, there is no confirmed in-the-wild exploitation, and no independent CVSS score has been published for HollowByte itself (unlike its release-train companion CVE-2026-45447, rated 9.8 Critical); the hunt/tracking rationale for this threat record is the pre-authentication, minimal-payload nature of the DoS combined with the breadth of downstream affected software and the fact that vendor patches already exist across five concurrently-supported release branches. Administrators are advised to upgrade their distribution's OpenSSL packages (or statically-linked application bundles) immediately, since standard connection-limiting defenses do not stop the attack, and to prioritize the same release train's CVE-2026-45447 (RCE-capable UAF) given its far higher severity.

## MITRE ATT&CK

- T1595 Active Scanning
- T1592 Gather Victim Host Information
- T1590 Gather Victim Network Information
- T1587 Develop Capabilities
- T1583 Acquire Infrastructure
- T1190 Exploit Public-Facing Application
- T1046 Network Service Discovery
- T1518 Software Discovery
- T1070 Indicator Removal
- T1203 Exploitation for Client Execution
- T1498 Network Denial of Service
- T1499 Endpoint Denial of Service
- T1489 Service Stop
- T1529 System Shutdown/Reboot

## Sources

- [HollowByte DDoS flaw bloats OpenSSL server memory with 11-byte payload](https://www.bleepingcomputer.com/news/security/hollowbyte-ddos-flaw-bloats-openssl-server-memory-with-11-byte-payload/)
- [OpenSSL HollowByte: A DoS Hiding in 11 Bytes](https://sec.okta.com/articles/2026/06/openssl-hollowbtye-a-dos-hiding-in-11-bytes/)
- [OpenSSL Release and Advisory Timeline](https://openssl-library.org/news/timeline/)
- [OpenSSL Vulnerabilities](https://openssl-library.org/news/vulnerabilities/)
- [openssl/openssl GitHub Releases](https://github.com/openssl/openssl/releases)
- [OpenSSL NEWS.md changelog](https://github.com/openssl/openssl/blob/master/NEWS.md)
- [OpenSSL CVE-2026-45447: Heap Use-After-Free in PKCS#7 Verification Enables S/MIME RCE, Discovered With AI](https://cybersecuritymarket.com/openssl-cve-2026-45447-pkcs7-rce/)
- [OpenSSL Fixes High-Severity Vulnerability That Can Lead to Remote Code Execution](https://vpncentral.com/openssl-fixes-high-severity-vulnerability-that-can-lead-to-remote-code-execution/)
- [Node.js — OpenSSL Security Advisory Assessment, January 2026](https://nodejs.org/en/blog/vulnerability/openssl-fixes-in-regular-releases-jan2026)
- [nginx security advisories](https://nginx.org/en/security_advisories.html)
- [CVE-2026-45447 Detail - NVD](https://nvd.nist.gov/vuln/detail/CVE-2026-45447)
- [CVE-2026-45447 - Heap Use-After-Free Enabling RCE - OpenSSL 1.0.2/1.1.1/3.0/3.4/3.5/3.6/4.0](https://www.ionix.io/threat-center/cve-2026-45447/)
- [OpenSSL Patches 16 Flaws Including Heap Use-After-Free RCE Risk](https://dailysecurityreview.com/resources/openssl-patches-16-flaws-including-heap-use-after-free-rce-risk/)
- [CVE-2026-34183 - Unbounded Memory Growth in the QUIC PATH_CHALLENGE Handler (Bugzilla)](https://bugzilla.redhat.com/show_bug.cgi?id=2481885)
- [CVE-2026-34183 - NVD](https://nvd.nist.gov/vuln/detail/CVE-2026-34183)

## Full data

Detection queries (Splunk SPL / Microsoft KQL / Sigma) and IOC values require the Threadlinqs MCP server (Purple tier): https://intel.threadlinqs.com/mcp

Canonical: https://intel.threadlinqs.com/threat/TL-2026-1457