# PixelSmash – FFmpeg MagicYUV Decoder Heap Out-of-Bounds Write (CVE-2026-8461)

> JFrog Research disclosed PixelSmash, a heap out-of-bounds write in FFmpeg's MagicYUV decoder caused by a rounding mismatch between the frame allocator and decoder chroma plane height calculations. A crafted AVI, MKV or MOV file achieved RCE against Jellyfin 10.11.9 and Nextcloud (ASLR disabled) and crashes in mpv, Kodi, ffmpegthumbnailer, OBS, Immich, PhotoPrism and vLLM.

- **Published:** 2026-11-03T00:48:43Z
- **Last reviewed:** 2026-11-03T00:48:43Z
- **Canonical:** https://intel.threadlinqs.com/threat/TL-2026-5110
- **Card image:** https://intel.threadlinqs.com/og/threat/TL-2026-5110.png
- **ID:** TL-2026-5110
- **Severity:** HIGH (CVSS 8.8)
- **Category:** VULNERABILITY
- **Status:** ACTIVE
- **Detections:** 9 (1 shown free below) · **IOCs:** 9
- **CVEs:** CVE-2026-8461

## Description

CVE-2026-8461 (PixelSmash) is a heap-based out-of-bounds write in the MagicYUV decoder (libavcodec/magicyuv.c) of FFmpeg, discovered by Ori Hollander of the JFrog Security Research team. magy_decode_frame reads slice_height from the bitstream and validates it only against INT_MAX. For YUV420P content the frame allocator sizes the chroma plane with standard rounding, while the decoder applies ceiling-rounded right shifts (AV_CEIL_RSHIFT) to the slice height. When slice_height is odd, each slice adds an extra chroma row and the extra rows accumulate past the end of the chroma buffer.

In JFrog's proof of concept, slice_height=31 with coded_height=32 yields two slices of 16 chroma rows each; the second slice writes at row 16 of a 16-row buffer, depositing roughly 640 attacker-controlled bytes onto the adjacent heap chunk. For code execution the write overwrites the adjacent AVBuffer structure: the free function pointer is replaced with the address of libc system() and the opaque argument with a pointer to an attacker command string. When av_frame_unref() later releases the frame, the indirect call becomes system(attacker_command), and the spawned shell runs before the parent process crashes from the remaining heap corruption.

The PoC file is about 50 KB and uses AVI, MKV (Matroska) or MOV containers; MP4 is not affected because MagicYUV is not registered in its codec registry. The RCE PoC requires ASLR to be disabled (hardcoded addresses), GDB-based heap layout calibration, preservation of 33 glibc malloc chunk metadata qwords and a matching file path length (a 60-character path was demonstrated). Demonstrated vectors: Jellyfin 10.11.9 library scan invoking ffprobe gives a reverse shell as the jellyfin user; Nextcloud's movie preview provider gives RCE as www-data; and a zero-click torrent vector where a seeded file lands in a monitored Jellyfin folder and is processed automatically. Other tested products showed DoS or heap corruption: mpv (SIGABRT), Kodi, ffmpegthumbnailer (SIGSEGV), Emby 4.8.11 (silent heap corruption), Immich and PhotoPrism (SIGABRT), OBS Studio, and vLLM through PyAV (3/3 SIGSEGV on video inference).

The MagicYUV decoder is enabled by default in upstream FFmpeg builds and in distribution packages (Ubuntu, Debian, Fedora, Arch, Alpine). Fixed in FFmpeg 8.1.2 (released 2026-06-17), which rejects slice_height values not aligned to the chroma vertical subsampling factor. The source does not report in-the-wild exploitation, attributed actors or network IOCs; the IOCs below are file, process and build-level indicators of the vulnerable component and exploit mechanics.

## MITRE ATT&CK

- T1587.004 Exploits
- T1190 Exploit Public-Facing Application
- T1195.001 Compromise Software Dependencies and Development Tools
- T1203 Exploitation for Client Execution
- T1204.002 Malicious File
- T1059.004 Unix Shell
- T1499.004 Application or System Exploitation

## Free detection (Splunk SPL, Microsoft KQL, Sigma)

**PixelSmash CVE-2026-8461 — Shell or Downloader Spawned by ffprobe/ffmpeg/ffmpegthumbnailer** (`TL-2026-5110-SPL-001`)

Detects a shell, netcat, curl, wget or scripting interpreter spawned directly by ffprobe, ffmpeg or ffmpegthumbnailer. In the PixelSmash (CVE-2026-8461) exploit the MagicYUV heap overwrite replaces AVBuffer.free with libc system(), so av_frame_unref() runs an attacker command as a child of the media-decoding process (Jellyfin library scan via ffprobe, Nextcloud movie preview via ffmpeg). Media decoders have no legitimate reason to launch a shell.

- **Languages:** Splunk SPL, Microsoft KQL, Sigma
- **ATT&CK:** T1190, T1203, T1059.004
- **Data sources:** sysmon_for_linux, index=linux sourcetype=sysmon:linux, DeviceProcessEvents
- **Severity:** high
- **Confidence:** high

### Splunk SPL

```spl
index=linux sourcetype=sysmon:linux EventCode=1 earliest=-24h (ParentImage=*/ffprobe OR ParentImage=*/ffmpeg OR ParentImage=*/ffmpegthumbnailer) (Image=*/sh OR Image=*/bash OR Image=*/dash OR Image=*/zsh OR Image=*/busybox OR Image=*/nc OR Image=*/ncat OR Image=*/curl OR Image=*/wget OR Image=*/perl OR Image=*/python3) | stats count min(_time) as first_seen max(_time) as last_seen values(Image) as child_images values(CommandLine) as child_cmdlines by host User ParentImage ParentCommandLine | convert ctime(first_seen) ctime(last_seen) | table host User ParentImage ParentCommandLine child_images child_cmdlines count first_seen last_seen
```

### Microsoft KQL

```kql
DeviceProcessEvents | where Timestamp > ago(24h) | where InitiatingProcessFileName in~ ('ffprobe', 'ffmpeg', 'ffmpegthumbnailer') | where FileName in~ ('sh', 'bash', 'dash', 'zsh', 'busybox', 'nc', 'ncat', 'curl', 'wget', 'perl', 'python3') | project Timestamp, DeviceName, AccountName, InitiatingProcessFileName, InitiatingProcessCommandLine, FileName, ProcessCommandLine | order by Timestamp desc
```

### Sigma

```yaml
title: PixelSmash CVE-2026-8461 Shell Spawned by FFmpeg Media Decoder
id: 5b0c2f6e-1a41-4d0e-9a7b-3e5d6c1f2a01
status: experimental
description: Detects a shell or downloader spawned by ffprobe, ffmpeg or ffmpegthumbnailer, consistent with the PixelSmash MagicYUV AVBuffer.free to system() hijack
references:
    - https://jfrog.com/blog/pixelsmash-critical-ffmpeg-vulnerability-turns-media-files-into-weapons/
    - https://nvd.nist.gov/vuln/detail/CVE-2026-8461
author: AII-Detector — ThreadLinqs Intelligence
date: 2026-11-03
tags:
    - attack.initial-access
    - attack.t1190
    - attack.execution
    - attack.t1203
    - attack.t1059.004
    - cve.2026-8461
logsource:
    product: linux
    category: process_creation
detection:
    selection_parent:
        ParentImage|endswith:
            - '/ffprobe'
            - '/ffmpeg'
            - '/ffmpegthumbnailer'
    selection_child:
        Image|endswith:
            - '/sh'
            - '/bash'
            - '/dash'
            - '/zsh'
            - '/busybox'
            - '/nc'
            - '/ncat'
            - '/curl'
            - '/wget'
            - '/perl'
            - '/python3'
    condition: selection_parent and selection_child
falsepositives:
    - Custom transcode wrappers that launch helper scripts through ffmpeg exec hooks
level: high
```

**Known false positives:**

- Custom transcode wrappers or exec hooks that legitimately launch a helper shell from ffmpeg
- Administrators running ffprobe through an interactive wrapper script when troubleshooting

**Tuning notes:** Adjust the index and sourcetype to your Sysmon for Linux or auditd ingestion. Allow-list known wrapper scripts by ParentCommandLine. Include bundled paths such as /usr/lib/jellyfin-ffmpeg/ffmpeg and emby-ffmpeg (already matched by the wildcard suffix). Treat any hit on a Jellyfin, Emby or Nextcloud host as an incident until proven otherwise.

### 8 more detection rules (Splunk SPL, Microsoft KQL, Sigma), unlock with TL Creds

- PixelSmash CVE-2026-8461 — FFmpeg Consumer Heap-Corruption Crash Signature (locked: unlock with TL Creds)
- PixelSmash CVE-2026-8461 — Host Reconnaissance and Credential-File Access by Media-Service Account (locked: unlock with TL Creds)
- PixelSmash CVE-2026-8461 — Torrent Client Writes AVI/MKV/MOV into Media Library Path (Zero-Click Delivery) (locked: unlock with TL Creds)
- PixelSmash CVE-2026-8461 — Shell Spawned by Other FFmpeg Consumers (mpv, Kodi, OBS, Emby) (locked: unlock with TL Creds)
- PixelSmash CVE-2026-8461 — MagicYUV Probing, Debugger Calibration and ASLR Disabling Around FFmpeg (Exploit Development and Staging) (locked: unlock with TL Creds)
- PixelSmash CVE-2026-8461 — Repeated auditd ANOM_ABEND Crashes of FFmpeg Consumers (locked: unlock with TL Creds)
- PixelSmash CVE-2026-8461 — Reverse Shell or Downloader Run by Media-Service Account (locked: unlock with TL Creds)
- PixelSmash CVE-2026-8461 — Outbound Connection from FFmpeg Process or Shell Under Media-Service Account (locked: unlock with TL Creds)

Sign up free to unlock the detection rules: https://threadlinqs.com/?modal=signup

## Sources

- [PixelSmash – Critical FFmpeg Vulnerability Turns Media Files into Weapons (JFrog)](https://jfrog.com/blog/pixelsmash-critical-ffmpeg-vulnerability-turns-media-files-into-weapons/)
- [JFrog Security Research advisory JFSA-2026-001676838 – FFmpeg MagicYUV heap OOB write](https://research.jfrog.com/vulnerabilities/ffmpeg-is-vulnerable-to-a-heap-out-of-bounds-write-in-the-magicyuv-decoder-cve-2026-8461/)
- [FFmpeg fix: reject unaligned MagicYUV slice_height (PR #23159)](https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23159)
- [NVD – CVE-2026-8461](https://nvd.nist.gov/vuln/detail/CVE-2026-8461)
- [CIRCL Vulnerability-Lookup – CVE-2026-8461](https://cve.circl.lu/cve/CVE-2026-8461)
- [GCVE – CVE-2026-8461 (GCVE-0-2026-8461)](https://db.gcve.eu/vuln/CVE-2026-8461)

## Full data

The other 8 detection rules (Splunk SPL / Microsoft KQL / Sigma) unlock with TL Creds; a free account includes daily creds: https://threadlinqs.com/?modal=signup
Indicator values unlock with TL Creds, at a lower price on Red and Purple; AI agents can query rules and IOCs through the Threadlinqs MCP server (Purple): https://intel.threadlinqs.com/mcp

Canonical: https://intel.threadlinqs.com/threat/TL-2026-5110