How Meta Scaled FFmpeg to Process Billions of Videos Daily

FFmpeg is the undisputed workhorse of media processing. At Meta's scale, it's executed tens of billions of times a day to handle video uploads, transcoding, and streaming. For years, the sheer volume and specific needs of live and on-demand video forced Meta to maintain a heavily modified internal fork of FFmpeg. This article delves into the engineering challenges of that fork, the two critical features it provided, and how collaborative work with the FFmpeg community allowed Meta to finally deprecate it and rely solely on upstream FFmpeg. You can read the original engineering deep dive as a key reference for this case study.

The Fork and Its Divergence

Maintaining an internal fork is a classic scaling dilemma. Meta's fork initially provided capabilities absent in mainline FFmpeg:

• Threaded Multi-Lane Transcoding: For adaptive streaming (DASH/HLS), multiple quality renditions (lanes) are created from one source. A naive approach runs separate FFmpeg processes, duplicating decode work. The internal fork allowed parallel encoding across all lanes within a single process, drastically improving CPU utilization.
• Real-Time Quality Metrics: Computing metrics like VMAF or SSIM typically requires a finished encode. For livestreaming, quality must be assessed during transcoding. The fork enabled 'in-loop' decoding, feeding compressed frames back for immediate comparison against the source.

As the fork diverged, it became harder to rebase and missed out on new codecs and stability fixes in upstream FFmpeg. Supporting two versions created complexity and risk.

Upstreaming for Community Impact

The decision to upstream is strategic. Features with broad utility are contributed; highly specific ones are kept internal.

What Was Upstreamed (Benefiting Everyone):

1. Efficient Multi-Lane Encoding: The threading model from Meta's fork inspired a major refactor in FFmpeg 6.0-8.0, making parallel encoding a native, optimized feature for all users.
2. Real-Time Quality Metrics: The 'in-loop' decoding capability landed in FFmpeg 7.0, enabling real-time metric computation for livestreaming pipelines everywhere.

What Stayed Internal (The Pragmatic Choice):

• Support for Meta's custom Scalable Video Processor (MSVP) ASIC was added via FFmpeg's standard hardware APIs. Since MSVP isn't publicly available, upstreaming it would burden maintainers without providing external value. Meta manages rebasing these internal patches themselves.

This approach mirrors best practices in cloud infrastructure, where contributing to core open-source projects is often more sustainable than maintaining private forks, as discussed in this analysis on enterprise-grade Kubernetes strategies.

Limitations and Considerations

While upstreaming is ideal, it's not always feasible. The MSVP case highlights a key limitation: contributing hardware-specific code without public hardware creates a support burden. The process also requires deep collaboration with project maintainers and can take years for complex changes to land.

Next Steps for Engineers

1. Audit Your Dependencies: Do you have internal forks? Quantify the maintenance cost versus the benefit of upstreaming key patches.
2. Engage with Communities: For widely useful features, present your use case to upstream maintainers early. Collaboration, as with FFlabs and VideoLAN, is key.
3. Design for Abstraction: Like Meta did with hardware APIs, wrap custom integrations behind standard interfaces to minimize lock-in and ease future migration.

The shift towards real-time, AI-enhanced media processing, as seen in developments like generative world models, will further push tools like FFmpeg. Meta's commitment underscores that investing in open-source core infrastructure is crucial for innovation at scale.