--- title: How to FFmpeg description: Guide for FFmpeg usage with Opus in m4a and video encoding --- # How to FFmpeg This guide is a collection of tips and tricks for using FFmpeg. It's not a comprehensive guide, but it's a good starting point for anyone who wants to use FFmpeg. This has been written by multiple engineers over many months, and is a constant work in progress. It's a good idea to take all this info with a grain of salt, as some of it may not be entirely accurate or up to date. If you do find issues with this info, please correct it for the next person who needs it. ## Opus in m4a ### Sources If you're going to chat with an LLM about all this, prompting it with these links helps. Generally, chatGPT and Gemini know ffmpeg better than Claude. - [Encapsulation of Opus in ISO Base Media File Format](https://www.opus-codec.org/docs/opus_in_isobmff.html) - [Troubleshooting Android audio](https://developer.android.com/media/media3/exoplayer/troubleshooting) - [ffmpeg-formats documentation](https://manpages.debian.org/stretch/ffmpeg/ffmpeg-formats.1.en.html) ### Misc Lessons - [Opus requires an ~80 ms preroll at every random-access point](https://www.opus-codec.org/docs/opus_in_isobmff.html#4.3.6.2); in MP4 that preroll must be signaled via the roll sample group (sgpd/roll with a non-zero roll_distance). FFmpeg does generate these for Opus in MP4, and Android/ExoPlayer generally handles them, but some Android stacks (notably older ExoPlayer builds and some OEM decoders) are sensitive to timeline "rewrites." Edit lists are the #1 place this bites. - Edit lists: ExoPlayer (Android) explicitly documents only partial support for applying MP4 edit lists; it won't preroll/truncate samples for some edits that don't start on a sync sample (all Opus samples are sync). This can manifest as the "preroll seems wrong" symptom. So for progressive fMP4 Opus, it's safer to disable them. - Fragmented mp4: [See this great comment for a description of fmp4.](https://stackoverflow.com/a/35180327) For audiopipe, there's a tough balance to find the right fragment size. Smaller fragments mean that we can start pushing audio sooner (decreasing time to first audio), but it also create lots of random-access points which makes seeks more expensive and forces preroll frequently. - Brands: The Opus-in-ISOBMFF spec says the file's brands should indicate support for roll groups; examples include iso2, and for roll groups in movie fragments, iso6. For safety, either don't override the brand (let FFmpeg set a compatible set), or set `-brand iso6`. Overriding to mp42 can drop the iso{2,6} compatibility hints some parsers look for. ### Non-audiopipe This is the simplest case. We still want "streaming" capabilities, but (progressive) mp4 natively supports this. ### Audiopipe #### ✅ Flags to Use **`-movflags +empty_moov`** - Writes an initial moov atom at the start. - Ensures the file looks like a valid MP4 from byte 0, so players can start parsing immediately. - Without this, the moov would be at the end → unplayable until fully written. **`-movflags +default_base_moof`** - Uses relative addressing for samples within fragments. - Improves compatibility with Android (ExoPlayer, Media3) and browsers when progressively downloading. - `default_base_moof` vs `omit_tfhd_offset`: both make fragments self-contained, but `default_base_moof` is the newer, clearer way (ISO/IEC 14496-12:2012) and avoids parsers needing to infer offsets from the end of the prior fragment. **`-use_editlist 0`** - Disables edit lists. - ExoPlayer only partially supports them, and they can interfere with Opus preroll signaling. - For progressive, growing files: safer to keep them off. **`-frag_duration 2000000` + `-min_frag_duration 2000000`** - Sets ~2 s fragment duration (tweakable 2–4 s). - Longer fragments reduce the number of random-access points → fewer preroll checks, more stable playback on Android. - `min_frag_duration` avoids tiny initial fragments that trip up parsers. **`-flush_packets 1 -muxdelay 0 -muxpreload 0`** - Reduces latency: pushes out data as soon as available. - Helps with "play while encoding." **`-movflags +dash`** (Optional) - Writes an sidx index box. - Many players can seek fine without sidx by scanning moofs, but some clients seek faster when a top-level sidx is present. - (Caveat: on a growing file the index will naturally only cover the portion written so far — that's okay; players fall back to scanning newer moofs.) #### ❌ Flags to Avoid **`-frag_discont`** - For timeline discontinuities (gaps, splices, restarts). - Not needed in steady, continuous encoding. - Can confuse players if used unnecessarily. - User scrubbing within the received portion is unaffected — no need for frag_discont. **`-separate_moof`** - Creates separate moof/mdat per track. - Only useful with multiple tracks. - Adds overhead and complexity with no benefit for single audio streams. **`-omit_tfhd_offset`** - Older way to simplify offsets, but brittle. - Superseded by `default_base_moof`, which is clearer and better supported. **`-use_editlist 1`** (the default) - Lets muxer rewrite the timeline. - ExoPlayer struggles with these for Opus. - Can break preroll → disable. **`-brand mp42`** - Opus-in-MP4 spec prefers iso6 (or leave default so FFmpeg writes the right compatible set). - Forcing mp42 can remove useful compatibility hints. ### HLS vs DASH vs Progressive mp4 ## Video Stuff ### Vocab - **PTS** = Presentation timestamp. This is the timestamp that a frame is meant to be presented at - **GOP** = Group of Pictures. A specific grouping of frames. Properly timed groups makes for cleaner division of the file during HLS processing - **I-frames (Intra-coded):** - Keyframes. Self-contained images that don't depend on other frames. - Larger in size but allow random access (seeking, segment starts). - **P-frames (Predicted):** - Encode differences relative to a *previous* I- or P-frame. - Smaller in size since only motion/changes are stored. - **B-frames (Bidirectionally predicted):** - Encode differences using both *previous* and *future* reference frames. - Often yield the best compression efficiency (10–20% bitrate savings) - Example: a B-frame between frame 1 and 3 can reference both, instead of encoding from scratch. - **IDR frames (Instantaneous Decoder Refresh):** A special type of I-frame in H.264/H.265. - Like any I-frame, it's self-contained and can be decoded without reference to earlier frames. - Unlike a normal I-frame, an IDR frame forces a **hard reset** of the decoding process. - After an IDR, the decoder must **discard all previous reference frames**. - No future P- or B-frames are allowed to reference anything before the IDR. - This makes IDRs the safest points for segment starts, seeking, or random access. - **SAR** = Sample Aspect Ratio ### Good to Know - Videos shot on phones often come with rotational metadata on them. FFmpeg mostly handles this rotation by default, though the big caveat is that the video dimensions from ffprobe might be flipped (eg. a video might look vertical with dimensions 1080x1920, but since it has a 90° rotation the ffprobe dimension reads 1920x1080) - When scaling, take note of the SAR. It's easy to accidentally stretch the SAR, which can result in a video that appears correct, but has consequences for downstream processing (usually resulting in strange stretching) ### Encoding Settings **`-video_track_timescale 120000`** - Clean, high‑precision integer timescale that divides common frame rates; prevents fractional PTS such as 0.041992. **`-vsync=cfr` and `-r=fps`** - Force constant frame rate at a clean integer fps to avoid fractional timebases that yield non‑zero start PTS. Not always necessary and can potentially cause stutter issues, but goot to know aobut. **`-fflags=+genpts`** - Regenerate PTSs if missing or unreliable. **`-muxpreload=0` / `-muxdelay=0`** - Prevent the muxer from buffering or padding the start. Helps to avoid non zero start PTS **`-g=gop_size` / `-keyint_min=gop_size`** - Fixed GOP cadence aligned to segment boundaries **`-forced-idr=1`** - Force IDR at GOP boundaries so HLS segments are independent. each GOP is entirely self contained **`-sc_threshold=0`** - Disable scene‑cut based keyframe insertion, which would otherwise break GOP cadence. **`-rc-lookahead=0`** - Remove encoder lookahead latency that can push first PTS {">"} 0. Has the side effect of disabling most b-frames. We *probably* want b-frames, but we were struggling with non-zero start PTS causing black frame padding in the HLS files, but that issue is resolved now so we can probably allow b-frames again. **`-itsoffset=`** - If the audio and video streams don't have the same start time, mixing in a new audio can cause the start of the video to shift. This can cause black frame padding in amazon media convert when they normalize our videos.