A muxer (short for multiplexer) is software that combines separate streams of video, audio, subtitles, and other data into a single file you can play or stream. Every time you watch a video, a muxer was responsible for weaving those individual components together so your media player receives them as one synchronized package.
How Muxing Works
Video and audio are created as separate streams. A camera records video frames, a microphone records audio samples, and subtitles exist as a text file. These streams need to be packaged together so a player can read them in sync. That packaging step is muxing.
A muxer takes chunks of data (called packets) from each stream and interleaves them into a single file. Each packet carries a small header identifying which stream it belongs to, whether that’s video, audio, or subtitles. The muxer arranges these packets so a player can pull out the right pieces at the right time during playback. Think of it like shuffling two decks of cards together in a specific order so someone on the other end can sort them back into separate decks.
This interleaving adds a small amount of overhead to the file. At normal video bitrates the overhead is negligible, typically 1 to 2 percent. At extremely low bitrates, though, the overhead from packet headers can balloon to 20 or even 40 percent of the total file size, since the headers stay roughly the same size regardless of how much actual media data each packet contains.
Keeping Audio and Video in Sync
The most critical job a muxer performs is synchronization. Each packet gets two timestamps: one that tells the decoder when to decode the frame, and another that tells the player when to display it. These two timestamps differ because modern video compression uses frames that reference other frames in the future, meaning some frames need to be decoded out of order. The decoder has to process a future reference frame before it can build the current one, so the “decode” timestamp comes earlier than the “display” timestamp for those frames.
To keep everything aligned, the muxer applies a small time offset to all streams so that audio and video start from the same reference point. Without this offset, you’d get the lip-sync drift that makes dialogue feel slightly “off.” The muxer ensures every stream shares the same starting timestamp so your player presents audio and video in lockstep.
Container Formats: Where Muxed Data Lives
The file a muxer produces is called a container. The container format determines what types of streams the file can hold and how they’re organized. Common containers include:
- MP4 (MPEG-4 Part 14): The most widely compatible format, used across web browsers, phones, and streaming platforms.
- MKV (Matroska): An open-source container that supports virtually unlimited audio, video, and subtitle tracks, popular for high-quality video collections.
- AVI (Audio Video Interleaved): An older Microsoft format, largely replaced by MP4 and MKV but still encountered in legacy files.
- WebM: A subset of Matroska designed specifically for web playback.
A container is not a codec. The codec compresses and decompresses the actual video or audio data. The container simply holds the compressed streams together. The same video codec can be muxed into an MP4, an MKV, or a different container entirely. Choosing a container affects compatibility and which features you can use (like multiple subtitle tracks), but it doesn’t change the underlying quality of the video or audio.
Multiple Tracks and Metadata
Containers can hold far more than one video and one audio stream. A single MKV file might contain six audio tracks across two languages, seven subtitle tracks for three languages, and metadata like chapter markers and font attachments for styled subtitles. The muxer bundles all of these into the file and tags each track with information like its language, whether it should play by default, and whether it’s a specialized track like a commentary or hearing-impaired subtitle.
This is where muxing becomes a practical, hands-on task for many users. If you want to swap in subtitles from a different source, add a second audio language, or strip out tracks you don’t need to save space, you’re remuxing. Remuxing copies streams from one container to another (or rearranges them within the same container) without re-encoding, so it’s fast and lossless. You’re rearranging the packaging, not reprocessing the contents.
Muxing in Live Streaming
Muxing plays a slightly different role in live and on-demand streaming. Protocols like HLS and DASH break video into small segments, each of which is its own muxed file. The streaming server muxes video and audio into these short chunks (often 2 to 10 seconds long), and the player downloads and plays them in sequence.
Adaptive bitrate streaming takes this further. The server muxes multiple versions of the same content at different resolutions and bitrates. Your player monitors your network speed and switches between versions mid-stream, picking higher quality when bandwidth is plentiful and dropping to lower quality to avoid buffering. Low-latency versions of these protocols reduce the delay from a typical 20 to 30 seconds down to 3 to 6 seconds by using partial segments and chunked transfers, which means the muxer is producing even smaller, more frequent output.
Demuxing: The Reverse Process
When you press play, the reverse happens. A demuxer (or demultiplexer) reads the container file, separates the interleaved packets back into individual streams, and routes each one to the correct decoder. The video packets go to the video decoder, the audio packets go to the audio decoder, and subtitle data goes to the subtitle renderer. The demuxer itself doesn’t decode anything. It’s a traffic director that reads the container structure and hands each stream off to the right place.
Every media player has demuxers built in or available as plugins. When a player fails to open a file, the problem is often a missing demuxer for that container format rather than a missing codec.
Common Muxing Tools
FFmpeg is the most widely used muxing tool in the world. It’s a command-line program that handles nearly every container and codec in existence, and it powers many commercial video platforms behind the scenes. If you’re comfortable typing commands, FFmpeg can mux, demux, remux, and transcode just about anything.
For a more visual approach, MKVToolNix provides a graphical interface specifically for working with MKV files. You drag in your source files, check or uncheck the tracks you want to keep, set language tags and default flags, and hit start. It’s the go-to tool for anyone who wants to combine or rearrange tracks without touching a command line. tsMuxeR fills a similar role for Blu-ray-compatible transport stream files. Most video editing software (Premiere, DaVinci Resolve, Handbrake) also performs muxing as the final step of its export process, even if it doesn’t call it that by name.