Codec — Compression/Decompression
Codec (Coder/Decoder)
For AV-over-IP codec selection, see control-systems/crestron-dm-nvx and networking/av-over-ip. For audio codecs in conferencing, see glossary/lpcm and glossary/ucc.
A codec (coder/decoder) is an algorithm — and the hardware or software that implements it — for encoding a signal into a compressed format for transmission or storage, and decoding it back to an uncompressed form for playback. In AV systems, codecs appear in two contexts: video/audio compression for AV-over-IP and streaming, and the hardware video conferencing appliance ("room codec") that handles video calls. Understanding codec characteristics — compression ratio, latency, quality, and bandwidth — is essential for specifying AV-over-IP systems, selecting streaming workflows, and troubleshooting video conferencing quality issues.
Video Compression Codecs
Intra-Frame vs. Inter-Frame
Intra-frame (I-frame only): each frame is compressed independently, similar to JPEG for still images. High quality, predictable bitrate, low decoder complexity, but higher bandwidth than inter-frame. Used in professional production (JPEG 2000, ProRes, JPEG XS).
Inter-frame (predictive): only the differences between frames are encoded. Much higher compression ratio than intra-frame, but requires decoding multiple reference frames — adds latency and increases decoder complexity. Used in consumer and conferencing (H.264, H.265, VP9, AV1).
Common AV Video Codecs
| Codec | Type | Bandwidth (1080p60) | Latency | Use Case |
|---|---|---|---|---|
| Uncompressed | None | 3 Gbps | < 1 ms | SDVoE, NDI (full) |
| JPEG XS | Intra-frame, near-lossless | 50–500 Mbps | 1–5 ms | Broadcast production, ST 2110 |
| JPEG 2000 | Intra-frame, visually lossless | 100–1500 Mbps | 1–20 ms | Digital cinema, broadcast |
| H.264 (AVC) | Inter-frame, lossy | 4–25 Mbps | 50–200 ms | Conferencing, streaming, NVX |
| H.265 (HEVC) | Inter-frame, lossy | 2–15 Mbps | 50–300 ms | 4K streaming, AV-over-IP |
| NDI | HX3 | Inter-frame | 15–100 Mbps | 50–100 ms |
| VP9 | Inter-frame, lossy | 2–10 Mbps | Variable | YouTube, WebRTC |
| AV1 | Inter-frame, lossy | 1–8 Mbps | Variable | Streaming (efficient, open) |
Latency Trade-offs
Compression latency includes encoding latency (time to compress a frame), transmission latency (time on the network), and decoding latency (time to decompress). For real-time video conferencing, total one-way latency should be < 150 ms; above 150 ms, conversations become awkward. For synchronized multi-room AV, latency must be consistent (low jitter) even if absolute latency is somewhat higher.
H.264 and H.265 use Group of Pictures (GOP) structures — one I-frame followed by many P/B frames (predicted frames). The GOP size (keyframe interval) affects latency: a 2-second GOP at 30 fps requires the decoder to buffer 2 seconds of frames. Reducing keyframe interval reduces latency but increases bitrate.
Audio Codecs in Conferencing
Video conferencing platforms use audio codecs that balance quality, bandwidth, and device compatibility:
| Codec | Bandwidth | Bitrate | Notes |
|---|---|---|---|
| G.711 | 300 Hz–3.4 kHz | 64 kbps | PSTN telephone quality; legacy |
| G.722 | 50 Hz–7 kHz | 64 kbps | Wideband HD voice; Teams/Zoom default |
| Opus | 6 Hz–20 kHz | 6–510 kbps | Adaptive; WebRTC standard; best quality at low bitrate |
| AAC-LD | 20 Hz–20 kHz | 96–256 kbps | Low-delay AAC; used in Apple FaceTime, some room systems |
Microsoft Teams and Zoom both use Opus as their primary audio codec in WebRTC mode. Hardware codecs (Poly, Cisco, Yealink) may use G.722 or proprietary codecs for SIP calls.
Room Codec (Hardware Appliance)
In AV industry usage, "codec" also refers to the hardware video conferencing appliance — the dedicated compute device that handles video conferencing call processing for a meeting room. Products:
- Poly Studio X30/X50/X70 — compact all-in-one codecs for small-to-large rooms
- Cisco Room Bar / Room Kit — Webex-native room codecs
- Logitech Rally Bar — camera+codec combo for Microsoft Teams or Zoom rooms
- Crestron Flex — Crestron-integrated Teams/Zoom room codec
- Neat Bar / Board — Teams and Zoom native appliances
These hardware codecs include the camera, microphone, and processing in one device, or connect to a room's separate camera and microphone array via USB, HDMI, or SDI.
Common Pitfalls
-
Selecting H.265 AV-over-IP for a system with mixed decode hardware. H.265 decoding requires hardware support that older GPUs and embedded systems may not provide. A system using H.265 encoders and older NVX decoders may produce no video or high CPU usage. Fix: verify all decoders in the system support the selected codec before specifying; H.264 has near-universal hardware decode support.
-
High GOP size causing large latency jumps on switching. An AV-over-IP system with a 5-second keyframe interval takes up to 5 seconds to display a clean image after switching to a new source — the decoder waits for the next I-frame. Fix: configure encoders with a short keyframe interval (0.5–1 s) to minimize switch latency; some AV-over-IP systems send an instantaneous keyframe on route change.
-
Assuming all "H.264" encoders produce compatible streams. H.264 has multiple profiles (Baseline, Main, High) and levels (3.0, 4.1, 5.2) — not all decoders support all profiles/levels. A 4K H.264 Main Profile Level 5.1 stream may not decode on a device that only supports Level 4.1. Fix: verify encoder output profile/level matches decoder capabilities; use Baseline or Main profile for maximum compatibility.