OSI Model for AV Systems

The OSI (Open Systems Interconnection) model is a conceptual framework that describes how network communication is divided into seven distinct layers. While it's often taught as an abstract concept, understanding which OSI layer a problem occurs at dramatically speeds up AV troubleshooting and helps explain why certain protocols behave the way they do.

The Seven Layers

Layer 1: Physical — Where Most AV Problems Start

The physical layer is the most overlooked and most common source of AV network problems. Bad cables, incorrect SFP transceivers, bent connector pins, cable runs that exceed maximum distance, or poor terminations all manifest as network issues that can look like software or configuration problems.

For AV networking, key physical layer considerations include:

• Cable category: CAT6 minimum for 1 Gbps; CAT6A or better for 10 Gbps runs. CAT5e is not recommended for new AV installations.
• Maximum run length: 100 meters (328 ft) for copper Ethernet. Fiber extends this to hundreds of meters or kilometers depending on type.
• PoE compatibility: Higher PoE classes (802.3bt) require all four cable pairs to carry current — use CAT6 or better.
• Patch cables: Pre-terminated patch cables vary dramatically in quality. Use rated, tested cables in professional installations.

Physical layer diagnostics: cable tester (Fluke DSX, ideal LANTEK), switch port link light, switch statistics showing CRC errors or input errors.

Layer 2: Data Link — VLANs, MAC Addresses, and Multicast

Most AV network configuration happens at Layer 2. VLANs (802.1Q tagging), MAC address tables, Spanning Tree Protocol, and IGMP snooping all operate here.

Dante audio flows are Layer 2 multicast on a local segment. When Dante can't find a device, or IGMP snooping isn't working, the problem is Layer 2. MAC address table overflow (when a switch has more active devices than its CAM table can hold) causes frames to be flooded — which can overwhelm Dante networks.

Layer 2 AV protocol stack:

• ARP — resolves IP addresses to MAC addresses
• 802.1Q — VLAN tagging on trunk ports
• 802.1p — Layer 2 CoS (Class of Service) QoS marking
• IGMP — multicast group membership (managed by switches via snooping)
• STP/RSTP — loop prevention

Layer 2 troubleshooting tools: switch web GUI, port statistics, MAC address table inspection, Wireshark on a mirrored port.

Layer 3: Network — IP Addressing and Routing

Layer 3 is where IP addressing, subnetting, and routing live. Inter-VLAN routing, routing between buildings, and DHCP all operate at Layer 3.

Dante can operate across Layer 3 boundaries using Dante Domain Manager (DDM), but standard Dante is a Layer 2 protocol and cannot be routed. NDI and other video-over-IP protocols are Layer 3 and can be routed between subnets — but require proper multicast routing (PIM) configuration if crossing router boundaries.

Key Layer 3 concepts for AV:

• Default gateway: Every AV VLAN interface on the Layer 3 switch is the gateway for devices in that subnet.
• Inter-VLAN routing: Traffic between AV VLANs (e.g., control reaching Dante endpoints) requires Layer 3 routing through the core switch.
• ACLs: Access Control Lists at Layer 3 can block specific traffic between VLANs — useful for isolating AV transport from corporate traffic.

Layer 3 troubleshooting: ping, traceroute, checking routing tables on the Layer 3 switch or router (show ip route on Cisco).

Layer 4: Transport — TCP vs. UDP

AV systems use both TCP and UDP, and understanding which protocol a system uses explains its behavior under network stress.

TCP is reliable: it guarantees delivery and retransmits lost packets. AV control systems (Crestron, AMX, QSC Q-SYS API commands) typically use TCP. The tradeoff is that retransmission introduces variable latency — unacceptable for real-time audio and video transport.

UDP is unreliable but fast: it sends packets without waiting for acknowledgment. Dante audio, AES67, NDI video, and RTP streams all use UDP. Lost packets in a UDP audio stream cause audible glitches — there is no retransmit. This is why network reliability (switch quality, proper QoS) matters so much for AV: there is no protocol-level safety net.

Common AV port numbers:

Layer 5: Session — Managing AV Communication Sessions

The session layer is responsible for establishing, maintaining, and terminating communication sessions. In AV systems:

• SIP (Session Initiation Protocol) — manages VoIP and video conferencing call setup/teardown; operates at Layer 5
• H.323 — older conferencing signaling protocol for endpoint registration and call control
• RTSP (Real Time Streaming Protocol) — manages streaming sessions for IP cameras and PTZ streams; used by cameras like Axis, Sony, and Panasonic
• TCP connection management — control system connections (Crestron CIP, AMX ICSP) maintain persistent TCP sessions; a session drop triggers reconnection logic in the control program

Session layer issues in AV: control processors that lose connection to devices intermittently, RTSP streams that don't reconnect after a network blip, or SIP registrations that expire without renewal.

Layer 6: Presentation — Encoding and Encryption

The presentation layer handles data formatting, encryption, and compression:

• HDCP (High-bandwidth Digital Content Protection) — encryption for HDMI/DisplayPort content; authentication happens at Layer 6, though transport uses lower layers
• H.264 / H.265 / AV1 — video codec compression and decompression
• HEVC / JPEG-XS — broadcast and low-latency video compression
• TLS/SSL — encrypts control API traffic (HTTPS, secure WebSocket)

Layer 6 issues in AV: HDCP handshake failures (often misdiagnosed as Layer 1 cable problems), codec negotiation mismatches between encoder and decoder, or TLS certificate errors preventing secure API connections.

Layer 7: Application — AV Protocols

At the application layer, AV protocols have their own discovery and session management:

• Dante: Uses mDNS/Bonjour for device discovery, then negotiates flows directly between devices
• NDI: Uses mDNS for discovery; video flows are unicast or multicast UDP at Layer 4/3
• AES67: Uses SDP (Session Description Protocol) for session announcement, RTP for transport
• Control systems: REST APIs, WebSocket, or proprietary TCP protocols for control commands
• SNMP: Network management protocol for monitoring switch and device status

When a control system can't discover an AV device, start at Layer 1 and work up.

OSI Troubleshooting Method for AV

The OSI model provides a systematic troubleshooting framework. Work bottom-up for connectivity problems; work top-down for application behavior problems:

Bottom-up (device can't be found / reached):

1. Layer 1 — Check link lights. Test the cable. Is the port physically active?
2. Layer 2 — Is the device on the correct VLAN? Does the switch MAC table show the device's MAC? Is IGMP snooping working?
3. Layer 3 — Does the device have the correct IP, subnet mask, and gateway? Can you ping it from the correct VLAN?
4. Layer 4 — Is the correct UDP/TCP port open? Is a firewall blocking the traffic?
5. Layer 7 — Is the application running on the device? Is the service configuration correct?

Common AV problem → OSI layer mapping:

Most AV problems resolve at Layers 1–3. Systematically ruling out lower layers before assuming a configuration or software problem saves significant time.

Common Pitfalls

• Jumping to Layer 7 troubleshooting before checking Layer 1 — Reinstalling software or factory-resetting devices while a bad patch cable is the actual problem wastes hours. Always physically verify the connection first.
• Confusing Layer 2 and Layer 3 VLANs — VLANs are a Layer 2 concept (802.1Q tagging), but they map to Layer 3 subnets for routing. A device can be on the correct VLAN but have the wrong IP subnet, causing inter-VLAN routing to fail.
• Not understanding UDP's unreliability — TCP control systems confirm delivery; UDP-based systems may not. Design control systems with acknowledgment checking at the application layer.
• Ignoring Layer 1 for intermittent issues — Intermittent cable faults cause bit errors that appear as random audio dropouts. These show up as CRC errors on the switch port. Check switch error counters during a dropout event.
• Overlooking Layers 5 and 6 — Session and Presentation layers are less discussed in AV, but HDCP failures, RTSP stream reconnection issues, and TLS certificate problems are all real-world Layer 5/6 issues that get misdiagnosed as wiring or configuration problems.