> ## Documentation Index
> Fetch the complete documentation index at: https://docs.altnautica.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Video Stack
> The full video pipeline from camera sensor to browser pixel, including WFB-ng, MediaMTX, and WebRTC.
# Video Stack
The ADOS video pipeline carries live HD video from the drone's camera to your browser with 40-100 ms latency, depending on the connection. It uses standard open-source tools at every stage: V4L2 for capture, ffmpeg for encoding, WFB-ng for radio transport, MediaMTX for local serving, and WebRTC for browser delivery. On the aircraft, the native Rust video service (`ados-video`, the `ados-video` unit) supervises the encoder and the local MediaMTX instance; the ground station receives the stream through `ados-mediamtx-gs`.
## Full pipeline
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
%%{init: {'theme': 'dark'}}%%
sequenceDiagram
participant Camera as Camera (CSI/USB)
participant V4L2 as V4L2 Driver
participant FFmpeg as ffmpeg Encoder
participant MTX_Air as MediaMTX (Air)
participant WFB_TX as wfb_tx (5 GHz)
participant WFB_RX as wfb_rx (5 GHz)
participant MTX_GS as MediaMTX (Ground)
participant Browser as Browser (WebRTC)
Camera->>V4L2: Raw frames
V4L2->>FFmpeg: YUV420p via V4L2 device
FFmpeg->>MTX_Air: H.264 RTSP push
MTX_Air->>WFB_TX: RTSP read + raw NALs
Note over WFB_TX,WFB_RX: 5 GHz monitor mode
FEC encoded packets
No WiFi association
WFB_TX->>WFB_RX: Radio link (50 km range)
WFB_RX->>MTX_GS: Decoded NALs
MTX_GS->>Browser: WebRTC WHEP
Note over Browser: Hardware H.264 decode
MediaSource or WebRTC
```
## Stage 1: Capture
The drone agent detects cameras at boot by scanning `/dev/video*` devices. It supports:
* **MIPI CSI cameras** via V4L2 (Radxa Camera 4K, Arducam modules)
* **USB UVC cameras** via V4L2 (any standard webcam)
The video service configures resolution, framerate, and pixel format before starting the encoder. Default: 1080p at 30 fps, YUV420p.
The encoder choice comes from the board's HAL profile. On boards with a Rockchip media engine (RK3588, RK3576), the agent can use `rkmpp` (Rockchip Media Process Platform) for zero-copy encode directly from the camera ISP. Boards that expose a V4L2 hardware encoder use it through ffmpeg. On boards without a usable hardware encoder, it falls back to `libx264`.
## Stage 2: Encode
ffmpeg encodes the raw camera output to H.264:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
ffmpeg \
-f v4l2 -input_format yuyv422 -video_size 1920x1080 -framerate 30 \
-fflags nobuffer -flags low_delay \
-probesize 32 -analyzeduration 0 \
-i /dev/video0 \
-c:v libx264 -preset ultrafast -tune zerolatency \
-profile:v high -level 4.1 \
-b:v 6000k -maxrate 6000k -bufsize 3000k \
-g 30 -keyint_min 30 \
-f rtsp rtsp://localhost:8554/ados
```
Key flags for low latency:
| Flag | Purpose |
| ------------------------------------- | ----------------------------------------- |
| `-fflags nobuffer` | Disable input buffering |
| `-flags low_delay` | Minimize encoder latency |
| `-probesize 32` | Tiny probe size (faster startup) |
| `-analyzeduration 0` | Skip format analysis |
| `-preset ultrafast -tune zerolatency` | Fastest possible x264 encode |
| `-g 30 -keyint_min 30` | Keyframe every 1 second (30 fps / 30 GOP) |
The encoded stream pushes to a local MediaMTX instance over RTSP.
## Stage 3: WFB-ng transport
WFB-ng (WiFi Broadcast next generation) uses an RTL8812EU adapter in monitor mode to broadcast FEC-encoded packets on 5 GHz. This is not standard WiFi. There is no association, no handshake, no retransmission, and no CSMA/CA backoff.
Key properties:
| Property | Value |
| --------------- | ---------------------------------- |
| Radio mode | 802.11 monitor mode (injection) |
| Frequency | 5 GHz (channels 36-165) |
| FEC | Reed-Solomon, configurable ratio |
| Encryption | WFB-ng key exchange (AES) |
| Max range | 50+ km (with directional antennas) |
| Typical latency | 2-5 ms one-way |
On the air side, `wfb_tx` reads the RTSP stream from MediaMTX and broadcasts it. On the ground side, `wfb_rx` receives and reassembles the stream, feeding it back into a ground-side MediaMTX instance.
The agent supervises `wfb_tx` and `wfb_rx` from native Rust services: the radio TX service (`ados-radio`, the `ados-wfb` unit) on the drone and the receiver service (`ados-groundlink`, the `ados-wfb-rx` unit) on the ground station. It does not use OpenHD as a runtime dependency. WFB-ng is the transport protocol, and the agent controls the underlying `wfb_tx` / `wfb_rx` binaries directly.
## Stage 4: Ground serving
The ground-side MediaMTX instance ingests the reassembled stream from WFB-ng RX and serves it to clients over WebRTC WHEP (WebRTC-HTTP Egress Protocol).
WHEP is an HTTP-based WebRTC signaling protocol. The browser sends a POST to the WHEP endpoint, and MediaMTX responds with an SDP answer. No custom signaling server needed.
| Endpoint | URL | Protocol |
| -------- | ------------------------------------- | ------------------------- |
| WHEP | `http://:8889/ados/whep` | WebRTC (H.264 RTP) |
| RTSP | `rtsp://:8554/ados` | RTSP (for tools like VLC) |
MediaMTX uses copy-codec (zero transcoding). The H.264 stream from WFB-ng passes through to WebRTC without re-encoding. CPU overhead is minimal: \~15% of one core on Pi 4B.
## Stage 5: Browser decode
The browser receives H.264 RTP over WebRTC and decodes it using the platform's hardware decoder. Chrome, Edge, Firefox, and Safari all support hardware H.264 decode on modern hardware.
The `webrtc-client.ts` module in Mission Control handles:
* WHEP negotiation (POST to the endpoint, receive SDP answer)
* ICE candidate gathering (STUN servers for NAT traversal)
* Track attachment to a `