Development
Notes for contributors working on the client codecs, protocol, or daemon.
Repository Layout
vtremoted/: Swift macOS daemon exposing VideoToolbox over TCP.ffmpeg/: FFmpeg fork containing the remote codec implementations.obs-plugin/: OBS plugin prototype using the same remote protocol.vaapi-driver/: Linux VA-API encode driver, static C SDK, packaging, and Plex smoke tooling.tests/: Integration tests and benchmarks.docs/: Protocol and architecture documentation.
Build Instructions
macOS (Server & Client)
The easiest way to build everything on a Mac:
make build
Release builds default to MACOSX_DEPLOYMENT_TARGET=13.0. Keep that default when building on macOS 27 or newer SDKs unless a release intentionally raises the minimum supported macOS version.
This produces:
vtremoted/.build/release/vtremotedffmpeg/ffmpeg(with local and remote VideoToolbox codecs enabled)
The vtremoted binary must not hard-link Homebrew liblz4 or libzstd; those libraries are runtime-only optional dependencies for LZ4/Zstd wire compression. The release smoke test checks both the macOS 13.0 minimum OS and the absence of absolute Homebrew dylib load commands.
Linux / Windows (Client Only)
Linux FFmpeg CI and release artifacts target x86_64 and arm64 (aarch64).
The arm64 job runs natively on GitHub-hosted ubuntu-24.04-arm, with the same
codec/filter, mock integration, and packaged-artifact smoke checks as x86_64.
See arm64 runtime requirements
for the Ubuntu 24.04 shared-library baseline. The project does not publish or test 32-bit i686 Linux builds.
The VA-API driver job runs on Ubuntu 24.04 and builds the pinned libva 2.22 /
VA-API 1.22 headers from source. Release artifacts target glibc 2.39.
Locally on Linux, run make test-vaapi-driver. The test suite uses
the shared repository mock and stock FFmpeg’s VA-API encoders.
FFmpeg sync candidates run the full CI matrix before promotion. After promotion, the release tag runs validation and packaging; the identical tagged sync push to main skips duplicate builds. FFmpeg compiler-cache checks require actual compilation hits or misses, so a bypassed cache fails CI.
- Install assembler and library dependencies (assemblers are needed for x86 builds):
nasm -v yasm --versionInstall current
nasmandyasmbefore building. FFmpeg’s x86 assembly can fail early with older or missing assemblers. - Configure:
cd ffmpeg ./configure --enable-videotoolbox-remote --enable-liblz4 --enable-libzstd \ --enable-libvmaf --enable-libaom --enable-libdav1d --enable-libsvtav1 - Build:
make -j$(nproc)
The repository Makefile wraps the same build with project defaults:
make build-ffmpeg
If a Linux x86_64 source build fails in FFmpeg x86 assembly after installing current assemblers, use the diagnostic compatibility fallback:
make clean-ffmpeg
make build-ffmpeg FFMPEG_DISABLE_X86ASM=1
This appends --disable-x86asm. It is slower and should not be the default path, but it confirms whether the failure is limited to the assembler/toolchain surface.
Testing & Benchmarks
Integration scripts are located in tests/integration/.
Setup
Export the path to your server binary:
export VTREMOTED="$PWD/vtremoted/.build/release/vtremoted"
Key Scripts
run_all.sh: Standard integration suite.performance.py: Current Intel VA-API, CPU fast/medium, local VideoToolbox and remote VideoToolbox comparison using Big Buck Bunny and FFmpeg signals.bench_vtremote.sh: Perform encoding/transcoding benchmarks.bench_sustained.py: Repeated natural-video and 1/2/4-session comparisons with identities, resource use, merged latency percentiles, counts and decode validation. See the runner instructions.run_transport_regressions.py,run_decode_duplex.py,run_session_reset.sh: Deadline, completion, simultaneous transport and codec context-reuse regressions.run_vtremoted_roundtrip.sh: Verify H.264/HEVC roundtrip correctness.run_mock_wire_compression.sh: Validate LZ4 and Zstd compressed frame payloads against the Python mock server.run_mock_protocol_capabilities.sh: Validate 0.4.1 configure-time capability negotiation and clear rejection of unsupported surfaces.run_mock_side_data_roundtrip.sh: Validate exact protocol side-data round-trips against the Python mock server.run_mock_hevc_pixfmt_negotiation.sh: Validate HEVCbgra,ayuv, andp210lenegotiation against the Python mock server.run_vtremoted_hardware_ingest.sh: Verify remote encoders accept local VideoToolbox hardware frames.run_vtremoted_transcode_hardware_ingest.sh: Verify a local VideoToolbox hardware-decode pipeline can feed remote HEVC encode.run_vtremoted_decode_hardware_output.sh: Verify remote decoders can return VideoToolbox-backed frames when requested.run_vtremoted_hevc_pixfmts.sh: Verify realvtremotedaccepts HEVCbgra,ayuv, andp210leinputs.run_vtremoted_hdr_side_data.sh: Verify realvtremotedpreserves HEVC HDR color signaling.run_vtremoted_hdr_to_sdr.py: Generate HDR10/PQ and HLG fixtures and verify SDR highlight detail, colour tags, HDR metadata removal, frame timing and bounded seeking with a macOS 15+ daemon.run_mock_hdr_to_sdr.py: Verify HDR capability negotiation and configuration validation on each client platform.run_obs_plugin_client_mock.sh: OBS plugin protocol smoke test against the Python mock server.
Measuring performance
Follow the performance comparison setup to reproduce the current results. performance.py dispatches workers to designated Linux and Mac hosts. Native and remote VideoToolbox use the same Mac; the coordinating computer does not encode video.
Keep raw captures outside Git. Publish only the runner’s allowlisted measurements, versions and hashes; never commit real server names, usernames, addresses or private paths.
Adaptive options
-vt_remote_wire_compression auto # pick LZ4 vs Zstd based on resolution/FPS
-vt_remote_inflight auto # adjust inflight over time
GitHub Metadata and Release Notes
FFmpeg sync uses a squashed upstream parent to preserve local changes in the
vendored tree. Keep git-subtree-dir and git-subtree-split trailers on that
upstream commit, not on the repository merge commit. Merge subtree updates and
tracking repairs with a merge commit; squashing or rebasing the PR can discard
the upstream parent. bash scripts/check_ffmpeg_subtree.sh checks the tracking
record before CI builds and automated syncs.
The repository includes small gh-based helpers for keeping the public GitHub surfaces in sync.
Prerequisite:
gh auth status
Key commands:
make sync-github-metadata
make release-notes TAG=v0.10.16
make release-notes-all
bash scripts/generate_release_notes.sh v0.10.16
bash tests/test_llms_release_version.sh
What they do:
make sync-github-metadataupdates the repo description, homepage, and topics.make release-notes TAG=...regenerates and applies the onboarding-style release body for one GitHub release.make release-notes-allbackfills all existing GitHub releases with the generated notes format.bash scripts/generate_release_notes.sh <tag>previews the generated release body locally without editing GitHub.bash tests/test_llms_release_version.shverifiesdocs/llms.txtmatches the latest changelog release.
Release checklist:
- Update
CHANGELOG.md. - Update
docs/llms.txtanddocs/_config.ymlwhen the public release changes. - Confirm the README, landing page CTA, and release notes still point to the latest release and getting-started flow.
- Run
make sync-github-metadataif description, homepage, or topics changed.
The dependency-free CI wiring regression runs with:
bash tests/test_ffmpeg_linux_arm64_ci.sh
Artifact Smoke Checks
CI smoke-tests packaged tarballs after creating them. To reproduce locally:
bash scripts/smoke_release_artifact.sh ffmpeg ffmpeg-linux-x86_64.tar.gz
bash scripts/smoke_release_artifact.sh ffmpeg ffmpeg-linux-arm64.tar.gz
bash scripts/smoke_release_artifact.sh vtremoted vtremoted-macos-arm64.tar.gz
The FFmpeg smoke check unpacks the tarball and verifies the remote encoders, remote decoders, vtremote_transcode, and quality filters are present. The vtremoted smoke check verifies --version and --help without binding a socket.
Documentation site
GitHub Pages builds docs/ with Jekyll. The release number in docs/_config.yml supplies the landing page, navigation and structured data; keep it aligned with docs/llms.txt, the daemon and changelog. Navigation lives in docs/_data/navigation.yml.
For a local preview with Jekyll and the configured plugins installed:
jekyll serve --source docs --host 127.0.0.1
Open http://127.0.0.1:4000/videotoolbox_remote/. Check desktop and mobile navigation, code copying, diagrams, local links and release metadata after theme/content edits. The Performance page contains the current measured comparison; remove obsolete result tables rather than relabeling them as fresh measurements.