perf: low-latency streaming pipeline, swapchain mailbox presentation, and macOS QoS #817

Open

whitestorm007 wants to merge 7 commits from whitestorm007/perf/low-latency-optimizations into main

whitestorm007 commented

2026-09-20 13:53:00 -05:00

(Migrated from github.com)

Summary of Changes

This PR introduces latency, throughput, and thread scheduling optimizations across the networking, decoding, and rendering pipeline:

Networking (lib/src/takion.c):
- Enlarged Socket Receive Buffer: Replaces the fixed buffer size with dynamic negotiation starting at 4 MB and stepping down (binary fallback to 256 KB) before falling back to a_rwnd. This absorbs microbursts and prevents UDP packet drops during high-bitrate frame bursts (e.g., 30–50 Mbps local PS5 HDR streams).
- macOS Real-Time Video QoS: Applies SO_NET_SERVICE_TYPE with NET_SERVICE_TYPE_RV on Darwin platforms. This informs the macOS kernel to map Remote Play UDP packets to Layer 2 802.11e WMM Video Access Category (AC_VI) and DSCP priority (CS4/AF41), minimizing latency and queueing over Wi-Fi.
Video Decoding (lib/src/ffmpegdecoder.c):
- Applies AV_CODEC_FLAG_LOW_DELAY and AV_CODEC_FLAG2_FAST to AVCodecContext prior to avcodec_open2().
- Tells FFmpeg and underlying hardware decoders (such as VideoToolbox) to disable B-frame reordering delay and output interactive video frames immediately.
Vulkan Swapchain Presentation (gui/src/qmlmainwindow.cpp):
- Updates swapchain creation to request VK_PRESENT_MODE_MAILBOX_KHR for VSync presentation, with an automatic fallback to VK_PRESENT_MODE_FIFO_KHR if mailbox mode is unsupported by the surface.
- Aligns with the existing code comment (// Prefer MAILBOX for vsync (lower) latency when supported). This eliminates VSync blocking stalls on high-refresh displays (such as 120Hz ProMotion / VRR panels).
Thread Scheduling Priorities (gui/src/qmlbackend.cpp, gui/src/qmlmainwindow.cpp, gui/src/streamsession.cpp):
- Elevates frame_thread and render_thread to QThread::TimeCriticalPriority and audio_out_drain_thread to QThread::HighPriority.
- Guides OS thread dispatchers (e.g. Darwin P-core / E-core assignment, Linux CFS, Windows priority classes) to schedule frame reception and rendering onto high-performance cores without preemption from background tasks.

Verification & Testing

Built and tested with CMake & Ninja on macOS Apple Silicon.
Passed 100% of unit tests in chiaki-unit (117 of 117 tests passing, 0 failures, 0 skipped).
Verified zero packet drops and smooth frame pacing on local 1080p 60fps HDR streams.

### Summary of Changes This PR introduces latency, throughput, and thread scheduling optimizations across the networking, decoding, and rendering pipeline: 1. **Networking (`lib/src/takion.c`)**: - **Enlarged Socket Receive Buffer**: Replaces the fixed buffer size with dynamic negotiation starting at 4 MB and stepping down (binary fallback to 256 KB) before falling back to `a_rwnd`. This absorbs microbursts and prevents UDP packet drops during high-bitrate frame bursts (e.g., 30–50 Mbps local PS5 HDR streams). - **macOS Real-Time Video QoS**: Applies `SO_NET_SERVICE_TYPE` with `NET_SERVICE_TYPE_RV` on Darwin platforms. This informs the macOS kernel to map Remote Play UDP packets to Layer 2 802.11e WMM Video Access Category (`AC_VI`) and DSCP priority (`CS4`/`AF41`), minimizing latency and queueing over Wi-Fi. 2. **Video Decoding (`lib/src/ffmpegdecoder.c`)**: - Applies `AV_CODEC_FLAG_LOW_DELAY` and `AV_CODEC_FLAG2_FAST` to `AVCodecContext` prior to `avcodec_open2()`. - Tells FFmpeg and underlying hardware decoders (such as VideoToolbox) to disable B-frame reordering delay and output interactive video frames immediately. 3. **Vulkan Swapchain Presentation (`gui/src/qmlmainwindow.cpp`)**: - Updates swapchain creation to request `VK_PRESENT_MODE_MAILBOX_KHR` for VSync presentation, with an automatic fallback to `VK_PRESENT_MODE_FIFO_KHR` if mailbox mode is unsupported by the surface. - Aligns with the existing code comment (`// Prefer MAILBOX for vsync (lower) latency when supported`). This eliminates VSync blocking stalls on high-refresh displays (such as 120Hz ProMotion / VRR panels). 4. **Thread Scheduling Priorities (`gui/src/qmlbackend.cpp`, `gui/src/qmlmainwindow.cpp`, `gui/src/streamsession.cpp`)**: - Elevates `frame_thread` and `render_thread` to `QThread::TimeCriticalPriority` and `audio_out_drain_thread` to `QThread::HighPriority`. - Guides OS thread dispatchers (e.g. Darwin P-core / E-core assignment, Linux CFS, Windows priority classes) to schedule frame reception and rendering onto high-performance cores without preemption from background tasks. ### Verification & Testing - Built and tested with CMake & Ninja on macOS Apple Silicon. - Passed 100% of unit tests in `chiaki-unit` (117 of 117 tests passing, 0 failures, 0 skipped). - Verified zero packet drops and smooth frame pacing on local 1080p 60fps HDR streams.

This pull request is broken due to missing fork information.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin 'whitestorm007/perf/low-latency-optimizations:whitestorm007/perf/low-latency-optimizations'

git switch 'whitestorm007/perf/low-latency-optimizations'

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch 'main'

git merge --no-ff 'whitestorm007/perf/low-latency-optimizations'

git switch 'whitestorm007/perf/low-latency-optimizations'

git rebase 'main'

git switch 'main'

git merge --ff-only 'whitestorm007/perf/low-latency-optimizations'

git switch 'whitestorm007/perf/low-latency-optimizations'

git rebase 'main'

git switch 'main'

git merge --no-ff 'whitestorm007/perf/low-latency-optimizations'

git switch 'main'

git merge --squash 'whitestorm007/perf/low-latency-optimizations'