gh-moonlight-stream/moonlight-common-c
Watch
1
0
Fork
You've already forked moonlight-common-c
 0
Code Issues 18 Pull requests 8 Releases Activity

feat: add microphone protocal #123

Open
Yundi339 wants to merge 1 commit from Yundi339/master into master
pull from: Yundi339/master
merge into: gh-moonlight-stream:master
Conversation 2 Commits 1 Files changed 7 +323 -9
Yundi339 commented 2026-01-31 09:06:09 -06:00 (Migrated from github.com)

The following is translated by AI.

Hi, I'm the developer of the Sunshine-Foundation and moonlight-vplus.

Moonlight has always been a one-way street for audio/video - downstream only, no microphone upstream. We implemented a microphone transmission protocol in Moonlight VPlus, and it's been in production with many users.

This PR contributes the code to the community, aiming to establish a unified specification. Better to standardize now before wider adoption to avoid microphone protocol conflicts down the road.

Everything defined here is open for modification - we'll follow the official guidelines.

Protocol Overview

Client captures mic → Opus encode → UDP send to host. Supports both plaintext and encrypted modes.

Packet Format

typedef struct {
    uint8_t  flags;           // Reserved, currently 0x00
    uint8_t  packetType;      // Fixed 0x61 (Opus audio)
    uint16_t sequenceNumber;  // Little-endian
    uint32_t timestamp;       // Little-endian
    uint32_t ssrc;            // Client identifier, little-endian
} MICROPHONE_PACKET_HEADER;  // 12 bytes

Followed by Opus-encoded audio (optionally encrypted). Total packet <= 1400 bytes.

Key Constants

#define MIC_PACKET_TYPE_OPUS  0x61        // Audio type
#define MAX_MIC_PACKET_SIZE   1400        // Max packet size
#define MIC_PACKET_MAGIC      0x12345678  // Client identifier
#define SS_ENC_MICROPHONE     0x08        // Encryption flag

Encryption Support

We use Sunshine-Foundation and Moonlight VPlus. The old version only supported plaintext MIC data transmission; the new version adds encrypted transmission.

For smooth upgrades (some users may still be on older versions), we implemented compatibility handling: RTSP handshake identifies whether both sides support encryption. If one side doesn't support it, falls back to plaintext. Developers can also force MIC to use encrypted transmission in code for absolute security.

  • Algorithm: AES-128-CBC, reuses remoteInputAesKey
  • IV Generation: riKeyId + sequenceNumber (big-endian) in first 4 bytes
  • Negotiation Logic: If audio encryption is enabled, mic encryption is automatically enabled; if one side doesn't support encryption, falls back to plaintext

Design Notes

MIC_PACKET_MAGIC

This value is sent in every packet's SSRC field. Sunshine currently doesn't validate it.

Design intent: If multiple mic implementations emerge (different codecs, configs), this value can identify client type and enable plugin-based processing.

Reserved Interfaces

These are defined but not currently used, reserved for scenarios where mic needs to be dynamically started/stopped via control stream:

#define SS_MICROPHONE_MAGIC   0x55000008
#define MIC_CONTROL_START     0x01
#define MIC_CONTROL_STOP      0x02

FEC and Packet Loss

Opus library has built-in FEC and packet loss concealment, default FEC level is 1. Not explicitly configured in code - can be added to RTSP handshake if stronger FEC protection is needed.

Files Changed

File Description
MicrophoneStream.c NEW - Core implementation
Limelight.h Added enableMic config and encryption flag
Limelight-internal.h Internal function declarations
Input.h Reserved interface definitions
Connection.c Lifecycle management
RtspConnection.c RTSP handshake
SdpGenerator.c Encryption negotiation

Compatibility

  • Mic initialization failure doesn't affect the connection, just mic unavailable
  • Legacy Sunshine handshake failure auto-disables mic feature
  • Encryption negotiation failure falls back to plaintext
**The following is translated by AI.** Hi, I'm the developer of the [Sunshine-Foundation](https://github.com/qiin2333/foundation-sunshine) and [moonlight-vplus](https://github.com/qiin2333/moonlight-vplus). Moonlight has always been a one-way street for audio/video - downstream only, no microphone upstream. We implemented a microphone transmission protocol in Moonlight VPlus, and it's been in production with many users. This PR contributes the code to the community, aiming to establish a unified specification. Better to standardize now before wider adoption to avoid microphone protocol conflicts down the road. **Everything defined here is open for modification** - we'll follow the official guidelines. --- ## Protocol Overview Client captures mic → Opus encode → UDP send to host. Supports both plaintext and encrypted modes. ### Packet Format ```c typedef struct { uint8_t flags; // Reserved, currently 0x00 uint8_t packetType; // Fixed 0x61 (Opus audio) uint16_t sequenceNumber; // Little-endian uint32_t timestamp; // Little-endian uint32_t ssrc; // Client identifier, little-endian } MICROPHONE_PACKET_HEADER; // 12 bytes ``` Followed by Opus-encoded audio (optionally encrypted). Total packet <= 1400 bytes. ### Key Constants ```c #define MIC_PACKET_TYPE_OPUS 0x61 // Audio type #define MAX_MIC_PACKET_SIZE 1400 // Max packet size #define MIC_PACKET_MAGIC 0x12345678 // Client identifier #define SS_ENC_MICROPHONE 0x08 // Encryption flag ``` --- ## Encryption Support We use Sunshine-Foundation and Moonlight VPlus. The old version only supported plaintext MIC data transmission; the new version adds encrypted transmission. For smooth upgrades (some users may still be on older versions), we implemented compatibility handling: RTSP handshake identifies whether both sides support encryption. If one side doesn't support it, falls back to plaintext. Developers can also force MIC to use encrypted transmission in code for absolute security. - **Algorithm**: AES-128-CBC, reuses `remoteInputAesKey` - **IV Generation**: `riKeyId + sequenceNumber` (big-endian) in first 4 bytes - **Negotiation Logic**: If audio encryption is enabled, mic encryption is automatically enabled; if one side doesn't support encryption, falls back to plaintext --- ## Design Notes ### `MIC_PACKET_MAGIC` This value is sent in every packet's SSRC field. Sunshine currently doesn't validate it. Design intent: If multiple mic implementations emerge (different codecs, configs), this value can identify client type and enable plugin-based processing. ### Reserved Interfaces These are defined but **not currently used**, reserved for scenarios where mic needs to be dynamically started/stopped via control stream: ```c #define SS_MICROPHONE_MAGIC 0x55000008 #define MIC_CONTROL_START 0x01 #define MIC_CONTROL_STOP 0x02 ``` ### FEC and Packet Loss Opus library has built-in FEC and packet loss concealment, default FEC level is 1. Not explicitly configured in code - can be added to RTSP handshake if stronger FEC protection is needed. --- ## Files Changed | File | Description | |------|-------------| | `MicrophoneStream.c` | **NEW** - Core implementation | | `Limelight.h` | Added `enableMic` config and encryption flag | | `Limelight-internal.h` | Internal function declarations | | `Input.h` | Reserved interface definitions | | `Connection.c` | Lifecycle management | | `RtspConnection.c` | RTSP handshake | | `SdpGenerator.c` | Encryption negotiation | --- ## Compatibility - Mic initialization failure doesn't affect the connection, just mic unavailable - Legacy Sunshine handshake failure auto-disables mic feature - Encryption negotiation failure falls back to plaintext
👍 2
Yundi339 commented 2026-01-31 09:08:58 -06:00 (Migrated from github.com)

Our microphone implementation is based on VBCable. While it shares functional similarities with another ongoing Sunshine Pull Request, the underlying mechanisms differ. We have considered merging the complete microphone codebase into the upstream repository; however, several challenges remain:

  • Feature Coupling: The microphone logic is deeply integrated with our other proprietary features, making it difficult to isolate for a clean merge.
  • Stability and Iteration: Having undergone several version updates, we believe the features require further maturation and optimization through continued real-world testing.
  • Technical Refinement: Certain aspects, such as the Acoustic Echo Cancellation (AEC), have not yet reached peak performance. Additionally, two-way microphone communication between Sunshine and Moonlight is currently unsupported.

Given these factors, our current priority is to continue refining these capabilities. Should the opportunity arise to integrate with the community version in the future, we intend to perform a complete rewrite of the microphone logic to ensure it meets community standards and maintains long-term maintainability.

Our microphone implementation is based on **VBCable**. While it shares functional similarities with another ongoing Sunshine Pull Request, the underlying mechanisms differ. We have considered merging the complete microphone codebase into the upstream repository; however, several challenges remain: * **Feature Coupling**: The microphone logic is deeply integrated with our other proprietary features, making it difficult to isolate for a clean merge. * **Stability and Iteration**: Having undergone several version updates, we believe the features require further maturation and optimization through continued real-world testing. * **Technical Refinement**: Certain aspects, such as the **Acoustic Echo Cancellation (AEC)**, have not yet reached peak performance. Additionally, two-way microphone communication between Sunshine and Moonlight is currently unsupported. Given these factors, our current priority is to continue refining these capabilities. Should the opportunity arise to integrate with the community version in the future, we intend to **perform a complete rewrite** of the microphone logic to ensure it meets community standards and maintains long-term maintainability.
jackson00w commented 2026-09-20 10:02:39 -05:00 (Migrated from github.com)

Hi, I'm a small developer and I'd like to know if you're still working on Moonlight's microphone support or if you've given up on the project. What was the last success you had with that project? Maybe I... I can help if you allow me to. I really want this to move forward because what the community wants most is a native microphone in Moonlight, right?

Hi, I'm a small developer and I'd like to know if you're still working on Moonlight's microphone support or if you've given up on the project. What was the last success you had with that project? Maybe I... I can help if you allow me to. I really want this to move forward because what the community wants most is a native microphone in Moonlight, right?
👍 1
This pull request has changes conflicting with the target branch.
  • src/SdpGenerator.c
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin 'Yundi339/master:Yundi339/master'
git switch 'Yundi339/master'

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch 'master'
git merge --no-ff 'Yundi339/master'
git switch 'Yundi339/master'
git rebase 'master'
git switch 'master'
git merge --ff-only 'Yundi339/master'
git switch 'Yundi339/master'
git rebase 'master'
git switch 'master'
git merge --no-ff 'Yundi339/master'
git switch 'master'
git merge --squash 'Yundi339/master'
git switch 'master'
git merge --ff-only 'Yundi339/master'
git switch 'master'
git merge 'Yundi339/master'
git push origin 'master'
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
bug

   
dependencies
Pull requests that update a dependency file

  
duplicate

   
enhancement

   
hacktoberfest-accepted

   
invalid

   
question

   
submodules
Pull requests that update submodules code

  
wontfix
No labels
bug
dependencies
duplicate
enhancement
hacktoberfest-accepted
invalid
question
submodules
wontfix
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
gh-moonlight-stream/moonlight-common-c!123
No description provided.