gh-moonlight-stream/moonlight-common-c
Watch
1
0
Fork
You've already forked moonlight-common-c
 0
Code Issues 18 Pull requests 8 Releases Activity

Add native trackpad input support #109

Open
cfalas wants to merge 1 commit from cfalas/trackpad into master
pull from: cfalas/trackpad
merge into: gh-moonlight-stream:master
Conversation 3 Commits 1 Files changed 4 +76
cfalas commented 2025-06-17 03:02:04 -05:00 (Migrated from github.com)

Current implementations (e.g. moonlight-android explicitly, and moonlight-qt implicitly) convert trackpad events into mouse actions, which inherently loses some information (e.g. gestures such as pinch-to-zoom and multi-finger swipes do not go through, scrolls are inprecise)

This PR adds a new channel for trackpad inputs. This is similar to a touch device (i.e. has positional "fingers"), with an additional button press for pressing the trackpad.

Similarly to touchscreen events, this is behind a feature-flag (which is currently implemented in my fork of sunshine, and my fork of moonlight-android, both of which I will be making a PR for).

Current implementations (e.g. moonlight-android explicitly, and moonlight-qt implicitly) convert trackpad events into mouse actions, which inherently loses some information (e.g. gestures such as pinch-to-zoom and multi-finger swipes do not go through, scrolls are inprecise) This PR adds a new channel for trackpad inputs. This is similar to a touch device (i.e. has positional "fingers"), with an additional button press for pressing the trackpad. Similarly to touchscreen events, this is behind a feature-flag (which is currently implemented in my fork of sunshine, and my fork of moonlight-android, both of which I will be making a PR for).
👍 1
cgutman commented 2025-07-05 19:12:11 -05:00 (Migrated from github.com)

How does this perform with different trackpad dimensions or aspect ratios?

One of the downsides of our normalized x/y coordinates is that it basically assumes a direct mapping between the input device and the screen, which is fine for a touchscreen but not correct for a trackpad. We have the same issue with our gamepad touchpad support, but we get away with it because those devices aren't particularly precise anyway (and the emulated gamepad usually matches the client's gamepad, so the apps interpret the coordinates properly anyway).

How does this perform with different trackpad dimensions or aspect ratios? One of the downsides of our normalized x/y coordinates is that it basically assumes a direct mapping between the input device and the screen, which is fine for a touchscreen but not correct for a trackpad. We have the same issue with our gamepad touchpad support, but we get away with it because those devices aren't particularly precise anyway (and the emulated gamepad _usually_ matches the client's gamepad, so the apps interpret the coordinates properly anyway).
cfalas commented 2025-07-06 01:04:07 -05:00 (Migrated from github.com)

I unfortunately don't have the hardware to test this with different trackpads (I only have one device with a trackpad, with a relatively "standard" aspect ratio). However, I do agree that this is an issue.

What we really want instead of resolution/aspect ratio is the physical dimensions of the trackpad (this is used by both libinput on Linux and Precision Trackpad on Windows). Once we have that, then using normalized coordinates in packets is OK, and also avoids any (rare I imagine) issues where the trackpad has different x- and y-axis pixel densities.

In order to do this, we would have to add some metadata to set the height/width. Would a new packet type (i.e. LI_TRACKPAD_EVENT_UPDATE_DIMENSIONS) be appropriate for this or would it be better placed elsewhere?

Sidenote: I don't know how easy it would be for clients to access the dimensions to advertise them (especially on non-linux/non-windows clients) - right now, I just set a udev rule on my machine to overwrite the dimensions of the trackpad I use over sunshine (which is obviously not very user-friendly). At the very least, we could have some sensible defaults and allow users to overwrite them as options in the clients for platforms which won't let you access the physical dimensions (e.g. Android unless I missed something).

I unfortunately don't have the hardware to test this with different trackpads (I only have one device with a trackpad, with a relatively "standard" aspect ratio). However, I do agree that this is an issue. What we really want instead of resolution/aspect ratio is the physical dimensions of the trackpad (this is used by both libinput on Linux and Precision Trackpad on Windows). Once we have that, then using normalized coordinates in packets is OK, and also avoids any (rare I imagine) issues where the trackpad has different x- and y-axis pixel densities. In order to do this, we would have to add some metadata to set the height/width. Would a new packet type (i.e. `LI_TRACKPAD_EVENT_UPDATE_DIMENSIONS`) be appropriate for this or would it be better placed elsewhere? Sidenote: I don't know how easy it would be for clients to access the dimensions to advertise them (especially on non-linux/non-windows clients) - right now, I just set a udev rule on my machine to overwrite the dimensions of the trackpad I use over sunshine (which is obviously not very user-friendly). At the very least, we could have some sensible defaults and allow users to overwrite them as options in the clients for platforms which won't let you access the physical dimensions (e.g. Android unless I missed something).
cgutman commented 2025-10-11 16:10:28 -05:00 (Migrated from github.com)

One option is to include the dimensions of the trackpad in each event. This is what GameStream does for absolute mouse input.

One option is to include the dimensions of the trackpad in each event. This is what GameStream does for absolute mouse input.
This pull request is broken due to missing fork information.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin 'cfalas/trackpad:cfalas/trackpad'
git switch 'cfalas/trackpad'

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch 'master'
git merge --no-ff 'cfalas/trackpad'
git switch 'cfalas/trackpad'
git rebase 'master'
git switch 'master'
git merge --ff-only 'cfalas/trackpad'
git switch 'cfalas/trackpad'
git rebase 'master'
git switch 'master'
git merge --no-ff 'cfalas/trackpad'
git switch 'master'
git merge --squash 'cfalas/trackpad'
git switch 'master'
git merge --ff-only 'cfalas/trackpad'
git switch 'master'
git merge 'cfalas/trackpad'
git push origin 'master'
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
bug

   
dependencies
Pull requests that update a dependency file

  
duplicate

   
enhancement

   
hacktoberfest-accepted

   
invalid

   
question

   
submodules
Pull requests that update submodules code

  
wontfix
No labels
bug
dependencies
duplicate
enhancement
hacktoberfest-accepted
invalid
question
submodules
wontfix
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
gh-moonlight-stream/moonlight-common-c!109
No description provided.