Android dual screen controls #820

Open

MicheleLoiDWR wants to merge 2 commits from MicheleLoiDWR/android-dual-screen-controls into main

MicheleLoiDWR commented

2026-09-29 04:56:47 -05:00

(Migrated from github.com)

Summary

Adds support for dual-screen Android devices (clamshell handhelds with a second, touch-enabled display): the touch controls can move to the second display, leaving the main display to show the stream video without an overlay on top. The second display is found through Android's DISPLAY_CATEGORY_PRESENTATION capability, so nothing is tied to a specific device.
It's opt-in and off by default, so there's no change for existing users unless they enable it.

Based on #819 (on-screen menu button). The stream menu is how the swap button below is reached on gesture-navigation devices. Until #819 is merged this PR also shows its commit; only the second commit belongs to this PR.

I saw there was a similar PR, #708 ("Feat/dual screen support"), but it was closed and didn't include a second-screen implementation, so this is a fresh take on the same idea.

What users get

Settings → Second Screen (only shown when a presentation-capable display is present):

Setting	Default
Controls on Second Screen	Off	Moves the touch controls to the second display
Menu Button	Off	Adds a button to the second screen that opens the stream menu on the main screen
Layout	Auto	Full (all controls), Simplified (touchpad + PS + swap), or Auto (Simplified when a gamepad is connected)
Opacity	100%	Opacity of the second-screen controls

Swap button (in the stream menu, and on the second screen in Simplified mode): hands the second display back to the system mid-stream, e.g. to use a browser on it, and takes it back again. The controls return to the main screen as the normal overlay in the meantime.

Implementation

stream/SecondDisplayController.kt (new): discovers the second display, listens for display and input-device changes, decides Full vs Simplified, owns the presentation's lifecycle (started/stopped from onResume/onPause), and exposes isActive / controllerState / enabled (release/reclaim).
touchcontrols/SecondScreenPresentation.kt (new): an android.app.Presentation hosting both layouts. It's FLAG_NOT_FOCUSABLE, so it doesn't take input focus from a physical gamepad, uses StreamTheme for the black background, and hides the system bars on the second display.
StreamActivity: creates the controller when enabled, hides its own touch controls while the second screen is active, and adds the swap button to the stream menu.
bindTouchControls() (TouchControlsFragment.kt): the full layout's button wiring, moved out of DefaultTouchControlsFragment so both screens share it. No behavior change.
fragment_controls.xml: the touchpad is now centered between Share and Options instead of the screen edges, keeping its 300dp cap. It's unchanged on wide screens and shrinks rather than overlapping other controls on narrow ones, such as a near-square second display.
TouchpadView.alwaysVisible: opt-in; draws the idle touchpad where there's no video behind it.
New preferences are all added to SettingsFragment's DataStore whitelist.

Testing

Tested manually on an AYN Thor (Android 13, dual-screen) with a PS5: both layouts, Auto switching, swap/release and reclaim, the second-screen menu button, opacity, and settings persistence.
New SecondScreenLayoutTest covers the Auto/Full/Simplified decision; all 19 unit tests pass.
./gradlew lintDebug: no new warnings on the changed files.

Known limitations

Pre-existing, not fixed here: on the main screen, the full on-screen controls light up but send no input on the device I tested on. I confirmed this on an unmodified main build on the Thor. I suspect the cause is in StreamActivity.onAttachFragment: it seems to replace the single forwarding job for each fragment it attaches, which would leave only the last-attached TouchpadOnlyFragment connected to the session. It also affects controls swapped back to the main screen. I'd rather look into it in a separate PR.
The Full layout's face buttons are vertically centered rather than placed relative to the touchpad, so on some near-square second screens of unusual sizes the touchpad can come within a few dp of the triangle button. On such devices Auto normally picks Simplified anyway.
Second-display availability in Settings is checked when the screen opens.

Notes

Developed with the assistance of Claude Code, and tested on real hardware as described above.

## Summary Adds support for dual-screen Android devices (clamshell handhelds with a second, touch-enabled display): the touch controls can move to the second display, leaving the main display to show the stream video without an overlay on top. The second display is found through Android's `DISPLAY_CATEGORY_PRESENTATION` capability, so nothing is tied to a specific device. It's opt-in and off by default, so there's no change for existing users unless they enable it. **Based on #819** (on-screen menu button). The stream menu is how the swap button below is reached on gesture-navigation devices. Until #819 is merged this PR also shows its commit; only the second commit belongs to this PR. I saw there was a similar PR, #708 ("Feat/dual screen support"), but it was closed and didn't include a second-screen implementation, so this is a fresh take on the same idea. ## What users get **Settings → Second Screen** (only shown when a presentation-capable display is present): | Setting | Default | | |---|---|---| | Controls on Second Screen | Off | Moves the touch controls to the second display | | Menu Button | Off | Adds a button to the second screen that opens the stream menu on the main screen | | Layout | Auto | **Full** (all controls), **Simplified** (touchpad + PS + swap), or **Auto** (Simplified when a gamepad is connected) | | Opacity | 100% | Opacity of the second-screen controls | **Swap button** (in the stream menu, and on the second screen in Simplified mode): hands the second display back to the system mid-stream, e.g. to use a browser on it, and takes it back again. The controls return to the main screen as the normal overlay in the meantime. ## Implementation - `stream/SecondDisplayController.kt` (new): discovers the second display, listens for display and input-device changes, decides Full vs Simplified, owns the presentation's lifecycle (started/stopped from `onResume`/`onPause`), and exposes `isActive` / `controllerState` / `enabled` (release/reclaim). - `touchcontrols/SecondScreenPresentation.kt` (new): an `android.app.Presentation` hosting both layouts. It's `FLAG_NOT_FOCUSABLE`, so it doesn't take input focus from a physical gamepad, uses `StreamTheme` for the black background, and hides the system bars on the second display. - `StreamActivity`: creates the controller when enabled, hides its own touch controls while the second screen is active, and adds the swap button to the stream menu. - `bindTouchControls()` (`TouchControlsFragment.kt`): the full layout's button wiring, moved out of `DefaultTouchControlsFragment` so both screens share it. No behavior change. - `fragment_controls.xml`: the touchpad is now centered between Share and Options instead of the screen edges, keeping its 300dp cap. It's unchanged on wide screens and shrinks rather than overlapping other controls on narrow ones, such as a near-square second display. - `TouchpadView.alwaysVisible`: opt-in; draws the idle touchpad where there's no video behind it. - New preferences are all added to `SettingsFragment`'s `DataStore` whitelist. ## Testing - Tested manually on an AYN Thor (Android 13, dual-screen) with a PS5: both layouts, Auto switching, swap/release and reclaim, the second-screen menu button, opacity, and settings persistence. - New `SecondScreenLayoutTest` covers the Auto/Full/Simplified decision; all 19 unit tests pass. - `./gradlew lintDebug`: no new warnings on the changed files. ## Known limitations - **Pre-existing, not fixed here:** on the main screen, the *full* on-screen controls light up but send no input on the device I tested on. I confirmed this on an unmodified `main` build on the Thor. I suspect the cause is in `StreamActivity.onAttachFragment`: it seems to replace the single forwarding job for each fragment it attaches, which would leave only the last-attached `TouchpadOnlyFragment` connected to the session. It also affects controls swapped back to the main screen. I'd rather look into it in a separate PR. - The Full layout's face buttons are vertically centered rather than placed relative to the touchpad, so on some near-square second screens of unusual sizes the touchpad can come within a few dp of the triangle button. On such devices Auto normally picks Simplified anyway. - Second-display availability in Settings is checked when the screen opens. ## Notes Developed with the assistance of Claude Code, and tested on real hardware as described above. <img width="1920" height="1080" alt="01-settings-second-screen" src="https://github.com/user-attachments/assets/070a182a-9a01-4fae-9b9c-55805e29f9cc" /> <img width="1920" height="1080" alt="05-top-stream-menu" src="https://github.com/user-attachments/assets/a8551e4f-c02a-470c-b169-771c9429353d" /> <img width="1240" height="1080" alt="04-bottom-simplified" src="https://github.com/user-attachments/assets/d7842807-2ac6-4577-86b4-9b10a67e095b" /> <img width="1240" height="1080" alt="06-bottom-full" src="https://github.com/user-attachments/assets/79f2d6eb-9c30-4dfb-92d2-f05c1f8c3167" />

This pull request is broken due to missing fork information.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin 'MicheleLoiDWR/android-dual-screen-controls:MicheleLoiDWR/android-dual-screen-controls'

git switch 'MicheleLoiDWR/android-dual-screen-controls'

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch 'main'

git merge --no-ff 'MicheleLoiDWR/android-dual-screen-controls'

git switch 'MicheleLoiDWR/android-dual-screen-controls'

git rebase 'main'

git switch 'main'

git merge --ff-only 'MicheleLoiDWR/android-dual-screen-controls'

git switch 'MicheleLoiDWR/android-dual-screen-controls'

git rebase 'main'

git switch 'main'

git merge --no-ff 'MicheleLoiDWR/android-dual-screen-controls'

git switch 'main'

git merge --squash 'MicheleLoiDWR/android-dual-screen-controls'