HDR support #93
Labels
No labels
bug
documentation
duplicate
enhancement
good first issue
help wanted
invalid
question
stale
wontfix
No milestone
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
gh-streetpea/chiaki-ng#93
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Add support for HDR image rendering in opengl for screens that support it such as the new Steam Deck OLED HDR screen.
I'm thinking of picking this up, got briefed by Florian Grill on how to enable HDR and will try to work out how to pass it onto the OLED deck's screen, which is going to take some detective work, since the whole Linux+HDR thing seems to be pretty new.
It might be neccessary to switch to Vulkan for rendering HDR content, since from what I can tell the support in VKD3d and GameScope hinges on the
EXT_HDR_METADATAVulkan extension, which is used to pass HDR metadata from the application to GameScope, which then passes it onto the display device via the Linux kernel's DRM.mpv also only seems to support it for Vulkan and D3D11 outputs:
github.com/haasn/libplacebo@426f346762/src/include/libplacebo/swapchain.h (L67-L81)EXT_HDR_METADATAextension API:github.com/haasn/libplacebo@426f346762/src/vulkan/swapchain.c (L273-L316)Qt5 has support for rendering to a
QSurfacevia Vulkan throughQVulkanInstance, which is the Vulkan analogue to theQOpenGLContextthat is currently used to render the frame received from the Playstation.Unfortunately the Deck is my only device that can render HDR content, so it'll take somebody else to test if the colors and contrast actually look "correct".
I could at least compare the images by eye. I have an hdr tv i can use and compare screenshots in-game vs streaming.
If I understood the gamescope code correctly, the HDR support is not limited to the OLED deck's internal screen, but should extend to displays attached via USB-C as well, as long as they declare HDR support in their EDID. So you should be able to confirm it's working by simply having both your deck and the PS5 connected to the display and then switching between them to see if the colors match.
However, this is probably going to take quite a lot of time, I'm basically starting from almost zero with graphics programming and especially color handling, so if there's anybody with experience in these areas, and especially Vulkan, and would like to collaborate (or take it over completely), get in touch :-)
--
Edit: After some more thinking, I think the fastest way to get HDR might be to simply use
libplacebo(already mentioned above) for rendering. It abstracts away all of the gnarly Vulkan stuff for us and on top offers a lot of goodies for rendering video that might come in handy. It's not exactly a lightweight dependency, but this way we could benefit from all of the hard lessons learned during the development of one of most advanced video players available (libplacebo grew out of mpv's rendering core).I personally have very limited graphics rendering experience so agree that libplacebo seems like a good idea.
Also, regarding testing the image I can set my tv to render different inputs on the same screen via a split screen feature so I can render the steam deck screen using remote play at the same time as the native ps stream for comparison purposes.
Really glad to see some guys are already working on this. Ratchet and Clank in HDR on the Deck would be amazing!
So I've made some progress and got Chiaki to render the 10bit HEVC stream of the PS55 using libplacebo with its Vulkan backend instead of the current OpenGL renderer.
There's only one problem: It seems that in order for the PS5 to send actual HDR10 signals along with the video packets, it needs to be hooked up to a HDR-capable display, at least according to this official Sony support article (my emphasis):
https://remoteplay.dl.playstation.net/remoteplay/lang/gb/ps5_win.html#:%7E:text=If%20your%20Windows%20PC%20and,HDR%5D%20to%20%5BAutomatic%5D
I can confirm that this seems to be the case, I am unable to enable the HDR option in games that have it (tried Cyberpunk 2077 and Ratchet & Clank).
Unfortunately I don't have such a display. I just ordererd a HDMI dongle that can fake the EDID to simulate a 4k HDR display and will see if I can make progress that way.
Additionally an observation: It seems that libplacebo + Vulkan is a bit more resource-intensive than the previous OpenGL approach, so the Deck might pull a bit more power. I'll test it with actual hardware once my OLED deck arrives (currently still waiting for pickup), but maybe we'll have to ship two renderers (OpenGL for non-HDR and LCD decks, libplacebo/Vulkan for HDR) to reduce the impact.
I have a hdr display so I could help with this when I get my oled deck in the mail in the next few days (it just got shipped today). Also unless I'm missing something (and I think you've already alluded to this), we'll have to use game mode to test this as hdr is supported by gamescope but not kde plasma (yet).
Thanks, but this is currently more about finding out where the HDR metadata is hiding (I have a strong suspicion but am not 100% certain), and for that I need to analyze the bitstream when I'm streaming a game with HDR enabled. I'll push my branch for testing when I've found the data and hooked it up to the rendering pipeline :-)
I have an initial version up on a branch: https://github.com/jbaiter/chiaki4deck/tree/hdr
Rendering via Placebo works and can be configured in the settings:
If somebody has a HDR-capable display and is able to build this for the deck, could you connect both the Deck and the PS5 to the display, configure Chiaki for Placebo rendering, set the codec to "H265 HDR" and then run Chiaki in game mode and check if the colors look OK?
Sorry if I sound like a noob, it is my first time on Github.
My deck arrives on friday and I got a LG C2 42 OLED. It got a really great HDR. I already use it for PS5 and PC so I can try connecting my Deck to it.
Bunch of questions:
I have never been on Github before this post, so I have no idea how to download this version of Chiaki above. If you could help me, I would appreciate it.
HDR of SteamOS only works in Gamemode, right? If yes, I will try this out after setting up everything on Friday. I probably need to change some TV settings because I "calibrated" color settings for my PS5 and PC.
Anything, I should watch out for?
Basically, I need to check if the screen looks the same when I switch between HDMI 1 (PS5) and HDMI 2 (Deck OLED) for example, right? If that works and looks the same, then Deck's screen should be outputing correct HDR, if I am not mistaken
Gave it a shot to compile, unfortunately the old tutorial with dependencies is out of date so I couldn't get it to build. Got the OLED Steamdeck in hand and a PS5 hooked up to an LG C9 so could test this now if I could work it out!
EDIT:Got it to build as a flatpak, added in the placebo lib to the yaml file and the program won't boot. Damn, so close. Learned a lot though!
Error while loading shared libraries: libSPIRV.so: cannot open shared object file: No such file or directory
^ That's what I get when I try to run the compiled flatpak through terminal
Thanks for that! The yaml probably needs to be modified to include the additional dependencies, I'll look into it!
@arshiatn @Hasney69 I uploaded a build you can use for testing download this file io.github.streetpea.Chiaki4deck.txt and rename it to io.github.streetpea.Chiaki4deck-hdr.flatpakref instead of .txt (I changed it to .txt for Github to allow me to upload it here) and then use the following in the directory where its located:
After running this file once to test with:
replace the config file with a copy of your chiaki4deck config file with:
and then open it again and select Placebeo (Vulkan, needed for hdr) from the Renderer settings on the bottom of the settings window.
Finally, add it to Steam as a non-steam game. Then launch it in game mode (hdr only works in game mode for now [apparently kde is planning hdr support for 2024])
A note when testing this it seems to segfault when exiting the stream causing the settings window to close instead of staying open (since it is when exiting it doesn't affect the experience but is something to note)
Thanks @streetpea
@jbaiter Copying your settings in the screenshot, I’m getting a very washed out picture with no HDR. Setting hardware decode method to none makes it work as expected with no HDR
Screenshots
vaapi: https://media.discordapp.net/attachments/853001265013719042/1177493346697089094/vaapi.jpg?ex=6572b522&is=65604022&hm=fbd9d0bf0266a5e7fb1189d3e926115b722f3886e9be06ec67565af75fe42694&=&format=webp&width=723&height=452
none: https://media.discordapp.net/attachments/853001265013719042/1177493347108134922/none.jpg?ex=6572b522&is=65604022&hm=f1c36ef5901bb989254414027c8182d94f8ea0fdb17ec1fc906d955d48e5d68a&=&format=webp&width=723&height=452
Thank you @streetpea!
I've seen the crash on exiting the stream with the settings open as well, doesn't happen always for me, and when it does it's somewhere in Qt's wayland stack, probably related to the destruction of the Vulkan resources allocated by the
AVPlaceboWidget. However, it doesn't crash on exiting for me when running with thestreamsubcommand, i.e. with no discovery window visible.And thank you as well for testing @Hasney69!
A few questions:
Running in game mode. Tried in desktop as well out of curiosity to see if it was different with HDR engaging but it wasn't apparent in GameOS, but it's the exact same picture rendering
Yes, but it's hit or miss just with how SteamOS is handling HDR games right now. Ori and the Will of the Wisps engages HDR, but Doom Eternal does not, even though it recognises it's an HDR compatible display. If the game requires Windows HDR to be enabled for your display, they tend to not even recognise it's HDR compatible on the Steam Deck
No problem @jbaiter ! https://cdn.discordapp.com/attachments/853001265013719042/1177507388509667398/chiaki_session_2023-11-24_07-06-59-181181.log?ex=6572c236&is=65604d36&hm=642c19213fe649296209e12c1737a21e2adeeb168acc70fad26df76f7b09e936&
It also seems like one of the side effects of this (maybe there is a way to make the qevents trigger with the avplacebowidget or a good workaround...I wonder if we could keep a different widget as the main widget and make it transparent or if there's a way to fix this) is that the way things are currently handled qevents aren't getting sent when interacting with the screen when using the libplacebo vulkan renderer (this means that touch events, the double click event to cause fullscreen and mouse events (also used for touch) don't work and it also seems that qkeyevents like the T key (for touchpad press) and ESC key for playstation home button intermittently fail (I'm not sure if this is related or not but think it probably is) ... when switching to the opengl renderer these things go away so I think it has to do with the renderer. The good news is that at least with my old steamdeck the renderer on high settings seems to be doing fine and not dropping too many frames (though of course experience will differ depending on the end user).
Edit: I think because it is create as a window container it doesn't have an event filter so we need to install an event filter for it and child widgets such as https://stackoverflow.com/questions/55986902/how-to-install-event-filter-on-a-custom-qt-class to get the events when using that widget
relevant article qt events and filters: https://doc.qt.io/qt-5/eventsandfilters.html
seemingly related issue (actually I think it's hitting at a different point): https://bugreports.qt.io/browse/QTBUG-40320
Update: for the key events such as the esc key if I open the stream in a small window and then click on my desktop (out of the screen window) the keyboard events work again so it probably does have to due with the event handling of the window as well and when its focused on that window it doesn't work but if it's not it does or something like that.
Update 2: A good way to see the difference between what should be happening and what's happening is to go the OpenGL window and double click on the streamsession screen in desktop mode. It will enter Fullscreen mode. Next, try the same thing with the vulkan renderer. It doesn't go into Fullscreen.
@Hasney69 thanks for the log!
Two things that caught my eye:
One: There are no 10 bit surface formats exposed:
Gamescope should add two 10 bit and one 16 bit surface format, if it determines that it should expose HDR: https://github.com/ValveSoftware/gamescope/blob/3.13.8/layer/VkLayer_FROG_gamescope_wsi.cpp#L388-L399
So the 10 bit image that is decoded is then converted to 8 bit for rendering to the surface, which is probably why the colors look so washed out?
Two: The device Vulkan extensions are missing the HDR extensions. Again, Gamescope should add these: https://github.com/ValveSoftware/gamescope/blob/3.13.8/layer/VkLayer_FROG_gamescope_wsi.cpp#L535-L544
The experiment with the HDMI dongle didn't work out, I just ordered the cheapest OLED screen I could get my hands on from Amazon, should arrive this afternoon and I'll experiment myself tonight.
@streetpea So that's why 💡 Thanks for looking into this, I'll fiddle around with it tonight.
Interesting! I tried playing around to see if there was anything on my end, and I did notice that gamescope now has an —enable-hdr flag. Tried adding it to the properties of the Steam entry like I would to change the scaling resolution and it refuses to start up with that in.
The washed out image also makes sense. I remember the official client doing something similar with the PS4 Pro around launch when the TV was turned on at the same time
@jbaiter I was able to fix the behavior by adding an event filter now it responds correctly to double click events, key events, and touch events. However, I see you have refactored the code with avplacebo widget window back on your main branch so not sure if you have already applied a fix with that. If not I can apply it to the updated code. I see you have an event filter but it still looks like it exhibits the behavior still currently since its applied to the container and not the underlying widget.
Update: I made a pull request to the hdr branch on your repo with the touch events, key events, and mouse events fixed.
I've been doing a lot of playing around with non Steam games on the OLED deck today and I have now gotten them working. It turns out the Heroic Launcher and Lutris stop HDR from being exposed to the games, even when identical versions of Proton are used. However, take the same .exes that those launchers downloaded, add them directly as non-steam games and HDR works as expected.
This leads me to believe that Valve aren't currently exposing the HDR capabilities of the Deck to native Linux applications from the GameOS, or it's expecting a launch argument or hook in the code to activate it. Doing a bit more digging with people that have been playing around since HDR was added to gamescope a couple of months ago.
I can corroborate that according to the logs libplacebo only sees the two surface configurations (which are the same desktop or game mode), but running
vulkaninfofrom game mode shows 12, including HDR ones (from desktop mode it only gives the same 2).Would that mean it might be a
libplaceboproblem?Here's the full output of
vulkaninfoboth as called from desktop mode and game mode, in case that's any help:vulkaninfo-desktop.txt
vulkaninfo-gamemode.txt
I might try exploring libplacebo later to see if I can find anything about what might be going on, but it's not something I have any prior experience with so no promises.
Thanks for the research @chipgw! That's strange, given that both are running under Gamescope 🤔
The preconditions for those surfaces and the Vulkan HDR extension to be exposed to clients are:
"GAMESCOPE_HDR_OUTPUT_FEEDBACK"svatom to a non-zero value.Question is: Who is the XCB client in this case?
vulkaninfolinks againstlibxcb.so, so is Valve maybe running a patchedvulkaninfothat sets that atom?Update: nvm, the XCB client is
steamcompmgr/thegamescopebinary, which sets the Atom to a truthy value if the display supports the ST2084 EOTF and HDR is enabled. And HDR is enabled if the--enable-hdrflag is passed or if the XCBGAMESCOPE_DISPLAY_HDR_ENABLEDatom is set to a non-zero value. And that atom is fetched from the display of the XWayland server exposed by Gamescope, where it's probably set by the client application? So maybe a patchedvulkaninfoafter all?Update 2: No
GAMESCOPE_DISPLAY_HDR_ENABLEDstring in the Deck'svulkaninfobinary, so it has to be defined somewhere else?Update 3: It's defined in
steamclient.soandsteamui.so, so it probably corresponds to the setting in the Steam UI? And then the hierarchy of applications running isgamescope -> xwayland -> steam -> $app. Which still begs the question, why isvulkaninfoseeing those extra surfaces and the HDR Vulkan extension, butlibplacebois not?just spitballing here but could the issue be with the way Flatpak handles Mesa drivers? if, for example, I install the GPU-Viewer Flatpak it gives a driver version of 23.1.9 where vulkaninfo gave 23.2.99; the former of which would be provided by Flatpak where the latter would be from Valve, at least as far as I understand it (TBH I'm in over my head a bit here)
EDIT: and I was testing using the hdr flatpak mentioned above in the thread; haven't tried building myself
EDIT2: I didn't see the surface info in GPUViewer itself, but I ran vulkaninfo in the container and it only has the two, even from game mode
That could well be it. There's the
FLATPAK_GL_DRIVERSvariable, which when set tohostseems to force it to use the host drivers, but it seems that it's bugged in combination with Steam :-((( https://github.com/ValveSoftware/steam-runtime/issues/474More info here: https://blog.tingping.se/2018/08/26/flatpak-host-extensions.html Maybe we can try forcing Flatpak to use the Deck's drivers?
I ran vulkan-tools inside of a flatpak and compared the vulkan-info between the flatpak version and the native (SteamOS) version for both desktop and game mode. Attached are the results.
vulkaninfo-flatpak-desktop.txt
vulkaninfo-flatpak-gamemode.txt
vulkaninfo-native-desktop.txt
vulkaninfo-native-gamemode.txt
tl;dr I get 12 surface formats using native vulkaninfo and 2 surface formats using flatpak vulkaninfo. I also noticed there were 12 layers for the native vs 2 for the flatpak
For reference I used this version (https://github.com/KhronosGroup/Vulkan-Tools/archive/refs/tags/sdk-1.3.261.1.tar.gz) of vulkan tools ... and setting FLATPAK_GL_DRIVERS=host didn't seem to make any changes for the flatpak version.
Regarding the layers in the flatpak i just get the mesa layers VK_LAYER_MESA_device_select (Linux device selection layer) Vulkan version 1.3.211, layer version 1:
whereas in native I get gamescope wsi layers like VK_LAYER_FROG_gamescope_wsi_x86 (Gamescope WSI (XWayland Bypass) Layer (x86)) Vulkan version 1.3.221, layer version 1:, mangoapp layers like VK_LAYER_MANGOAPP_overlay (Mangoapp Layer) Vulkan version 1.3.0, layer version 1: Mangohud layers, renderdoc layers, and valve layers
Update: @tvidal-net created a static steamos build of chiaki4deck (https://github.com/tvidal-net/chiaki4deck-build) if this could be updated to add lib placebo by someone perhaps we could use that to see if the HDR is then detected. I can also try an appimage build and see if I have any luck there while we try to run down how to get it working in flatpak.
So, afer sleeping on it a bit, I think that we might not actually need the system-level Mesa drivers to get this working. All that is needed from my understanding is the Vulkan layer that gamescope provides, which takes care of providing those 10bit and 16bit Vulkan surface types and the Vulkan HDR extension.
Now there is a gamescope Flatpak that we could use to get access to the layer, however this would pose a problem with version conflicts. For example, if a user is running a Preview build of SteamOS on her Deck, but the Vulkan layer provides by the gamescope flatpak is running an older version, there might be issues with compatibility.
So the best way to do this would probably be to mount the system's
/usrdirectory to/var/run/host/usr, then set theVK_LAYER_PATHto the system-level layer directory (/var/run/host/usr/share/vulkan/implicit_layer.d), and then setVK_INSTANCE_LAYERSto include theVkLayer_FROG_gamescope_wsilayer. Can you give that a shot @streetpea? My OLED hasn't arrived yet, unfortunately :-/@jbaiter /usr is blacklisted to use as a filesystem mount so we can't mount it directly via --filesystem (see https://docs.flatpak.org/en/latest/sandbox-permissions.html#filesystem-access)
But aren't they available under
/var/run/host? I thought the blacklisting only applied if they are mounted directly to/usr:so I just needed to add
--filesystem=hostwhich is why I wasn't seeing them there when I checked before.So when I run the command:
I get the gamescope layer but it isn't able to load because the shared object file isn't in the path specified in the file (since /usr is mounted under /var/run/host) so I get the following output and error when I grep for layers:
I can make my own .json that links to the so at the right path and try that but then I'm not sure if the steamos so file will be compatible with the kde flatpak runtime so we may have to build our own or use one built with the runtime and link to that ... I'll mess around with this tonight.
Bad news on the HDMI dongle front: It seems that even when I flash the EDID of the popular and well supported LG C2 TV onto the dongle, the PS5 refuses to recognize it and makes it impossible to enable HDR.
This means that, when we get this working, it's only going to be possible to be used by people with access to a HDR display that can be connected (and remain connected) to the PS5 to enable HDR on the PS5 :-(
hmm @jbaiter other than going into gamescope do you have a better idea of how this is activating? (since in desktop mode it shows 2 layers and in game mode it shows 12 layers) I pointed the file to the .so for the layer from SteamOS which at least didn't throw any errors and did show that it loaded the layer. However, I'm still only getting 2 surfaces in both desktop mode and game mode when running vulkaninfo from the flatpak. This has me wondering if the vklayer doesn't know it's running under gamescope so doesn't activate the hdr (maybe that doesn't make sense, I have a very limited understanding of how the vulkan stuff works.)
You need to run it through gamescope I'm afraid. That or you try set the necessary X11 atom values (see above) in Chiaki, that might make the layer expose the HDR stuff.
So I am running it via game mode (I created a script and run that script to launch the flatpak in gamemode and log the output of vulkaninfo).
I looked at the logs and found this error:
This error doesn't occur in desktop mode (only in game mode) so I think it's detecting that it should use the Gamescode WSI layer but it can't connect to the socket.
In flatpak, I specified the allowed sockets as wayland and x11 and the available ones are from the docs:
so I'm not sure if I can access the gamescope one
I got a static steamos build of the HDR branch building on the steamdeck OLED through the arch user repository starting from
git clone https://aur.archlinux.org/yay-bin.gitcd yay-binmakepkg -sithen using yay -Ss to find and install libplacebo dependencies that pacman lacks (ex: it lacked libdovi so I used AUR/libdovi-git and AUR/libplacebo-git)
a bit more involved than what I wrote here; if it's a testing path worth trying I can provide more detailed instructions.
I don't have vulkan working on either build though, the flatpak version gives me a black screen with 0FPS while the static version returns "avcodec_get_hw_config failed" after failing to initialied FFMPEG decoder and won't start the stream at all.
Edit: okay went through the logs and libplacebo is picking VK_FORMAT_R16G16B16A16_UNORM + VK_COLOR_SPACE_SRGB_NONLINEAR_KHR though it does show HDR10 as an option and has the full 12 surface configs.
It looks like you can use colorspace hinting to prefer an HDR colorspace (see https://code.videolan.org/videolan/libplacebo/-/merge_requests/165) which replaced the old prefer_hdr parameter
I'm not quite sure how to make it pick up HDR on its own; libplacebo has a built-in scoring system to pick the surface configuration https://code.videolan.org/videolan/libplacebo/-/blob/master/src/vulkan/swapchain.c#L209-L249 and for some reason it prioritizes SRGB nonlinear for me.
Naturally, hard coding the best_id to use one of the configurations in the VK_COLOR_SPACE_HDR10_ST2084_EXT color space prioritizes HDR instead and it's providing an output similar to what I'd see directly from the PS5 to an HDR display. There's still plenty of artifacts but that happens on any configuration so HDR is still an improvement over that. I am using "none" for the hardware decode method as vaapi mutes the colors on any setting I've tried.
Hdr isn't activating via the flatpak but I see the same washed out effect when I choose vaapi and h265 hdr as the option. If I choose a non hdr option with vaapi or deselect vaapi it's not washed out.
I asked on the flatpak channel on Matrix and all I needed was to add the socket via a filesystem command: specifically
--filesystem=xdg-run/gamescope-0. I now see all 12 surfaces in the flatpak when running in game mode. I'll update the flatpak build with the necessary changes to get this working so others can try it as well in addition to the static build.Update: I updated the chiaki4deck-hdr repo with the updated files and now it shows 12 surface format and is picking the same VK_FORMAT_R16G16B16A16_UNORM that the static build is picking
If you want to see HDR working for testing all you have to do is set L246 of libplacebo/src/vulkan/swapchain.cs as I linked above. I have my pkgbuild replacing from the videolan copy before building with
sed -i 's/best_id= i/best_id = 9/' ./libplacebo/src/vulkan/swapchain.csI don't have a lot of PS5 games to test with but it's a night and day difference on Final Fantasy XVI which has a demo available; you'll know when HDR is working.
A more versatile solution that doesn't change libplacebo might require filtering the 12 surface formats list and remove any instance of COLOR_SPACE_SRGB_NONLINEAR_KHR when the PS5 HDR toggle is chosen as setting the .surface to FORMAT_A2R10G10B10_UNORM_PACK32 on chiaki's end will still result with SRGB given libplacebo's current scoring.
Since I haven't figured out how to filter the list fully a quicker solution should work by changing libplacebo's scoring to default towards HDR when the best surface is FORMAT_A2R10G10B10_UNORM_PACK32 and have chiaki swap between the two surfaces based on the PS5 HDR selection.
@panthurrr looking at the code you posted earlier showing the scoring system and looking at the libplacebo demo code there's a hinting system we can use within libplacebo for that scoring algorithm. One of the hints we can give is to prefer hdr by picking a hdr transfer function in our colorspace hint. I will try this out tonight and see if it results in the hdr colorspace getting picked.
Amazing work @streetpea @panthurrr! My OLED deck arrived today, I'll hopefully have it set up by tomorrow and then I'll try getting HDR to work on it myself.
In the meantime, since it seems we're close to having this working, should I open a WIP PR so we can move the discussion there and keep it close to the code?
On a related note: Given that placebo has a nice API for adding overlays to the screen, I'll probably work on a small connection quality overlay next, like mangohud, but with ping/RTT, packet loss and bandwidth graphs 🙃
Once I added the colorspace hint, it initially selects the color space with the highest bit depth but changes to the HDR colorspace when streaming starts (the hint is updated based on the incoming frame) if H265 HDR is selected. If H265 or H264 (SDR formats) are selected it will stick with the SDR colorspace. I updated the io.github.streetpea.Chiaki4deck-hdr flatpak repo so now when you use it you should see (I do on my Steam Deck OLED) the HDR label if you hit the three buttons once streaming starts (av frames are received) to open the game overlay. Thanks for all your help in figuring that out @panthurrr.
I think the biggest issue now is that if vaapi is selected the colors are extremely washed out. If vaapi isn't selected the overall stream quality (I.e.,number of dropped frames) is worse (at least for me) so I would prefer to use hw decoding. Maybe this has to do with the different av pixel format used for vaapi or maybe it's something else?
@jbaiter you can open a WIP PR but before you do I just sent you a PR for the updates to the colorspace and scripts.
Opened the PR here: https://github.com/streetpea/chiaki4deck/pull/104
I just noticed when I open in desktop mode I get the error for the flatpak:
it works fine in gamemode though
when I override the VK_LAYER_PATH with
flatpak override --user --unset-env=VK_LAYER_PATH io.github.streetpea.Chiaki4deck-hdrit works again ... we can either have users specify the VK_LAYER_PATH or just mention that you have to override that if using desktop mode. Since hdr is only available in gamescope right now and we still have the opengl renderer for non-hdr it probably makes sense to leave things the way they are and just note that this is the case in case users encounter it. This seems to only be an issue on the Steam Deck itself as otherwise the gamescope.so for the Vulkan gamescope WSI layer doesn't exist and so isn't loaded (i.e., the layer is skipped)Yeah I'm getting a massive amount of dropped frames without hardware decoding; would be nice to figure out how to use vaapi. So far I've tried to change the render_params.tone_mapping_function and render_params.color_adjustment in avplacebowidget.cpp but touching any value at all in avplacebowidget.cpp is destroying the entire render for me.
I haven't been able to get the flatpak to use HDR yet to test, I'm guessing my launch options are missing something beyond the gamescope-0 socket
@panthurrr you can use the flatpak I created using the instructions above in this thread and it should work in game mode without needing to add any arguments if you are on the latest version (I updated it). If you have an earlier version use flatpak upgrade io.github.streetpea.Chiaki4deck-hdr. Also if you want to see the differences you can take a look at the new flatpak builder file it is in the updated version of jbaiter's chiaki4deck repo on the hdr branch under scripts/flatpak/chiaki4deck.yaml. Also I switched to 720p for now since vaapi is off and I no longer get dropped frames but ymmv depending on your connection with vaapi enabled.
(A probably dumb question: Does it mean, it is ready and it only needs to get implemented to Flatpak version of chiaki4deck in discover app?)
Thank you in advance, Kapitän Haddock and others.
@arshiatn it's a work in progress PR as there are still some things to get ironed out. Having said that, HDR should be working in game mode if you want to test it in the io.github.streetpea.Chiaki4deck-hdr flatpak. If you previously downloaded this flatpak and added it to game mode you can update it with flatpak upgrade io.github.streetpea.Chiaki4deck-hdr to get the updated version. At this time you should set hardware decoding to none as otherwise the colors are extremely washed out when selecting H265 (HDR). Progress will be tracked on that PR.
Alright, I just installed it with flatpak just you said in #104. It opened up and I changed the settings too. However I have no idea how to add it to Steam. I cannot find its exe file.
Just add it from the start menu. You will probably have 2 chiaki4deck entries on there now and figure out which one is the HDR branch
I got it. Thanks. I will go and check out in Gamemode :D
I tested HDR on my deck - with software decoding there's a lot of stuttering but it's works (finally color banding is gone). Good job! I'll wait for version with hardware decoding and I think it's a new level of steam deck streaming.
I didn't want to interfere in #104 so I am gonna ask you here:
did you guys really fix hardware encoding for HDR :O?
Nevermind. I just went to desktop mode and discover app updated HDR Chiaki4deck and holy moly. Vulkan Hardware Encoding + HDR runs soo good. It shows 8-10w power usage (I am using high quality) but only 3-4w is from APU. I played about 1 hour of of Ghost of Tsushima and it ran very well. (Bunch of time it lagged a bit but it is because of my router) :))
edit: Nice job :D
after updating it's back to not detecting the HDR surface formats for me (it's the same two it was finding at the start)
I uninstalled, cleared data, & reinstalled to make sure it wasn't anything lingering from changes I had made to test things and it's still only outputting in SDR
EDIT: it's worth noting that before resetting, I still had my own json files and env vars set on the flatpak to force loading the vulkan layers, and whatever changed that made it stop working broke it even with that still in place.
@chipgw I'm not sure if the update is what caused this. Can you try reverting to the previous update
and see if you are still having problems
btw if you want to switch to an even earlier update you can use
to see all the commits you can choose from
Update: I am also facing the same issue as you and found that if I was docked to an external monitor hdr turned on in chiaki4deck. This was especially weird because I was docked into a regular (non-hdr) monitor.
switching to that version did not work either, however, I opened Flatseal to look at the settings there and it lost the
--filesystem=xdg-run/gamescope-0 --env=VK_LAYER_PATH=/app/vulkan-layers/settings for some reason, and after adding them back it's working again. (it looks like that and--filesystem=hostare missing from thechiaki4deck-devel.yamlfile, so if that's what was used to make this flatpak then I expect that's the culprit).and I would guess what broke my local overrides was the incorrect separator mentioned here
I’ve played around with this quite a bit on the OLED. It’s spectacular so far.
I did notice something yesterday however. I played for about 20 mins and all was smoothish… the odd glitch and once where audio went out of sync by 2-3 seconds, a reconnect fixed that.
When I returned an hour later I connected again and the screen was garbled. Not green (which happens occasionally, not so much in standard Chiaki4deck) but multicoloured blocky nonsense. I disconnected and restarted Chiaki and nothing happened. Restarted the deck and it was solid for 3 hours straight. Not sure what lingered in the deck to make a restart fix it completely, but I did the last couple of Spider-man 2 main quests and rolled credits with zero issues after the restart.
Using streetpeas latest flatpak build.
Thanks for the bugreport @edge3uk!
Looks like there can be weird issues with HDR, judging from the SteamOS 3.5.8 changelog:
Would be great to know the details to that, maybe somebody from Valve is reading this? 🙃
Can you try switching to the Preview update channel and see if you can reproduce the issue?
You’re welcome @jbaiter, I’ll try and drop as much info as I can if I find anything interesting.
thank you so much for taking this on!
i've got an LE OLED and an LG C1 to test this on, so i thought I'd share my first impressions and things i've noticed after testing for a bit. I installed the latest flatpak and pointed the automation script from the vanilla chiaki4deck install to the hdr binary and all of that works just fine. I tested a couple of games last night and noticed a few things:
(i've configured h265-hdr and vaapi at 50Mb on the deck, wifi6E unifi network, PS5 hardwired to switch.)
Super excited about this and happy to keep testing. I'm really excited about how much more stable the stream seems to be overall, HDR or not. Keep up the great work!!
Any plan on when this will be closed and released to Flathub? I’m debating manually installing the HDR Flatpack and just waiting for release.
@mattinm it will be out by Monday just testing some other changes now before the next release. The pr has already been merged to the main branch.
Released with version 1.5.0
I'm a bit confused if the HDR is actually working now or not? When I play a Steam game that supports HDR, there is a little blue/purple "HDR" badge in the top right above the brightness sliders that you can bring up in Steam Deck Quick Settings. But that badge doesn't show up when starting the stream in Chiaki.
@natanji If that badge isn't showing up hdr isn't activating. Make sure you are using h265 hdr as your codec. For a full list of things w.r.t hdr see https://streetpea.github.io/chiaki4deck/setup/configuration/#hdr-high-dynamic-range
I realized three days ago that the maximum nits I am getting isn't around 1000nits but 1150nits with the settings I told you (16-16-0 steps for HDR calibration). It seems like I am a blind and deaf human being that heard 16 instead of 15 steps in HDTVTest's video (https://www.youtube.com/watch?v=FwcSCgW47rY) :D
In short: it should be 15-15-0 steps to get around 975 nits which gets us to almost 1000 nits. 16-16-10 steps give us maximum 1150 nits and we don't want that. It makes our highlights overly bright.
=> People should set their HDR calibration with 15-15-0
I would appreciate if you guys @streetpea @jbaiter could fix that guide of yours.
Another important note is that the deck can reach its maximum brightness (1000nits) at 75% already. With 75% I mean the brightness slider of the deck itself. The sldier got 101 steps (0-100). To get to 75% you need to go 25 steps to left when the brightness is at max. Why? Everyhing above 75% brightness is making your display "brighter" by playing around with the gamma. It makes the HDR highlights overly bright and thus shouldn't be used in HDR.
(https://github.com/ValveSoftware/SteamOS/issues/1314#issuecomment-1861988729)
You can see the digital gain above 75% if you use moonlight HDR (still in Beta) and open up Windows HDR Calibration in it. Everything below 75% makes deck's screen clip at around 950nits, however everything above 75% brightness gets digital gain and thus makes the display clip at below 950nits. At 100% it clips around 600nits => 1000nits objects look overly bright at 100% Brightness.
=> HDR brightness of Steam Deck should be set at 75%. (Everything above that is digital gain)
I would appreciate if you could add this to your guide too.
Thank you in advance.
Thank you, somehow I missed setting the codec because the PS5 showed me the option to calibrate the display for HDR after I updated chiaki4deck, even when the HDR codec wasn't selected. Is that supposed to happen or is chiaki now telling the TV "I can do HDR" regardless of the codec being selected?
Also, I have been trying the God of War Ragnarok DLC and I'm running into an issue where the PS5 will automatically pause the game and show an error message saying I should connect a different TV to run it. I can immediately click the game again and keep playing (it only pauses), but it happens rather frequently, like once in every 20 minutes. Is that a known issue when using HDR? I'm using a Wifi 6 connection in the same room so tha shouldn't be an issue.
I just get the adjust display screen if I unplug and re plug my tv or a different tv or adapter to the ps5. I don't think that has to do with remote play. I also haven't heard of the issue where it tells you to plugin a different tv. It seems like maybe something is wrong with the PS5 connection to the tv but idk. Do you get the same issue when playing directly on the tv without remote play?
No, it never happens directly on my TV. Only over chiaki.
Does HDR streaming work for PS4/Pro?
No, this is a feature exclusive to PS5 and only with the H265 codec.
Awesome work!
May I ask if it can be ported to upstream chiaki? Macbook also have an HDR-capable displayNevermind, just discovered there're already macos builds, will give a try