Improve streaming error handling

streetpea commented

2023-12-08 10:04:50 -06:00

(Migrated from github.com)

Improve handling of streaming errors such as corrupted frames that currently causes flashes sometimes and green or gray covering the screen other times.

streetpea commented

2023-12-08 21:30:38 -06:00

(Migrated from github.com)

@nowrep have you made enough progress on this to add a patch towards this in the upcoming release or do you think any progress on this should wait for the next release (could make a bug fix release later for progress on this).

nowrep commented

2023-12-09 03:41:04 -06:00

(Migrated from github.com)

github.com/nowrep/chiaki4deck@3b7dbf7b35

This is what I have so far. Not perfect, but seems to behave a bit better so I'd say it's at least worth testing.

https://github.com/nowrep/chiaki4deck/commit/3b7dbf7b358ea5114f5e7e6124f1c9a77504bc00 This is what I have so far. Not perfect, but seems to behave a bit better so I'd say it's at least worth testing.

🚀 2

nowrep commented

2023-12-09 14:28:44 -06:00

(Migrated from github.com)

What it does is simply re-submit last good frame on FEC error. But on its own it wouldn't work, because ffmpeg would error out on duplicate frame number / POC. So it also includes ffmpeg patch to deal with this, it could be done without patching ffmpeg but it's way more code so I did it the "lazy" way 😃

What it does is simply re-submit last good frame on FEC error. But on its own it wouldn't work, because ffmpeg would error out on duplicate frame number / POC. So it also includes ffmpeg patch to deal with this, it could be done without patching ffmpeg but it's way more code so I did it the "lazy" way :smiley:

❤️ 1

jbaiter commented

2023-12-09 15:07:48 -06:00

(Migrated from github.com)

thanks for the explanation, I deleted my comment because it was based on an older state of the code, sorry about that.

panthurrr commented

2023-12-09 18:57:50 -06:00

(Migrated from github.com)

I get a lot of whole screen flashes on this fec-replay branch: github.com/nowrep/chiaki4deck@3b7dbf7b35

If anyone knows a good way to run a debugger like gdb with gamemode I'll give that a try to see which part is directly producing the missing frames. Tried locally building and running a debugger which crashed desktop mode so that didn't get me very far but if anyone has experience with remote debugging on Steam Deck then hopefully that would be useful.

Using hw decoder of none + 90000 bitrate to produce more egregious data:
https://github.com/panthurrr/chiaki4deck/blob/logs/chiaki_session_2023-12-09_17-46-44-867867.log

I get a lot of whole screen flashes on this fec-replay branch: https://github.com/nowrep/chiaki4deck/commit/3b7dbf7b358ea5114f5e7e6124f1c9a77504bc00 If anyone knows a good way to run a debugger like gdb with gamemode I'll give that a try to see which part is directly producing the missing frames. Tried locally building and running a debugger which crashed desktop mode so that didn't get me very far but if anyone has experience with remote debugging on Steam Deck then hopefully that would be useful. Using hw decoder of none + 90000 bitrate to produce more egregious data: https://github.com/panthurrr/chiaki4deck/blob/logs/chiaki_session_2023-12-09_17-46-44-867867.log

streetpea commented

2023-12-09 19:13:18 -06:00

(Migrated from github.com)

@panthurrr you can follow this https://streetpea.github.io/chiaki4deck/diy/steamdeckdev/ for using gdb on steam deck builds.Install 23.08 not 22.08 though as it changed since the last release. You can use desktop mode for this if you follow those instructions.

streetpea commented

2023-12-09 19:46:45 -06:00

(Migrated from github.com)

@panthurr I would also suggest printing out the logs in stdout and stderr to a file like:

flatpak run io.github.streetpea.Chiaki4deck-hdr &> chiaki4deck.log

(I updated Chiaki4deck-hdr repo to add the fixing POC change)
so you can see the ffmpeg logs which aren't part of the Chiaki logs. You can see my output doing this below. You can see the Fixing duplicate POC coming into effect but you can still see Could not find ref with POC of earlier numbers as well
chiaki4deck.log

I also get a lot of white flashes with vaapi off and bitrate set to 90000 (log above). Using an older version of the code without the change is just as bad if not worse so at the very least this change doesn't seem to make things worse but there is still work to be done to eliminate the flashing.

@panthurr I would also suggest printing out the logs in stdout and stderr to a file like: ``` flatpak run io.github.streetpea.Chiaki4deck-hdr &> chiaki4deck.log ``` (I updated Chiaki4deck-hdr repo to add the fixing POC change) so you can see the ffmpeg logs which aren't part of the Chiaki logs. You can see my output doing this below. You can see the Fixing duplicate POC coming into effect but you can still see Could not find ref with POC of earlier numbers as well [chiaki4deck.log](https://github.com/streetpea/chiaki4deck/files/13626207/chiaki4deck.log) I also get a lot of white flashes with vaapi off and bitrate set to 90000 (log above). Using an older version of the code without the change is just as bad if not worse so at the very least this change doesn't seem to make things worse but there is still work to be done to eliminate the flashing.

nowrep commented

2023-12-10 03:37:43 -06:00

(Migrated from github.com)

Alright, updated branch. Should be better now, it also survives the "settings->internet" test now.

Use vaapi with this please. Also the ffmpeg patches are required, it won't work at all without it.

Alright, updated [branch](https://github.com/nowrep/chiaki4deck/commits/fec-replay/). Should be better now, it also survives the "settings->internet" test now. Use vaapi with this please. Also the ffmpeg patches are required, it won't work at all without it.

🚀 1

streetpea commented

2023-12-10 13:17:28 -06:00

(Migrated from github.com)

@nowrep I tested this out and now it only has fixing duplicate POC and afterwards does not have the could not find ref with POC. There is also no more white flashing in these instances. I did encounter one white flash at the very beginning of one of the test streaming sessions but that was just could not find ref with POC with negative numbers like

[hevc @ 0x557806a9df40] Could not find ref with POC -7
[hevc @ 0x557806a9df40] Could not find ref with POC -8
[hevc @ 0x557806a9df40] Could not find ref with POC -9
[hevc @ 0x557806a9df40] Could not find ref with POC -10
[hevc @ 0x557806a9df40] Could not find ref with POC -11
[hevc @ 0x557806a9df40] Could not find ref with POC -12
[hevc @ 0x557806a9df40] Could not find ref with POC -13
[hevc @ 0x557806a9df40] Could not find ref with POC -14
[hevc @ 0x557806a9df40] Could not find ref with POC -15

and didn't have the fixing duplicate POC so I think that happened in a different error case. Anyway, the streaming errors are definitely much improved versus the current state so I will add this to the release. If there are any more advancements you can make on this @nowrep I can add them to a patch release afterwards. Otherwise, this seem like a great improvement. Thanks for all your contributions @nowrep!

@nowrep I tested this out and now it only has fixing duplicate POC and afterwards does not have the could not find ref with POC. There is also no more white flashing in these instances. I did encounter one white flash at the very beginning of one of the test streaming sessions but that was just could not find ref with POC with negative numbers like ``` [hevc @ 0x557806a9df40] Could not find ref with POC -7 [hevc @ 0x557806a9df40] Could not find ref with POC -8 [hevc @ 0x557806a9df40] Could not find ref with POC -9 [hevc @ 0x557806a9df40] Could not find ref with POC -10 [hevc @ 0x557806a9df40] Could not find ref with POC -11 [hevc @ 0x557806a9df40] Could not find ref with POC -12 [hevc @ 0x557806a9df40] Could not find ref with POC -13 [hevc @ 0x557806a9df40] Could not find ref with POC -14 [hevc @ 0x557806a9df40] Could not find ref with POC -15 ``` and didn't have the fixing duplicate POC so I think that happened in a different error case. Anyway, the streaming errors are definitely much improved versus the current state so I will add this to the release. If there are any more advancements you can make on this @nowrep I can add them to a patch release afterwards. Otherwise, this seem like a great improvement. Thanks for all your contributions @nowrep!

nowrep commented

2023-12-11 03:20:33 -06:00

(Migrated from github.com)

Hmm no idea how the negative numbers happened. But I guess if the very first frame is corrupted it could happen?

Also offtopic here: you can get vulkan video on deck (export RADV_PERFTEST=video_decode) but it's currently broken for 10bit (hdr) and I can reproduce the same on my desktop. It's either ffmpeg or mesa bug.

Hmm no idea how the negative numbers happened. But I guess if the very first frame is corrupted it could happen? Also offtopic here: you can get vulkan video on deck (`export RADV_PERFTEST=video_decode`) but it's currently broken for 10bit (hdr) and I can reproduce the same on my desktop. It's either ffmpeg or mesa bug.

👀 1

Rows
Columns

Improve streaming error handling #108