OBS/CEF segfaults consistently on application exit #501

Open
opened 2025-09-11 09:13:41 -05:00 by mihawk90 · 10 comments
mihawk90 commented 2025-09-11 09:13:41 -05:00 (Migrated from github.com)

edit:
Offending commit and workaround found, see comment 7.


Operating System Info

Other

Other OS

Fedora 42

OBS Studio Version

Other

OBS Studio Version (Other)

32.0.0-beta1

OBS Studio Log URL

https://github.com/user-attachments/files/22277058/2025-09-11.16-08-07.txt

OBS Studio Crash Log URL

https://github.com/user-attachments/files/22277059/gdb.txt

Expected Behavior

OBS should be closing the browserdocks and exiting cleanly.

Current Behavior

OBS segfaults on exit, resulting in the "unclean shutdown" dialogue showing each launch.

Steps to Reproduce

Unfortunately reproduction is a bit murky since last time during testing I wasn't able to get it consistently either (which is why I didn't open an issue then), so I can only infer from my main profile where it happens consistently.

  1. Login to Twitch to get the Twitch docks
  2. Add a couple more custom browser docks (I have 5 additional)
    • https://dashboard.twitch.tv/popout/u/<user>/stream-manager/stream-health
    • https://dashboard.twitch.tv/popout/u/<user>/stream-manager/quick-actions
    • https://dashboard.twitch.tv/popout/u/<user>/stream-manager/activity-feed?uuid=<uuid> (not sure why I still have this TBH)
    • https://dashboard.twitch.tv/popout/u/<user>/stream-manager/reward-queue?uuid=<uuid>
    • https://play.pretzel.rocks/my-music/playlists
    • I also have various browser sources, but from what I understand they should not be relevant here
  3. Close OBS
  4. Get lucky(?)
  5. ...
  6. Profit(?)/segfault

Anything else we should know?

This used to be a widespread issue (happening with just the Twitch docks even), but was fixed in (IIRC) OBS 30. It started happening again with the OBS 31.1 beta cycle.

On Discord I was troubleshooting with Matt and we ended up increasing the timeout here:
github.com/obsproject/obs-browser@910617b499/panel/browser-panel.cpp (L169-L176)

... to 5000, which fixed it for the most part. However it seems to me that increasing the timeout is really only a bandaid and not fixing the actual problem?

Regardless, during testing with a new profile I was unable to reproduce this consistently, which is likely due to not adding all the docks I have in my main profile, where I do get this consistently.

To my knowledge the exact cause of this crash is still unknown, but it was "fixed" with OBS 30 introducing the timeout loop. I have yet to figure out what happened between 31 and 31.1 beta1 to have this reappear, since I didn't get to bisect considering I couldn't reproduce it reliably in testing.

I don't remember if the backtrace from back then was the same/similar, you'd have to check the old beta discord channels for it, pretty sure I posted it.

This crash is likely tangentially related to #353

edit: Offending commit and workaround found, see [comment 7](#issuecomment-3283090433). --- ### Operating System Info Other ### Other OS Fedora 42 ### OBS Studio Version Other ### OBS Studio Version (Other) 32.0.0-beta1 ### OBS Studio Log URL https://github.com/user-attachments/files/22277058/2025-09-11.16-08-07.txt ### OBS Studio Crash Log URL https://github.com/user-attachments/files/22277059/gdb.txt ### Expected Behavior OBS should be closing the browserdocks and exiting cleanly. ### Current Behavior OBS segfaults on exit, resulting in the "unclean shutdown" dialogue showing each launch. ### Steps to Reproduce Unfortunately reproduction is a bit murky since last time during testing I wasn't able to get it consistently either (which is why I didn't open an issue then), so I can only infer from my main profile where it happens consistently. 1. Login to Twitch to get the Twitch docks 2. Add a couple more custom browser docks (I have 5 additional) * `https://dashboard.twitch.tv/popout/u/<user>/stream-manager/stream-health` * `https://dashboard.twitch.tv/popout/u/<user>/stream-manager/quick-actions` * `https://dashboard.twitch.tv/popout/u/<user>/stream-manager/activity-feed?uuid=<uuid>` (not sure why I still have this TBH) * `https://dashboard.twitch.tv/popout/u/<user>/stream-manager/reward-queue?uuid=<uuid>` * `https://play.pretzel.rocks/my-music/playlists` * I also have various browser sources, but from what I understand they should not be relevant here 3. Close OBS 4. Get lucky(?) 5. ... 6. Profit(?)/segfault ### Anything else we should know? This used to be a widespread issue (happening with just the Twitch docks even), but was fixed in (IIRC) OBS 30. It started happening again with the OBS 31.1 beta cycle. On Discord I was troubleshooting with Matt and we ended up increasing the timeout here: https://github.com/obsproject/obs-browser/blob/910617b4997b0f53f2b3f95c7d2fa0f4a0753a92/panel/browser-panel.cpp#L169-L176 ... to 5000, which fixed it for the most part. However it seems to me that increasing the timeout is really only a bandaid and not fixing the actual problem? Regardless, during testing with a new profile I was unable to reproduce this consistently, which is likely due to not adding all the docks I have in my main profile, where I do get this consistently. To my knowledge the exact cause of this crash is still unknown, but it was "fixed" with OBS 30 introducing the timeout loop. I have yet to figure out what happened between 31 and 31.1 beta1 to have this reappear, since I didn't get to bisect considering I couldn't reproduce it reliably in testing. I don't remember if the backtrace from back then was the same/similar, you'd have to check the old beta discord channels for it, pretty sure I posted it. This crash is likely tangentially related to #353
mihawk90 commented 2025-09-11 14:35:12 -05:00 (Migrated from github.com)

tytan suggested testing with 6533v6 as well (the previous log was with v5), so here we are:

https://github.com/user-attachments/files/22282679/gdb.txt

To me these GDB backtraces are all hieroglyphs but at first glance it looks to be the same(-ish), except for

#0  0x0000000000000000 in ??? ()

Which I don't know what that is because I have all the debug symbols I could get...

tytan suggested testing with 6533v6 as well (the previous log was with v5), so here we are: https://github.com/user-attachments/files/22282679/gdb.txt To me these GDB backtraces are all hieroglyphs but at first glance it looks to be the same(-ish), except for ``` #0 0x0000000000000000 in ??? () ``` Which I don't know what that is because I have all the debug symbols I could get...
RytoEX commented 2025-09-11 14:45:48 -05:00 (Migrated from github.com)

tytan suggested testing with 6533v6 as well (the previous log was with v5), so here we are:

https://github.com/user-attachments/files/22282679/gdb.txt

To me these GDB backtraces are all hieroglyphs but at first glance it looks to be the same(-ish), except for

#0  0x0000000000000000 in ??? ()

Which I don't know what that is because I have all the debug symbols I could get...

Due to the presence of QMainWindowTabBar in the stack, this almost sounds like QTBUG-137755:

The stack in that QTBUG is the same as yours. Fixed in Qt 6.9.2 and 6.10+.

> tytan suggested testing with 6533v6 as well (the previous log was with v5), so here we are: > > https://github.com/user-attachments/files/22282679/gdb.txt > > To me these GDB backtraces are all hieroglyphs but at first glance it looks to be the same(-ish), except for > > ``` > #0 0x0000000000000000 in ??? () > ``` > > Which I don't know what that is because I have all the debug symbols I could get... Due to the presence of `QMainWindowTabBar` in the stack, this almost sounds like [QTBUG-137755](https://bugreports.qt.io/browse/QTBUG-137755): * https://github.com/qt/qtbase/commit/cc1ddebf63a2b6c051529d1d3127534aa4c1d55a The stack in that QTBUG is the same as yours. Fixed in Qt 6.9.2 and 6.10+.
mihawk90 commented 2025-09-11 15:03:47 -05:00 (Migrated from github.com)

This is with Qt 6.9.2 (runtime, I didn't rebuild):

https://github.com/user-attachments/files/22283813/gdb.txt

This is with Qt 6.9.2 (runtime, I didn't rebuild): https://github.com/user-attachments/files/22283813/gdb.txt
RytoEX commented 2025-09-11 15:19:50 -05:00 (Migrated from github.com)

This is with Qt 6.9.2 (runtime, I didn't rebuild):

https://github.com/user-attachments/files/22283813/gdb.txt

# 0 0x0000555558004000 in ??? ()
# 1 0x00007fff2a89625c in QCefWidgetInternal::unsetToplevelXdndProxy (this=0x555559b8e400) at /usr/src/debug/obs-studio-32.0.0~beta1-11.fc42.x86_64/plugins/obs-browser/panel/browser-panel.cpp:249
[...]
# 13 0x00007fff2a88e712 in BrowserManagerThread () at /usr/src/debug/obs-studio-32.0.0~beta1-11.fc42.x86_64/plugins/obs-browser/obs-browser-plugin.cpp:415

The only context I can provide is here:
github.com/obsproject/obs-browser@c3b1e81a12/panel/browser-panel.cpp (L237-L253)

Line 249 is the offending line in the stack here. Perhaps somehow cefBrowser still exists here, but something else is wrong here.

> This is with Qt 6.9.2 (runtime, I didn't rebuild): > > https://github.com/user-attachments/files/22283813/gdb.txt > > \# 0 0x0000555558004000 in ??? () > \# 1 0x00007fff2a89625c in QCefWidgetInternal::unsetToplevelXdndProxy (this=0x555559b8e400) at /usr/src/debug/obs-studio-32.0.0\~beta1-11.fc42.x86_64/plugins/obs-browser/panel/browser-panel.cpp:249 > [...] > \# 13 0x00007fff2a88e712 in BrowserManagerThread () at /usr/src/debug/obs-studio-32.0.0\~beta1-11.fc42.x86_64/plugins/obs-browser/obs-browser-plugin.cpp:415 The only context I can provide is here: https://github.com/obsproject/obs-browser/blob/c3b1e81a12380cd3d3f2cac65ef50289f144b2b6/panel/browser-panel.cpp#L237-L253 Line 249 is the offending line in the stack here. Perhaps somehow `cefBrowser` still exists here, but something else is wrong here.
RytoEX commented 2025-09-11 15:37:34 -05:00 (Migrated from github.com)

Some further pondering:
What's puzzling to me is that unsetToplevelXdndProxy() seems like it should only be called on initialization. Or at least, it's added to the CEF task queue on initialization. Not sure why it's being called on application exit, unless it's flushing the task queue and it's still stuck in there.

github.com/obsproject/obs-browser@c3b1e81a12/panel/browser-panel.cpp (L276-L281)

github.com/obsproject/obs-browser@c3b1e81a12/panel/browser-panel.cpp (L326-L328)

Some further pondering: What's puzzling to me is that `unsetToplevelXdndProxy()` seems like it should only be called on initialization. Or at least, it's added to the CEF task queue on initialization. Not sure why it's being called on application exit, unless it's flushing the task queue and it's still stuck in there. https://github.com/obsproject/obs-browser/blob/c3b1e81a12380cd3d3f2cac65ef50289f144b2b6/panel/browser-panel.cpp#L276-L281 https://github.com/obsproject/obs-browser/blob/c3b1e81a12380cd3d3f2cac65ef50289f144b2b6/panel/browser-panel.cpp#L326-L328
mihawk90 commented 2025-09-11 16:53:57 -05:00 (Migrated from github.com)

OK so I've been trying to bisect and started with bisecting obs-browser first because it is the smaller commit count compared to OBS proper.

The relevant history is this:
b56fd78936761891475458447c1cc9058bb9c2d4...5ee805695d337cbea4180fd25e5b86bfff4af33b

And here's how it went:

All testing with CEF 6533v6

31.0.3^0
5ee805695d
build failure :<

31.0.3^1
a54e7c4cb0
good

31.0.3^2
0fb70b6a9a
good

31.0.3^3
f617868ab1
build failure :<

The build failures are this:

/builddir/build/BUILD/obs-studio-31.0.3_3-build/obs-studio/plugins/obs-browser/obs-browser-plugin.cpp:719:21: error: ‘gs_get_driver_version’ was not declared in this scope
  719 |         glVersion = gs_get_driver_version();
      |                     ^~~~~~~~~~~~~~~~~~~~~

And this is beyond me to fix because I have no idea about the codebase.

However, f617868ab1 is windows-specific so should be irrelevant. The commit after that (5ee805695d) only increases the version, and the commit before (0b13eb9b91) introduces the build failure.
So unless the problem is the glVersion change I think it's safe to say obs-browser is not the issue.

OK so I've been trying to bisect and started with bisecting `obs-browser` first because it is the smaller commit count compared to OBS proper. The relevant history is this: b56fd78936761891475458447c1cc9058bb9c2d4...5ee805695d337cbea4180fd25e5b86bfff4af33b And here's how it went: All testing with CEF 6533v6 31.0.3^0 5ee805695d337cbea4180fd25e5b86bfff4af33b build failure :< 31.0.3^1 a54e7c4cb0d7e93c02f8f52bf412645ccee2bf5f good 31.0.3^2 0fb70b6a9afaeb7424b60371987ffd4661a4ba9a good 31.0.3^3 f617868ab1299ea19a01f0f3c1ae24c22cebe8e7 build failure :< The build failures are this: ``` /builddir/build/BUILD/obs-studio-31.0.3_3-build/obs-studio/plugins/obs-browser/obs-browser-plugin.cpp:719:21: error: ‘gs_get_driver_version’ was not declared in this scope 719 | glVersion = gs_get_driver_version(); | ^~~~~~~~~~~~~~~~~~~~~ ``` And this is beyond me to fix because I have no idea about the codebase. However, f617868ab1299ea19a01f0f3c1ae24c22cebe8e7 is windows-specific so should be irrelevant. The commit after that (5ee805695d337cbea4180fd25e5b86bfff4af33b) only increases the version, and the commit before (0b13eb9b91a91e4862948627dce1b65e580bb4b2) introduces the build failure. So unless the problem is the `glVersion` change I think it's safe to say obs-browser is not the issue.
mihawk90 commented 2025-09-11 19:37:38 -05:00 (Migrated from github.com)

Finally done with bisecting OBS commits.

TLDR:

github.com/obsproject/obs-studio@bdebea3d85

bdebea3d858deeccf1900fe8272d1f40bb9501e3 is the first bad commit
commit bdebea3d858deeccf1900fe8272d1f40bb9501e3
Author: tytan652
Date:   Wed Sep 11 14:01:36 2024 +0200

    frontend: Enable browser hw accel toggle under Linux

 frontend/OBSApp.cpp                    |  6 ++++--
 frontend/settings/OBSBasicSettings.cpp | 10 +++++-----
 2 files changed, 9 insertions(+), 7 deletions(-)

edit:
It's late so I didn't think of the next logical step which Matt pointed out...
Disabling Hardware Acceleration (in the settings, not by reverting the commit) does indeed fix the crash. Across 8 restarts there was no crash, turned it back on and I immediately got one again.


git bisect good 31.0.3

git bisect bad 31.1.0-beta1

git bisect good/bad && git submodule update --init --recursive after each

Test builds

All testing with CEF 6533v6

31.0.3^4
github.com/obsproject/obs-studio@f03280c4fd
good

31.0.3^5
github.com/obsproject/obs-studio@8f1bcc1798
good

31.0.3^6
github.com/obsproject/obs-studio@a19945b53a
good

31.0.3^7
github.com/obsproject/obs-studio@6204c22853
good

31.0.3^8
github.com/obsproject/obs-studio@b46e5bb1c0
good

31.0.3^9
github.com/obsproject/obs-studio@b9cd9f30b1
good

31.0.3^10
github.com/obsproject/obs-studio@f6c6b3f9d1
good

31.0.3^11
github.com/obsproject/obs-studio@bdebea3d85
bad (3/10 good)

31.0.3^12
github.com/obsproject/obs-studio@e8d1834f68
good

Finally done with bisecting OBS commits. TLDR: https://github.com/obsproject/obs-studio/commit/bdebea3d858deeccf1900fe8272d1f40bb9501e3 ``` bdebea3d858deeccf1900fe8272d1f40bb9501e3 is the first bad commit commit bdebea3d858deeccf1900fe8272d1f40bb9501e3 Author: tytan652 Date: Wed Sep 11 14:01:36 2024 +0200 frontend: Enable browser hw accel toggle under Linux frontend/OBSApp.cpp | 6 ++++-- frontend/settings/OBSBasicSettings.cpp | 10 +++++----- 2 files changed, 9 insertions(+), 7 deletions(-) ``` --- edit: It's late so I didn't think of the next logical step which Matt pointed out... Disabling Hardware Acceleration (in the settings, not by reverting the commit) does indeed fix the crash. Across 8 restarts there was no crash, turned it back on and I immediately got one again. --- `git bisect good 31.0.3` `git bisect bad 31.1.0-beta1` `git bisect good/bad && git submodule update --init --recursive` after each ## Test builds All testing with CEF 6533v6 31.0.3^4 https://github.com/obsproject/obs-studio/commit/f03280c4fd449d1591363291fd2762ffca878e6a good 31.0.3^5 https://github.com/obsproject/obs-studio/commit/8f1bcc179869ee864ab0dee70dc3ccdf6bec9e11 good 31.0.3^6 https://github.com/obsproject/obs-studio/commit/a19945b53a8d37244a599f3afd44f473853c3ee6 good 31.0.3^7 https://github.com/obsproject/obs-studio/commit/6204c228531eb51b01a6b8e27b832c566fe4aeec good 31.0.3^8 https://github.com/obsproject/obs-studio/commit/b46e5bb1c08fb385d19f024199557c792a4e6126 good 31.0.3^9 https://github.com/obsproject/obs-studio/commit/b9cd9f30b1be44b4b3862c694d88d3fcb5324f1d good 31.0.3^10 https://github.com/obsproject/obs-studio/commit/f6c6b3f9d1f7fee92652165e1bf850f4eb343f08 good 31.0.3^11 https://github.com/obsproject/obs-studio/commit/bdebea3d858deeccf1900fe8272d1f40bb9501e3 bad (3/10 good) 31.0.3^12 https://github.com/obsproject/obs-studio/commit/e8d1834f683a3e1b723104aac7864aef7aedbc75 good
RytoEX commented 2025-09-15 18:51:15 -05:00 (Migrated from github.com)

Acknowledging that the above conclusion seems to be that the browser source hardware acceleration setting seems to be somehow causing this. Unfortunately, debugging that is beyond me, as I primarily work on Windows.

Acknowledging that the above conclusion seems to be that the browser source hardware acceleration setting seems to be somehow causing this. Unfortunately, debugging that is beyond me, as I primarily work on Windows.
RytoEX commented 2025-09-17 14:12:22 -05:00 (Migrated from github.com)

The build failures are this:

/builddir/build/BUILD/obs-studio-31.0.3_3-build/obs-studio/plugins/obs-browser/obs-browser-plugin.cpp:719:21: error: ‘gs_get_driver_version’ was not declared in this scope
  719 |         glVersion = gs_get_driver_version();
      |                     ^~~~~~~~~~~~~~~~~~~~~

And this is beyond me to fix because I have no idea about the codebase.

For completeness, this should be because gs_get_driver_version() was only added in OBS Studio 31.1, so obviously the function is not available when building against 31.0.x (before github.com/obsproject/obs-studio@abb8cdf0da).

> The build failures are this: > > ``` > /builddir/build/BUILD/obs-studio-31.0.3_3-build/obs-studio/plugins/obs-browser/obs-browser-plugin.cpp:719:21: error: ‘gs_get_driver_version’ was not declared in this scope > 719 | glVersion = gs_get_driver_version(); > | ^~~~~~~~~~~~~~~~~~~~~ > ``` > > And this is beyond me to fix because I have no idea about the codebase. For completeness, this should be because `gs_get_driver_version()` was only added in OBS Studio 31.1, so obviously the function is not available when building against 31.0.x (before https://github.com/obsproject/obs-studio/commit/abb8cdf0dad2267b59072b1b83da24e5fcd0b19f).
PatTheMav commented 2026-01-24 07:20:28 -06:00 (Migrated from github.com)

This crash might be related to similar crashes we see on macOS:

When a browser dock is destroyed it creates a nested event loop to effectively "block" the current call stack to wait up to 1 second for CEF to finish its own shut down.

But this nested event loop will receive all application events (it effectively becomes the "main" event loop for that second) and while CEF might be shutting down all other app events will be handled.

The main issue with this is that this nested event loop is spawned within OBSBasic's destructor, which is batshit insane on an architectural level (destructors have to be "safe"), but at least on macOS this can lead to a Quit event being handled within the destructor, which then triggers the same destructor on a half-destructed object again.

It doesn't seem this is the specific issue here as well, but having that nested event loop while the "main" event loop is busy draining itself of all remaining events is just a recipe for disaster.

This crash might be related to similar crashes we see on macOS: When a browser dock is destroyed it creates a nested event loop to effectively "block" the current call stack to wait up to 1 second for CEF to finish its own shut down. But this nested event loop will receive _all_ application events (it effectively becomes the "main" event loop for that second) and while CEF might be shutting down all other app events will be handled. The main issue with this is that this nested event loop is spawned within `OBSBasic`'s destructor, which is batshit insane on an architectural level (destructors have to be "safe"), but at least on macOS this can lead to a `Quit` event being handled within the destructor, which then triggers the same destructor on a half-destructed object again. It doesn't seem this is the specific issue here as well, but having that nested event loop while the "main" event loop is busy draining itself of all remaining events is just a recipe for disaster.
Sign in to join this conversation.
No milestone
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
gh-obsproject/obs-browser#501
No description provided.