sleep_ms(900) stuck forever when compiled with Debug Build Type (FreeRTOS) #2319
Labels
No labels
bazel
bluetooth
board settings
boot_stage2
bug
build
c++
can't reproduce
clang
clib-interface
cmsis
code cleanliness
code size
compiler version issues
cyw43_driver
debug
discussion
documentation
duplicate
dynamic_clock
elf2uf2
enhancement
errata
filesystem
flash layout
floating point
FreeRTOS
hardware_adc
hardware_claim
hardware_clocks
hardware_dma
hardware_exception
hardware_flash
hardware_gpio
hardware_i2c
hardware_irq
hardware_pio
hardware_pwm
hardware_rcp
hardware_rtc
hardware_spi
hardware_sync
hardware_uart
hardware_vreg
hardware_watchdog
hardware_xosc
help wanted
host
IAR
installation
invalid
linker scripts
lto
lwip
mbedtls
missing API
needs reporter detail
networking
newlib
newlib-multicore
non-c
non-cmake
non-gcc
openocd
orphan PR
OS integration
pico_async_context
pico_base
pico_binary_info
pico_flash
picolibc
pico_lwip
pico_malloc
pico_multicore
pico_printf
pico_runtime
pico_stdio
pico_stdio_usb
pico_sync
pico_time
pico_util
pioasm
port
question
review
review2
rtos interop
runtime init
static library
svd
test
tinyusb
vscode
wifi integration
wontfix
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
gh-raspberrypi/pico-sdk#2319
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
I have the issue that a
sleep_ms(900)never returns when code is compiled withDebugBuild Type and uses BLE. I could reduce my code to a bt-pico-freeze-issue sample, with which I can reproduce the problem.When I run the sample with
ReleaseBuild Type, all works perfect, BLE is advertising and the LED is blinking.When I run the sample with
DebugBuild Type, most of the time, the firstsleep_ms(900)in the loop is stuck forever no LED blinking and also USB serial output stops, but BLE advertisement work. I can see them with a BLE scanner like nRF Connect.Very interesting is that when I then connect via BLE the
sleep_ms(900)returns (even after being stuck for 5 minutes) and then the loop and with this the LED blinking works perfect. The same also happens when I disconnect and reconnect the USB serial port. So for me, it looks like some kind of external event fixes the stucksleep_ms().Serial output (USB) of
DebugBuild Type stuck at thesleep_ms(900):Serial output (USB) of
Debugwhen I do a BLE connection after ~8 minutes :serial output (USB) of
ReleaseBuild Type:When I debug it and break while the
sleep_ms()is stuck, it's always at thetimer_time_reachedmethod (before the IRQ).When looking at the variable values, the method would actually return
trueso the sleep_until loop would end, but somehow it never gets over that line as if all CPU is used by IRQ handlers?You don't mention what version of the SDK you are using? Can you try the LATEST master branch 2.1.1 that we released last week as it includes some timer fixes.
I use SDK 2.1.1 installed via VsCode it is
bddd20f928which is 1 commit behind current master, but to what I see that just changed a comment.Ok, sorry, I see you had the version in the log.
What device are you using Pico W (rp2040) or Pico 2 W (rp2350)?
I used a Pico 2 W (rp2350).
I now also run it with a Pico W (rp2040) and it does not happen, also with
DebugBuild Type, on the Pico W all works perfect, BLE is advertising and the LED is blinking.I can't immediately reproduce your issue. I'm testing Pico 2 W on Windows and choosing "Pico Debug (Cortex Debug)". Is that what you're using? The is flashing and I see console output.
It sounds like there are too many interrupts going off constantly. I would guess cyw43. I've no idea why it would only affect debug. You could try defining PICO_BTSTACK_CYW43_MAX_HCI_PROCESS_LOOP_COUNT=10 to limit the work done in hci_transport_cyw43_process in case that's constantly busy.
Defining
PICO_BTSTACK_CYW43_MAX_HCI_PROCESS_LOOP_COUNT=10did not help.I'm working on macOS, but hope that should not make a difference. It's not about launching with a debugger but building the Debug Version like
or setting
CMAKE_BUILD_TYPE=Debug. It's also happening when flashing via USB with picotool.So for me the following reproduces it:
As then, the output of screen is
and stuck at the last line of sleep 900
I'm getting this also, with the current head of main on an RP2040. It seems that get_absolute_time() always returns 0, so the sleep functions never advance. gcc arm-none-eabi 13.2.rel1 supplied by Ubuntu 24.04 (on an x86_64 host). Not certain about debug vs release configurations, I do have PICO_DEOPTIMIZED_DEBUG set. Minimum reproducible code:
output:
Giving the loop a few hundred million iterations doesn't change the output at all.
clarifying information: this only seems to happen with a debugger attached, if you just let it run without launching with a debugger, it does give the expected output:
Time: 494
incrementingTime: 17800515
Time2: 17800517
Time After Sleep: 17815898
I may have figured it out, thanks to https://github.com/raspberrypi/debugprobe/issues/45
the vscode extension's launch configuration has this
adding in another monitor reset seems to have kicked off the timers again.
and sleep_ms() works again while a debugger is attached.
It wasn't specific to 2.1.1 either, this happened on 2.0.0 (tested just now). The underlying issue for the OP may be that one core was stuck in a halted state after the firmware was loaded (possibly from a previous debugging session?). For me, it was one core was stuck in a stopped state due to the openocd load command, and never restarted the timers again.
@digiexchris I suspect that your issue may be different to the OP's issue, as I think your issue means that every
sleep_mswould lock up, whereas the OP's output logging indicates that their initialsleep_ms(100)works, and it's only the latersleep_ms(900)that locks up?yes you may be right, but I wonder if it is related to when the default timer gets started, and on which core the sleep is called on. but yeah it doesn't look like the OP's reproduction code does the sleeps on the second core. I wonder if the behaviour is any different if a full power cycle is done after flashing? That is what pointed me to the realization that my issue was actually related to the cores halting for the load function.
Also in my case it does seem like it's only the default timer that's affected, since the FreeRTOS tick timer that gets started when the scheduler starts works totally fine.
@digiexchris I's a bit OffTopic from the original issue here, but see #1152 for more info.
Interestingly, I added an additional sleep_ms into my duplication code above, and despite not having a debugger connected, I am having the same symptoms as the OP! The second sleep stalls!
Callstack:
vPortLockInternalSpinUnlockWithWait@0x1004505c (/home/chris/repos/PicoMillPowerFeed/src/FreeRTOS-Kernel/portable/ThirdParty/GCC/RP2040/port.c:1047)
sleep_until@0x10035ee0 (/toolchain/pico-sdk/src/common/pico_time/time.c:410)
sleep_us@0x10035f3e (/toolchain/pico-sdk/src/common/pico_time/time.c:424)
sleep_ms@0x10035f70 (/toolchain/pico-sdk/src/common/pico_time/time.c:440)
main@0x10001c12 (/home/chris/repos/PicoMillPowerFeed/src/main.cxx:146)
I do have freertos included, but the scheduler is not active at the time when these sleep_ms are called. I wonder if there's an issue with the freertos port.
What version of FreeRTOS (upstream, or rpi repo)?
is this on RP2040?
what does "not active" mean - you haven't started it yet?
Correct, I have not started the scheduler yet when calling the pico sdk sleep functions in the above repro code. I am using the head of upstream, since the rp2040 and smp code has been merged in there. Or so I read, maybe I'm wrong, but SMP and all other expected features appear to be working once I start the scheduler, but I do not use any normal sleep_ms type functions in the freertos tasks and use vTaskDelay() in those. The sleep_ms() calls are just during setup, like giving a short delay after GPIO setup to allow an SSD1306 to settle before starting everything that's going to use it.
I've since moved to a deterministic order of starting of freertos tasks instead of using sleep_ms, but it's probably worth addressing since there was no indication anything was broken other than the default timer never firing.
@digiexchris i'm a bit confused reading this; do you have a simple example which shows the problem without the misconfiguration of debug under VSCode?
Yes, my post on March 9 is the code I was using to do this, without the debugger attached. Just add a second sleep_ms call.
I tried this
On RP2040 with both release and debug builds (both under the debugger and not); linked against FreeRTOS and it does not hang for me
What compiler version are you using; can you attach an ELF that hangs
closing as not reproduced.. if still a problem retry with latest develop where there are some FreeRTOS interaction fixes