Issues with flash writes when VGA PIO output is running (Pico stock speed: 125MHz) #1984
Labels
No labels
bazel
bluetooth
board settings
boot_stage2
bug
build
c++
can't reproduce
clang
clib-interface
cmsis
code cleanliness
code size
compiler version issues
cyw43_driver
debug
discussion
documentation
duplicate
dynamic_clock
elf2uf2
enhancement
errata
filesystem
flash layout
floating point
FreeRTOS
hardware_adc
hardware_claim
hardware_clocks
hardware_dma
hardware_exception
hardware_flash
hardware_gpio
hardware_i2c
hardware_irq
hardware_pio
hardware_pwm
hardware_rcp
hardware_rtc
hardware_spi
hardware_sync
hardware_uart
hardware_vreg
hardware_watchdog
hardware_xosc
help wanted
host
IAR
installation
invalid
linker scripts
lto
lwip
mbedtls
missing API
needs reporter detail
networking
newlib
newlib-multicore
non-c
non-cmake
non-gcc
openocd
orphan PR
OS integration
pico_async_context
pico_base
pico_binary_info
pico_flash
picolibc
pico_lwip
pico_malloc
pico_multicore
pico_printf
pico_runtime
pico_stdio
pico_stdio_usb
pico_sync
pico_time
pico_util
pioasm
port
question
review
review2
rtos interop
runtime init
static library
svd
test
tinyusb
vscode
wifi integration
wontfix
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
gh-raspberrypi/pico-sdk#1984
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
I'm seeing this in someone else's project (a proper SDK one, but I usually use the Arduino-Pico environment as shown here)...
Back to the real project: I've created a flash memory update routine, and I'm flushing the cache to verify a write, but sometimes it fails to read back any written data - this sketch has the least code which shows what's failing.
Eventually, the memory compare doesn't get back what's in the flash memory chip and the LED stops flashing.
After some more testing, it seems important to say I was running at 252MHz (what I've heard is "safe" overclocking) but when I run this sketch at 133MHz, it seems quite stable. It may have failed once, but wasn't repeatable. It really starts to show repeatable issues at 200MHz and above. Note there's a lot going on in Arduino-Pico, and the project's SDK firmware
In the Pico SDK firmware, which is copy-to-RAM, I've disabled everything else, no interrupts or anything else going on, and it seems stable at any frequency the main project uses (up to 376MHz - it's a VGA output project, so can use increments of every ~25MHz).
Could this be one part of a RP2040 that just doesn't overclock well in a noisy environment?
It consistently fails for me at 133MHz when I configure Arduino-Pico "boards.txt" for rpipicow to use the DIV 4 flash boot2 object instead of the default DIV 2 (I know, it should be more stable, but it isn't for me).
Opened a discussion here:
https://github.com/earlephilhower/arduino-pico/discussions/2546
Is this DIV 4 for the PIO clock?
Assuming it is have you tried DIV 4 when you're overclocking the CPU? It doesn't surprise me that you can't set the pio clock to DIV 4 at 133MHz.
No it's not the PIO clock, it's the SSI clock. Nevertheless, I tested 200MHz with the DIV 4 boot2 and it also fails the same way. Interestingly, it doesn't fail (at least not soon) at 250MHz, but my special build of Arduino-Pico which can overclock to 300MHz (which does lower the PIO clock so I can still control the LED via the WiFi chip) also fails with the DIV 4 boot2.
You might have to attach a full example to demonstrate the problem.
Yeah, you're probably right, which is why I opened the discussion in the Arduino-Pico repo - since it's running enough additional code in the background to show the issue with the minimal example. Hopefully someone (maybe @earlephilhower ) will try it . Then again, take any moderately complex SDK project (using interrupts, pio and dma) and add in the cache flush & memcmp I've got... and I think that might show up the instability.
This still uses the VGA library files (fires interrupts, and sends messages to milticore_fifo) but it's pretty concise, the fifo is deep, and the SDK should be discarding messages targeting the fifo if it's full anyway...
This shows my Pico W with an LED on I/O pin 1 lit solidly, until the issue occurs and it then slow-flashes, showing the XIP device hasn't re-engaged and all I got back were 0x00's.
The VGA files are here:
https://github.com/visrealm/pico9918/tree/main/src/vga
Ok, that was a bad link to the VGA files, github did that. But you can see where the link should go.
Here is a full example... It sets up PIO, DMA and Ints along with CPU1 doing nothing in particular. This is all copy-to-RAM so nothing is using Flash Memory. Upon loading the example, it may show the LED dimly lit, so it's running and may not show the issue. Unplug the USB port, plug it back in. Again, it might do the same thing, but one in 10 times, it will slow flash soon after power-up. Feel free to use a non-W pico with an LED on IO25 (switch the defines).
The Pico Datasheet says that the Pico uses a Winbond W25Q16JV flash chip; and looking up the datasheet for that chip it says "133MHz Single, Dual/Quad SPI clocks". So maybe you simply need to slow the system clock down to (at least) 133MHz before trying to write any data to flash (and flushing the cache), and then switch it back up to a higher frequency after you've finished writing? (that's just my uneducated guess 😉 )
It's a good idea @lurch but the project this is in, is a VGA output device, so when doing flash updates you would lose sync.
I did try though, leaving the speed at the SDK default (I assume 125MHz - by not calling set_sys_clock_pll) along with DIV 4 on the flash chip (32MHz in that case) and my last program I posted kills the flash XIP engine 1 in 3 times at power-up. I've also tried memory-to-memory DMA along with timer Interrupts, but those alone work flawlessly, the XIP engine never crashes at power-up. It's just this combination of PIO, DMA and Ints.
I should also say, this is "copy_to_ram" via the CMakeList.txt file - even though I've got the functions all marked as "not_in_flash" - that's just because Arduino-Pico doesn't support copy_to_ram - but I'm not using that anymore, this is all SDK now.
I got 2 more Pico W's - one is new and one is really quite old, that came from a disassembled project... Both of these units have no issue with the program I posted, the XIP engine seems to be working solidly on every power-up.
So, 2 devices with issues (one isn't a Pico) and 2 devices without issues :(
Do you have problems on normal Pico's (not Pico W)?
Are you referring to the internal led? Some of your posts look like you're talking about an external led which would imply this has nothing to do with Pico W.
If you're having issues with the Pico W internal led then it's likely communication with the cyw43 chip has broken down. Do you see any relevant debug on the uart? If you're messing about with the system clock this will impact the pio used for comms to cyw43.
I'm not sure you'll get more help unless you can attach a full working example including cmake files.
I don't have any "normal Pico's" to try and I only used my Pico W's internal LED on the first Arduino sketch, where I quickly realised that might have those issues, so I moved to an external LED with a current limiting resistor on I/O pin 1. The fact that a cyw43 build can't be fully run from RAM on a RP2040 (500KB to control an LED?) made me abandon that.
The Pico (W) isn't crashing, the non-Pico device isn't either, XIP just isn't being re-engaged. In both devices (the Pico W and the non-Pico VGA device) I have worked around the issue by calling the flash-write a 2nd time (I don't think the 256 byte write actually fails, but calling the function a 2nd time doesn't ruin the 1st write, and does give it a 2nd chance to re-engage XIP) and that works... In the real project and the test code. My code change tries a 3rd time too in the project if needed, but I've never seen that happen.
My bad Pico W shows, on a bad boot, it switching successfully in and out of XIP, getting an error, switching back for a very short while, then getting errors again... Repeat.. On a good boot, it just works.
I think the clock is somehow broken, on both these bad devices. All my Pico's have come from the same authorised distributor too, the bad one isn't a fake.
Seems unlikely to me that "the XIP engine" (which is internal to the RP2040) would be "good" in some chips but "bad" in others? It might be the flash chip that is determining the difference in behaviour between your "good" and "bad" devices?
I changed the issue title to be more relevant to the reported problem?
Yeah, no problem here with a title change... I was hoping for that, not knowing what's going on and all (neither device showing issues is mine). EDIT: mine, as in my design.. obviously I own them.
I've changed the issue title, here's an example project that just has the VGA PIO component running while performing all the SDK ROM calls that disengage and reengage XIP. All 3 of my Pico W's show it failing to reengage, if it doesn't happen soon after booting you should cycle power and try again. This project isn't changing the stock speed from whatever the default is (125MHz?)
pico9918-gpu-dev_f.zip
Let me know, if it doesn't present as a problem on your Pico/Pico W. Peter.
We (the project owner and I) are now testing on RP2350B chips, and I'm happy to report this issue doesn't affect that chip, only RP2040 ones.
Please let me know if you need me refine the test example.
The MCVE from the arduino-pico issue seems (based on my reading) to be able to reproduce this without any PIO involvement, just core 0 doing flash operations. Is that right, or is there some configuration I'm missing?
I can look into this but I'll remove it from the 2.1 release as it's not even clear it's an SDK issue.
I only tested the Arduino version on one particular Pico-W and it was overclocked to 252MHz. At speeds below 200MHz, it was fine. When I had more Pico-Ws available to try, I was only testing the non-Arduino example code, but most of the Picos worked OK with that, to my surprise. The final example I posted is the one which shows issues on all the Pico-Ws I have access to.
Just for the record, you can leave Bluetooth firmware and combined WiFi firmware or just WiFi (if using WiFi only) firmware in flash. Example for Bluetooth firmware:
or just add
__in_flash("CYW43439_FIRMWARE")and that's it. And then firmwares won't be copied over to SRAM even forcopy_to_rambinaries.