Cannot drag a UF2 to RPI-RP2 on macOS Ventura #1081
Labels
No labels
bazel
bluetooth
board settings
boot_stage2
bug
build
c++
can't reproduce
clang
clib-interface
cmsis
code cleanliness
code size
compiler version issues
cyw43_driver
debug
discussion
documentation
duplicate
dynamic_clock
elf2uf2
enhancement
errata
filesystem
flash layout
floating point
FreeRTOS
hardware_adc
hardware_claim
hardware_clocks
hardware_dma
hardware_exception
hardware_flash
hardware_gpio
hardware_i2c
hardware_irq
hardware_pio
hardware_pwm
hardware_rcp
hardware_rtc
hardware_spi
hardware_sync
hardware_uart
hardware_vreg
hardware_watchdog
hardware_xosc
help wanted
host
IAR
installation
invalid
linker scripts
lto
lwip
mbedtls
missing API
needs reporter detail
networking
newlib
newlib-multicore
non-c
non-cmake
non-gcc
openocd
orphan PR
OS integration
pico_async_context
pico_base
pico_binary_info
pico_flash
picolibc
pico_lwip
pico_malloc
pico_multicore
pico_printf
pico_runtime
pico_stdio
pico_stdio_usb
pico_sync
pico_time
pico_util
pioasm
port
question
review
review2
rtos interop
runtime init
static library
svd
test
tinyusb
vscode
wifi integration
wontfix
No milestone
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
gh-raspberrypi/pico-sdk#1081
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
This results in:
It is still possible to copy UF2 files from the command line
Note this isn't an issue with the SDK, but this seems like as good a place as any for discussion
Note the error is
kPOSIXErrorENOATTR ("Attribute not found")This (and the fact that
cpcommand doesn't have the problem) makes me suppose the issue relates to Finder trying to write and then read additional metadata (or a resource or perhaps .DS_Store) to/from the drive.Note that the USB MSC volume (RPi-RP2) that the the RP2040 exposes from the ROM bootloader when BOOTSEL mode is entered is a "fake" volume. There is no mutable storage backing the volume. Instead the UF2 file sectors are recognized and processed as they are written. Other writes to the volume will succeed, but reading back the data will give the wrong contents.
NOTE: This is a common mechanism employed by embedded devices which support firmware updates over USB. They simply do not have resources to back the state of the entire drive. Note; this is the reason for the UF2 format to exist in the first place (see https://github.com/microsoft/uf2)
Something has clearly changed in macOS Ventura; either
In any case, i would suggest the new behavior of macOS Ventura is incorrect; if you drag a file onto a USB drive, the intention is to copy the file, and that should succeed (if there is no disk failure) irrespective of other metadata
We're tracking a very similar (likely the same...) issue with DAPLink, which is used for mbed and micro:bit devices amongst others. https://github.com/ARMmbed/DAPLink/issues/982
I took an image of the RPI-RP2 drive from a Pico using dd, and then wrote it to a normal flash drive. The contents after dragging blink.uf2 are:
I don't know how this compares to a previous version. Perhaps I can spin up a VM of Monterey to compare.
If you're in the office tomorrow you can grab someone (any of) the people from the Comms team that might be in, everyone there has a Mac, and none of them will be running Ventura at this point.
In case it helps, I had issues creating a VM with a VMWare Fusion from Monterey on an x86 Macbook. I created one from the Monterey recovery partition and upgraded it, but it constantly crashed and I was completely unable to enter Recovery Mode (with any method I found online) to disable System Integrity Protection, which is useful for using additional utilities like ptrace/dtrace/strace/wireshark.
On the other hand on a different x86 macbook already upgraded to Ventura, I was able to easily create a working VM from the recovery partition with Parallels and disable SIP. I think Parallels then lets you easily transfer the VM to a different system, so that might be a useful way to upgrade only one computer and then use it to generate VMs that could be used from computers using Monterey or older.
I'm not an MacOS expert, but I saw the tweet from @aallan via RT. My Macs aren't scheduled for Ventura just yet, but I may have a suggestion of at least which pond to fish in.
My GUESS (former OS guy...) is that MacOS is creating the .fseventd, Spotlight-vt100, .Trashes, etc. indexing and metadata on the drive and getting grumpy when they don't persist. (Breaking this is bad for all kinds of reasons, but it's not totally unreasonable for an OS to throw errors when it tries to write files to a "disk" and those files don't get written.)
I used to work with a GPS that would emit a solid beeeeeeeep on bootup and then crash any time it would see an SD card that had been touched by a Mac with the Finder (CLI was fine) because it didn't like seeing those files that it couldn't understand. It may have been something about them not being 8.3, but it was just a combination of obscure, annoying behaviour.
If it's not indexing data for search, it's also possible it's the extended attribute thing that it's trying to creae to indicate the file is busy. Maybe that was only on network drives. There was some externally visible trait that could be seen on MacOS when it had a file open for write. It's possible that the inability to set THAT flag/attribute/file blows it up, though the creation/write of the file itself is fine.
As just things to try - and these are pain killers, not cures - try:
Let your pseudofilesystem have an empty file in the root directory ".metadata_never_index". Similarly, a zero-length file ".Trashes" will stop it from trying to create a directory of that name. If you have your own filesystem code, maybe try mocking up (even if it's of the form "if filename == "/.Trashes" touch /.Trashes; return) to let your removable device implement just enough to make Ventura happy.
Experiment with the mdiutil options for -d, -i, -X.
Experiment with the big hammer of "mdutil -a -i [on|off]" This is a systemwide toggle that changes how it handles metadata. It also seriously hoses the finder/mdfind search experience. Once you've disabled mdutil, be prepared for the system to burn serious battery and CPU to bring it back as it might need to reindex everything. (Maybe not true these days...)
You may find joy in combining one or more of these - or no joy at all. These are just my memories of managing other devices that didn't play nice with MacOS removable media.
None of these are long term fixes and I fully recognize the backward compatibility problems of changing your SD cards and/or your firmware to work around a problem that isn't years. I'm just offering ideas tht I hope are helpful to at least get a stuck developer moving again.
There was some other OS that used to leave little dingleberry files on filesystems once a device passed through it, but that was years ago and I don't even remember with certainty which it was. Maybe it was only when the removable volumes were further shared over a network or something bizarre.
Good luck!
At least for RP2040, the firmware is on ROM on the silicon, we literally can't change it.
Oh, I'm sure there are thousands of devices in this same approximate boat.
I feel 'ya.
Hopefully there are some other ideas in that list that can at least lead
someone closer to a workaround. This is CLEARLY something that needs fixed
in MacOS.
On Fri, Oct 28, 2022 at 7:04 PM Alasdair Allan @.***>
wrote:
Try the ditto command instead of cp.. maybe like so:
ditto --norsrc --noextattr --noacl ~/Desktop/demo.py /Volumes/RPI-RP2/
I just tried to verify if the drag still produces the error. On the new beta, 13.1 (22C5033e) persists.
Just to confirm for folks that aren't directly involved,
works as expected, copying the UF2 file to the virtual volume.
Just to keep tabs on them, other folks that have Github issues open:
We've published this on the Adafruit blog, which also goes to Twitter. We can revise the post if needed:
https://blog.adafruit.com/2022/10/31/uploading-uf2-files-with-macos-13-0-ventura-apple-microbit_edu-raspberry_pi-circuitpython/
We've also now written this up as a blog post, https://www.raspberrypi.com/news/the-ventura-problem/.
I've hacked together a quick macOS Shortcut. If you add it as a Quick Action — you'll need to give it permission to run shell scripts — then you can just right-click on your UF2 and select "Quick Actions > Copy to RPI-RP2" and your UF2 will be copied to your Pico. It's not as nice as drag-and-drop, but it's better than opening a Terminal window I guess?
https://www.icloud.com/shortcuts/989b90c87fec4c11965dd517685f87cd
I gave feedback to Apple today. The number they gave me was FB11762117. I used your FB from the blog post to link the issue to RasPi.
Hi there, this no longer works for me after updating to 13.0.1 (hoping it would resolve the drag/drop issue). For a complete beginner, it's pretty frustrating to say the least.
It works for me on 13.0.1. Try,
instead?
Just to be sure, your Terminal window is in the same directory as the UF2 binary is in?
According to comments on ARMmbed/DAPLink#982,
cp -Xdoesn't work for firmware files >1MB in size. Howeverrsyncdoes work. I'd therefore recommend people look to usersyncrather thancpuntil there is a proper fix.Update: This shouldn't be a problem for UF2 files as the file format is designed to avoid problems like this, as each block stores its own address as well as the total number of blocks so the fact they're being written non-sequentially shouldn't matter.
Confirmed that
cp -Xworks as expected on 13.0.1. Sorry, but you must be experiencing some other problem that isn't related.Thank you for checking, @aallan. I've been able to transfer a blink.UF2 file downloaded from the official website using the above method. Unfortunately, I'm still unable to detect and connect to the board (Arduino IDE), view or manage whatever files may be stored on there. I will keep troubleshooting...
Just to show it working!
https://user-images.githubusercontent.com/53625/203112367-31ddaaca-8a55-44af-824a-4c461dd1e54d.mov
macOS 13.0.1, nothing works.
drag & drop 'error 100093'.
cp -X 'disk not ejected properly', same thing with rsync.
I'm running 13.0.1 and both "cp -X" and "rsync" on the command line work to copy the file to the "fake" volume. As noted in he blog post you should expect error messages, but you can ignore them, the copy completes correctly.
You could try the short cut action if the messages bother you however?
Hey there, I built a macOS app that makes it really easy to upload UF2 files to the Pico (and any other RP2040 board technically). You can get it for free here:
https://github.com/stuartziane/Pico-UF2-Uploader-Mac
You'll still get the system notification about the Pico being ejected, but I decided to leave it like that, just because it's closer to how things normally happen. If it's annoying, I'll sort that out.
Please let me know what you think of the app!
Cheers,
Stuart
Sorry if you had trouble earlier trying to download the app - For some reason, the app itself was uploaded as a folder, rather than a .app. It's fixed now.
According to a comment on the ARMmbed/DAPLink#982 issue on this, "…I've tested macOS Ventura 13.1 beta.4 and it looks like it resolves the issue."
I've not yet had a chance to check this myself, or heard anything directly back from Apple, but if anyone that has access to macOS beta releases wants to try out the latest beta and report back?
This is fixed on macOS Ventura 13.1
Drag and drop in Finder starts promising, with a grayed out name in the Pico window and a small progress icon.
Then the RPI-RP2 unmounts and does not reappear. I need to disconnect and reconnect (with the button down) the uf2 file has not copied.
Using rsync, cp -X also result in unmounting and disappearing and file not copying.
The unmount is the correct behaviour, at that point the Pico should have rebooted and run your software so will no longer appear as a device. Rebooting with the button pressed makes it go back in to boot mode, the UF2 does NOT appear in the file list as its a virtual file system only intended to support the copying of files. Are you sure your program is not running straight after the copy?
I'm... not sure. Maybe my expectation is wrong.. (for instance Adafruit Metro Express Circuit py mounts as a device and runs the code.) Is there any way to tell the uf2 is installing properly?
If your code outputs anything on serial then monitor that, or turn on the onboard LED when you code starts running. I tend to flash the onboard if I need to check things are starting up.
Based on the file filename (v1.19.1) it looks like you might be loading a MicroPython UF2 file. In that case, and if you haven't somehow attached your own code to the build, then it probably doesn't do anything "visible" when running and there is no MSD drive to load.
The easiest way to check it's running correctly is to connect to the serial port and it should drop you into the MicroPython REPL.
CircuitPython presents the CIRCUITPY drive. MicroPython does not present a drive, but serial access to the REPL should be available. You should see a
/dev/cu.somethingdevice appear that you can connect to with a terminal program, Mu, etc.Expectations adjusted :) With https://thonny.org 4.01 for Mac and the settings that go with it, I have a blinking led. Thank you for the helpful comments!