HIDAPI testing device with known good firmware, including using Virtual HID device for CI Testing #478
Labels
No labels
3rdparty
Android
API
Author does not respond
bsd
bug
build system/CI
ci-virtual-device
CMake
Contributions Welcome
Core
dependencies
device-specific
documentation
don't_merge
duplicate
enhancement
Example
Haiku
hidraw
hotplug
info required
invalid
libusb
macOS
Misc
Not an issue
question
Release
Test
Test GUI
Windows
wontfix
No milestone
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
gh-libusb/hidapi#478
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
There is an example device for hidraw from umockdev project. Maybe we can integrate it for CI under Linux.
https://github.com/martinpitt/umockdev/tree/main/devices/hidraw
Ref: libusb project uses umockdev for Linux testing using a container.
https://github.com/libusb/libusb/blob/master/tests/umockdev.c
https://github.com/libusb/libusb/blob/master/.github/workflows/linux.yml
The alternative:
uhid.In one of the issues there was a sample. The sample implementation actually has several issues/bugs, but the overall aproach is fine.
I am not so sure if this is possible or not.
https://github.com/vpelletier/python-functionfs
BTW, other than CI, it is also good to see if we have some known good HID device or FW to test HIDAPI.
For libusb, here is the reference, it can be the same for USB HID device development. But for Bluetooth/BLE and even I2C/SPI HID device, we need different tools.
https://github.com/libusb/libusb/wiki/FAQ#do-you-have-a-list-of-known-good-devices-to-test-libusb
Other than vendor provided USB stacks (sometimes with proprietary licenses), there are also Opensource MCU USB stacks.
Just wondering what is the test device you are using to test HIDAPI?
For me I usually use whatever HID device I have with me like the following for general test using hidtest.
For more real tests with hidapitester or simple test codes. I do not know much about the FW development other than simple modifications of the USB PIC USB codes and maybe a little bit with USB AVRs.
Then I use some HID devices to test hidapi indirectly using avrdude and OpenOCD (CMSIS DAP v1 compliant adapters).
Unfortunately I don't have any devices dedicated for testing, only a few of my peripherals, and a few devices from my employer (I'm under NDA not to disclose what those are), which have a few fixed input/output/interrupt reports, that I usually use for some sanity testing.
Yeah, would be great to have 1-2 FW samples for some well known simple/cheap device to be able to test most (all?) of the HIDAPI.
@todbot has some test firmware for hidapitester which can be used for hidapi testing as well.
https://github.com/todbot/hidapitester/tree/master/test_hardware
TinyUSB
https://github.com/hathach/tinyusb
node-hid has some examples as well.
https://github.com/node-hid/node-hid#examples
Jan Axelson has quite some HID examples for things like Cypress EZ-USB FX2LP and Microchip USB PIC.
http://janaxelson.com/hidpage.htm
Simple test for Jan Axelson FX2HID example (0925:1234) -- loop back of Input/Output report (no report ID) using hidapitest.
For this FW, I have to explicitly send output report ID 0.
Her USB PIC generic HID example has feature report loopback as well.
I have done the modification of the generic HID FW to add report ID (Input 1, Output 2, Feature OUT 3, Feature IN 4). I used that to test libusb Windows HID backend many year ago during the initial stage of the libusb Windows backend development.
Simple test using hidapitester for the Microchip Simple HID demo (which is the base for the FW used in hidtest). I am using the mla version for the 8bit/16bit USB PICs.
@Youw
For libusb testing, I usually recommend Cypress EZ-USB FX3 (SuperSpeed). Then EZ-USB FX2LP (High-Speed).
https://github.com/libusb/libusb/wiki/FAQ#do-you-have-a-list-of-known-good-devices-to-test-libusb
I think for HIDAPI testing, maybe FX3 is a bit overkill but you may want to get cheap EZ-USB FX2LP breakout from online marketplace like AliExpress at about US$5. For example, I have the following board.
I just did a simple modification of Jan Axelson's fx2hid example to extend the Input/Output report size to 64 bytes.
fx2hid_64bytes.zip
Target.zip
Edit: take note that in order to develop high speed HID USB device using FX2LP with wMaxPacketSize > 64 Bytes, we can not use EP1IN or EP1OUT, as their buf size is only 64 bytes. We should probably use EP2 and EP6 (buffer size equals to 512 Bytes or 1024 Bytes). So the modification is not as simple. But if we just need to get report size >64 bytes, then it is still possible to use EP1 IN and EP1 OUT. We need to handle the fragamentation in firmware.
For EZ-USB FX2LP, previous examples are using Keil UV2 and Keil C51 toolchain (the FW is rather small so the free 4K version bundled will work).
Lastes from Cypress (now part of Infineon) is using Eclipse based IDE with Open Source SDCC toolchain.
https://community.infineon.com/t5/Knowledge-Base-Articles/FX2-FX2LP-SDCC-Eclipse-Project-Interrupt-not-Serviced-KBA231372/ta-p/250672
Then there are quite some open source FW library available, using sdcc.
https://github.com/djmuhlestein/fx2lib
https://github.com/whitequark/libfx2
It should be a device, where you can program any HID ReportDescriptor, PhysicalDescriptor and StringDescriptor on low-level (byte-wise). Not something where you configure everything on high-level, because for testing it makes a difference how we order the items in the ReportDescriptor.
I believe most of the MCU development board will allow this, including the EZ-USB FX2LP (actually the example code from Jan Axelson uses assembly language for the descriptor, same for examples from Cypress).
Unfortunately I am not a programmer so I can only carry out minor modification of existing FW codes.
I need to check but the following FX2LP FW seems to work, EP1 IN and EP1 OUT have wMacPacketSize = 64 bytes, but INPUT/OUTPUT report size are both 128 bytes.
fx2hid_128bytes_report.zip
Target.zip
Similar thing for 512Bytes report.
fx2hid_512bytes_report.zip
I combined the codes from BulkLoop (change to Interrupt Endpoint) and FX2HID and create the new FX2HID FW using EP2 (OUT) and EP6 (IN) and it seems to work fine, with wMaxPacketSize = 512 Bytes and Report Size = 512 Bytes.
fx2hid_512bytes_packet.zip
1024 bytes wMaxPacketSize and HID INPUT/OUTPUT report size.
It is a bit strange for the last two bytes. I am not so sure what is the problem of the extra bytes of 0x48.
fx2hid_1024bytes_packet.zip
BTW, I think the above implementation is not fully correct as I have not implemented getting the Input/Output report using Control Transfer. I will try that as well later. But I think it is good enough for Input/Output transfer using the Interrupt IN/OUT endpoint.
I will also try to add Feature reports as well.
For this purpose, I have the modified Generic HID Device Firmware example from Jan Axelson, using USB PIC (I have PIC18F87J50 USB PIM, other USB PICs should work).
Testing results of two examples from hidapitester.
https://github.com/todbot/hidapitester/issues/20
For Feature report, hidapitester needs to add first 0 in order to work with HID device without Feature report ID (tested under Windows). The zero will not appear on the wire and the device will not receive the firs zero.
Arduino serial monitor output.
There are four test FWs in hidapitester. I have only tested the first two examples: ProMicroRawHID and TeensyRawHid. I will try to get the other tinyusb based examples working later.
https://github.com/todbot/hidapitester/tree/master/test_hardware
Here are the test results for SAMD21 (using Arduino M0 clone) hidtest-tinyusb example.
Here are the test results for SAMD21 (using Arduino M0 clone) hidtest_tinyusb_static example.
Mode: blink1 mode, echo ON.
teensy mode, echo ON.
BTW, Adafruit Arduino TinyUSB library has a generic hid example as well and it should work on many boards. The following test is using the Arduino M0 clone (Microchip/Atmel SAMD21).
The following result is from Raspberry Pi Pico.
@Youw and @todbot
I tend to think the two examples from hidapitester can be of good use to hidapi project testing.
https://github.com/todbot/hidapitester/tree/master/test_hardware
https://github.com/todbot/hidapitester/blob/master/test_hardware/ProMicroRawHID/ProMicroRawHID.ino
Reference: the library is said to support USB AVR and SAMD21.
https://github.com/NicoHood/HID
https://github.com/todbot/hidapitester/blob/master/test_hardware/hidtest_tinyusb/hidtest_tinyusb.ino
Reference:
https://github.com/adafruit/Adafruit_TinyUSB_Arduino
This is the underlying library which should work with Raspberry Pico and ESP32 S2/S3/C3/C6 with USB support and Microchip SAMD21 based boards as well as Nodic nRF52 based boards.
Maybe we can adapt hidtest.c to work with one of them and use Readme to document the example code for another one.
For BT/BLE HID, I will try to see if there are good examples. It seems to me not easy to get I2C/SPI HID development boards.
As for CI, I think we can continue to explore uhid.
"3" That would be great.
"5" Agree
For HID over I2C/SPI, I tend to think it is difficult to do under Windows as a driver may be needed.
https://learn.microsoft.com/en-us/windows-hardware/drivers/hid/hid-over-i2c-guide
https://learn.microsoft.com/en-us/windows-hardware/drivers/hid/hid-over-spi
There are some examples of HID over I2C for Linux (using things as Raspberry Pi 4/400 as Linux host)
https://github.com/benedekkupper/stm32-i2c-hid.git (STM32)
https://github.com/NordicPlayground/nrf52-i2c-hid-demo (Nodic NRF52).
For HID over SPI, it seems to me the support under Linux is quite new.
@todbot
Sorry to disturb you. I believe there is a stupid mistake here in
code.pyso maybe you can help. I am not familiar with Circuit Python.Code.py is working fine for Generic USB HID device emulation.
But sending input report (looping back from output report) does not work.
hidapitester output
Hi @mcuee,
The code looks correct from inspection. I'll look into this when I get back to my bench tomorrow.
As for why you need two descriptors in
enable(), you're probably running into a Python-ism where one-item tuples are converted to a non-list item. To get around this, add a comma to your one-item tuple. That is, do this:@todbot
Yes you are right. The following works.
Relevant discussion here.
I have had problems with 64-byte reports with report IDs. Not sure if this is a MCU buffer limitation (which is 64 bytes usually), an issue with hidapi, or something else.
Here is a modified set of above code that does work, up to 63 byte reports:
And running hidapitester:
Note that hidapitester is given
-l 64to make a report that is REPORT_COUNT+1 long.When run, the CircuitPython device outputs this on its REPL:
@todbot
Thanks a lot for the help.
Interesting and this is exactly the same problem I have with my USB PIC using Jan Axelson's generic HID example as well. 63 bytes works but not 64 bytes.
Firmware_generic_hid4_63B.zip
I think there are a few potential reasons.
This is one potential issue of hidapitester I would like to check with you and @Youw. I am thinking this should be worked-around inside hidapitest for Windows (report ID 0 needs to be in the buffer but not really appear in the USB BUS. That is probably why we need one more bytes of bufer than the length of report size. I believe this is not necessary for Linux and Windows. I could be wrong though.
This reflects how
hidapiworks, where the first byte of the send buffer is the reportId if it exists, which I believe came from how libusb works. It's always bugged me and has led to so much confusion, but I'm not sure how we could get around it.@Youw
Please take a look when you got the time. Thanks.
@Youw and @todbot and @JoergAtGithub
I think we have gone through this report length previously. Maybe we have to revisit them.
I think the following PR fixes the issue for reading back the input report and it is correct.
I think we need to revisit for Feature report. I think the following commit is kind of correct if we look at from Windows driver point of view.
https://sourceforge.net/p/libusb/mailman/message/37303650/
But from a cross-platform API point of view, we may need to revist to align with Linux and macOS, which means we need to focus on the data on the USB bus and we need to drop the extra 0 in front when the report ID is 0 for Windows.
github.com/libusb/hidapi@6fcb0bb228Same for Output report. Right now we may report one more byte than real data on the bus due to Windows driver behavior, when the report ID is 0.
All in all, we may need to align Windows HIDAPI behavior along with Linux and macOS. Need to check with the libusb backend as well.
Another related old PR for macOS.
There are many confusions under libusb Windows HID backend as well. I think there are more problems with libusb Windows HID backend (libusb project recommends people to use HIDAPI and not libusb, so it is a legacy thingy).
There is another interesting issue there as well (hopefully not affecting HIDAPI).
@tormodvolden
FYI as well. I believe https://github.com/libusb/libusb/pull/1217 is incorrect.
Unfortunately I've never had good test device(s) to check all: input/output/feature/interrupt, with and w/o report ID and with different lengths.
All being said, I think we need to focus to make the behavior (e.g. in terms of the return length, etc.) be consistent on all platforms, specifically to match the documentation.
I'm a little lost in the context and worst of all - cannot properly check it locally...
I will recommend the following devices which will work with the examples here. They are of relative low cost from online stores like AliExpress.
https://github.com/todbot/hidapitester/tree/master/test_hardware
Yes I agree.
I understand. I will try to summarize and create another issue. And hopefully you can get one or two of the test devices without too much difficulties.
Hm, I actually have an Arduino somewhere. Haven't used it for years. Maybe I could use it.
Now it is only the question if finding fime for it.
No problem, take your time.
The Arduino Uno and Maga 2560 (official version and some clones) come with ATmega16U2. It is a bit tricky to get the rawhid example working.
https://github.com/NicoHood/HID/wiki/RawHID-API
https://github.com/NicoHood/HoodLoader2
Arduino Leonardo or Arduino Micro or Pro Micro or clones come with ATmega32U4 and then you should have no issues to get rawhid example working.
https://github.com/todbot/hidapitester/tree/master/test_hardware/ProMicroRawHID
I am going to move this to discussion and create a new issue to make this one clean. Sorry I have put too much info here and people may get confused.