Native BSD backend using uhid #608

New issue

Closed

opened 2023-08-12 08:54:42 -05:00 by ghost · 25 comments

ghost commented 2023-08-12 08:54:42 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

I was curious about the pragmatics of supporting various BSDs. There seems to be a fairly common API for HID over USB on most common BSDs called usbhid which is intended to be used with the uhid kernel driver for generic HID devices. Support for this is likely a necessity because several BSD ports of libusb cannot be used with USB devices already reserved by a kernel driver.

I'm too unfamiliar with HID myself to work on this at present, but is there anyone else willing to put in the effort to add this as a hidapi backend?

I was curious about the pragmatics of supporting various BSDs. There seems to be a fairly common API for HID over USB on most common BSDs called `usbhid` which is intended to be used with the `uhid` kernel driver for generic HID devices. Support for this is likely a necessity because several BSD ports of `libusb` cannot be used with USB devices already reserved by a kernel driver. I'm too unfamiliar with HID myself to work on this at present, but is there anyone else willing to put in the effort to add this as a `hidapi` backend?

Youw commented 2023-08-12 11:07:22 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

Duplicate of #543 ?

Duplicate of #543 ?

ghost commented 2023-08-12 11:45:59 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

Duplicate of #543 ?

I don't believe so. There's more BSDs than just FreeBSD. And #543 is about the hidraw compatibility layer that is only on FreeBSD. This is about the HID layer that exists on multiple BSDs, though they have diverged a bit over the years. It may be feasible to manage that with some clever use of the preprocessor.

> Duplicate of #543 ? I don't believe so. There's more BSDs than just FreeBSD. And #543 is about the `hidraw` compatibility layer that is only on FreeBSD. This is about the HID layer that exists on multiple BSDs, though they have diverged a bit over the years. It may be feasible to manage that with some clever use of the preprocessor.

Youw commented 2023-08-12 15:38:42 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

For those who potentially want to start implementing this but don't know where to start (from HIDAPI POV) - here is an example me adding an experimental WinRT backend: https://github.com/libusb/hidapi/compare/winrt

For those who potentially want to start implementing this but don't know where to start (from HIDAPI POV) - here is an example me adding an experimental WinRT backend: https://github.com/libusb/hidapi/compare/winrt

ghost commented 2023-08-12 16:21:31 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

Some relevant links for uhid:

https://man.openbsd.org/uhid.4
https://man.netbsd.org/uhid.4
https://man.freebsd.org/cgi/man.cgi?query=uhid&sektion=4
https://leaf.dragonflybsd.org/cgi/web-man?command=uhid&section=4

Some relevant links for `uhid`: https://man.openbsd.org/uhid.4 https://man.netbsd.org/uhid.4 https://man.freebsd.org/cgi/man.cgi?query=uhid&sektion=4 https://leaf.dragonflybsd.org/cgi/web-man?command=uhid&section=4

ghost commented 2023-08-12 22:05:16 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

I don't believe a native port for DragonFly is feasible unless they fix their UHID ioctl situation. It doesn't appear to support getting any information about the device behind the UHID device such as the USB device information. The other 3 still appear to be viable.

I don't believe a native port for DragonFly is feasible unless they fix their UHID ioctl situation. It doesn't appear to support getting any information about the device behind the UHID device such as the USB device information. The other 3 still appear to be viable.

mcuee commented 2023-08-12 22:31:15 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

I am not so sure if there is any interest to have the uhid backend for FreeBSD as the existing libusb backedn works fine (libusb is part of FreeBSD now, and it is using FreeBSD's own libusb implementation with BSD license). So you can arguably call it native for FreeBD as well. hidraw backend seems to be more interesting for FreeBSD.

But I agree that it will be good to have uhid backend for NetBSD and OpenBSD, if it is possible. HIDAPI libusb backend for NetBSD/OpenBSD require rebuilding the kernel last time I checked, so it is not really that useful.

I am not so sure if there is any interest to have the uhid backend for FreeBSD as the existing libusb backedn works fine (libusb is part of FreeBSD now, and it is using FreeBSD's own libusb implementation with BSD license). So you can arguably call it native for FreeBD as well. hidraw backend seems to be more interesting for FreeBSD. But I agree that it will be good to have uhid backend for NetBSD and OpenBSD, if it is possible. HIDAPI libusb backend for NetBSD/OpenBSD require rebuilding the kernel last time I checked, so it is not really that useful.

ghost commented 2023-08-13 15:19:29 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

Further research reveals a problem with the current hidapi public API. There's a symbol conflict between hidapi and usbhid. The function hid_init is used in both. At the very least this makes it impossible to combine hidapi and usbhid in the same program. I can see two options. Just don't use usbhid and hope the developer doesn't either or rename hid_init in hidapi which would break the library ABI. usbhid mainly exists as an optional BSD library to handle a subset of what hidapi does but I was wanting to use its report parsing functionality.

Further research reveals a problem with the current `hidapi` public API. There's a symbol conflict between `hidapi` and `usbhid`. The function `hid_init` is used in both. At the very least this makes it impossible to combine `hidapi` and `usbhid` in the same program. I can see two options. Just don't use `usbhid` and hope the developer doesn't either or rename `hid_init` in `hidapi` which would break the library ABI. `usbhid` mainly exists as an optional BSD library to handle a subset of what `hidapi` does but I was wanting to use its report parsing functionality.

Youw commented 2023-08-13 15:45:03 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

Instead of linking against uhid usbhid HIDAPI could load it dynamically - that way we avoid symbol conflict.
Also, for HIDAPI v1 I think hid_init will be renamed to hidapi_initialize (and it will become context-specific initialisation, similar to what libusb uses).

Instead of linking against ~`uhid`~ `usbhid` HIDAPI could load it dynamically - that way we avoid symbol conflict. Also, for HIDAPI v1 I think `hid_init` will be renamed to `hidapi_initialize` (and it will become context-specific initialisation, similar to what `libusb` uses).

ghost commented 2023-08-13 17:14:40 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

Actually uhid API is entirely contained in headers since it all goes through ioctl. What I was referring to it the optional high level library usbhid. I think it will be a non-issue since I can just avoid using it.

Actually `uhid` API is entirely contained in headers since it all goes through `ioctl`. What I was referring to it the optional high level library `usbhid`. I think it will be a non-issue since I can just avoid using it.

ghost commented 2023-08-14 14:33:47 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

I have done a fair amount of work on a NetBSD port via uhid and have discovered uhid appears to be intended to be used with usbhid at the same time. The uhid API appears to assign a single device with multiple report IDs to separate uhid devices, and you are expected to get the report ID and descriptor from the uhid kernel APIs and to then parse them with usbhid or another HID parser to get the missing information for how to properly use the device.

Is there even an API in hidapi to handle this? It seems to assume you can just pass any arbitrary report ID through the API and it will just handle it but uhid already decides which one you are assigned and expects you to structure your reports in accordance to the exposed information. In the case of a device with multiple report IDs being split across multiple devices, how should I approach this?

I have done a fair amount of work on a NetBSD port via `uhid` and have discovered `uhid` appears to be intended to be used with `usbhid` at the same time. The `uhid` API appears to assign a single device with multiple report IDs to separate `uhid` devices, and you are expected to get the report ID and descriptor from the `uhid` kernel APIs and to then parse them with `usbhid` or another HID parser to get the missing information for how to properly use the device. Is there even an API in `hidapi` to handle this? It seems to assume you can just pass any arbitrary report ID through the API and it will just handle it but `uhid` already decides which one you are assigned and expects you to structure your reports in accordance to the exposed information. In the case of a device with multiple report IDs being split across multiple devices, how should I approach this?

Youw commented 2023-08-15 11:59:22 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

What I was referring to it the optional high level library usbhid

Excuse me for confusion, I really meant usbhid.

Is there even an API in hidapi to handle this?
In the case of a device with multiple report IDs being split across multiple devices, how should I approach this?

This is an interesting problem.
As per current HIDAPI design, each hid_device is only as unique as VID/PID/SerialNumber/UsagePage/Usage combination (and it is not even always true for the UsagePage/Usage part - but that's a different story).

HIDAPI doesn't support "device-per-report" abstraction as it assumes that a single device may support multiple reports (that's why a report ID is explicitly passed as part of data payload).

To support the uhid abstraction, the HIDAPI implementation would have to do it under the hood:

• parse the report descriptor;
• find a list of reports supported by the device;
• first time a report is tried to read/write - open a corresponding per-report uhid device and use it to send/receive data.

I don't see any better way.

> What I was referring to it the optional high level library `usbhid` Excuse me for confusion, I really meant `usbhid`. > Is there even an API in `hidapi` to handle this? > In the case of a device with multiple report IDs being split across multiple devices, how should I approach this? This is an interesting problem. As per current HIDAPI design, each `hid_device` is only as unique as VID/PID/SerialNumber/UsagePage/Usage combination (and it is not even always true for the UsagePage/Usage part - but that's a different story). HIDAPI doesn't support "device-per-report" abstraction as it assumes that a single device may support multiple reports (that's why a report ID is explicitly passed as part of data payload). To support the `uhid` abstraction, the HIDAPI implementation would have to do it under the hood: - parse the report descriptor; - find a list of reports supported by the device; - first time a report is tried to read/write - open a corresponding per-report `uhid` device and use it to send/receive data. I don't see any better way.

ghost commented 2023-08-15 15:59:05 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

I am considering an approach that opens all the uhid devices and uses their USB device information to construct a proper abstraction. We could use an internal per device lookup table to quickly see if there's a corresponding device for the requested report ID to know where to forward the IO requests to.

I am considering an approach that opens all the `uhid` devices and uses their USB device information to construct a proper abstraction. We could use an internal per device lookup table to quickly see if there's a corresponding device for the requested report ID to know where to forward the IO requests to.

Youw commented 2023-08-15 16:04:28 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

I believe opening all devices at once would be overkill, if even possible (what about exclusive/shared access, read/write permissions, etc.?). Lookup table - yes, plus lazy device opening, only if/when needed.

I believe opening all devices at once would be overkill, if even possible (what about exclusive/shared access, read/write permissions, etc.?). Lookup table - yes, plus lazy device opening, only if/when needed.

ghost commented 2023-08-15 16:44:29 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

I believe opening all devices at once would be overkill, if even possible (what about exclusive/shared access, read/write permissions, etc.?). Lookup table - yes, plus lazy device opening, only if/when needed.

Well, we have to do that anyway to properly enumerate devices, don't we? I only intended this to be done during an initial search for all the uhid devices connected to the same HID device or during enumeration.

> I believe opening all devices at once would be overkill, if even possible (what about exclusive/shared access, read/write permissions, etc.?). Lookup table - yes, plus lazy device opening, only if/when needed. Well, we have to do that anyway to properly enumerate devices, don't we? I only intended this to be done during an initial search for all the `uhid` devices connected to the same HID device or during enumeration.

ghost commented 2023-08-15 17:09:16 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

Ok. I am beginning to think a uhid implementation may be untenable due to serious limitations. Devices are opened exclusively. So how does one even do enumeration when another program might be using them? And I know of no other way to interact with the HID layer. I think I am done exploring this. I can see now why hidraw on FreeBSD is considered a big deal.

Ok. I am beginning to think a `uhid` implementation may be untenable due to serious limitations. Devices are opened exclusively. So how does one even do enumeration when another program might be using them? And I know of no other way to interact with the HID layer. I think I am done exploring this. I can see now why `hidraw` on FreeBSD is considered a big deal.

👍 1

Youw commented 2023-08-16 06:04:15 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

Nevertheless thanks for great investigation.
No one has check it this deep in context of HIDAPI.

I'll keep this ticket open for now. Maybe there will be more findings/details.

Nevertheless thanks for great investigation. No one has check it this deep in context of HIDAPI. I'll keep this ticket open for now. Maybe there will be more findings/details.

ghost commented 2023-08-16 14:23:17 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

Don't fret. I at least did learn that uhid is going to need a lot of help from other platform specific APIs to integrate well with hidapi. After looking at usbdevs source code on NetBSD I discovered you can use drvctl API combined with usb API to enumerate the uhid parent device, uhidev. This avoids the major problem with the exclusive lock enforced by the kernel and you only have to open an uhid device to get the report descriptor now, which is just needed for the the usage fields. This can be left out if the device is already in use.

Don't fret. I at least did learn that `uhid` is going to need a lot of help from other platform specific APIs to integrate well with `hidapi`. After looking at `usbdevs` source code on NetBSD I discovered you can use `drvctl` API combined with `usb` API to enumerate the `uhid` parent device, `uhidev`. This avoids the major problem with the exclusive lock enforced by the kernel and you only have to open an `uhid` device to get the report descriptor now, which is just needed for the the usage fields. This can be left out if the device is already in use.

ghost commented 2023-08-17 04:30:54 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

That should do it for the majority of the work. I thought I might as well put my research into actual code.

That should do it for the majority of the work. I thought I might as well put my research into actual code.

ghost commented 2023-08-18 16:40:55 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

I also looked into making a native one for OpenBSD but despite having similar APIs to NetBSD there is no apparent way to connect the uhid parent device to the uhid devices it is exposing so I cannot see myself being able to write a port that would be comparable to the level that the NetBSD one is at.

Without the ability to explore the device relationships like this I cannot see a way to workaround the uhid limitations and design differences like I did for NetBSD. You need something akin to NetBSD's drvctl interface which allows you to walk the device tree. I used this extensively in the port to avoid needlessly opening uhid devices and also to connect the parent device uhidev to its uhid children.

All that said I believe a port to FreeBSD is still possible due to how advanced its sysctl interface is. It should be possible to explore the device tree sufficiently to get the needed information for a hidraw or uhid interface. In this situation it might be a good idea to share code between the two since they are operating on the same platform just different native APIs.

I also looked into making a native one for OpenBSD but despite having similar APIs to NetBSD there is no apparent way to connect the `uhid` parent device to the `uhid` devices it is exposing so I cannot see myself being able to write a port that would be comparable to the level that the NetBSD one is at. Without the ability to explore the device relationships like this I cannot see a way to workaround the `uhid` limitations and design differences like I did for NetBSD. You need something akin to NetBSD's `drvctl` interface which allows you to walk the device tree. I used this extensively in the port to avoid needlessly opening `uhid` devices and also to connect the parent device `uhidev` to its `uhid` children. All that said I believe a port to FreeBSD is still possible due to how advanced its `sysctl` interface is. It should be possible to explore the device tree sufficiently to get the needed information for a `hidraw` or `uhid` interface. In this situation it might be a good idea to share code between the two since they are operating on the same platform just different native APIs.

👀 1

ghost commented 2023-08-27 04:32:29 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

@Youw Does the libusb backend on FreeBSD have to completely disable the HID kernel driver to work? I was considering that in some niche situations it may be good to have the option to use the native kernel interface since devices in some cases will expose multiple HID interfaces.

@Youw Does the libusb backend on FreeBSD have to completely disable the HID kernel driver to work? I was considering that in some niche situations it may be good to have the option to use the native kernel interface since devices in some cases will expose multiple HID interfaces.

mcuee commented 2023-08-27 05:13:32 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

@Youw Does the libusb backend on FreeBSD have to completely disable the HID kernel driver to work? I was considering that in some niche situations it may be good to have the option to use the native kernel interface since devices in some cases will expose multiple HID interfaces.

No. That is the beauty of FreeBSD USB stack. It exposes ugen driver along with uhid for generic USB HID devices, so that you can use libusb backend out of box.

Unfortunately Hans Petter Selasky passed away recently. If not I would ask him to explain how it works.

> @Youw Does the libusb backend on FreeBSD have to completely disable the HID kernel driver to work? I was considering that in some niche situations it may be good to have the option to use the native kernel interface since devices in some cases will expose multiple HID interfaces. No. That is the beauty of FreeBSD USB stack. It exposes ugen driver along with uhid for generic USB HID devices, so that you can use libusb backend out of box. Unfortunately Hans Petter Selasky [passed away recently](https://forums.freebsd.org/threads/in-memoriam-hans-petter-william-sirevag-selasky.89697/). If not I would ask him to explain how it works.

ghost commented 2023-08-27 05:16:39 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

No. That is the beauty of FreeBSD USB stack. It exposes ugen driver along with uhid for generic USB HID devices, so that you can use libusb backend out of box.

Unfortunately Hans Petter Selasky passed away recently. If not I would ask him to explain how it works.

I see. How unfortunate. I think I will just work on OpenBSD backend and consider uhid work complete. I have an idea for how to make it work. It won't be ideal but it is still an improvement due to how limited libusb on OpenBSD is. Notable limitations would be that it would not be possible to enumerate uhid devices that are already in use.

> No. That is the beauty of FreeBSD USB stack. It exposes ugen driver along with uhid for generic USB HID devices, so that you can use libusb backend out of box. > > Unfortunately Hans Petter Selasky [passed away recently](https://forums.freebsd.org/threads/in-memoriam-hans-petter-william-sirevag-selasky.89697/). If not I would ask him to explain how it works. I see. How unfortunate. I think I will just work on OpenBSD backend and consider `uhid` work complete. I have an idea for how to make it work. It won't be ideal but it is still an improvement due to how limited `libusb` on OpenBSD is. Notable limitations would be that it would not be possible to enumerate `uhid` devices that are already in use.

mcuee commented 2023-08-27 05:22:37 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

I see. How unfortunate. I think I will just work on OpenBSD backend and consider uhid work complete. I have an idea for how to make it work. It won't be ideal but it is still an improvement due to how limited libusb on OpenBSD is. Notable limitations would be that it would not be possible to enumerate uhid devices that are already in use.

Having a uhid backend for OpenBSD will be great. Thanks. As you mentioned, it will still be much better than libusb backend.

In fact I consider HIDAPI libusb backend not so useful (in fact kind of useless) for NetBSD/OpenBSD users.

> I see. How unfortunate. I think I will just work on OpenBSD backend and consider `uhid` work complete. I have an idea for how to make it work. It won't be ideal but it is still an improvement due to how limited `libusb` on OpenBSD is. Notable limitations would be that it would not be possible to enumerate `uhid` devices that are already in use. Having a uhid backend for OpenBSD will be great. Thanks. As you mentioned, it will still be much better than libusb backend. In fact I consider HIDAPI libusb backend not so useful (in fact kind of useless) for NetBSD/OpenBSD users.

mcuee commented 2025-03-27 20:56:31 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

• https://github.com/libusb/hidapi/pull/612 has been merged.

* https://github.com/libusb/hidapi/pull/612 has been merged.

Youw commented 2025-03-28 05:24:35 -05:00 (Migrated from github.com)

• Copy link
• Copy Markdown

Right, closing

Right, closing

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

master

read-interrupt

connection-callback-2

hotplug-async-enumerate-libusb

hotplug-tests-integration

hotplug-async-enumerate-mac

hotplug-tests

hotplug-async-enumerate-windows

hotplug-async-enumerate-linux

hotplug-docs-consistency

archive/20260923T145431Z-rescue/falkTX/linux-dynamic-libudev

gh-pages

archive/20260923T145431Z-rescue/aokblast/use_hid_raw

archive/20260923T145431Z-rescue/sudobash1/libusb_input_size

hotplug-integration

connection-callback

read-interrupt-win-test

archive/20260923T145431Z-rescue/auxcorelabs/feature/input-report-buffer-size

archive/20260923T145431Z-rescue/FrameworkComputer/hid-firmware-ver

archive/20260923T145431Z-rescue/slouken/0001-hidapi-linux-retry-hid_send_feature_report-if-the-io.patch

archive/20260923T145431Z-rescue/slouken/0001-hidapi-libusb-maintain-in-memory-cache-of-vendor-pro.patch

archive/20260923T145431Z-rescue/slouken/0001-hidapi-windows-fixed-PS4-controllers-over-Bluetooth-.patch

winrt

hidapi-0.15.0

hidapi-0.14.0

hidapi-0.13.1

hidapi-0.13.0

hidapi-0.12.0

hidapi-0.12.0-rc2

hidapi-0.12.0-rc1

hidapi-0.11.2

hidapi-0.11.1

hidapi-0.11.0

cmake-support-deprecated

libusb_wrap_device-deprecated

hidapi-0.10.1

hidapi-0.10.0

hidapi-0.9.0

hidapi-0.9.0-rc1

hidapi-0.8.0-rc1

hidapi-0.7.0

hidapi-0.6.0

hidapi-0.5.2

hidapi-0.5.1

hidapi-0.5.0

hidapi-0.3.0

hidapi-0.2.4

hidapi-0.2.3

hidapi-0.2.2

hidapi-0.1.0

Labels

Clear labels   

3rdparty

3rd-party related issue/question

  

Android

Android OS related

  

API

API change, Version 1 stuff

  

Author does not respond

  

bsd

FreeBSD, NetBSD, OpenBSD, etc

  

bug

Something isn't working

  

build system/CI

Anything related to building the project or running on CI

  

ci-virtual-device

Run the virtual HID device CI jobs (Windows driver + libusb raw-gadget)

  

CMake

Related to CMake build system

  

Contributions Welcome

Any external help is welcome

  

Core

Related to common codes like hidapi.h

  

dependencies

Pull requests that update a dependency file

  

device-specific

Issues/PRs related to a specific device

  

documentation

Improvements or additions to documentation

  

don't_merge

Don't merge this PR as is

  

duplicate

This issue or pull request already exists

  

enhancement

New feature or request

  

Example

Related to hidtest or new examples

  

Haiku

Related to Haiku OS (using libusb backend)

  

hidraw

Related to Linux/hidraw backend

  

hotplug

Related to hotplug

  

info required

Need more input from PR/issue author

  

invalid

Not a HIDAPI issue

  

libusb

Related to libusb backend

  

macOS

Related to macOS backend

  

Misc

Typo, whitespace, etc.

  

Not an issue

  

question

Information is requested

  

Release

Related to new release

  

Test

Related to built-in test (not hidtest)

  

Test GUI

Related to a Test GUI application

  

Windows

Related to Windows backend

  

wontfix

This will not be worked on

No labels

3rdparty

Android

API

Author does not respond

bsd

bug

build system/CI

ci-virtual-device

CMake

Contributions Welcome

Core

dependencies

device-specific

documentation

don't_merge

duplicate

enhancement

Example

Haiku

hidraw

hotplug

info required

invalid

libusb

macOS

Misc

Not an issue

question

Release

Test

Test GUI

Windows

wontfix

Milestone

Clear milestone

No items

No milestone

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

gh-libusb/hidapi#608

No description provided.