Add hid_set_num_input_buffers() API #787
No reviewers
Labels
No labels
3rdparty
Android
API
Author does not respond
bsd
bug
build system/CI
ci-virtual-device
CMake
Contributions Welcome
Core
dependencies
device-specific
documentation
don't_merge
duplicate
enhancement
Example
Haiku
hidraw
hotplug
info required
invalid
libusb
macOS
Misc
Not an issue
question
Release
Test
Test GUI
Windows
wontfix
No milestone
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
gh-libusb/hidapi!787
Loading…
Reference in a new issue
No description provided.
Delete branch "auxcorelabs/feature/input-report-buffer-size"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Exposes a new public function
hid_set_num_input_buffers(dev, num_buffers)to resize the per-device input report queue.High-throughput HID devices (medical telemetry, high-poll-rate gaming peripherals, data acquisition hardware) can emit bursts of input reports that exceed the current hardcoded queue sizes (30 on macOS and Linux/libusb, 64 on Windows). When a burst exceeds the queue, reports are silently dropped with no error indication to the caller. Observed on hardware emitting ~90 reports in 200 ms bursts: both the macOS and Linux/libusb backends silently drop ~20% of frames.
Per-backend behavior
HidD_SetNumInputBuffers(parity with existing behavior)@noteon the API declaration inhidapi.h)Safety
[1, HID_API_MAX_NUM_INPUT_BUFFERS]return -1 and register a descriptive error on the device (via the backend'sregister_device_errorhelper where available)dev->mutex/dev->thread_state) as the respective report callbackDesign notes
hid_deviceis opaque inhidapi.h, so struct field additions in backend-private files are not an ABI break.INT_MAX(malicious or buggy). We've measured bursts up to ~90 reports deep in practice; 1024 leaves room for devices ~10× more demanding. Overridable at build time via-DHID_API_MAX_NUM_INPUT_BUFFERS=Nfor memory-constrained targets.hid_get_input_report()instead? That API issues a host-initiatedGET_REPORTrequest (via the control endpoint or OS equivalent). It does not drain the interrupt-endpoint queue that streaming devices fill, and in practice many devices either don't implementGET_REPORTfor input reports or respond incorrectly. The two APIs use different USB transfer mechanisms (interrupt endpoint vs. control endpoint); this PR fixes the interrupt-streaming path.Userspace queue: flat pre-allocated ring buffer
The userspace queue is a fixed-size ring with inline-slot storage:
capacity × sizeof(size_t)bytes) followed by the slot data region (capacity × slot_sizebytes), allocated once at device open. The struct holds three pointers:storageowns the allocation, whilelengthsanddataare non-owning views into the lengths and slot-data regions respectively.kIOHIDMaxInputReportSizeKey(IOKit-parsed HID report descriptor; authoritative)wMaxPacketSize(matches the existing single-packet transfer sizing)memcpy-only (no per-reportmalloc); pop is copy-out (pop_into(r, dst, dst_len)) — the ring owns its storage end-to-end, no borrowed-pointer contract.hidapi_input_ring_resize()allocates new storage, memcpys survivors in FIFO order, frees old, swaps under the caller-held mutex. Shrink-below-count drops oldest, matching push-time policy.core/hidapi_input_ring.h; backend.cfiles include it as a static-in-header helper.tests/test_hidapi_input_ring.c— test cases covering init/destroy idempotency, push/pop FIFO, drop-oldest, varying lengths, resize grow/shrink/wrap/empty/post-wrap/invalid-args, plus a 10k-iteration stress run.References
HidD_SetNumInputBuffersfunctionality to callersAssisted-by: Claude Code:opus-4.6, Claude Code:opus-4.7, Codex:gpt-5.4
@ -1199,6 +1199,17 @@ HID_API_EXPORT const wchar_t * HID_API_CALL hid_read_error(hid_device *dev)return dev->last_read_error_str;remove this check (from all backends) - device functions do not check for validity of the device handle
the only exception is
hid_close@ -1199,6 +1199,17 @@ HID_API_EXPORT const wchar_t * HID_API_CALL hid_read_error(hid_device *dev)return dev->last_read_error_str;too bad we don't have a logging mechanism
this looks like a perfect spot for a warning
making this completely silent might confuse some users
anyway - even though documentation mentiones that there is no user-space queue for linux backend - we should explicitly document that this function for linux and bsd backend is a no-op
remove
@ -955,6 +955,17 @@ HID_API_EXPORT const wchar_t* HID_API_CALL hid_read_error(hid_device *dev)return dev->last_read_error_str;remove
@ -1257,6 +1257,20 @@ HID_API_EXPORT const wchar_t * HID_API_CALL hid_read_error(hid_device *dev)return dev->last_read_error_str;remove
refer to comments, otherwise looks good
@ -1199,6 +1199,17 @@ HID_API_EXPORT const wchar_t * HID_API_CALL hid_read_error(hid_device *dev)return dev->last_read_error_str;Thanks. done.
Thanks. done.
@ -955,6 +955,17 @@ HID_API_EXPORT const wchar_t* HID_API_CALL hid_read_error(hid_device *dev)return dev->last_read_error_str;Thanks done.
@ -1257,6 +1257,20 @@ HID_API_EXPORT const wchar_t * HID_API_CALL hid_read_error(hid_device *dev)return dev->last_read_error_str;Thanks. done.
@ -1199,6 +1199,17 @@ HID_API_EXPORT const wchar_t * HID_API_CALL hid_read_error(hid_device *dev)return dev->last_read_error_str;Have updated the comment to explicitly call out no-op. Thanks
Thanks for reviewing. I have made the changes.
I think the name of the Win32-API
HidD_SetNumInputBuffersdescribes better what it does thanhid_set_input_report_buffer_size, as it does change the number of buffers and not the size of the bufferin bytes.Good callout @JoergAtGithub. Have renamed the function to
hid_set_num_input_buffers()and made the associated changes.@Youw - Please let me know if any further changes required.
Thanks both.
@auxcorelabs
Just wondering if you can enhance the hidtest application to include this new API.
If not, do you have a simple test to share? Thanks.
@ -1199,6 +1199,17 @@ HID_API_EXPORT const wchar_t * HID_API_CALL hid_read_error(hid_device *dev)return dev->last_read_error_str;I meant having this section in the .h as a
NOTEsection.not much point having it here as verbose versino, specifically no point mentioning other backends from within (like mentioning BSD backend inside of the hidraw implementation)
@ -1199,6 +1199,17 @@ HID_API_EXPORT const wchar_t * HID_API_CALL hid_read_error(hid_device *dev)return dev->last_read_error_str;@Youw - have moved this to .h file as a Note section.
@mcuee - have updated the hidtest app to include the new API.
So far so good.
Pull request overview
Adds a new public HIDAPI entry point to let callers increase the per-device input report queue depth, addressing silent report drops during bursty/high-throughput input on backends that queue reports in userspace (macOS + Linux/libusb) and exposing the existing Windows kernel buffering control.
Changes:
hid_set_num_input_buffers(dev, num_buffers)and documents it inhidapi.h(with a max-cap macro).num_input_buffersvalue (default 30).hidtestto exercise the new API when available.Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.
Show a summary per file
hidapi/hidapi.hmac/hid.clibusb/hid.cwindows/hid.cHidD_SetNumInputBuffers.linux/hid.cnetbsd/hid.chidtest/test.c💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
@ -426,0 +433,4 @@-DHID_API_MAX_NUM_INPUT_BUFFERS=<value>.*/#ifndef HID_API_MAX_NUM_INPUT_BUFFERS#define HID_API_MAX_NUM_INPUT_BUFFERS 1024HID_API_MAX_NUM_INPUT_BUFFERSis defined unconditionally, which makes the “overridable via -D…” behavior described in the PR text impossible (command-line defines will be overwritten by this header definition). If the intent is to allow build-time overrides, wrap this in#ifndef HID_API_MAX_NUM_INPUT_BUFFERS(or similar) so downstreams can set the cap without patching the header.@ -426,0 +470,4 @@backend-specific error). Call hid_error(dev) fordetails where supported.*/int HID_API_EXPORT HID_API_CALL hid_set_num_input_buffers(hid_device *dev, int num_buffers);PR description mentions a new public API named
hid_set_input_report_buffer_size(dev, size), but the code addshid_set_num_input_buffers(dev, num_buffers)instead. Please either update the PR description to match the shipped API name, or rename the API in code (including docs) so they are consistent.Same concern as the macOS backend: with
num_input_buffersnow configurable up to 1024, this per-report linked-list tail scan is O(n) and can become a significant CPU cost during bursts. Also, the currentnum_queuedcounting makes the effective cap differ fromdev->num_input_buffers. Consider keeping a tail pointer + queued-count (or a ring buffer) so enqueue/drop is O(1) and the limit matches the API’s meaning.@ -1199,6 +1199,17 @@ HID_API_EXPORT const wchar_t * HID_API_CALL hid_read_error(hid_device *dev)return dev->last_read_error_str;This backend has a
register_device_error()helper for settingdev->last_error_str, but this new API writesdev->last_error_strviaregister_error_str()directly. For consistency (and to keep any future behavior in the helper centralized), consider usingregister_device_error(dev, "num_buffers out of range")here.The queue-length enforcement here becomes both expensive and semantically fuzzy now that callers can raise
num_input_buffersup to 1024. Each report traverses the entire linked list to find the tail (O(n)), which can become a hot path at high report rates, and the currentnum_queuedaccounting means the steady-state queue length is not exactlydev->num_input_buffers. Consider tracking a tail pointer and an explicit queued-count (or switching to a ring buffer) so enqueue/drop is O(1) and the limit corresponds precisely tonum_input_buffers.@ -955,6 +955,17 @@ HID_API_EXPORT const wchar_t* HID_API_CALL hid_read_error(hid_device *dev)return dev->last_read_error_str;Similar to other NetBSD APIs, prefer using the
register_device_error()helper (or the same error helper used elsewhere in this backend) rather than writingdev->last_error_strviaregister_error_str()directly. This keeps error-setting behavior consistent across the file.@ -1199,6 +1199,17 @@ HID_API_EXPORT const wchar_t * HID_API_CALL hid_read_error(hid_device *dev)return dev->last_read_error_str;agree
@ -955,6 +955,17 @@ HID_API_EXPORT const wchar_t* HID_API_CALL hid_read_error(hid_device *dev)return dev->last_read_error_str;agree
@ -426,0 +470,4 @@backend-specific error). Call hid_error(dev) fordetails where supported.*/int HID_API_EXPORT HID_API_CALL hid_set_num_input_buffers(hid_device *dev, int num_buffers);minor: need update PR description
@ -426,0 +433,4 @@-DHID_API_MAX_NUM_INPUT_BUFFERS=<value>.*/#ifndef HID_API_MAX_NUM_INPUT_BUFFERS#define HID_API_MAX_NUM_INPUT_BUFFERS 1024true - either a PR descriptino or code needs to be updated, I don't really have a preference
Someone needs to double-check this, but I'm pretty sure Copilot is right here.
We might even need to consider one step further and switch from linked list to a ring buffer as an optimisation
@ -955,6 +955,17 @@ HID_API_EXPORT const wchar_t* HID_API_CALL hid_read_error(hid_device *dev)return dev->last_read_error_str;Updated.
@ -1199,6 +1199,17 @@ HID_API_EXPORT const wchar_t * HID_API_CALL hid_read_error(hid_device *dev)return dev->last_read_error_str;Updated.
@ -426,0 +470,4 @@backend-specific error). Call hid_error(dev) fordetails where supported.*/int HID_API_EXPORT HID_API_CALL hid_set_num_input_buffers(hid_device *dev, int num_buffers);Updated PR description.
@ -426,0 +433,4 @@-DHID_API_MAX_NUM_INPUT_BUFFERS=<value>.*/#ifndef HID_API_MAX_NUM_INPUT_BUFFERS#define HID_API_MAX_NUM_INPUT_BUFFERS 1024Updated code.
Updated to ring buffer
Updated to ring buffer.
This is an impressive amount of new code contributed in a short amount of time.
What AI tool has been used to generate it? I will not believe it was hand-written.
This was a relatively large change, developed using a structured workflow with human oversight & guidance. The design and approach/choices were planned and reviewed (via AI-assisted passes and manually) before any code change. Two AI models were then used in complementary roles—one for code drafting and the other for iterative review to refine the result. The output went through human review and feedback, and the final changes were manually validated, including building across available platforms to verify there were no errors before committing.
From my experience, the effectiveness comes primarily from the structured workflow and repeated AI review iterations, followed by human review & guidance.
I generally don't mind as I'm using similar tools and aproaches myself in various other projects.
But since HIDAPI is open-source well-know public repo with many users and this is a first huge contribution to HIDAPI using AI-tools, I'd like to mention this explicitly in a form suggested by https://docs.kernel.org/process/coding-assistants.html, specifically the commit message/PR description attribution (at least I'm planning to include it in the final squash-merge commit into master), e.g.:
Assisted-by: AGENT_NAME:MODEL_VERSION [TOOL1] [TOOL2]
that's why I asked what AI tool/model was used to produce this.
I'll make a generic NOTE about this somewhere in the README a bit later.
Thanks for the information. Please find details below:
Assisted-by: Claude Code CLI:claude-opus-4.6
Assisted-by: Claude Code CLI:claude-opus-4.7
Assisted-by: Codex CLI:gpt-5.4
@ -877,48 +867,22 @@ static void hid_report_callback(void *context, IOReturn result, void *sender,(void) report_type;as per thread-safety notes posted on Wiki -
register_device_erroritself is not thread-safe (and not intended to be) and accessing error string from different threads (hid_report_callbackthread in this case) can cause a data race@ -0,0 +1,269 @@/*******************************************************I believe this doesn't have to be a separate allocation either. There could be a single
storageand two non-owning poiinters tolengthsanddataThis looks great, thanks!
I did a quick review, but I want to go over it again what I have a bit more free time than now.
Thanks for the contribution! I'll have it merged some time soon.
@ -877,48 +867,22 @@ static void hid_report_callback(void *context, IOReturn result, void *sender,(void) report_type;Done. Dropped register_device_error() from the hid_report_callback thread.
@ -0,0 +1,269 @@/*******************************************************Done. Updated.
Thanks!
@Youw - just wanted to check whether you’ve had a chance to review this. Thanks.
Somehow the added test is not built under Windows, only libusb and macOS.
@mcuee - The tests/test_hidapi_input_ring.c are unit tests, all of them exercising the ring buffer (hidapi_input_ring.h). The ring buffer only links into mac and libusb backends — Windows wraps HidD_SetNumInputBuffers natively (our hid_set_num_input_buffers is a thin wrapper there). Hence these tests were gated to only libusb and mac.
Since hidtest/test.c builds everywhere, that's where symbol-availability check and a runtime smoke call to hid_set_num_input_buffers were added.
@auxcorelabs
I see. Thanks for the explanation.
View command line instructions
Manual merge helper
Use this merge commit message when completing the merge manually.
Checkout
From your project repository, check out a new branch and test the changes.Merge
Merge the changes and update on Forgejo.Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.