various floating point fixes #2849

Merged
kilograham merged 14 commits from float_fixes into develop 2026-03-06 13:48:11 -06:00
kilograham commented 2026-03-05 15:03:42 -06:00 (Migrated from github.com)
  • fix issues with and improve tests for dcp conversions of INFINITES to fixed point values
  • fixes missing DCP state saves (fix #2793)
- fix issues with and improve tests for dcp conversions of INFINITES to fixed point values - fixes missing DCP state saves (fix #2793)
Wren6991 (Migrated from github.com) reviewed 2026-03-05 15:03:42 -06:00
lurch (Migrated from github.com) reviewed 2026-03-06 08:54:27 -06:00
@ -179,6 +181,15 @@ int test() {
test_checki(double2fix(-0.0, 16), 0, "double2fix13");
lurch (Migrated from github.com) commented 2026-03-06 08:54:27 -06:00
    u64d.u = 0x7fe0000000012345ull;
    test_checki(double2fix(u64d.d, 0), INT32_MAX, "double2fix16a");
    test_checki(double2fix(u64d.d, 1), INT32_MAX, "double2fix16b");
    test_checki(double2fix(u64d.d, 2), INT32_MAX, "double2fix16c");
    u64d.u = 0xffe0000000012345ull;
    test_checki(double2fix(u64d.d, 0), INT32_MIN, "double2fix17a");
    test_checki(double2fix(u64d.d, 1), INT32_MIN, "double2fix17b");
    test_checki(double2fix(u64d.d, 2), INT32_MIN, "double2fix17c");

?

```suggestion u64d.u = 0x7fe0000000012345ull; test_checki(double2fix(u64d.d, 0), INT32_MAX, "double2fix16a"); test_checki(double2fix(u64d.d, 1), INT32_MAX, "double2fix16b"); test_checki(double2fix(u64d.d, 2), INT32_MAX, "double2fix16c"); u64d.u = 0xffe0000000012345ull; test_checki(double2fix(u64d.d, 0), INT32_MIN, "double2fix17a"); test_checki(double2fix(u64d.d, 1), INT32_MIN, "double2fix17b"); test_checki(double2fix(u64d.d, 2), INT32_MIN, "double2fix17c"); ``` ?
lurch (Migrated from github.com) reviewed 2026-03-06 08:54:59 -06:00
@ -189,13 +200,23 @@ int test() {
test_checku(double2ufix(-INFINITY, 16), 0, "double2ufix5b");
lurch (Migrated from github.com) commented 2026-03-06 08:54:58 -06:00
    u64d.u = 0xffe0000000012345ull;
    test_checku(double2ufix(u64d.d, 0), 0, "double2ufix17a");
    test_checku(double2ufix(u64d.d, 1), 0, "double2ufix17b");
    test_checku(double2ufix(u64d.d, 2), 0, "double2ufix17c");
```suggestion u64d.u = 0xffe0000000012345ull; test_checku(double2ufix(u64d.d, 0), 0, "double2ufix17a"); test_checku(double2ufix(u64d.d, 1), 0, "double2ufix17b"); test_checku(double2ufix(u64d.d, 2), 0, "double2ufix17c"); ```
lurch (Migrated from github.com) reviewed 2026-03-06 08:55:32 -06:00
@ -239,6 +262,14 @@ int test() {
test_checki64(double2fix64(u64d.d, 19), -13ll * (1ll << 17) - 1ll, "double2fix6415j");
lurch (Migrated from github.com) commented 2026-03-06 08:55:32 -06:00
    u64d.u = 0xffe0000000012345ull;
    test_checki64(double2fix64(u64d.d, 0), INT64_MIN, "double2fix6417a");
    test_checki64(double2fix64(u64d.d, 1), INT64_MIN, "double2fix6417b");
    test_checki64(double2fix64(u64d.d, 2), INT64_MIN, "double2fix6417c");
```suggestion u64d.u = 0xffe0000000012345ull; test_checki64(double2fix64(u64d.d, 0), INT64_MIN, "double2fix6417a"); test_checki64(double2fix64(u64d.d, 1), INT64_MIN, "double2fix6417b"); test_checki64(double2fix64(u64d.d, 2), INT64_MIN, "double2fix6417c"); ```
lurch (Migrated from github.com) reviewed 2026-03-06 08:56:18 -06:00
@ -250,11 +281,22 @@ int test() {
test_checku64(double2ufix64(-INFINITY, 16), 0, "double2ufix645b");
lurch (Migrated from github.com) commented 2026-03-06 08:56:18 -06:00
    u64d.u = 0xffe0000000012345ull;
    test_checku64(double2ufix64(u64d.d, 0), 0, "double2ufix6412a");
    test_checku64(double2ufix64(u64d.d, 1), 0, "double2ufix6412b");
    test_checku64(double2ufix64(u64d.d, 2), 0, "double2ufix6412c");
```suggestion u64d.u = 0xffe0000000012345ull; test_checku64(double2ufix64(u64d.d, 0), 0, "double2ufix6412a"); test_checku64(double2ufix64(u64d.d, 1), 0, "double2ufix6412b"); test_checku64(double2ufix64(u64d.d, 2), 0, "double2ufix6412c"); ```
lurch (Migrated from github.com) reviewed 2026-03-06 08:57:33 -06:00
@ -275,6 +319,14 @@ int test() {
test_checki(double2fix_z(-0.0, 16), 0, "double2fix_z13");
lurch (Migrated from github.com) commented 2026-03-06 08:57:33 -06:00
    u64d.u = 0xffe0000000012345ull;
    test_checki(double2fix_z(u64d.d, 0), INT32_MIN, "double2fix_z15a");
    test_checki(double2fix_z(u64d.d, 1), INT32_MIN, "double2fix_z15b");
    test_checki(double2fix_z(u64d.d, 2), INT32_MIN, "double2fix_z15c");

(for consistency)

```suggestion u64d.u = 0xffe0000000012345ull; test_checki(double2fix_z(u64d.d, 0), INT32_MIN, "double2fix_z15a"); test_checki(double2fix_z(u64d.d, 1), INT32_MIN, "double2fix_z15b"); test_checki(double2fix_z(u64d.d, 2), INT32_MIN, "double2fix_z15c"); ``` (for consistency)
lurch (Migrated from github.com) reviewed 2026-03-06 08:58:39 -06:00
@ -283,8 +335,10 @@ int test() {
test_checku(double2ufix_z(65536.0, 16), UINT32_MAX, "double2ufix_z4");
lurch (Migrated from github.com) commented 2026-03-06 08:58:39 -06:00
    u64d.u = 0xffe0000000012345ull;
    test_checku(double2ufix_z(u64d.d, 0), 0, "double2ufix_z14a");
    test_checku(double2ufix_z(u64d.d, 1), 0, "double2ufix_z14b");
    test_checku(double2ufix_z(u64d.d, 2), 0, "double2ufix_z14c");
```suggestion u64d.u = 0xffe0000000012345ull; test_checku(double2ufix_z(u64d.d, 0), 0, "double2ufix_z14a"); test_checku(double2ufix_z(u64d.d, 1), 0, "double2ufix_z14b"); test_checku(double2ufix_z(u64d.d, 2), 0, "double2ufix_z14c"); ```
lurch (Migrated from github.com) reviewed 2026-03-06 08:59:11 -06:00
@ -333,6 +397,14 @@ int test() {
test_checki64(double2fix64_z(u64d.d, 19), -13ll * (1ll << 17), "double2fix64_z15j");
lurch (Migrated from github.com) commented 2026-03-06 08:59:11 -06:00
    u64d.u = 0xffe0000000012345ull;
    test_checki64(double2fix64_z(u64d.d, 0), INT64_MIN, "double2fix64_z17a");
    test_checki64(double2fix64_z(u64d.d, 1), INT64_MIN, "double2fix64_z17b");
    test_checki64(double2fix64_z(u64d.d, 2), INT64_MIN, "double2fix64_z17c");
```suggestion u64d.u = 0xffe0000000012345ull; test_checki64(double2fix64_z(u64d.d, 0), INT64_MIN, "double2fix64_z17a"); test_checki64(double2fix64_z(u64d.d, 1), INT64_MIN, "double2fix64_z17b"); test_checki64(double2fix64_z(u64d.d, 2), INT64_MIN, "double2fix64_z17c"); ```
kilograham (Migrated from github.com) reviewed 2026-03-06 09:00:33 -06:00
@ -179,6 +181,15 @@ int test() {
test_checki(double2fix(-0.0, 16), 0, "double2fix13");
kilograham (Migrated from github.com) commented 2026-03-06 09:00:33 -06:00

oops; i got heavy handed in fixing duplicates somewhere

oops; i got heavy handed in fixing duplicates somewhere
lurch (Migrated from github.com) reviewed 2026-03-06 09:00:48 -06:00
@ -342,15 +414,25 @@ int test() {
test_checki64(double2ufix64_z(65536.0 * 65536.0 * 65536.0, 16), UINT64_MAX, "double2ufix64_z4b");
lurch (Migrated from github.com) commented 2026-03-06 09:00:48 -06:00
    test_checku64(double2ufix64_z(3.0, -1), 1, "double2ufix64_z8"); // not very useful
```suggestion test_checku64(double2ufix64_z(3.0, -1), 1, "double2ufix64_z8"); // not very useful ```
lurch (Migrated from github.com) reviewed 2026-03-06 09:01:11 -06:00
@ -301,22 +367,48 @@ int test() {
test_checki64(float2fix64_z(65536.0f, 16), 65536ll << 16, "float2fix64_z4");
lurch (Migrated from github.com) commented 2026-03-06 09:01:11 -06:00
    test_checki64(float2ufix64_z(3.0f, -1), 1, "float2ufix64_z8"); // not very useful
```suggestion test_checki64(float2ufix64_z(3.0f, -1), 1, "float2ufix64_z8"); // not very useful ```
lurch (Migrated from github.com) reviewed 2026-03-06 09:03:10 -06:00
@ -342,15 +414,25 @@ int test() {
test_checki64(double2ufix64_z(65536.0 * 65536.0 * 65536.0, 16), UINT64_MAX, "double2ufix64_z4b");
lurch (Migrated from github.com) commented 2026-03-06 09:03:10 -06:00
    u64d.u = 0xffe0000000012345ull;
    test_checku64(double2ufix64_z(u64d.d, 0), 0, "double2ufix64_z14a");
    test_checku64(double2ufix64_z(u64d.d, 1), 0, "double2ufix64_z14b");
    test_checku64(double2ufix64_z(u64d.d, 2), 0, "double2ufix64_z14c");
```suggestion u64d.u = 0xffe0000000012345ull; test_checku64(double2ufix64_z(u64d.d, 0), 0, "double2ufix64_z14a"); test_checku64(double2ufix64_z(u64d.d, 1), 0, "double2ufix64_z14b"); test_checku64(double2ufix64_z(u64d.d, 2), 0, "double2ufix64_z14c"); ```
lurch (Migrated from github.com) reviewed 2026-03-06 09:05:30 -06:00
@ -216,6 +216,10 @@ int test() {
test_checki(float2fix(-65536.0f, 16), INT32_MIN, "float2fix4b");
lurch (Migrated from github.com) commented 2026-03-06 09:05:29 -06:00
    test_checki(float2fix(u32f.f, 0), INT32_MIN, "float2fix13a");
    test_checki(float2fix(u32f.f, 1), INT32_MIN, "float2fix13b");
    test_checki(float2fix(u32f.f, 2), INT32_MIN, "float2fix13c");
```suggestion test_checki(float2fix(u32f.f, 0), INT32_MIN, "float2fix13a"); test_checki(float2fix(u32f.f, 1), INT32_MIN, "float2fix13b"); test_checki(float2fix(u32f.f, 2), INT32_MIN, "float2fix13c"); ```
lurch (Migrated from github.com) reviewed 2026-03-06 09:06:08 -06:00
@ -216,6 +216,10 @@ int test() {
test_checki(float2fix(-65536.0f, 16), INT32_MIN, "float2fix4b");
lurch (Migrated from github.com) commented 2026-03-06 09:06:08 -06:00
    u32f.u = 0xff012345;
    test_checku(float2ufix(u32f.f, 0), 0, "float2ufix10a");
    test_checku(float2ufix(u32f.f, 1), 0, "float2ufix10b");
    test_checku(float2ufix(u32f.f, 2), 0, "float2ufix10c");
```suggestion u32f.u = 0xff012345; test_checku(float2ufix(u32f.f, 0), 0, "float2ufix10a"); test_checku(float2ufix(u32f.f, 1), 0, "float2ufix10b"); test_checku(float2ufix(u32f.f, 2), 0, "float2ufix10c"); ```
lurch (Migrated from github.com) reviewed 2026-03-06 09:06:39 -06:00
@ -245,11 +266,24 @@ int test() {
test_checki64(float2fix64(2147483648.0f, 16), 2147483648ll << 16, "float2fix644b");
lurch (Migrated from github.com) commented 2026-03-06 09:06:39 -06:00
    u32f.u = 0xff012345;
    test_checki64(float2fix64(u32f.f, 0), INT64_MIN, "float2fix6412a");
    test_checki64(float2fix64(u32f.f, 1), INT64_MIN, "float2fix6412b");
    test_checki64(float2fix64(u32f.f, 2), INT64_MIN, "float2fix6412c");
```suggestion u32f.u = 0xff012345; test_checki64(float2fix64(u32f.f, 0), INT64_MIN, "float2fix6412a"); test_checki64(float2fix64(u32f.f, 1), INT64_MIN, "float2fix6412b"); test_checki64(float2fix64(u32f.f, 2), INT64_MIN, "float2fix6412c"); ```
lurch (Migrated from github.com) reviewed 2026-03-06 09:07:11 -06:00
@ -258,9 +292,24 @@ int test() {
test_checku64(float2ufix64(65536.0f, 16), 65536ull << 16, "float2ufix644");
lurch (Migrated from github.com) commented 2026-03-06 09:07:10 -06:00
    u32f.u = 0xff012345;
    test_checku64(float2ufix64(u32f.f, 0), 0, "float2ufix6410a");
    test_checku64(float2ufix64(u32f.f, 1), 0, "float2ufix6410b");
    test_checku64(float2ufix64(u32f.f, 2), 0, "float2ufix6410c");
```suggestion u32f.u = 0xff012345; test_checku64(float2ufix64(u32f.f, 0), 0, "float2ufix6410a"); test_checku64(float2ufix64(u32f.f, 1), 0, "float2ufix6410b"); test_checku64(float2ufix64(u32f.f, 2), 0, "float2ufix6410c"); ```
lurch (Migrated from github.com) reviewed 2026-03-06 09:07:43 -06:00
@ -269,6 +318,10 @@ int test() {
test_checki(float2fix_z(65536.0f, 16), INT32_MAX, "float2fix_z4");
lurch (Migrated from github.com) commented 2026-03-06 09:07:43 -06:00
    test_checki(float2fix_z(u32f.f, 0), INT32_MIN, "float2fix_z13a");
    test_checki(float2fix_z(u32f.f, 1), INT32_MIN, "float2fix_z13b");
    test_checki(float2fix_z(u32f.f, 2), INT32_MIN, "float2fix_z13c");
```suggestion test_checki(float2fix_z(u32f.f, 0), INT32_MIN, "float2fix_z13a"); test_checki(float2fix_z(u32f.f, 1), INT32_MIN, "float2fix_z13b"); test_checki(float2fix_z(u32f.f, 2), INT32_MIN, "float2fix_z13c"); ```
lurch (Migrated from github.com) reviewed 2026-03-06 09:08:07 -06:00
@ -269,6 +318,10 @@ int test() {
test_checki(float2fix_z(65536.0f, 16), INT32_MAX, "float2fix_z4");
lurch (Migrated from github.com) commented 2026-03-06 09:08:06 -06:00
    test_checku(float2ufix_z(u32f.f, 0), 0, "float2ufix_z10a");
    test_checku(float2ufix_z(u32f.f, 1), 0, "float2ufix_z10b");
    test_checku(float2ufix_z(u32f.f, 2), 0, "float2ufix_z10c");
```suggestion test_checku(float2ufix_z(u32f.f, 0), 0, "float2ufix_z10a"); test_checku(float2ufix_z(u32f.f, 1), 0, "float2ufix_z10b"); test_checku(float2ufix_z(u32f.f, 2), 0, "float2ufix_z10c"); ```
lurch (Migrated from github.com) reviewed 2026-03-06 09:08:31 -06:00
@ -301,22 +367,48 @@ int test() {
test_checki64(float2fix64_z(65536.0f, 16), 65536ll << 16, "float2fix64_z4");
lurch (Migrated from github.com) commented 2026-03-06 09:08:31 -06:00
    u32f.u = 0xff012345;
    test_checki64(float2fix64_z(u32f.f, 0), INT64_MIN, "float2fix64_z12a");
    test_checki64(float2fix64_z(u32f.f, 1), INT64_MIN, "float2fix64_z12b");
    test_checki64(float2fix64_z(u32f.f, 2), INT64_MIN, "float2fix64_z12c");
```suggestion u32f.u = 0xff012345; test_checki64(float2fix64_z(u32f.f, 0), INT64_MIN, "float2fix64_z12a"); test_checki64(float2fix64_z(u32f.f, 1), INT64_MIN, "float2fix64_z12b"); test_checki64(float2fix64_z(u32f.f, 2), INT64_MIN, "float2fix64_z12c"); ```
lurch (Migrated from github.com) reviewed 2026-03-06 09:09:01 -06:00
@ -301,22 +367,48 @@ int test() {
test_checki64(float2fix64_z(65536.0f, 16), 65536ll << 16, "float2fix64_z4");
lurch (Migrated from github.com) commented 2026-03-06 09:09:01 -06:00
    u32f.u = 0xff012345;
    test_checku64(float2ufix64_z(u32f.f, 0), 0, "float2ufix64_z10a");
    test_checku64(float2ufix64_z(u32f.f, 1), 0, "float2ufix64_z10b");
    test_checku64(float2ufix64_z(u32f.f, 2), 0, "float2ufix64_z10c");
```suggestion u32f.u = 0xff012345; test_checku64(float2ufix64_z(u32f.f, 0), 0, "float2ufix64_z10a"); test_checku64(float2ufix64_z(u32f.f, 1), 0, "float2ufix64_z10b"); test_checku64(float2ufix64_z(u32f.f, 2), 0, "float2ufix64_z10c"); ```
lurch (Migrated from github.com) reviewed 2026-03-06 09:20:01 -06:00
@ -301,22 +367,48 @@ int test() {
test_checki64(float2fix64_z(65536.0f, 16), 65536ll << 16, "float2fix64_z4");
lurch (Migrated from github.com) commented 2026-03-06 09:20:00 -06:00

Should the test_checki64 here be test_checku64?

Should the `test_checki64` here be `test_checku64`?
lurch (Migrated from github.com) reviewed 2026-03-06 09:21:57 -06:00
@ -250,11 +281,22 @@ int test() {
test_checku64(double2ufix64(-INFINITY, 16), 0, "double2ufix645b");
lurch (Migrated from github.com) commented 2026-03-06 09:21:57 -06:00

Should these two lines be using test_checku64?

Should these two lines be using `test_checku64`?
kilograham (Migrated from github.com) reviewed 2026-03-06 13:43:19 -06:00
@ -250,11 +281,22 @@ int test() {
test_checku64(double2ufix64(-INFINITY, 16), 0, "double2ufix645b");
kilograham (Migrated from github.com) commented 2026-03-06 13:43:19 -06:00
    test_checku64(double2ufix64(0.0, 16), 0, "double2ufix649");
```suggestion test_checku64(double2ufix64(0.0, 16), 0, "double2ufix649"); ```
kilograham (Migrated from github.com) reviewed 2026-03-06 13:43:44 -06:00
@ -250,11 +281,22 @@ int test() {
test_checku64(double2ufix64(-INFINITY, 16), 0, "double2ufix645b");
kilograham (Migrated from github.com) commented 2026-03-06 13:43:44 -06:00
    test_checku64(double2ufix64(-0.0, 16), 0, "double2ufix6410");
```suggestion test_checku64(double2ufix64(-0.0, 16), 0, "double2ufix6410"); ```
kilograham (Migrated from github.com) reviewed 2026-03-06 13:44:05 -06:00
@ -301,22 +367,48 @@ int test() {
test_checki64(float2fix64_z(65536.0f, 16), 65536ll << 16, "float2fix64_z4");
kilograham (Migrated from github.com) commented 2026-03-06 13:44:05 -06:00
    test_checku64(float2ufix64_z(3.0f, -1), 1, "float2ufix64_z8"); // not very useful
```suggestion test_checku64(float2ufix64_z(3.0f, -1), 1, "float2ufix64_z8"); // not very useful ```
Sign in to join this conversation.
No reviewers
No milestone
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
gh-raspberrypi/pico-sdk!2849
No description provided.