Faster conversion to String

jwilm commented

2018-04-16 11:35:01 -05:00

(Migrated from github.com)

Hello, I had a need for much faster printing of UUIDs, and I created a library uuidtoa to prototype and use immediately in another project. I would like to send a PR to include the improved write speed here, but a couple of questions first.

Is the uuid project interested?
Not going through the format! machinery is a requirement for me. Would you be open to adding a public API like what's used in uuidtoa in addition to the fmt::Display way of printing currently included in uuid?

Thanks!

Hello, I had a need for much faster printing of UUIDs, and I created a library [uuidtoa](https://github.com/jwilm/uuidtoa) to prototype and use immediately in another project. I would like to send a PR to include the improved write speed here, but a couple of questions first. 1. Is the uuid project interested? 2. Not going through the format! machinery is a requirement for me. Would you be open to adding a public API like what's used in uuidtoa _in addition_ to the `fmt::Display` way of printing currently included in uuid? Thanks!

kinggoesgaming commented

2018-04-16 22:58:47 -05:00

(Migrated from github.com)

If we decide to go ahead with this... I would suggest waiting for about a month (currently working a whole set of breaking changes, and holding up on new features)

KodrAus commented

2018-04-25 17:06:34 -05:00

(Migrated from github.com)

Thanks for building this @jwilm! I'm on board with pulling it in and improving our Display implementation as well as offering an alternative lightweight way to write out UUIDs. Maybe we could add a fmt module that includes the write_* methods?

Thanks for building this @jwilm! I'm on board with pulling it in and improving our `Display` implementation as well as offering an alternative lightweight way to write out UUIDs. Maybe we could add a `fmt` module that includes the `write_*` methods?

kinggoesgaming commented

2018-04-25 17:28:27 -05:00

(Migrated from github.com)

for the record me and @Dylan-DPC are onboard for this aswell...

@jwilm mind joining our gitter to discuss this?

for the record me and @Dylan-DPC are onboard for this aswell... @jwilm mind joining our [gitter](https://gitter.im/uuid-rs/Lobby) to discuss this?

kinggoesgaming commented

2018-05-12 23:17:50 -05:00

(Migrated from github.com)

@jwilm can start this for Uuid type?

I still haven't managed to refactor the adapter types yet, so we can leave them for a later date

@jwilm can start this for `Uuid` type? I still haven't managed to refactor the adapter types yet, so we can leave them for a later date

Dylan-DPC-zz commented

2018-05-31 03:05:56 -05:00

(Migrated from github.com)

@jwilm any updates on this?

jwilm commented

2018-05-31 12:30:28 -05:00

(Migrated from github.com)

I won't personally have time to do this for a while, but you are welcome to pull in the code.

Dylan-DPC-zz commented

2018-05-31 12:44:30 -05:00

(Migrated from github.com)

Uhmm no issues. We still have time for the next major release (0.7)

jwilm commented

2018-05-31 12:49:06 -05:00

(Migrated from github.com)

The format changes won't have breakage anyhow. The only API difference would be the addition of a fmt module.

The format changes won't have breakage anyhow. The only API difference would be the _addition_ of a `fmt` module.

kinggoesgaming commented

2018-05-31 13:17:18 -05:00

(Migrated from github.com)

At this point, we have frozen 0.6.x cycle sans bug fixes... It's just keep the load light on us... :)

At this point, we have frozen `0.6.x` cycle sans bug fixes... It's just keep the load light on us... :)

kinggoesgaming commented

2018-08-31 15:42:19 -05:00

(Migrated from github.com)

@huonw I think your PR fixes this right? we just don't have to have separate functions anymore for the similar speedups

huonw commented

2018-08-31 16:27:13 -05:00

(Migrated from github.com)

It probably puts uuid similar to uuidtoa, but I suspect uuidtoa will still be slightly faster, and the uuid code still goes through the fmt machinery. Maybe @jwilm could compare the performance again.

It probably puts uuid similar to uuidtoa, but I suspect uuidtoa will still be slightly faster, and the uuid code still goes through the `fmt` machinery. Maybe @jwilm could compare the performance again.

kinggoesgaming commented

2018-08-31 16:37:15 -05:00

(Migrated from github.com)

for reference: PR #303

available in uuid@0.7.0-beta and onwards

for reference: PR #303 available in `uuid@0.7.0-beta` and onwards

huonw commented

2018-09-02 22:42:43 -05:00

(Migrated from github.com)

I tried benchmarking it myself, using github.com/huonw/uuid@e8504e4f5f.

Results without LTO:

running 4 tests
test bench_hyphen  ... bench:          49 ns/iter (+/- 4)
test bench_simple  ... bench:          40 ns/iter (+/- 4)
test bench_urn     ... bench:          56 ns/iter (+/- 3)
test bench_uuidtoa ... bench:          66 ns/iter (+/- 7)

test result: ok. 0 passed; 0 failed; 0 ignored; 4 measured; 0 filtered out

Results with LTO:

running 4 tests
test bench_hyphen  ... bench:          42 ns/iter (+/- 6)
test bench_simple  ... bench:          34 ns/iter (+/- 2)
test bench_urn     ... bench:          47 ns/iter (+/- 8)
test bench_uuidtoa ... bench:          14 ns/iter (+/- 1)

test result: ok. 0 passed; 0 failed; 0 ignored; 4 measured; 0 filtered out

That is, in the full-optimisations case, there's ~30ns of overhead using uuid directly (maybe 5ns of this is due to using Formatter::write_str and thus calling str::from_utf8 instead of str::from_utf8_unchecked).

One possibility for an API might be something like along the lines of https://doc.rust-lang.org/std/primitive.char.html#method.encode_utf8 e.g.

impl Hyphenated {
    fn encode_lower(&self, v: &mut [u8]) -> &mut str { ... }
    fn encode_upper(&self, v: &mut [u8]) -> &mut str { ... }
}

where the v must have length at least 36 (or 32, or 45, depending on the adapter). This can then be (somewhat) fluently used for writing to an intermediate buffer:

let mut buf: &mut [u8] = ...;

let written = uuid.to_hyphenated().encode_lower(&mut buf).len();
buf = &mut buf[written..];

And it can also be used naturally if the str encoded value is all that's useful:

fn foo(s: &str) { ... }
foo(uuid.to_hyphenated().encode_lower(&mut [0; 36]))

This also avoids double buffering: unlike both uuid and uuidtoa, this new form doesn't need to create its own [u8; N] buffer internally that it then copies into the real output.

Variations/additions:

take a flag for upper/lower as a parameter instead of two separate functions
provide a function for creating the buffer, e.g. Uuid::encode_buffer() -> [u8; 45], so the latter case can be foo(uuid.to_hyphenated().encode_lower(&mut Uuid::encode_buffer())) without the programmer having think about the size of a printed UUID
offer encode_... on Uuid itself, probably using the hyphenated form (this would probably best be in addition to the others, meaning the common case is uuid.encode_lower(&mut [0; 36]), but the simple and urn forms are still easily accessible).

Thoughts?

char - Rust
API documentation for the Rust `char` primitive in crate `std`.

I tried benchmarking it myself, using https://github.com/huonw/uuid/commit/e8504e4f5f6ccf606b9cb134431c1ed1dd625e84. Results without LTO: ```txt running 4 tests test bench_hyphen ... bench: 49 ns/iter (+/- 4) test bench_simple ... bench: 40 ns/iter (+/- 4) test bench_urn ... bench: 56 ns/iter (+/- 3) test bench_uuidtoa ... bench: 66 ns/iter (+/- 7) test result: ok. 0 passed; 0 failed; 0 ignored; 4 measured; 0 filtered out ``` Results with LTO: ```txt running 4 tests test bench_hyphen ... bench: 42 ns/iter (+/- 6) test bench_simple ... bench: 34 ns/iter (+/- 2) test bench_urn ... bench: 47 ns/iter (+/- 8) test bench_uuidtoa ... bench: 14 ns/iter (+/- 1) test result: ok. 0 passed; 0 failed; 0 ignored; 4 measured; 0 filtered out ``` That is, in the full-optimisations case, there's ~30ns of overhead using `uuid` directly (maybe 5ns of this is due to using `Formatter::write_str` and thus calling `str::from_utf8` instead of `str::from_utf8_unchecked`). --- One possibility for an API might be something like along the lines of https://doc.rust-lang.org/std/primitive.char.html#method.encode_utf8 e.g. ```rust impl Hyphenated { fn encode_lower(&self, v: &mut [u8]) -> &mut str { ... } fn encode_upper(&self, v: &mut [u8]) -> &mut str { ... } } ``` where the `v` must have length at least 36 (or 32, or 45, depending on the adapter). This can then be (somewhat) fluently used for writing to an intermediate buffer: ```rust let mut buf: &mut [u8] = ...; let written = uuid.to_hyphenated().encode_lower(&mut buf).len(); buf = &mut buf[written..]; ``` And it can also be used naturally if the `str` encoded value is all that's useful: ```rust fn foo(s: &str) { ... } foo(uuid.to_hyphenated().encode_lower(&mut [0; 36])) ``` This also avoids double buffering: unlike both `uuid` and `uuidtoa`, this new form doesn't need to create its own `[u8; N]` buffer internally that it then copies into the real output. Variations/additions: - take a flag for upper/lower as a parameter instead of two separate functions - provide a function for creating the buffer, e.g. `Uuid::encode_buffer() -> [u8; 45]`, so the latter case can be `foo(uuid.to_hyphenated().encode_lower(&mut Uuid::encode_buffer()))` without the programmer having think about the size of a printed UUID - offer `encode_...` on `Uuid` itself, probably using the hyphenated form (this would probably best be in addition to the others, meaning the common case is `uuid.encode_lower(&mut [0; 36])`, but the simple and urn forms are still easily accessible). Thoughts? <blockquote><img src="https://doc.rust-lang.org/favicon.ico" width="48" align="right"><div><strong><a href="https://doc.rust-lang.org/std/primitive.char.html#method.encode_utf8">char - Rust</a></strong></div><div>API documentation for the Rust `char` primitive in crate `std`.</div></blockquote>

Rows
Columns

Faster conversion to String #196