Fix memory corruption in AVX-512 codepath #6

Merged

cgutman merged 1 commit from fix_avx512_oob into master

2024-07-10 21:54:32 -05:00

cgutman commented

2024-07-10 21:09:42 -05:00

(Migrated from github.com)

The AVX-512 codepath incorrectly calculates the end of the caller's buffer using sizeof(__m256i) but the elements being processed are actually sizeof(__m512i).

As a result, reed_solomon_encode() and reed_solomon_decode() may write outside the bounds of the caller's buffer for certain block sizes when compiled with AVX-512 optimizations enabled.

This was originally reported in https://github.com/LizardByte/Sunshine/issues/2543

The AVX-512 codepath incorrectly calculates the end of the caller's buffer using `sizeof(__m256i)` but the elements being processed are actually `sizeof(__m512i)`. As a result, `reed_solomon_encode()` and `reed_solomon_decode()` may write outside the bounds of the caller's buffer for certain block sizes when compiled with AVX-512 optimizations enabled. This was originally reported in https://github.com/LizardByte/Sunshine/issues/2543

sleepybishop (Migrated from github.com) approved these changes

2024-07-10 21:54:27 -05:00

sleepybishop (Migrated from github.com) left a comment