gh-Cyan4973/xxHash
Watch
1
0
Fork
You've already forked xxHash
 0
Code Issues 7 Pull requests 3 Releases 26 Wiki Activity

Does not accept unicode file path #293

New issue
Closed
opened 2020-01-22 04:25:45 -06:00 by KonstantinMogilev · 36 comments
KonstantinMogilev commented 2020-01-22 04:25:45 -06:00 (Migrated from github.com)

Does not accept unicode file path, for example:

xxhsum.exe "еÐÌ.pdf"
Error: Could not open '?╡?I.pdf': Invalid argument.
Does not accept unicode file path, for example: ``` xxhsum.exe "еÐÌ.pdf" Error: Could not open '?╡?I.pdf': Invalid argument. ```
👍 1
Cyan4973 commented 2020-01-22 04:35:28 -06:00 (Migrated from github.com)

Unfortunately, the code handling file names does nothing special, and relies entirely on standard C fopen().

I have no idea how fopen() behaves with unicode characters. This may be system dependent.

I presume the issue happens on Windows ?

Unfortunately, the code handling file names does nothing special, and relies entirely on standard C `fopen()`. I have no idea how `fopen()` behaves with unicode characters. This may be system dependent. I presume the issue happens on Windows ?
KonstantinMogilev commented 2020-01-22 04:47:58 -06:00 (Migrated from github.com)

Yes, the problem is in Windows, checked on 2 systems:
Windows Server 2008 R2
Windows 7
Maybe fopen () has options for working correctly with unicode, or use another function that can correctly handle such names?

Yes, the problem is in Windows, checked on 2 systems: Windows Server 2008 R2 Windows 7 Maybe fopen () has options for working correctly with unicode, or use another function that can correctly handle such names?
Cyan4973 commented 2020-01-22 05:18:39 -06:00 (Migrated from github.com)

How was the binary produced ?
Did you compiled it using Visual Studio ?

How was the binary produced ? Did you compiled it using Visual Studio ?
KonstantinMogilev commented 2020-01-22 05:25:19 -06:00 (Migrated from github.com)

I used your latest compiled version from here: https://github.com/Cyan4973/xxHash/releases/download/v0.7.2/xxhsum_win64_0_7_2_avx2.zip

I used your latest compiled version from here: https://github.com/Cyan4973/xxHash/releases/download/v0.7.2/xxhsum_win64_0_7_2_avx2.zip
👍 1
easyaspi314 commented 2020-01-31 19:52:48 -06:00 (Migrated from github.com)

Oof, this will be fun.

fopen on Windows uses ANSI encoding, while files are stored in UTF-16.

I might take a look into this when I get a chance. It might be as simple as _wfopen with a mbtowc, but maybe not.

Oof, this will be fun. `fopen` on Windows uses ANSI encoding, while files are stored in UTF-16. I might take a look into this when I get a chance. It might be as simple as `_wfopen` with a `mbtowc`, but maybe not.
t-mat commented 2020-02-05 15:45:52 -06:00 (Migrated from github.com)

Hi, I've made small code which introduces UTF-8 compatibility for Windows+MSVC build.
@easyaspi314 since I don't test this code much, I don't send PR so far. But hope it helps.

Hi, I've made [small code](https://github.com/t-mat/xxHash/commit/56914231a8c12b04cd7ee241cce95bd43ef889b3) which introduces UTF-8 compatibility for Windows+MSVC build. @easyaspi314 since I don't test this code much, I don't send PR so far. But hope it helps.
t-mat commented 2020-02-05 16:34:21 -06:00 (Migrated from github.com)

Here's some notes.

(1) We can safely use UTF-8 to represent Unicode filename for modern systems (except Windows).

I have no idea how fopen() behaves with unicode characters. This may be system dependent.

Nowadays as for modern Linux, *NIX and OS X systems, basically we can say all possible filenames are sequence of chars. So in these systems, fopen() is fine for Unicode filenames.
Some OS X file system/API has Unicode normalization issues and it may cause issue of interoperability of xxhsum -c. But I think we can address this problem later.

(2) I believe "UTF-8 Everywhere" is good approach.
For xxhsum, I mean that almost all part of program should think about and be written for UTF-8. And only input/output functions are taking care about other encodings.

(3) I'd like to avoid to use low level WIN32 API such as GetCommandLineW() or CommandLineToArgvW().
For example, Microsoft's ucrt has wildcard expansion in its startup routine (see argv_parsing.cpp in Windows Kits). To support these functions, it'd be nice to avoid to use low level APIs.

(4) New entry point should be wmain().
Since mapping between Unicode and MBCS isn't perfect, MBCS may introduce other problem. So, I strongly recommend to use Unicode directly.
One possible idea is that redefine main as main_utf8 by macro. wmain() receives UTF-16 argv[], convert them to UTF-8 and passes UTF-8 argv[] to main_utf8().

(5) For stdout and stderr, _setmode(x, _O_U8TEXT) looks promising.
But according to the document, there're many restrictions. One of the biggest restriction is:

Caution
Unicode mode is for wide print functions (for example, wprintf) and is not supported for narrow print functions. Use of a narrow print function on a Unicode mode stream triggers an assert.

Here, "Unicode mode" includes UTF-8 (_O_U8TEXT). Therefore, we must use wprintf() family which only accepts UTF-16 string. So, we must convert UTF-8 to UTF-16 to output UTF-8!

Other restriction is it doesn't allow to use narrow print functions such as printf(). If we use narrow print function, program crashes immediately. For xxhsum it means we should keep using DISPLAY() macro family.

(6) Windows Powershell has problematic default encoding method for redirection.
You can't redirect UTF-8 encoding stdout/stderr with default settings of the Windows PowerShell. Please read this Q&A at StackOverflow: Displaying Unicode in Powershell.
As this answer shows, you can use UTF-8 encoding in redirection by the following command:

$OutputEncoding = [console]::InputEncoding = [console]::OutputEncoding = New-Object System.Text.UTF8Encoding
Here's some notes. (1) We can safely use UTF-8 to represent Unicode filename for modern systems (except Windows). > I have no idea how fopen() behaves with unicode characters. This may be system dependent. Nowadays as for modern Linux, *NIX and OS X systems, basically we can say all possible filenames are sequence of `char`s. So in these systems, `fopen()` is fine for Unicode filenames. Some OS X file system/API has [Unicode normalization issues](https://en.wikipedia.org/wiki/Filename#Unicode_interoperability) and it may cause issue of interoperability of `xxhsum -c`. But I think we can address this problem later. (2) I believe "[UTF-8 Everywhere](http://utf8everywhere.org/)" is good approach. For `xxhsum`, I mean that almost all part of program should think about and be written for UTF-8. And only input/output functions are taking care about other encodings. (3) I'd like to avoid to use low level WIN32 API such as `GetCommandLineW()` or `CommandLineToArgvW()`. For example, Microsoft's ucrt has wildcard expansion in its startup routine (see `argv_parsing.cpp` in Windows Kits). To support these functions, it'd be nice to avoid to use low level APIs. (4) New entry point should be `wmain()`. Since mapping between Unicode and MBCS isn't perfect, MBCS may introduce other problem. So, I strongly recommend to use Unicode directly. One possible idea is that redefine `main` as `main_utf8` by macro. `wmain()` receives UTF-16 `argv[]`, convert them to UTF-8 and passes UTF-8 `argv[]` to `main_utf8()`. (5) For stdout and stderr, `_setmode(x, _O_U8TEXT)` looks promising. But [according to the document](https://docs.microsoft.com/cpp/c-runtime-library/reference/setmode), there're many restrictions. One of the biggest restriction is: > Caution > Unicode mode is for wide print functions (for example, wprintf) and is not supported for narrow print functions. Use of a narrow print function on a Unicode mode stream triggers an assert. Here, "Unicode mode" includes UTF-8 (`_O_U8TEXT`). Therefore, we must use `wprintf()` family which only accepts UTF-16 string. So, we must convert UTF-8 to UTF-16 to output UTF-8! Other restriction is it doesn't allow to use narrow print functions such as `printf()`. If we use narrow print function, program crashes immediately. For `xxhsum` it means we should keep using `DISPLAY()` macro family. (6) Windows Powershell has problematic default encoding method for redirection. You can't redirect UTF-8 encoding stdout/stderr with default settings of the Windows PowerShell. Please read this Q&A at StackOverflow: [Displaying Unicode in Powershell](https://stackoverflow.com/a/49481797/2132223). As this answer shows, you can use UTF-8 encoding in redirection by the following command: ```Powershell $OutputEncoding = [console]::InputEncoding = [console]::OutputEncoding = New-Object System.Text.UTF8Encoding ```
👍 2
flawrist commented 2020-02-11 19:48:29 -06:00 (Migrated from github.com)

Windows 7, CHCP set to 65001.
0.7.0 outputs привет.txt, whereas 0.7.1+ outputs ������.txt

Windows 7, CHCP set to 65001. [0.7.0](https://github.com/Cyan4973/xxHash/files/4189514/xxhsum070sse2.zip) outputs привет.txt, whereas 0.7.1+ outputs ������.txt
easyaspi314 commented 2020-02-12 12:26:11 -06:00 (Migrated from github.com)

0.7.0 doesn't seem to work, at least not in MinGW64:

$ ./xxhsum "ファイル.txt"
Error: Could not open '????.txt': Invalid argument.
0.7.0 doesn't seem to work, at least not in MinGW64: ``` $ ./xxhsum "ファイル.txt" Error: Could not open '????.txt': Invalid argument. ```
easyaspi314 commented 2020-02-12 13:25:42 -06:00 (Migrated from github.com)

wmain causes a linker error on MinGW, but I think I found a solution.

There has to be a better way to do this, though.

static int XXH_main(int argc, char **argv)
{
   ...
}

int main(int argc, char **argv)
{
    int result;
    /* We need to get the real argv on Windows for Unicode. */
#ifdef _WIN32
    const int oldStdoutMode = _setmode(_fileno(stdout), _O_U8TEXT);
    const int oldStderrMode = _setmode(_fileno(stderr), _O_U8TEXT);
    wchar_t **utf16_argv = CommandLineToArgvW(GetCommandLineW(), &argc);
    argv = createUtf8Argv(argc, utf16_argv);
#endif /* _WIN32 */

    result = XXH_main(argc, argv);

    /* Cleanup */
#ifdef _WIN32
    freeUtf8Argv(argc, argv);
    fflush(stdout); _setmode(_fileno(stdout), oldStdoutMode);
    fflush(stderr); _setmode(_fileno(stderr), oldStderrMode);
#endif /* _WIN32 */
    return result;
}
`wmain` causes a linker error on MinGW, but I think I found a solution. There has to be a better way to do this, though. ```c static int XXH_main(int argc, char **argv) { ... } int main(int argc, char **argv) { int result; /* We need to get the real argv on Windows for Unicode. */ #ifdef _WIN32   const int oldStdoutMode = _setmode(_fileno(stdout), _O_U8TEXT);   const int oldStderrMode = _setmode(_fileno(stderr), _O_U8TEXT);   wchar_t **utf16_argv = CommandLineToArgvW(GetCommandLineW(), &argc); argv = createUtf8Argv(argc, utf16_argv); #endif /* _WIN32 */   result = XXH_main(argc, argv); /* Cleanup */ #ifdef _WIN32   freeUtf8Argv(argc, argv);   fflush(stdout); _setmode(_fileno(stdout), oldStdoutMode);   fflush(stderr); _setmode(_fileno(stderr), oldStderrMode); #endif /* _WIN32 */ return result; } ```
👍 1
easyaspi314 commented 2020-02-24 12:20:14 -06:00 (Migrated from github.com)

This should be fixed by #304.

This should be fixed by #304.
👍 1
flawrist commented 2020-02-25 01:27:21 -06:00 (Migrated from github.com)

@easyaspi314, err… where is a Windows binary to verify the fix?

@easyaspi314, err… where is a Windows binary to verify the fix?
easyaspi314 commented 2020-02-25 12:21:54 -06:00 (Migrated from github.com)

Here are a few binaries for the people who are too lazy to compile their own code (jk):

  • MSVC 2019 i686
  • MSVC 2019 x64
  • GCC 8.2.0 i686 (classic MinGW32)
  • GCC 9.2.0 i686 (Mingw-w64 via MSYS2)
  • GCC 9.2.0 x64 (Mingw-w64 via MSYS2)

xxhsum-windows-binaries-eba72be9.zip

Here are a few binaries for the people who are too lazy to compile their own code (jk): - MSVC 2019 i686 - MSVC 2019 x64 - GCC 8.2.0 i686 (classic MinGW32) - GCC 9.2.0 i686 (Mingw-w64 via MSYS2) - GCC 9.2.0 x64 (Mingw-w64 via MSYS2) [xxhsum-windows-binaries-eba72be9.zip](https://github.com/Cyan4973/xxHash/files/4251432/xxhsum-windows-binaries-eba72be9.zip)
easyaspi314 commented 2020-02-25 12:32:29 -06:00 (Migrated from github.com)

At least on my PC, Windows 10, they all pass the Unicode test I made, even if the terminal/font cannot display it:

Testing filename provided on command line...
xxhsum.exe "ユニコード.unicode"
2d7f1808da1fa63c  ユニコード.unicode
Testing a checksum file...
xxhsum.exe -c unicode_test.xxh64
ユニコード.unicode: OK
At least on my PC, Windows 10, they all pass the Unicode test I made, even if the terminal/font cannot display it: ``` Testing filename provided on command line... xxhsum.exe "ユニコード.unicode" 2d7f1808da1fa63c ユニコード.unicode Testing a checksum file... xxhsum.exe -c unicode_test.xxh64 ユニコード.unicode: OK ```
easyaspi314 commented 2020-02-25 16:39:46 -06:00 (Migrated from github.com)

@KonstantinMogilev @t-mat @sergeevabc When you get a chance, can you check to make sure this works on your machines?

@KonstantinMogilev @t-mat @sergeevabc When you get a chance, can you check to make sure this works on your machines?
flawrist commented 2020-02-26 07:12:46 -06:00 (Migrated from github.com)

@easyaspi314, that version is still better as its output is consistent no matter what code page is, i.e.

chcp 65001 
xxhsum070sse2.exe привет.txt
a2a5260369b4da80  привет.txt
chcp 1251
xxhsum070sse2.exe привет.txt
a2a5260369b4da80  привет.txt

whereas your approach outputs

chcp 65001 
xxhsum-msvc-x64.exe привет.txt
a2a5260369b4da80    привет.txt
chcp 1251
xxhsum-msvc-x64.exe привет.txt
a2a5260369b4da80    привет.txt
@easyaspi314, [that version][1] is still better as its output is consistent no matter what code page is, i.e. ``` chcp 65001 xxhsum070sse2.exe привет.txt a2a5260369b4da80 привет.txt chcp 1251 xxhsum070sse2.exe привет.txt a2a5260369b4da80 привет.txt ``` whereas your approach outputs ``` chcp 65001 xxhsum-msvc-x64.exe привет.txt a2a5260369b4da80 привет.txt chcp 1251 xxhsum-msvc-x64.exe привет.txt a2a5260369b4da80 привет.txt ``` [1]: https://github.com/Cyan4973/xxHash/issues/293#issuecomment-584976693
😕 1
easyaspi314 commented 2020-02-26 09:04:38 -06:00 (Migrated from github.com)

I honestly don't understand why 0.7.0 works — AFAICT, nothing changed that would cause it to work, and also, I am pretty sure I tried compiling 0.7.0 and it did not work.

I honestly don't understand why 0.7.0 works — AFAICT, nothing changed that would cause it to work, and also, I am pretty sure I tried compiling 0.7.0 and it did not work.
easyaspi314 commented 2020-03-02 17:44:35 -06:00 (Migrated from github.com)

OK, I think I see the difference.

I analyzed the binary, and it looks like the 0.7.0 binary was compiled by MSYS2 instead of MinGW or MSVC.

MSYS2 and Cygwin both use a wrapper which handles everything UTF-8 automagically.

OK, I think I see the difference. I analyzed the binary, and it looks like the 0.7.0 binary was compiled by MSYS2 instead of MinGW or MSVC. MSYS2 and Cygwin both use a wrapper which handles everything UTF-8 automagically.
👍 1
Cyan4973 commented 2020-03-02 17:47:43 -06:00 (Migrated from github.com)

Well, strangely, I thought that all Windows binaries provided in the release section were compiled using mingw64 from msys2 environment. Maybe I missed or confused something...

Well, strangely, I thought that _all_ Windows binaries provided in the _release_ section were compiled using `mingw64` from `msys2` environment. Maybe I missed or confused something...
easyaspi314 commented 2020-03-02 17:57:39 -06:00 (Migrated from github.com)

The difference between MinGW and MSYS2/Cygwin is that MinGW basically compiles entirely against the MSVC CRT and acts like Windows, but MSYS2/Cygwin try to emulate a POSIX system, which requires many wrappers. It basically hijacks MSVC CRT and replaces it with its own Newlib libc.

Command line conversion is done here:
github.com/msys2/Cygwin@bede85ba6d/winsup/cygwin/dcrt0.cc (L947)

The difference between MinGW and MSYS2/Cygwin is that MinGW basically compiles entirely against the MSVC CRT and acts like Windows, but MSYS2/Cygwin try to emulate a POSIX system, which requires many wrappers. It basically hijacks MSVC CRT and replaces it with its own Newlib libc. Command line conversion is done here: https://github.com/msys2/Cygwin/blob/bede85ba6d90b9383f3c83a6e99152284ca90f6a/winsup/cygwin/dcrt0.cc#L947
Cyan4973 commented 2020-03-02 18:00:49 -06:00 (Migrated from github.com)

Btw, I don't see a v0.7.0 Windows binary in the release section ...

Btw, I don't see a `v0.7.0` Windows binary in the _release_ section ...
😕 1
easyaspi314 commented 2020-03-02 18:04:07 -06:00 (Migrated from github.com)

Weird. Maybe it was a test binary.

Weird. Maybe it was a test binary.
Cyan4973 commented 2020-03-02 18:07:48 -06:00 (Migrated from github.com)

Anyway, is there a preferred environment for the production of next version of Windows binaries ?

Anyway, is there a preferred environment for the production of next version of Windows binaries ?
easyaspi314 commented 2020-03-03 20:00:30 -06:00 (Migrated from github.com)

xxhsum-i686-mingw32-test.zip

New test based more off of @t-mat's code

Compiled on classic MinGW-w32, which links directly to msvcrt.dll which is system specific.

@sergeevabc can you please test this on your Windows 7 machine?

Also, if anyone has XP installed for some reason, I'd like to know if it works too, out of curiosity.

[xxhsum-i686-mingw32-test.zip](https://github.com/Cyan4973/xxHash/files/4284632/xxhsum-i686-mingw32-test.zip) New test based more off of @t-mat's code Compiled on classic MinGW-w32, which links directly to `msvcrt.dll` which *is* system specific. @sergeevabc can you please test this on your Windows 7 machine? Also, if anyone has XP installed for some reason, I'd like to know if it works too, out of curiosity.
easyaspi314 commented 2020-03-04 10:51:54 -06:00 (Migrated from github.com)

@Cyan4973 @KonstantinMogilev This should be completely solved by #323.

@Cyan4973 @KonstantinMogilev This should be completely solved by #323.
👍 2
easyaspi314 commented 2020-03-04 17:04:47 -06:00 (Migrated from github.com)

Anyway, is there a preferred environment for the production of next version of Windows binaries ?

@Cyan4973 Uh, well MSVC sucks at XXH3, so I'm guessing either clang-cl or mingw-w64.

> Anyway, is there a preferred environment for the production of next version of Windows binaries ? @Cyan4973 Uh, well MSVC sucks ~~at XXH3~~, so I'm guessing either `clang-cl` or `mingw-w64`.
👍 1
flawrist commented 2020-03-04 22:11:07 -06:00 (Migrated from github.com)

@easyaspi314, about XP… it works here, but in a peculiar way.

@easyaspi314, about XP… it works here, but in a peculiar way. [![](https://i4.imageban.ru/out/2020/03/05/f8d29d0a649a6010205195d208b1d99f.gif)][1] [1]: https://ipfstube.erindachtler.me/v/QmcJcBRiYEFZxkSrLiHk7X4AMuLbQijArM4m7HPmMC6UDz
👀 2
easyaspi314 commented 2020-03-04 23:48:32 -06:00 (Migrated from github.com)

Thanks!

That is actually a crash. On most Windows terminals, when a console program crashes, the last line is blanked out and the program exits silently. If you watch in slow motion, the first bit of the hash is displayed:
Screenshot_20200305-005043

I'd kill for the helpful Segmentation fault or Aborted messages on Linux, or at least an error message box.

This blankout doesn't happen when redirecting the output to a file, which is why you see that.

We don't need to fully support XP. We didn't need to since 2014.

But the important thing is that ASCII works.

I would be more concerned if the program crashed immediately.

Plus, it might actually work if we link against something Microsoft wanted us to link against.

Thanks! That is actually a crash. On most Windows terminals, when a console program crashes, the last line is blanked out and the program exits silently. If you watch in slow motion, the first bit of the hash is displayed: ![Screenshot_20200305-005043](https://user-images.githubusercontent.com/6258309/75951995-75e2af00-5e7b-11ea-99af-4ee2b1ca6a84.png) I'd kill for the helpful `Segmentation fault` or `Aborted` messages on Linux, or at least an error message box. This blankout doesn't happen when redirecting the output to a file, which is why you see that. We don't need to fully support XP. We didn't need to since 2014. But the important thing is that ASCII works. I would be more concerned if the program crashed immediately. Plus, it might actually work if we link against something [Microsoft wanted us to link against](https://devblogs.microsoft.com/oldnewthing/?p=1273).
easyaspi314 commented 2020-03-04 23:59:54 -06:00 (Migrated from github.com)

Untested possible solution:

int oldstdoutmode, oldstderrmode;
/* Try to switch to UTF-8 */
oldstdoutmode = _setmode(_fileno(stdout), _O_U8TEXT);
/* On error (likely with XP), switch back to normal text */
if (oldstdoutmode == -1)
    oldstdoutmode = _setmode(_fileno(stdout), _O_TEXT);
/* Repeat for stderr */
oldstderrmode = _setmode(_fileno(stderr), _O_U8TEXT);
if (oldstderrmode == -1)
    oldstderrmode = _setmode(_fileno(stderr), _O_TEXT);

In this case, what would probably happen is UTF-8 appears as mojibake.

Untested possible solution: ```c int oldstdoutmode, oldstderrmode; /* Try to switch to UTF-8 */ oldstdoutmode = _setmode(_fileno(stdout), _O_U8TEXT); /* On error (likely with XP), switch back to normal text */ if (oldstdoutmode == -1) oldstdoutmode = _setmode(_fileno(stdout), _O_TEXT); /* Repeat for stderr */ oldstderrmode = _setmode(_fileno(stderr), _O_U8TEXT); if (oldstderrmode == -1) oldstderrmode = _setmode(_fileno(stderr), _O_TEXT); ``` In this case, what would probably happen is UTF-8 appears as mojibake.
easyaspi314 commented 2020-03-05 09:44:17 -06:00 (Migrated from github.com)

Nevermind, I'm a massive idiot…if _setmode errors out, it wouldn't change from _O_TEXT...

I don't immediately see the solution, but I don't think this should delay v0.7.3.

XP support is good enough for now, and there are higher priorities.

Nevermind, I'm a massive idiot…if `_setmode` errors out, it wouldn't change from `_O_TEXT`... I don't immediately see the solution, but I don't think this should delay `v0.7.3`. XP support is good enough for now, and there are higher priorities.
easyaspi314 commented 2020-03-09 22:26:04 -05:00 (Migrated from github.com)

I think we can safely close this.

While this patch may crash XP on Unicode text with the default msvcrt.dll, that is not much worse than it was before the patch where Unicode also didn't work. And XP is 19 years old and EOL for 6 years so yeah.

I think we can safely close this. While this patch may crash XP on Unicode text with the default msvcrt.dll, that is not much worse than it was before the patch where Unicode also didn't work. And XP is 19 years old and EOL for 6 years so yeah.
👍 1 😕 1
flawrist commented 2020-03-10 09:24:35 -05:00 (Migrated from github.com)

@easyaspi314, I wholeheartedly disagree with that EOL reasoning. It means something for relatively wealthy people in industrially developed areas, but what about the rest of mankind who are being happy on little and have limited means to upgrade at once by an order of Silicon Valley hegemon? Otherwise, let’s appeal to extremes and abandon x86 in favour of x64, because, you know, “the progress calls”.

Firstly, XP is still patched by enthusiasts like Simplix. Secondly, there are projects such as DOSBox and ScummVM (accepted for Google Summer of Code, for that matter), which give a second life to pre-Win apps with an amazing benefit-to-every-byte ratio. Thirdly, the stats about XP machines exist, at least Internet-connected ones (not always the case), which show they are in no hurry to leave top 10 OS list.

That’s why I urge you to see this issue as an intellectual challenge like demoscene craftsmen have with 64K intros, like visual artists have with pixel paintings, like algorithm researchers have with code golfing, like Tom Cruise has with “Mission impossible”. Pull it together, you can do it.

@easyaspi314, I wholeheartedly disagree with that EOL reasoning. It means something for relatively wealthy people in industrially developed areas, but what about the rest of mankind who are being happy on little and have limited means to upgrade at once by an order of Silicon Valley hegemon? Otherwise, let’s appeal to extremes and abandon x86 in favour of x64, because, you know, “the progress calls”. Firstly, XP is still patched by enthusiasts like [Simplix][1]. Secondly, there are projects such as [DOSBox][2] and [ScummVM][3] (accepted for Google Summer of Code, for that matter), which give a second life to pre-Win apps with an amazing benefit-to-every-byte ratio. Thirdly, the [stats][8] about XP machines exist, at least Internet-connected ones (not always the case), which show they are in no hurry to leave top 10 OS list. That’s why I urge you to see this issue as an intellectual challenge like demoscene craftsmen have with [64K intros][4], like visual artists have with [pixel paintings][5], like algorithm researchers have with [code golfing][6], like Tom Cruise has with “Mission ~~im~~possible”. Pull it together, you can [do it][7]. [1]: https://translate.yandex.ru/translate?url=https%3A%2F%2Fblog.simplix.info%2Fupdatepack%2F&lang=ru-en [2]: https://www.dosbox.com/ [3]: https://www.scummvm.org/ [4]: https://en.wikipedia.org/wiki/Demoscene [5]: http://pixeljoint.com/ [6]: https://codegolf.stackexchange.com/ [7]: https://youtube.com/watch?v=5-sfG8BV8wU [8]: https://www.netmarketshare.com/operating-system-market-share.aspx?options={%22filter%22%3A{%22%24and%22%3A[{%22deviceType%22%3A{%22%24in%22%3A[%22Desktop%2Flaptop%22]}}]}%2C%22dateLabel%22%3A%22Custom%22%2C%22attributes%22%3A%22share%22%2C%22group%22%3A%22platformVersion%22%2C%22sort%22%3A{%22share%22%3A-1}%2C%22id%22%3A%22platformsDesktopVersions%22%2C%22dateInterval%22%3A%22Monthly%22%2C%22dateStart%22%3A%222018-03%22%2C%22dateEnd%22%3A%222020-02%22%2C%22plotKeys%22%3A[{%22platformVersion%22%3A%22Windows%2010%22}%2C{%22platformVersion%22%3A%22Windows%207%22}%2C{%22platformVersion%22%3A%22Windows%20XP%22}%2C{%22platformVersion%22%3A%22Linux%22}%2C{%22platformVersion%22%3A%22Mac%20OS%20X%2010.13%22}]%2C%22pageLength%22%3A25%2C%22segments%22%3A%22-1000%22}
👍 1 😕 1
easyaspi314 commented 2020-03-10 09:58:17 -05:00 (Migrated from github.com)

memed

Well the only thing I have left to try is mucking with Win32 API. 🤢 Apparently, Git does this.

I might try to set up an XP VM.

![memed](https://user-images.githubusercontent.com/6258309/76325680-f8c19a80-62bd-11ea-9e0a-f8c213959312.jpg) Well the only thing I have left to try is mucking with Win32 API. :nauseated_face: [Apparently, Git does this.](https://github.com/git/git/blob/b4374e96c84ed9394fed363973eb540da308ed4f/compat/winansi.c#L132) I might try to set up an XP VM.
easyaspi314 commented 2020-03-10 11:35:45 -05:00 (Migrated from github.com)

Holy shit, WriteConsoleW works.
unicode progress

Mostly.

Holy shit, `WriteConsoleW` works. ![unicode progress](https://user-images.githubusercontent.com/6258309/76335551-1812f480-62cb-11ea-8833-539853a7d05e.png) Mostly.
easyaspi314 commented 2020-03-10 13:24:03 -05:00 (Migrated from github.com)

unicode fixed

Yatta! @sergeevabc The new fix is up in #335

![unicode fixed](https://user-images.githubusercontent.com/6258309/76345959-9a56e500-62da-11ea-9595-80ebc1d33eca.png) Yatta! @sergeevabc The new fix is up in #335
easyaspi314 commented 2020-03-10 13:40:33 -05:00 (Migrated from github.com)

I don't want to see a wchar_t ever again.

I don't want to see a `wchar_t` ever again.
😆 2
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
dev
dependabot/github_actions/actions/upload-artifact-7.0.2
dependabot/github_actions/actions/download-artifact-8.0.2
dependabot/github_actions/actions/setup-node-7.1.0
archive/20261011T062114Z-rescue/fhgffy/fix/cmake-dispatch-off
dependabot/github_actions/msys2/setup-msys2-2.33.0
archive/20261011T062114Z-rescue/filtede98/fix-null-guards
ci/release-windows
fix/mingw-exe-symlink
gh-pages
release
ci/pin-actions
docs/refresh-readme
archive/20261011T062114Z-recovered/docs/refresh-readme
v0.8.4
v0.8.3
v0.8.2
v0.8.1
v0.8.0
v0.7.4
v0.7.3
v0.7.2
v0.7.1
v0.7.0
graphs
v0.6.5
v0.6.4
v0.6.3
v0.6.2
v0.6.1
v0.6.0
v0.5.1
v0.5.0
r42
r41
r40
r39
r38
r37
r36
r35
Labels
Clear labels   
announcement

   
bug

   
build

   
CI
Continuous Integration tests

  
dependencies
Pull requests that update a dependency file

  
documentation

   
duplicate

   
enhancement

   
experience feedback

   
feature request

   
github_actions
Pull requests that update GitHub Actions code

  
help wanted

   
install

   
invalid

   
Long-term

   
optimization

   
portability

   
question

   
target-specific
issue triggered in a specific environment (harware, compiler, OS)

  
test issue

   
wontfix
No labels
announcement
bug
build
CI
dependencies
documentation
duplicate
enhancement
experience feedback
feature request
github_actions
help wanted
install
invalid
Long-term
optimization
portability
question
target-specific
test issue
wontfix
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
gh-Cyan4973/xxHash#293
No description provided.