No wildcards on Windows CMD.EXE

easyaspi314 commented

2020-03-19 23:28:08 -05:00

(Migrated from github.com)

Hooray, another Windows bug.

The current GetCommandLineW method does not expand wildcards on cmd.exe:

v0.7.2:

C:\Code\xxHash>xxhsum.exe *.c
04504bbceb58a519  test.c
1eed5ddd8e02d18b  xxhash.c
45f1dab554385494  xxhsum.c

v0.7.3:

C:\Code\xxHash>xxhsum.exe *.c
Error: Could not open '*.c': Invalid argument.

Note that this is not a problem with wmain.

I think the solution is to use __wgetmainargs with _DoWildCard = 1.

This also avoids linking against shell32.dll, which is a lot of bloat:
CommandLineToArgvW:

$ ldd xxhsum.exe
        ntdll.dll => /c/Windows/SYSTEM32/ntdll.dll (0x7ffbf8140000)
        KERNEL32.DLL => /c/Windows/System32/KERNEL32.DLL (0x7ffbf7ef0000)
        KERNELBASE.dll => /c/Windows/System32/KERNELBASE.dll (0x7ffbf5ad0000)
        msvcrt.dll => /c/Windows/System32/msvcrt.dll (0x7ffbf7e50000)
        SHELL32.dll => /c/Windows/System32/SHELL32.dll (0x7ffbf69e0000)
        ucrtbase.dll => /c/Windows/System32/ucrtbase.dll (0x7ffbf5d80000)
        cfgmgr32.dll => /c/Windows/System32/cfgmgr32.dll (0x7ffbf5e80000)
        shcore.dll => /c/Windows/System32/shcore.dll (0x7ffbf79b0000)
        RPCRT4.dll => /c/Windows/System32/RPCRT4.dll (0x7ffbf77e0000)
        combase.dll => /c/Windows/System32/combase.dll (0x7ffbf6540000)
        bcryptPrimitives.dll => /c/Windows/System32/bcryptPrimitives.dll (0x7ffbf5ed0000)
        windows.storage.dll => /c/Windows/System32/windows.storage.dll (0x7ffbf5120000)
        msvcp_win.dll => /c/Windows/System32/msvcp_win.dll (0x7ffbf5f50000)
        sechost.dll => /c/Windows/System32/sechost.dll (0x7ffbf7910000)
        advapi32.dll => /c/Windows/System32/advapi32.dll (0x7ffbf7ac0000)
        profapi.dll => /c/Windows/System32/profapi.dll (0x7ffbf5080000)
        powrprof.dll => /c/Windows/System32/powrprof.dll (0x7ffbf5030000)
        UMPDC.dll => /c/Windows/System32/UMPDC.dll (0x7ffbf5000000)
        shlwapi.dll => /c/Windows/System32/shlwapi.dll (0x7ffbf7700000)
        GDI32.dll => /c/Windows/System32/GDI32.dll (0x7ffbf63a0000)
        win32u.dll => /c/Windows/System32/win32u.dll (0x7ffbf50d0000)
        gdi32full.dll => /c/Windows/System32/gdi32full.dll (0x7ffbf5930000)
        USER32.dll => /c/Windows/System32/USER32.dll (0x7ffbf7560000)
        kernel.appcore.dll => /c/Windows/System32/kernel.appcore.dll (0x7ffbf5010000)
        cryptsp.dll => /c/Windows/System32/cryptsp.dll (0x7ffbf5100000)

__wgetmainargs:

$ ldd xxhsum.exe
        ntdll.dll => /c/Windows/SYSTEM32/ntdll.dll (0x7ffbf8140000)
        KERNEL32.DLL => /c/Windows/System32/KERNEL32.DLL (0x7ffbf7ef0000)
        KERNELBASE.dll => /c/Windows/System32/KERNELBASE.dll (0x7ffbf5ad0000)
        msvcrt.dll => /c/Windows/System32/msvcrt.dll (0x7ffbf7e50000)

I still need to do more research, but this seems to work great on classic MinGW, even skipping the borked shell globbing (still doesn't support typing the filename):

$ ls *.txt
ls: ??????.txt: No such file or directory

$ ./xxhsum.exe *.txt
2d7f1808da1fa63c  привет.txt

### Hooray, another Windows bug. The current `GetCommandLineW` method does not expand wildcards on `cmd.exe`: v0.7.2: ``` C:\Code\xxHash>xxhsum.exe *.c 04504bbceb58a519 test.c 1eed5ddd8e02d18b xxhash.c 45f1dab554385494 xxhsum.c ``` v0.7.3: ``` C:\Code\xxHash>xxhsum.exe *.c Error: Could not open '*.c': Invalid argument. ``` Note that this is not a problem with `wmain`. I think the solution is to use [`__wgetmainargs`](https://docs.microsoft.com/en-us/cpp/c-runtime-library/getmainargs-wgetmainargs?view=vs-2019) with `_DoWildCard = 1`. This also avoids linking against `shell32.dll`, which is a lot of bloat: `CommandLineToArgvW`: ``` $ ldd xxhsum.exe ntdll.dll => /c/Windows/SYSTEM32/ntdll.dll (0x7ffbf8140000) KERNEL32.DLL => /c/Windows/System32/KERNEL32.DLL (0x7ffbf7ef0000) KERNELBASE.dll => /c/Windows/System32/KERNELBASE.dll (0x7ffbf5ad0000) msvcrt.dll => /c/Windows/System32/msvcrt.dll (0x7ffbf7e50000) SHELL32.dll => /c/Windows/System32/SHELL32.dll (0x7ffbf69e0000) ucrtbase.dll => /c/Windows/System32/ucrtbase.dll (0x7ffbf5d80000) cfgmgr32.dll => /c/Windows/System32/cfgmgr32.dll (0x7ffbf5e80000) shcore.dll => /c/Windows/System32/shcore.dll (0x7ffbf79b0000) RPCRT4.dll => /c/Windows/System32/RPCRT4.dll (0x7ffbf77e0000) combase.dll => /c/Windows/System32/combase.dll (0x7ffbf6540000) bcryptPrimitives.dll => /c/Windows/System32/bcryptPrimitives.dll (0x7ffbf5ed0000) windows.storage.dll => /c/Windows/System32/windows.storage.dll (0x7ffbf5120000) msvcp_win.dll => /c/Windows/System32/msvcp_win.dll (0x7ffbf5f50000) sechost.dll => /c/Windows/System32/sechost.dll (0x7ffbf7910000) advapi32.dll => /c/Windows/System32/advapi32.dll (0x7ffbf7ac0000) profapi.dll => /c/Windows/System32/profapi.dll (0x7ffbf5080000) powrprof.dll => /c/Windows/System32/powrprof.dll (0x7ffbf5030000) UMPDC.dll => /c/Windows/System32/UMPDC.dll (0x7ffbf5000000) shlwapi.dll => /c/Windows/System32/shlwapi.dll (0x7ffbf7700000) GDI32.dll => /c/Windows/System32/GDI32.dll (0x7ffbf63a0000) win32u.dll => /c/Windows/System32/win32u.dll (0x7ffbf50d0000) gdi32full.dll => /c/Windows/System32/gdi32full.dll (0x7ffbf5930000) USER32.dll => /c/Windows/System32/USER32.dll (0x7ffbf7560000) kernel.appcore.dll => /c/Windows/System32/kernel.appcore.dll (0x7ffbf5010000) cryptsp.dll => /c/Windows/System32/cryptsp.dll (0x7ffbf5100000) ``` `__wgetmainargs`: ``` $ ldd xxhsum.exe ntdll.dll => /c/Windows/SYSTEM32/ntdll.dll (0x7ffbf8140000) KERNEL32.DLL => /c/Windows/System32/KERNEL32.DLL (0x7ffbf7ef0000) KERNELBASE.dll => /c/Windows/System32/KERNELBASE.dll (0x7ffbf5ad0000) msvcrt.dll => /c/Windows/System32/msvcrt.dll (0x7ffbf7e50000) ``` I still need to do more research, but this seems to work great on classic MinGW, even skipping the borked shell globbing (still doesn't support typing the filename): ``` $ ls *.txt ls: ??????.txt: No such file or directory $ ./xxhsum.exe *.txt 2d7f1808da1fa63c привет.txt ```

👍 1

easyaspi314 commented

2020-03-23 11:57:19 -05:00

(Migrated from github.com)

__wgetmainargs works on XP. Will open a PR once I clean things up.

`__wgetmainargs` works on XP. Will open a PR once I clean things up.

👍 1

Cyan4973 commented

2020-04-07 18:24:11 -05:00

(Migrated from github.com)

I presume this issue has been fixed by one of your patches @easyaspi314 ?

For information, I had the same issue a long time ago, while working on lz4 on Visual Studio.
It was solved by adding a link dependency in the builder, something like setargv.obj. It worked well, requiring no change of code, though the dependency was specified in, and therefore only enabled from, the Visual Studio script.

I presume this issue has been fixed by one of your patches @easyaspi314 ? For information, I had the same issue a long time ago, while working on `lz4` on Visual Studio. It was solved by adding a link dependency in the builder, something like [`setargv.obj`](https://docs.microsoft.com/en-us/cpp/c-language/expanding-wildcard-arguments?view=vs-2019). It worked well, requiring no change of code, though the dependency was specified in, and therefore only enabled from, the Visual Studio script.

easyaspi314 commented

2020-04-08 15:03:27 -05:00

(Migrated from github.com)

Hmm. They really make it confusing

Either way, AFAICT everything is working.

Hmm. They really make it confusing Either way, AFAICT everything is working.

flawrist commented

2020-06-10 07:22:49 -05:00

(Migrated from github.com)

Err…

$ xxhsum-x64-mingw-w64 --version
0.7.3 (64-bit x86_64 + SSE2 little endian), GCC 9.2.0, by Yann Collet

$ xxhsum-x64-mingw-w64 *.txt
Error: Could not open '*.txt': Invalid argument.               <--------- ???

$ md5sum *.txt
8a669e9418750c81ab90ae159a8ec410 *a.txt
5d089de07c1c2733a1adde0399423ad1 *b.txt

Err… ```console $ xxhsum-x64-mingw-w64 --version 0.7.3 (64-bit x86_64 + SSE2 little endian), GCC 9.2.0, by Yann Collet $ xxhsum-x64-mingw-w64 *.txt Error: Could not open '*.txt': Invalid argument. <--------- ??? $ md5sum *.txt 8a669e9418750c81ab90ae159a8ec410 *a.txt 5d089de07c1c2733a1adde0399423ad1 *b.txt ``` [1]: https://github.com/Cyan4973/xxHash/pull/335#issuecomment-597348448

easyaspi314 commented

2020-06-10 07:27:23 -05:00

(Migrated from github.com)

0.7.3 was released before this patch.

👀 1

Rows
Columns

No wildcards on Windows CMD.EXE #341

Hooray, another Windows bug.