xxhsum.exe Long MAX_PATH support over 260 characters in the path #1060

New issue

Closed

opened 2025-08-20 01:26:18 -05:00 by 29039 · 1 comment

29039 commented

2025-08-20 01:26:18 -05:00

(Migrated from github.com)

I was having an issue with xxhsum not reading files over 260 characters in Windows despite LongPathsEnabled in the registry being enabled.

Using the Microsoft documentation here and here and then using resource hacker to change the application manifest inside the .exe, I was able to get this working perfectly.

I poked around the source but I don't know enough about how to change the manifest file it compiles with to be able to propose a patch, but if someone can work this out, here is my Before and After.

Before:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
  <trustInfo xmlns="urn:schemas-microsoft-com:asm.v3">
    <security>
      <requestedPrivileges>
        <requestedExecutionLevel level="asInvoker"/>
      </requestedPrivileges>
    </security>
  </trustInfo>
  <compatibility xmlns="urn:schemas-microsoft-com:compatibility.v1">
    <application>
      <!--The ID below indicates application support for Windows Vista -->
      <supportedOS Id="{e2011457-1546-43c5-a5fe-008deee3d3f0}"/>
      <!--The ID below indicates application support for Windows 7 -->
      <supportedOS Id="{35138b9a-5d96-4fbd-8e2d-a2440225f93a}"/>
      <!--The ID below indicates application support for Windows 8 -->
      <supportedOS Id="{4a2f28e3-53b9-4441-ba9c-d69d4a4a6e38}"/>
      <!--The ID below indicates application support for Windows 8.1 -->
      <supportedOS Id="{1f676c76-80e1-4239-95bb-83d0f6d0da78}"/> 
      <!--The ID below indicates application support for Windows 10 -->
      <supportedOS Id="{8e0f7a12-bfb3-4fe8-b9a5-48fd50a15a9a}"/> 
    </application>
  </compatibility>
</assembly>

After my changes:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0" xmlns:asmv3="urn:schemas-microsoft-com:asm.v3">
  <asmv3:application>
    <windowsSettings xmlns:ws2="http://schemas.microsoft.com/SMI/2016/WindowsSettings">
        <ws2:longPathAware>true</ws2:longPathAware>
    </windowsSettings>
  </asmv3:application>
  <trustInfo xmlns="urn:schemas-microsoft-com:asm.v3">
    <security>
      <requestedPrivileges>
        <requestedExecutionLevel level="asInvoker"/>
      </requestedPrivileges>
    </security>
  </trustInfo>
  <compatibility xmlns="urn:schemas-microsoft-com:compatibility.v1">
    <application>
      <!--The ID below indicates application support for Windows Vista -->
      <supportedOS Id="{e2011457-1546-43c5-a5fe-008deee3d3f0}"/>
      <!--The ID below indicates application support for Windows 7 -->
      <supportedOS Id="{35138b9a-5d96-4fbd-8e2d-a2440225f93a}"/>
      <!--The ID below indicates application support for Windows 8 -->
      <supportedOS Id="{4a2f28e3-53b9-4441-ba9c-d69d4a4a6e38}"/>
      <!--The ID below indicates application support for Windows 8.1 -->
      <supportedOS Id="{1f676c76-80e1-4239-95bb-83d0f6d0da78}"/> 
      <!--The ID below indicates application support for Windows 10 -->
      <supportedOS Id="{8e0f7a12-bfb3-4fe8-b9a5-48fd50a15a9a}"/> 
    </application>
  </compatibility>
</assembly>

I was having an issue with xxhsum not reading files over 260 characters in Windows despite LongPathsEnabled in the registry being enabled. Using the Microsoft documentation [here](https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry#application-manifest-updates-to-declare-long-path-capability) and [here](https://learn.microsoft.com/en-us/windows/win32/sbscs/application-manifests#longpathaware) and then using [resource hacker](https://scoop.sh/#/apps?q=resource-hacker) to change the application manifest inside the .exe, I was able to get this working perfectly. I poked around the source but I don't know enough about how to change the manifest file it compiles with to be able to propose a patch, but if someone can work this out, here is my Before and After. Before: ``` <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0"> <trustInfo xmlns="urn:schemas-microsoft-com:asm.v3"> <security> <requestedPrivileges> <requestedExecutionLevel level="asInvoker"/> </requestedPrivileges> </security> </trustInfo> <compatibility xmlns="urn:schemas-microsoft-com:compatibility.v1"> <application>  <supportedOS Id="{e2011457-1546-43c5-a5fe-008deee3d3f0}"/>  <supportedOS Id="{35138b9a-5d96-4fbd-8e2d-a2440225f93a}"/>  <supportedOS Id="{4a2f28e3-53b9-4441-ba9c-d69d4a4a6e38}"/>  <supportedOS Id="{1f676c76-80e1-4239-95bb-83d0f6d0da78}"/>  <supportedOS Id="{8e0f7a12-bfb3-4fe8-b9a5-48fd50a15a9a}"/> </application> </compatibility> </assembly> ``` After my changes: ``` <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0" xmlns:asmv3="urn:schemas-microsoft-com:asm.v3"> <asmv3:application> <windowsSettings xmlns:ws2="http://schemas.microsoft.com/SMI/2016/WindowsSettings"> <ws2:longPathAware>true</ws2:longPathAware> </windowsSettings> </asmv3:application> <trustInfo xmlns="urn:schemas-microsoft-com:asm.v3"> <security> <requestedPrivileges> <requestedExecutionLevel level="asInvoker"/> </requestedPrivileges> </security> </trustInfo> <compatibility xmlns="urn:schemas-microsoft-com:compatibility.v1"> <application>  <supportedOS Id="{e2011457-1546-43c5-a5fe-008deee3d3f0}"/>  <supportedOS Id="{35138b9a-5d96-4fbd-8e2d-a2440225f93a}"/>  <supportedOS Id="{4a2f28e3-53b9-4441-ba9c-d69d4a4a6e38}"/>  <supportedOS Id="{1f676c76-80e1-4239-95bb-83d0f6d0da78}"/>  <supportedOS Id="{8e0f7a12-bfb3-4fe8-b9a5-48fd50a15a9a}"/> </application> </compatibility> </assembly> ```

t-mat commented

2025-08-20 09:35:39 -05:00

(Migrated from github.com)

Hi @29039 , thanks for the report and your investigation.

I put additional information here for further integration in future.

Reproduction procedure

cmd.exe
cd /d "%PUBLIC%"
git clone --depth 1 --branch v0.8.3 https://github.com/Cyan4973/xxHash
cd xxHash
mkdir         build
cmake -B      build -S cmake_unofficial
cmake --build build --config Release
.\build\Release\xxhsum.exe --version

pushd
mkdir 0---------1---------2---------3---------4---------5---------6---------7---------8---------9---------
cd    0---------1---------2---------3---------4---------5---------6---------7---------8---------9---------
mkdir a---------b---------c---------d---------e---------f---------g---------h---------i---------j---------
cd    a---------b---------c---------d---------e---------f---------g---------h---------i---------j---------
mkdir k---------l---------m---------n---------o---------p---------q---------r---------s---------t---------
cd    k---------l---------m---------n---------o---------p---------q---------r---------s---------t---------
popd
copy /y README.md          0---------1---------2---------3---------4---------5---------6---------7---------8---------9---------\a---------b---------c---------d---------e---------f---------g---------h---------i---------j---------\k---------l---------m---------n---------o---------p---------q---------r---------s---------t---------\
.\build\Release\xxhsum.exe 0---------1---------2---------3---------4---------5---------6---------7---------8---------9---------\a---------b---------c---------d---------e---------f---------g---------h---------i---------j---------\k---------l---------m---------n---------o---------p---------q---------r---------s---------t---------\README.md

Error: Could not open '0---------1---------2---------3---------4---------5---------6---------7---------8---------9---------\a---------b---------c---------d---------e---------f---------g---------h---------i---------j---------\k---------l---------m---------n---------o---------p---------q---------r---------s---------t---------\README.md': No such file or directory.

Possible patch for `XSUM_fopen() (wchar_t)`

NOTE : XSUM_stat also needs to support long path.

XSUM_API FILE* XSUM_fopen(const char* filename, const char* mode)
{
    FILE* f = NULL;
    wchar_t* const wide_filename = XSUM_widenString(filename, NULL);
    if (wide_filename != NULL) {
        wchar_t* const wide_mode = XSUM_widenString(mode, NULL);
        if (wide_mode != NULL) {
#if 0
            // short path
            f = _wfopen(wide_filename, wide_mode);
#else
            // long path
            size_t const absPathSizeInBytes = 65536;    // 32767 wchar_ts + (wchar_t)0
            wchar_t* const absPath = (wchar_t*) malloc(absPathSizeInBytes);
            if(absPath != NULL) {
                DWORD const absResult = GetFullPathNameW(wide_filename, (DWORD) absPathSizeInBytes/sizeof(wchar_t), absPath, NULL);
                if(absResult > 0 && absResult < absPathSizeInBytes/sizeof(wchar_t)) {
                    wchar_t* const uncPrefix = L"\\\\?\\";
                    wchar_t* const uncPath = (wchar_t*) malloc(absPathSizeInBytes);
                    if(uncPath != NULL) {
                        if (wcsncmp(absPath, uncPrefix, wcslen(uncPrefix)) == 0) {
                            wcscpy(uncPath, absPath);
                        } else {
                            swprintf(uncPath, absPathSizeInBytes/sizeof(wchar_t), L"%s%s", uncPrefix, absPath);
                        }
                        f = _wfopen(uncPath, wide_mode);
                        free(uncPath);
                    }
                }
                free(absPath);
            }
#endif
            free(wide_mode);
        }
        free(wide_filename);
    }
    return f;
}

.\build\Release\xxhsum.exe 0---------1---------2---------3---------4---------5---------6---------7---------8---------9---------\a---------b---------c---------d---------e---------f---------g---------h---------i---------j---------\k---------l---------m---------n---------o---------p---------q---------r---------s---------t---------\README.md

\021486ed356c9bbc  0---------1---------2---------3---------4---------5---------6---------7---------8---------9---------\\a---------b---------c---------d---------e---------f---------g---------h---------i---------j---------\\k---------l---------m---------n---------o---------p---------q---------r---------s---------t---------\\README.md

Hi @29039 , thanks for the report and your investigation. I put additional information here for further integration in future. Reproduction procedure ---------------------- ```bat cmd.exe cd /d "%PUBLIC%" git clone --depth 1 --branch v0.8.3 https://github.com/Cyan4973/xxHash cd xxHash mkdir build cmake -B build -S cmake_unofficial cmake --build build --config Release .\build\Release\xxhsum.exe --version pushd mkdir 0---------1---------2---------3---------4---------5---------6---------7---------8---------9--------- cd 0---------1---------2---------3---------4---------5---------6---------7---------8---------9--------- mkdir a---------b---------c---------d---------e---------f---------g---------h---------i---------j--------- cd a---------b---------c---------d---------e---------f---------g---------h---------i---------j--------- mkdir k---------l---------m---------n---------o---------p---------q---------r---------s---------t--------- cd k---------l---------m---------n---------o---------p---------q---------r---------s---------t--------- popd copy /y README.md 0---------1---------2---------3---------4---------5---------6---------7---------8---------9---------\a---------b---------c---------d---------e---------f---------g---------h---------i---------j---------\k---------l---------m---------n---------o---------p---------q---------r---------s---------t---------\ .\build\Release\xxhsum.exe 0---------1---------2---------3---------4---------5---------6---------7---------8---------9---------\a---------b---------c---------d---------e---------f---------g---------h---------i---------j---------\k---------l---------m---------n---------o---------p---------q---------r---------s---------t---------\README.md ``` ``` Error: Could not open '0---------1---------2---------3---------4---------5---------6---------7---------8---------9---------\a---------b---------c---------d---------e---------f---------g---------h---------i---------j---------\k---------l---------m---------n---------o---------p---------q---------r---------s---------t---------\README.md': No such file or directory. ``` Possible patch for `XSUM_fopen() (wchar_t)` ------------------------------------------- NOTE : `XSUM_stat` also needs to support long path. ```C XSUM_API FILE* XSUM_fopen(const char* filename, const char* mode) { FILE* f = NULL; wchar_t* const wide_filename = XSUM_widenString(filename, NULL); if (wide_filename != NULL) { wchar_t* const wide_mode = XSUM_widenString(mode, NULL); if (wide_mode != NULL) { #if 0 // short path f = _wfopen(wide_filename, wide_mode); #else // long path size_t const absPathSizeInBytes = 65536; // 32767 wchar_ts + (wchar_t)0 wchar_t* const absPath = (wchar_t*) malloc(absPathSizeInBytes); if(absPath != NULL) { DWORD const absResult = GetFullPathNameW(wide_filename, (DWORD) absPathSizeInBytes/sizeof(wchar_t), absPath, NULL); if(absResult > 0 && absResult < absPathSizeInBytes/sizeof(wchar_t)) { wchar_t* const uncPrefix = L"\\\\?\\"; wchar_t* const uncPath = (wchar_t*) malloc(absPathSizeInBytes); if(uncPath != NULL) { if (wcsncmp(absPath, uncPrefix, wcslen(uncPrefix)) == 0) { wcscpy(uncPath, absPath); } else { swprintf(uncPath, absPathSizeInBytes/sizeof(wchar_t), L"%s%s", uncPrefix, absPath); } f = _wfopen(uncPath, wide_mode); free(uncPath); } } free(absPath); } #endif free(wide_mode); } free(wide_filename); } return f; } ``` ``` .\build\Release\xxhsum.exe 0---------1---------2---------3---------4---------5---------6---------7---------8---------9---------\a---------b---------c---------d---------e---------f---------g---------h---------i---------j---------\k---------l---------m---------n---------o---------p---------q---------r---------s---------t---------\README.md ``` ``` \021486ed356c9bbc 0---------1---------2---------3---------4---------5---------6---------7---------8---------9---------\\a---------b---------c---------d---------e---------f---------g---------h---------i---------j---------\\k---------l---------m---------n---------o---------p---------q---------r---------s---------t---------\\README.md ```

👍 1

No milestone

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

gh-Cyan4973/xxHash#1060

No description provided.

Rows
Columns

xxhsum.exe Long MAX_PATH support over 260 characters in the path #1060

Reproduction procedure

Possible patch for XSUM_fopen() (wchar_t)

Possible patch for `XSUM_fopen() (wchar_t)`