gh-Cyan4973/xxHash

Watch

Fork 0

xxhsum aren't Behaving Like shasum on Checking #724

New issue

Closed

opened 2022-06-16 03:27:01 -05:00 by ghost · 3 comments

ghost commented

2022-06-16 03:27:01 -05:00

(Migrated from github.com)

I.g. sha512sum will not check sha256sum's checksums, it'll output an error. xxHash checks all, not following Coreutils' behaviour.

Please would it be possible for "xxh64sum" to reject "xxh128sum" checksums and vice versa? For example, a symbolic link or the "-H" option modifies the behaviour, but "xxhsum" called alone behaves broadly like usual.

The step additionally might help with #645 in some way.

I.g. sha512sum will not check sha256sum's checksums, it'll output an error. xxHash checks all, not following Coreutils' behaviour. Please would it be possible for "xxh64sum" to reject "xxh128sum" checksums and vice versa? For example, a symbolic link or the "-H" option modifies the behaviour, but "xxhsum" called alone behaves broadly like usual. The step additionally might help with #645 in some way.

👍 1 👀 1

t-mat commented

2022-07-14 07:12:52 -05:00

(Migrated from github.com)

Hi ghost 👻 , thanks for the report.

I've confirmed your issue. Since they're long, I just put them in this post though.

Repro and details of the difference between sha*sum and xxh*sum

Clone & build

$ git clone https://github.com/Cyan4973/xxHash.git
$ cd xxHash/
$ make

Create sha256sum and sha512sum

$ sha256sum README.md > out.sha256sum
$ sha512sum README.md > out.sha512sum

$ cat out.sha256sum
936bb323f8a500ae6396858041c647a14f8d0230eafb3873cd8498537fab0c75 README.md
$ cat out.sha512sum
cd63147103fa14c82c04f925f197c7d3bbf5232529f952e9d450f469104014a1715185bcf1308f431f5f24172a5f7f97b8b0998e56fe2d93d0821c0db2107552 README.md

Test sha256sum

$ sha256sum -c out.sha256sum
README.md: OK
$ sha256sum -c out.sha512sum
sha256sum: out.sha512sum: no properly formatted SHA256 checksum lines found

Test sha512sum

$ sha512sum -c out.sha256sum
sha512sum: out.sha256sum: no properly formatted SHA512 checksum lines found
$ sha512sum -c out.sha512sum
README.md: OK

$ ./xxh64sum README.md > out.xxh64sum
$ ./xxh128sum README.md > out.xxh128sum

$ cat out.xxh64sum
bb03b8a8d9ac2202 README.md
$ cat out.xxh128sum
1c6c0b96b536a13f58b36ce50a91b1b9 README.md

$ ./xxh64sum -c out.xxh64sum
README.md: OK
$ ./xxh64sum -c out.xxh128sum
README.md: OK

$ ./xxh128sum -c out.xxh64sum
README.md: OK
$ ./xxh128sum -c out.xxh128sum
README.md: OK

How `digest.c` outputs the "no properly lines found" message.

// https://git.savannah.gnu.org/cgit/coreutils.git/tree/src/digest.c?h=v9.1#n1198

/* With --check, exit with a non-zero return code if any line is
   improperly formatted. */
static bool strict = false;

static bool
split_3 (char *s, size_t s_len,
         unsigned char **hex_digest, int *binary, char **file_name)
{
  /* @t-mat : Basically, this function returns false if something wrong. */
  bool ok = ...;
  return ok;
}


static bool
digest_check (char const *checkfile_name)
{
  ...
  bool properly_formatted_lines = false;
  do
    {
      if (! (split_3 (line, line_length, &hex_digest, &binary, &filename)
             && ! (is_stdin && STREQ (filename, "-"))))
        {
          ++n_misformatted_lines;
          ...
          ++n_improperly_formatted_lines;
        }
      else
        {
          ...
          properly_formatted_lines = true;
          ...
        }
    }
  ...
  if (! properly_formatted_lines)
    {
      /* Warn if no tests are found.  */
      error (0, 0, _("%s: no properly formatted checksum lines found"),
             quotef (checkfile_name));
    }
  ...
  /* @t-mat : When "--strict" is specified and some lines are wrong
              formatted, this functions returns false. */
  return (properly_formatted_lines
          ...
          && n_mismatched_checksums == 0
          ...
          && (!strict || n_improperly_formatted_lines == 0));
}


int
main (int argc, char **argv)
{
  ...
  while ((opt = getopt_long (argc, argv, short_opts, long_options, NULL)) != -1)
    switch (opt)
      {
      ...
      case STRICT_OPTION:
        strict = true;
      ...
      }
  ...
}

Hi ghost 👻 , thanks for the report. I've confirmed your issue. Since they're long, I just put them in this post though. <details> <summary>Repro and details of the difference between sha*sum and xxh*sum</summary> # Clone & build $ git clone https://github.com/Cyan4973/xxHash.git $ cd xxHash/ $ make # Create sha256sum and sha512sum $ sha256sum README.md > out.sha256sum $ sha512sum README.md > out.sha512sum $ cat out.sha256sum 936bb323f8a500ae6396858041c647a14f8d0230eafb3873cd8498537fab0c75 README.md $ cat out.sha512sum cd63147103fa14c82c04f925f197c7d3bbf5232529f952e9d450f469104014a1715185bcf1308f431f5f24172a5f7f97b8b0998e56fe2d93d0821c0db2107552 README.md # Test sha256sum $ sha256sum -c out.sha256sum README.md: OK $ sha256sum -c out.sha512sum sha256sum: out.sha512sum: no properly formatted SHA256 checksum lines found # Test sha512sum $ sha512sum -c out.sha256sum sha512sum: out.sha256sum: no properly formatted SHA512 checksum lines found $ sha512sum -c out.sha512sum README.md: OK $ ./xxh64sum README.md > out.xxh64sum $ ./xxh128sum README.md > out.xxh128sum $ cat out.xxh64sum bb03b8a8d9ac2202 README.md $ cat out.xxh128sum 1c6c0b96b536a13f58b36ce50a91b1b9 README.md $ ./xxh64sum -c out.xxh64sum README.md: OK $ ./xxh64sum -c out.xxh128sum README.md: OK $ ./xxh128sum -c out.xxh64sum README.md: OK $ ./xxh128sum -c out.xxh128sum README.md: OK </details> <details> <summary>How `digest.c` outputs the "no properly lines found" message.</summary> ```C // https://git.savannah.gnu.org/cgit/coreutils.git/tree/src/digest.c?h=v9.1#n1198 /* With --check, exit with a non-zero return code if any line is improperly formatted. */ static bool strict = false; static bool split_3 (char *s, size_t s_len, unsigned char **hex_digest, int *binary, char **file_name) { /* @t-mat : Basically, this function returns false if something wrong. */ bool ok = ...; return ok; } static bool digest_check (char const *checkfile_name) { ... bool properly_formatted_lines = false; do { if (! (split_3 (line, line_length, &hex_digest, &binary, &filename) && ! (is_stdin && STREQ (filename, "-")))) { ++n_misformatted_lines; ... ++n_improperly_formatted_lines; } else { ... properly_formatted_lines = true; ... } } ... if (! properly_formatted_lines) { /* Warn if no tests are found. */ error (0, 0, _("%s: no properly formatted checksum lines found"), quotef (checkfile_name)); } ... /* @t-mat : When "--strict" is specified and some lines are wrong formatted, this functions returns false. */ return (properly_formatted_lines ... && n_mismatched_checksums == 0 ... && (!strict || n_improperly_formatted_lines == 0)); } int main (int argc, char **argv) { ... while ((opt = getopt_long (argc, argv, short_opts, long_options, NULL)) != -1) switch (opt) { ... case STRICT_OPTION: strict = true; ... } ... } ``` </details>

t-mat commented

2022-07-14 07:19:09 -05:00

(Migrated from github.com)

I think there're some possible solutions.

(1) Mimic sshXXXsum (digest.c). When xxhsum is invoked via symlink, it just accept specific length of hash value. (Breaking change)
(2) Add new command line switch for more strict mode which mimics sshXXXsum. --pretty-strict ? (Compatible. But not compatible with sshXXXsum)

I think there're some possible solutions. (1) Mimic sshXXXsum (digest.c). When `xxhsum` is invoked via symlink, it just accept specific length of hash value. (Breaking change) (2) Add new command line switch for *more* strict mode which mimics sshXXXsum. `--pretty-strict` ? (Compatible. But not compatible with sshXXXsum)

Cyan4973 commented

2022-07-21 09:46:43 -05:00

(Migrated from github.com)

A more restrictive acceptance policy for xxh128sum and similar is likely appropriate.

A more restrictive acceptance policy for `xxh128sum` and similar is likely appropriate.

👍 1

No milestone

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

gh-Cyan4973/xxHash#724

No description provided.

Rows
Columns

xxh*sum aren't Behaving Like sha*sum on Checking #724

Clone & build

Create sha256sum and sha512sum

Test sha256sum

Test sha512sum

xxhsum aren't Behaving Like shasum on Checking #724