Use cpuid to detect CPU support for hardware instruction.
Fall back to slice by 8 algorithm if not supported.
A 500MiB file improves from 1.40s to 0.67s on an i3-2310M
* configure.ac [USE_PCLMUL_CRC32]: A new conditional,
set when __get_cpuid() and clmul compiler intrinsics are supported.
* src/cksum.c (pclmul_supported): A new function using __get_cpuid()
to determine if pclmul instructions are supported.
(cksum): A new function refactored from cksum_slice8(),
which calls pclmul_supported() and then cksum_slice8()
or cksum_pclmul() as appropriate.
* src/cksum.h: Export the crctab array for use in the new module.
* src/cksum_pclmul.c: A new module to implement using pclmul intrinsics.
* po/POTFILES.in: Reference the new cksum_pclmul module.
* src/local.mk: Likewise. Note we build it as a separate library
so that it can be portably built with separate -mavx etc. flags.
* tests/misc/cksum.sh: Add new test modes for pertinent buffer sizes.
GNU/Linux is unusual here in that rmdir("symlink/") returns ENOTDIR,
whereas Solaris and FreeBSD at least, will follow the symlink
and remove the target directory. We don't make the behavior
on Linux kernels consistent, but at least clarify
the confusing error message.
* src/rmdir (main): Output a specific error message for the above case.
(remove_parents): In the error message, don't assume intermediate paths
are directories, as they could be symlinks.
* tests/rmdir/symlink-errors.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the improvement.
This regression was introduced in commit COREUTILS-6_8-58-g553d347d3
* src/pr.c (init_parameters): Process tabs for multiple columns.
* tests/pr/pr-tests.pl: Add test cases.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/46422
- \r\n is common a line end combination
- catting such a file without options causes it to display normally
- overwriting the first char with $, loses info
* src/cat.c (cat): Convert \r preceeding a \n to ^M.
* tests/misc/cat-E.sh: New test.
* tests/local.mk: Reference new test.
* tests/misc/cat-proc.sh: Fix typo.
* doc/coreutils.texi (cat invocation): Mention the new behavior.
* NEWS: Mention the improvement.
This functionality regressed with the adjustments
in commit v8.25-4-g62e7af032
* src/split.c (bytes_chunk_extract): Account for already read data
when seeking into the file.
* tests/split/b-chunk.sh: Use the hidden ---io-blksize option,
to test this functionality.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/46048
Problem reported by David McCall (Bug#45886).
I introduced this problem when fixing Bug#14371.
* NEWS: Mention the fix.
* src/mkdir.c (struct mkdir_options): New members umask_ancestor,
umask_self, replacing umask_value.
(make_ancestor): Use them when temporarily adjusting umask.
(main): Set them, and set the umask to umask_self instead
of leaving it alone.
* tests/mkdir/perm.sh (tests): Add test case for bug.
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
* src/nl.c (main): Enforce the POSIX specified
behavior of assuming ':' is specified after a single
character argument to -d.
* tests/misc/nl.sh: Add a test case.
* NEWS: Mention the bug fix.
* doc/coreutils.texi (nl invocation): Mention the GNU extensions
of allowing arbitrary length and empty delimiter strings.
* src/nl.c (usage): Likewise.
* tests/misc/nl.sh: Add test cases for the GNU extensions.
The format can be determined from --options or the locale,
so it's useful to output the format string being used.
* src/date.c (show_date): Show the output format
along with the date being shown.
* tests/misc/date-debug.sh: Adjust accordingly.
Addresses https://bugs.gnu.org/44960
When removing a directory fails for some reason, and that directory
is empty, the rm_fts code gets the return value of the excise call
confused with the return value of its earlier call to prompt,
causing fts_skip_tree to be called again and the next file
that rm would otherwise have deleted to survive.
* src/remove.c (rm_fts): Ensure we only skip a single fts entry,
when processing empty dirs. I.e. only skip the entry
having successfully removed it.
* tests/rm/empty-immutable-skip.sh: New root-only test.
* tests/local.mk: Add it.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/44883
This crash was identified by Cyber Independent Testing Lab:
https://cyber-itl.org/2020/10/28/citl-7000-defects.html
and was introduced with commit v8.5-163-g3f48829c2
* src/tr.c (validate_case_classes): Don't apply these
extra case alignment checks in the --complement case,
which is even more restrictive as to the contents of SET2.
* tests/misc/tr-case-class.sh: Add a test case,
for a large SET1, which caused the length adjustment
in validate_case_classes to underflow and trigger the assert.
* NEWS: Mention the bug fix.
This crash was identified by Cyber Independent Testing Lab:
https://cyber-itl.org/2020/10/28/citl-7000-defects.html
and was introduced with commit v6.9.90-11-g4245876e2
* src/ls.c (gobble_file): Ensure scontext is initialized
in the case where files are not statable.
* tests/ls/selinux-segfault.sh: Renamed from proc-selinux-segfault.sh,
and added test case for broken symlinks.
* tests/local.mk: Adjust for the renamed test.
* NEWS: Mention the bug fix.
Previously we would have failed immediately upon internal overflow,
which didn't output the full line being processed, and assumed
there would be another numbered line.
* src/nl.c (line_no_overflow): A new global to track overflow.
(print_lineno): Only fail if about to output an overflowed number.
(reset_lineno): A new function to refactor resetting of the number,
and which also clears line_no_overflow.
* tests/misc/nl.sh: Add a test case.
* src/csplit.c (process_regexp): Process the line suppression
in all invocations so that the last match is suppressed.
Previously with a non infinite match count,
the last regex pattern was not suppressed.
* NEWS: Mention the bug fix.
* tests/misc/csplit-suppress-matched.pl: Add a test case.
Fixes https://bugs.gnu.org/42764
Parts of this test expect that the rmdir syscall returns with EPERM,
but the root user does not see that.
* tests/rmdir/ignore.sh: Add uid_is_privileged_ guards around parts
of the test which expect rmdir() to fail with EPERM.
Reported by Nick Alcock <nix@esperi.org.uk> in
https://bugs.gnu.org/42633
* src/factor.c (strto2uintmax): Instead of here ...
(print_factors): ... skip spaces and '+' here, so that
bignums are treated like non-bignums.
* tests/misc/factor.pl (bug-gmp-plus_2_sup_128_plus_1): New test.
* tests/cp/fiemap-FMR.sh: Avoid FICLONE ioctl,
which would avoid the point of the test (fiemap testing).
Also it avoids a valgrind bug with this ioctl:
https://bugs.kde.org/show_bug.cgi?id=397605
Since -LONG_MIN results in LONG_MIN again, the operation itself is
a signed integer overflow.
This can be observed with the following calls (best if compiled
with -ftrapv or -fsanitize=undefined):
$ numfmt --padding=-9223372036854775808
$ seq 1e-9223372036854775808
Technically, the change in seq "reduces" the precision, but a double
or long double that small would be represented as 0 anyway.
* src/numfmt.c: Explicitly disallow --padding=LONG_MIN.
* src/seq.c: Treat 1e$LONG_MIN as 1e-$LONG_MAX.
* tests/misc/numfmt.pl: Add a test case.
* tests/misc/seq-precision.sh: Likewise.
Fixes https://bugs.gnu.org/41850
The previous attempt to skip that test on NFS (commit 4181fc5183)
made the test fail; it introduced two problems:
a) In the good case, i.e., when the subshell returns with exit status 0,
the test ran into framework_failure_.
b) As the subshell also runs with 'set -x', the later comparison of
/dev/null with 'err' would fail.
* tests/ls/removed-directory.sh: Revert to the style without subshell,
and add 'test -d .' to verify that 'ls' can read the removed dir.
Have the `ls` `--classify` option take an optional argument for when to
classify ("always", "auto", "never"), just like the optional argument
for `--color`. When the optional argument is not specified, default to
"always" for backwards compatibility.
* src/ls.c (usage): Update help text.
(decode_switches): Support an optional argument for --classify.
* tests/ls/classify.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the new feature.
* tests/ls/removed-directory.sh: Remove host_triplet test.
Skip this test if one cannot remove the working directory.
From a suggestion by Bernhard Voelker (Bug#39929).
* NEWS: Mention this.
* src/ls.c: Do not include <sys/sycall.h>
(print_dir): Don't worry about whether the directory is removed.
* tests/ls/removed-directory.sh: Adjust to match new (i.e., old)
behavior.
* tests/misc/env-S.pl: `env -i env` will call the system env
due to the path being cleared, so pass the absolute path
of our env binary under test to avoid that. This was seen
to be an issue on Guix where /usr/bin/env was not available.
* tests/misc/timeout-parameters.sh: Split the large timeout
handling to ...
* tests/misc/timeout-large-parameters.sh: ... here, so that
the 3 second delay is contained in its own test, and if
the test is skipped due invalid handling within timeout(1),
it will be more apparent.
Also adjust the check so we skip whenever the kernel timer
fires immediately, to handle the buggy OpenIndiana 11 kernel also.
Reported by Bruno Haible.
* init.cfg (require_bash_as_SHELL_): A new function to replace
SHELL for the current test, with bash if available.
This is useful on OpenIndiana 11 where /bin/sh was seen
to have races in handling of SIGPIPE.
* tests/misc/seq-epipe.sh: Use the new function to enforce bash.
* tests/misc/env-signal-handler.sh: Likewise.
Reported by Bruno Haible
* tests/ls/stat-free-color.sh: Check for the availability
of various stat calls individually, and add statx() and fstatat64()
to the list to check. Fix the stat counting logic to
ignore lines like "+++ exited with 0 +++".
* tests/ls/stat-free-symlinks.sh: Check syscalls other than stat().
With these adjustments, all tests pass on macOS Catalina.
* tests/dd/sparse.sh: Adjust so that systems like apfs that
don't create holes < 16 MiB do not fail erroneously.
* tests/touch/trailing-slash.sh: Darwin was seen to dereference
symlinks to files when given a trailing slash, so avoid
that particular case.
If the current directory has been removed, then "ls" confusingly
produced no output and no error message, indistinguishable from
running on an empty directory.
* src/ls.c (print_dir): Report ENOENT on GNU/Linux if readdir
finds no directory entries at all, not even "." or "..",
and a recheck with the getdents syscall returns ENOENT.
We recheck with getdents() as POSIX states that
"The directory entries for dot and dot-dot are optional".
* tests/ls/removed-directory.sh: New file.
* tests/local.mk (all_tests): Add new test.
* NEWS: Mention the change in behavior.
Reported by Owen Thomas.
strcoll() is only significant to uniq(1) if it returns 0,
and it generally only does so with buggy locales or mismatched
locales and data. Some systems may have strcoll()
return 0 for equivalent normalized unicode forms,
but for consistency across platforms strcoll() is avoided.
The various cases are defined in the new test.
This is consistent with newer POSIX standards as discussed at:
https://www.austingroupbugs.net/view.php?id=963
* src/uniq.c: s/xstrcoll/memcmp/.
* tests/local.mk: Reference the new test.
* tests/misc/uniq-collate.sh: Add a new test.
* NEWS: Mention the change in behavior.
Fixes https://bugs.gnu.org/38627
* tests/cp/proc-short-read.sh: Switch to using /proc/cpuinfo,
rather than /proc/kallsyms which was seen to vary in some cases.
Fixes https://bugs.gnu.org/39357
Since v6.10-21-ged5c4e7 `rmdir --ignore-fail-on-non-empty`
had reversed the failure status for directories that failed
to be removed for permissions reasons. I.E. it would have
returned a failure status for such non empty dirs, and vice versa.
* src/rmdir.c (errno_may_be_non_empty): Rename from the
more confusing errno_may_be_empty(), and remove the EEXIST
case (specific to Solaris), which is moot here since
handled in errno_rmdir_non_empty().
(ignorable_failure): Fix the logic error so that
_non_ empty dirs are deemed to have ignorable failures.
(main): Fix clobbering of errno by is_empty_dir().
(remove_parents): Likewise.
* tests/rmdir/ignore.sh: Add a test case.
* THANKS.in: Add reporter who fixed the errno handling.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/39364
* src/ls.c (usage): Reorganize help for --time,
and add description for --time=birth.
(do_statx): Store btime in mtime if available.
(get_stat_btime): A new function to read the creation time
from the appropriate stat structure member.
(cmp_btime): A new function to compare birth time.
(print_long_format): Output '?' when birth time unavailable.
* doc/coreutils.texi: Document --time={birth,creation}.
* tests/local.mk: Reference the new test.
* tests/ls/birthtime.sh: Add a new test.
* NEWS: Mention the new feature.
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
‘shuf -r -n 0 file’ would mistakenly read from standard input.
Problem reported by my student Jingnong Qu while reimplementing a
shuf subset in Python as an exercise in UCLA Computer Science 35L:
https://web.cs.ucla.edu/classes/fall19/cs35L/assign/assign3.html
* NEWS: Mention the fix. Also, ASCIIfy a previous item.
* src/shuf.c (main): Fix bug.
* tests/misc/shuf.sh: Add a test case for the bug.
* src/seq.c: (seq_fast): Accept STEP as a parameter and use that
to skip the output of generated numbers.
(main): Relax to using seq_fast for integer steps between 1 and 200.
For larger steps the throughput was faster using the standard
incrementing procedure.
(cmp): Use the equivalent but faster memcmp for equal len strings.
* tests/misc/seq.pl: Update fast path cases.
Addresses https://bugs.gnu.org/37241
