* tests/ls/stat-vs-dirent.sh: Skip the test if we can't stat(1),
as the file may have been removed, or have a malformed name
due to '\n' etc. in the file name.
Adjust to output the file name if any name parameter is passed.
This is consistent with sum -s, cksum, and sum implementations
on other platforms. This should not cause significant compat
issues, as multiple fields are already output, and so already
need to be parsed.
* src/sum.c (bsd_sum_file): Output the file name
if any name parameter is passed.
* tests/misc/sum.pl: Adjust accordingly.
* doc/coreutils.texi (sum invocation): Likewise.
* NEWS: Mention the change in behavior.
* tests/misc/help-version.sh: Test that /dev/full causes
shell printf to fail. This ports better to NetBSD 9.88.46,
where it doesn’t. Problem reported by Nelson H. F. Beebe.
* tests/misc/help-version.sh: Merge gzip-related changes
back from gzip/tests/help-version. This fixes problems
when TERM is not 'dumb', and should simplify maintenance.
Emil Lundberg <lundberg.emil@gmail.com> reports in
https://bugs.gnu.org/49741 about a 'basenc --base64 -d' decoding bug.
The input buffer length was not divisible by 3, resulting in
decoding errors.
* NEWS: Mention fix.
* src/basenc.c (DEC_BLOCKSIZE): Change from 1024*5 to 4200 (35*3*5*8)
which is divisible by 3,4,5,8 - satisfying both base32 and base64;
Use compile-time verify() macro to enforce the above.
* tests/misc/basenc.pl: Add test.
This patch modifies basenc to prefer signed integers to
unsigned, as signed are less error-prone.
This patch also updates Gnulib to to latest, which updates Gnulib’s
base32 and base64 modules to prefer signed to unsigned integers.
* src/basenc.c: Include idx.h.
(struct base2_decode_context): Use unsigned char, not unsigned
for an octet that must fit in an unsigned char.
(base_encode, struct base_decode_context)
(base64_decode_ctx_wrapper, prepare_inbuf, base64url_encode)
(base64url_decode_ctx_wrapper, base32_decode_ctx_wrapper)
(base32hex_encode, base32hex_decode_ctx_wrapper, base16_encode)
(base16_decode_ctx, z85_encode, Z85_HI_CTX_TO_32BIT_VAL)
(z85_decoding, z85_decode_ctx, base2msbf_encode)
(base2lsbf_encode, base2lsbf_decode_ctx, base2msbf_decode_ctx)
(wrap_write, do_encode, do_decode, main):
Prefer signed integers to unsigned.
(main): Treat extremely large wrap columns as if they were
infinite; that’s good enough. Since we’re now using xstrtoimax,
this allows ‘-w -0’ (same as ‘-w 0’).
* tests/misc/base64.pl (gen_tests): -w-0 is no longer an error.
This better tests the SEEK_HOLE logic which
replaced the original fiemap hole identification logic.
Also it avoids a false failure in sparse-2.sh
on reflink supporting file systems, where we
try to correlate the file sizes produced by cp and dd.
* tests/cp/sparse-2.sh: s/cp/cp --reflink=never/
* tests/cp/sparse-extents-2.sh: Likewise.
* tests/cp/sparse-extents.sh: Likewise.
* tests/cp/sparse-perf.sh: Likewise.
* tests/cp/sparse.sh: Likewise.
Fixes https://github.com/coreutils/coreutils/issues/54
We must delay handling when \r is the last character
of the buffer being processed, as the next character
may or may not be \n.
* src/cat.c (pending_cr): A new global to record whether
the last character processed (in -E mode) is '\r'.
(cat): Honor pending_cr when processing the start of the buffer.
(main): Honor pending_cr if no more files to process.
* tests/misc/cat-E.sh: Add test cases.
Fixes https://bugs.gnu.org/49925
In documentation and comments, don’t assume that secondary storage
devices are disk devices. Similarly, don’t assume that main memory
uses magnetic cores, which became obsolete in the 1970s.
* src/du.c (usage):
* src/ls.c (usage):
* src/shred.c (usage): Reword to avoid “disk” in usage messages.
* NEWS, doc/coreutils.texi (General output formatting):
* src/ls.c (usage):
Document this.
* src/ls.c (ZERO_OPTION): Rename from NULL_OPTION.
All uses changed.
(long_options): Rename --null to --zero.
(dired_dump_obstack, main, print_dir): Use '\n' instead of
eolbyte where eolbyte must equal '\n'.
(decode_switches): Decode --zero instead of --null.
--zero also implies -1, -N, --color=none, --show-control-chars.
Use easier-to-decipher code to set ‘format’ and ‘dired’.
Reject attempts to combine --dired and --zero.
* tests/local.mk: Adjust to test script renaming.
* tests/ls/zero-option.sh: Rename from tests/ls/null-option.sh,
and test --zero instead of --null.
In preparation for changing the default device number
representation (to decomposed decimal), provide more
formatting options for device numbers.
These new (FreeBSD compat) formatting options are added:
%Hd major device number in decimal (st_dev)
%Ld minor device number in decimal (st_dev)
%Hr major device type in decimal (st_rdev)
%Lr minor device type in decimal (st_rdev)
%r (composed) device type in decimal (st_rdev)
%R (composed) device type in hex (st_rdev)
* doc/coreutils.texi (stat invocation): Document new formats.
* src/stat.c (print_it): Handle the new %H and %L modifiers.
(print_statfs): Adjust to passing the format as two chars
rather than an int. Using an int was introduced in commit db42ae78,
but using separate chars is cleaner and more extensible.
(print_stat): Likewise. Handle any modifiers and the new 'r' format.
(usage): Document the new formats.
* tests/misc/stat-fmt.sh: Add a test case for new modifiers.
Addresses https://bugs.gnu.org/48960
* tests/cp/sparse-2.sh: Double check cp --sparse=always,
with dd conv=sparse, in the case where the former didn't
create a sparse file. Now that this test is being newly run
on macos, we're seeing a failure due to seek() not creating
holes on apfs unless the size is >= 16MiB.
fiemap is no longer the default copy implementation,
so check for SEEK_DATA support instead as that's preferred.
This will ensure better test coverage on systems without fiemap.
* init.cfg: Replace fiemap_capable_ with seek_data_capable_.
This is best supported with python 3 so prefer that.
* tests/seek-data-capable: A new test script checking for
SEEK_DATA support on the passed file name,
called from seek_data_capable_.
* tests/fiemap-capable: Remove no longer used probing script.
* tests/cp/fiemap-perf.sh: Renamed to tests/cp/sparse-perf.sh
* tests/cp/fiemap-2.sh: Renamed to tests/cp/sparse-2.sh
* tests/cp/fiemap-extents.sh: Renamed to tests/cp/sparse-extents.sh
* tests/cp/sparse-fiemap.sh: Renamed to tests/cp/sparse-extents-2.sh
* tests/cp/fiemap-FMR.sh: Renamed to tests/cp/copy-FMR.sh
* tests/local.mk: Reference the renamed tests.
On newer systems like Fedora 34 and openSUSE Tumbleweed, ls(1) calls
newfstatat(STDOUT_FILENO, ...), but only when there is something to
output.
* tests/ls/stat-free-color.sh: Add -a option to the reference invocation
of ls, thus enforcing something gets output.
This helps identify the outliers for long filenames, and also produces
a more compact display of columns when listing a directory with many
entries of various widths.
* src/ls.c (sort_type, sort_types, sort_width): New sort_width sort
type.
(sort_args): Add "width" sort arg.
(cmp_width, fileinfo_width): New sort function and helper for file name
width.
(quote_name_width): Add function prototype declaration.
(usage): Document --sort=width option.
* doc/coreutils.texi: Document --sort=width option.
* tests/ls/sort-width-option.sh: New test for --sort=width option.
* tests/local.mk: Reference new test.
* NEWS: Mention the new feature.
Use cpuid to detect CPU support for hardware instruction.
Fall back to slice by 8 algorithm if not supported.
A 500MiB file improves from 1.40s to 0.67s on an i3-2310M
* configure.ac [USE_PCLMUL_CRC32]: A new conditional,
set when __get_cpuid() and clmul compiler intrinsics are supported.
* src/cksum.c (pclmul_supported): A new function using __get_cpuid()
to determine if pclmul instructions are supported.
(cksum): A new function refactored from cksum_slice8(),
which calls pclmul_supported() and then cksum_slice8()
or cksum_pclmul() as appropriate.
* src/cksum.h: Export the crctab array for use in the new module.
* src/cksum_pclmul.c: A new module to implement using pclmul intrinsics.
* po/POTFILES.in: Reference the new cksum_pclmul module.
* src/local.mk: Likewise. Note we build it as a separate library
so that it can be portably built with separate -mavx etc. flags.
* tests/misc/cksum.sh: Add new test modes for pertinent buffer sizes.
GNU/Linux is unusual here in that rmdir("symlink/") returns ENOTDIR,
whereas Solaris and FreeBSD at least, will follow the symlink
and remove the target directory. We don't make the behavior
on Linux kernels consistent, but at least clarify
the confusing error message.
* src/rmdir (main): Output a specific error message for the above case.
(remove_parents): In the error message, don't assume intermediate paths
are directories, as they could be symlinks.
* tests/rmdir/symlink-errors.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the improvement.
This regression was introduced in commit COREUTILS-6_8-58-g553d347d3
* src/pr.c (init_parameters): Process tabs for multiple columns.
* tests/pr/pr-tests.pl: Add test cases.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/46422
- \r\n is common a line end combination
- catting such a file without options causes it to display normally
- overwriting the first char with $, loses info
* src/cat.c (cat): Convert \r preceeding a \n to ^M.
* tests/misc/cat-E.sh: New test.
* tests/local.mk: Reference new test.
* tests/misc/cat-proc.sh: Fix typo.
* doc/coreutils.texi (cat invocation): Mention the new behavior.
* NEWS: Mention the improvement.
This functionality regressed with the adjustments
in commit v8.25-4-g62e7af032
* src/split.c (bytes_chunk_extract): Account for already read data
when seeking into the file.
* tests/split/b-chunk.sh: Use the hidden ---io-blksize option,
to test this functionality.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/46048
Problem reported by David McCall (Bug#45886).
I introduced this problem when fixing Bug#14371.
* NEWS: Mention the fix.
* src/mkdir.c (struct mkdir_options): New members umask_ancestor,
umask_self, replacing umask_value.
(make_ancestor): Use them when temporarily adjusting umask.
(main): Set them, and set the umask to umask_self instead
of leaving it alone.
* tests/mkdir/perm.sh (tests): Add test case for bug.
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
* src/nl.c (main): Enforce the POSIX specified
behavior of assuming ':' is specified after a single
character argument to -d.
* tests/misc/nl.sh: Add a test case.
* NEWS: Mention the bug fix.
* doc/coreutils.texi (nl invocation): Mention the GNU extensions
of allowing arbitrary length and empty delimiter strings.
* src/nl.c (usage): Likewise.
* tests/misc/nl.sh: Add test cases for the GNU extensions.
The format can be determined from --options or the locale,
so it's useful to output the format string being used.
* src/date.c (show_date): Show the output format
along with the date being shown.
* tests/misc/date-debug.sh: Adjust accordingly.
Addresses https://bugs.gnu.org/44960
When removing a directory fails for some reason, and that directory
is empty, the rm_fts code gets the return value of the excise call
confused with the return value of its earlier call to prompt,
causing fts_skip_tree to be called again and the next file
that rm would otherwise have deleted to survive.
* src/remove.c (rm_fts): Ensure we only skip a single fts entry,
when processing empty dirs. I.e. only skip the entry
having successfully removed it.
* tests/rm/empty-immutable-skip.sh: New root-only test.
* tests/local.mk: Add it.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/44883
This crash was identified by Cyber Independent Testing Lab:
https://cyber-itl.org/2020/10/28/citl-7000-defects.html
and was introduced with commit v8.5-163-g3f48829c2
* src/tr.c (validate_case_classes): Don't apply these
extra case alignment checks in the --complement case,
which is even more restrictive as to the contents of SET2.
* tests/misc/tr-case-class.sh: Add a test case,
for a large SET1, which caused the length adjustment
in validate_case_classes to underflow and trigger the assert.
* NEWS: Mention the bug fix.
This crash was identified by Cyber Independent Testing Lab:
https://cyber-itl.org/2020/10/28/citl-7000-defects.html
and was introduced with commit v6.9.90-11-g4245876e2
* src/ls.c (gobble_file): Ensure scontext is initialized
in the case where files are not statable.
* tests/ls/selinux-segfault.sh: Renamed from proc-selinux-segfault.sh,
and added test case for broken symlinks.
* tests/local.mk: Adjust for the renamed test.
* NEWS: Mention the bug fix.
Previously we would have failed immediately upon internal overflow,
which didn't output the full line being processed, and assumed
there would be another numbered line.
* src/nl.c (line_no_overflow): A new global to track overflow.
(print_lineno): Only fail if about to output an overflowed number.
(reset_lineno): A new function to refactor resetting of the number,
and which also clears line_no_overflow.
* tests/misc/nl.sh: Add a test case.
* src/csplit.c (process_regexp): Process the line suppression
in all invocations so that the last match is suppressed.
Previously with a non infinite match count,
the last regex pattern was not suppressed.
* NEWS: Mention the bug fix.
* tests/misc/csplit-suppress-matched.pl: Add a test case.
Fixes https://bugs.gnu.org/42764
Parts of this test expect that the rmdir syscall returns with EPERM,
but the root user does not see that.
* tests/rmdir/ignore.sh: Add uid_is_privileged_ guards around parts
of the test which expect rmdir() to fail with EPERM.
Reported by Nick Alcock <nix@esperi.org.uk> in
https://bugs.gnu.org/42633
* src/factor.c (strto2uintmax): Instead of here ...
(print_factors): ... skip spaces and '+' here, so that
bignums are treated like non-bignums.
* tests/misc/factor.pl (bug-gmp-plus_2_sup_128_plus_1): New test.
* tests/cp/fiemap-FMR.sh: Avoid FICLONE ioctl,
which would avoid the point of the test (fiemap testing).
Also it avoids a valgrind bug with this ioctl:
https://bugs.kde.org/show_bug.cgi?id=397605
Since -LONG_MIN results in LONG_MIN again, the operation itself is
a signed integer overflow.
This can be observed with the following calls (best if compiled
with -ftrapv or -fsanitize=undefined):
$ numfmt --padding=-9223372036854775808
$ seq 1e-9223372036854775808
Technically, the change in seq "reduces" the precision, but a double
or long double that small would be represented as 0 anyway.
* src/numfmt.c: Explicitly disallow --padding=LONG_MIN.
* src/seq.c: Treat 1e$LONG_MIN as 1e-$LONG_MAX.
* tests/misc/numfmt.pl: Add a test case.
* tests/misc/seq-precision.sh: Likewise.
Fixes https://bugs.gnu.org/41850
The previous attempt to skip that test on NFS (commit 4181fc5183)
made the test fail; it introduced two problems:
a) In the good case, i.e., when the subshell returns with exit status 0,
the test ran into framework_failure_.
b) As the subshell also runs with 'set -x', the later comparison of
/dev/null with 'err' would fail.
* tests/ls/removed-directory.sh: Revert to the style without subshell,
and add 'test -d .' to verify that 'ls' can read the removed dir.
Have the `ls` `--classify` option take an optional argument for when to
classify ("always", "auto", "never"), just like the optional argument
for `--color`. When the optional argument is not specified, default to
"always" for backwards compatibility.
* src/ls.c (usage): Update help text.
(decode_switches): Support an optional argument for --classify.
* tests/ls/classify.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the new feature.
