* tests/numfmt/mb-non-utf8.sh: Test GB18030 delimiter search.
* tests/local.mk: Reference the new test, and move
the existing numfmt.pl test from tests/misc to tests/numfmt.
Rename to less redundant names, now that we use
a separate test directory per util.
* tests/basenc/basenc-bounded-memory.sh -> .../bounded-memory.sh
* tests/basenc/basenc-large.sh -> .../large-input.sh
* tests/local.mk: Reference new names.
* NEWS: Mention the bug.
* src/digest.c (split_3): Check that the base64 digest matches the
length supported by the algorithm.
(digest_check): Check that the read digest matches the base64 length of
the algorithm's digest. The previous condition would not work for
'cksum -a blake2b -l 8 ...'.
* tests/cksum/cksum-base64-untagged.sh: New file.
* tests/local.mk (all_tests): Add the new test.
A coverage report indicated these weren't tested
(as generally the test shell builtin is used).
* tests/test/test-file.sh: Add a new test.
* tests/local.mk: Reference the new test.
* src/tail.c (file_lines): Seek to the previous block instead of the
beginning (or a little before) of the block that was just scanned.
Otherwise, the same block is read and scanned (at least partially)
again. This bug was introduced by commit v9.7-219-g976f8abc1.
* tests/tail/basic-seek.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: mention the bug fix.
base58_length() operated naively on an int
which resulted in an overflow to a negative number
for any input > 2^31-1/138, i.e. 15,561,475 bytes.
* src/basenc.c (base_length): Change input and output
parameter types from int to idx_t since this needs to
cater for the full input size in the base58 case.
(base58_length): Likewise. Also reorder the calculation
to be less exact, but doing the division first
to minimize the chance of overflow (which now on 64 bit
would only happen for inputs > around 6 Exa bytes).
* tests/basenc/basenc-large.sh: Add a new test,
that triggers with valgrind or ASAN.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
* src/fold.c (fold_file): Prefer putchar ('\n') to copying characters.
If we do not have room in the output buffer print it since it is not a
full line of text.
* tests/fold/fold-zero-width.sh: New test case.
* tests/local.mk (all_tests): Add it.
This is useful to give better test coverage at least,
and may be useful for users to tune their environment.
* bootstrap.conf: Reference the cpu-supports gnulib module.
* src/cksum.c: Use cpu_supports() rather than __builtin_cpu_supports().
* src/wc.c: Likewise.
* tests/cksum/cksum.sh: Adjust to testing all implementations.
* tests/wc/wc-cpu.sh: A new test to do likewise.
* tests/local.mk: Reference the new wc test.
This implicitly tests the previous commit to
adjust how date(1) handles multiple named format options.
Currrently it tests the following are supported:
chown --quiet --silent
date --rfc-email --rfc-822 --rfc-2822
date --uct --utc --universal
dircolors --bourne-shell --sh
dircolors --csh --c-shell
head --quiet --silent
* tests/misc/option-aliases.sh: A new test to ensure all
option aliases supported by a command are supported.
* Reference the new test.
* src/digest.c: Include sha3.h.
(BLAKE2B_MAX_LEN): Rename to
DIGEST_MAX_LEN since it is also used for SHA-3.
(sha3_sum_stream): New function.
(enum Algorithm, algorithm_args, algorithm_args, algorithm_types)
algorithm_tags, algorithm_bits, cksumfns, cksum_output_fns): Add entries
for SHA-3.
(usage): Mention that SHA-3 is supported. Mention requirements for
--length with SHA-3.
(split_3): Use DIGEST_MAX_LEN instead of BLAKE2B_MAX_LEN. Determine the
length of the digest for SHA-3. Make sure it is 224, 256, 384, or 512.
(digest_file): Set the digest length in bytes. Use DIGEST_MAX_LEN
instead of BLAKE2B_MAX_LEN. Always append the digest length to SHA3 in
the output.
(main): Allow the use of --length with 'cksum -a sha3'. Use
DIGEST_MAX_LEN instead of BLAKE2B_MAX_LEN. Make sure it is 224, 256,
384, or 512.
* tests/cksum/cksum-base64.pl (@pairs): Add expected sha3 output.
(fmt): Modify the output to use SHA3-512 since that is the default.
(@Tests): Modify arguments for sha3 to use --length=512.
* tests/cksum/cksum-sha3.sh: New test, based on tests/cksum/b2sum.sh.
* tests/local.mk (all_tests): Add the test.
* bootstrap.conf: Add crypto/sha3.
* gnulib: Update to latest commit.
* NEWS: Mention the change.
* doc/coreutils.texi (cksum general options): Mention sha3 as a
supported argument to the -a option. Mention that 'cksum -a sha3'
supports the --length option. Mention that SHA-3 is considered secure.
* src/fold.c: Include mcel.h.
(count_bytes): Remove variable.
(counting_mode, last_character_width): New variables.
(shortopts, long_options): Add the option.
(adjust_column): If --characters is in used account for number of
characters instead of their width.
(fold_file): Use getline and iterate over the result with mcel functions
to handle multibyte characters.
(main): Check for the option.
* src/local.mk (src_fold_LDADD): Add $(LIBC32CONV), $(LIBUNISTRING), and
$(MBRTOWC_LIB).
* tests/fold/fold-characters.sh: New file.
* tests/fold/fold-spaces.sh: New file.
* tests/fold/fold-nbsp.sh: New file.
* tests/local.mk (all_tests): Add the tests.
* NEWS: Mention the new option.
* doc/coreutils.texi (fold invocation): Likewise.
* src/readlink.c (main): Set verbose if the POSIXLY_CORRECT environment
variable is set.
* tests/readlink/readlink-posix.sh: New file.
* tests/local.mk (all_tests): Add it.
* NEWS: Mention the change.
* doc/coreutils.texi (readlink invocation): Document the behavior of
POSIXLY_CORRECT.
Suggested for consideration by Torbjörn Granlund in:
https://lists.gnu.org/r/coreutils/2025-01/msg00000.html
* src/factor.c (PROVE_PRIMALITY): Now defaults to false.
(mp_prime_p): Help the compiler by telling it mpz_prob_prime_p
returns nonnegative.
* tests/factor/create-test.sh (bigprime): Test 2^400 - 593,
since that’s now practical.
* tests/local.mk (factor_tests): Add new test.
* src/od.c (print_n_spaces, pad_at, pad_at_overflow):
New static functions.
(struct tspec, PRINT_FIELDS, print_named_ascii, print_ascii)
(decode_one_format, write_block, main):
Use idx_t, not int, for counts that depend on the number
of bytes in an object.
(decode_one_format): Use print_n_spaces to output spaces.
(PRINT_FIELDS, print_named_ascii, print_ascii):
Use pad_at to avoid integer overflow.
(write_block): Do not use %*s to pad, as the total pad might
exceed INT_MAX. Instead, pad by hand with putchar (' ').
(main): Use pad_at_overflow to report integer overflow due to
oversize -w. Use better way to tell whether -w is used,
without needing IF_LINT.
* tests/od/big-w.sh: New test.
* tests/local.mk (all_tests): Add it.
* tests/stty/stty-invalid.sh: Adjust to what is now invalid.
* tests/stty/stty.sh: Add checks for valid speed variants.
* tests/stty/bad-speed.sh: New test to ensure unsupported speeds
are diagnosed.
* src/sort.c (begfield): Check pointer adjustment
to avoid Out-of-range pointer offset (CWE-823).
(limfield): Likewise.
* tests/sort/sort-field-limit.sh: Add a new test,
which triggers with ASAN or Valgrind.
* tests/local.mk: Reference the new test.
* NEWS: Mention bug fix introduced in v7.2 (2009).
Fixes https://bugs.gnu.org/78507
* src/tail.c (tail_file): Open files with O_NONBLOCK
if we might need async processing.
(pipe_bytes): Ignore EAGAIN read() errors.
(pipe_lines): Likewise.
* tests/tail/pid-pipe.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
Reported by Berhard Voelker.
Update to latest gnulib with new copyright year.
Run "make update-copyright" and then...
* gnulib: Update included in this commit as copyright years
are the only change from the previous gnulib commit.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
* tests/ls/selinux-segfault.sh: Move recent addition to ...
* tests/ls/selinux.sh: ... this new test that uses require_selinux_
to skip appropriately when we've built without selinux support.
Also add a non root test that checks we output '.' along with the
mode for files, to indicate a security context is present.
* tests/local.mk (Reference the new test).
* tests/ls/no-cap.sh: Move to being a root only test, since
commit v9.5-132-g2a6bed933 we now need to call setcap
to make the test effective. Otherwise we would have always
just skipped the test.
* src/factor.c (mod2): Work even if cntd <= cnta. The old version
of the code assumed that shifts by N had unspecified behavior
unless 0 <= N < wordsize. Although this assumption is portable to
all known practical platforms, the C standard says these shifts
have undefined behavior and some pedantic platforms check this.
* tests/factor/create-test.sh:
* tests/local.mk (factor_tests): New test t37.
* src/printf.c (print_formatted): Add support for %i$ indexed args.
* tests/printf/printf-indexed.sh: Add a new file of test cases.
* tests/local.mk: Reference the new test file.
* doc/coreutils.texi (printf invocation): Mention how mixed
processing of indexed and sequential references are supported,
unlike the printf(2) library function.
* NEWS: Mention the new (POSIX:2024) feature.
Addresses https://bugs.gnu.org/73068
* src/copy.h (struct cp_options): New member 'exchange'.
* src/copy.c (copy_internal): Support the new member.
* src/mv.c (EXCHANGE_OPTION): New constant.
(long_options): Add --exchange.
(usage): Document --exchange.
(main): Support --exchange.
* tests/mv/mv-exchange.sh: New test case.
* tests/local.mk (all_tests): Add it.
There have been various requests to add -h to avoid following symlinks
for security reasons. This wasn't provided previously as chmod(1)
already ignored symlinks unless specified on the command line.
Note chmod defaults to -H mode rather than the chown default of -P,
as usually chown can work directly on symlinks and so defaults
to not traversing those specified on the command line.
Note FreeBSD chmod does default to -P mode, but we retain the -H mode
default also for compatibility with existing chmod behavior.
Adding -HLP will allow chmod to disable traversing CLI symlinks to dirs.
Adding -h will allow to disable following CLI symlinks to files/dirs,
also operating on all symlinks on systems that support that.
Adding --dereference will be significant with -H (the default). I.e.
symlinks to dirs not recursed, but symlinks are dereferenced.
Adding these options will also be consistent with chown(1), chgrp(1),
and chmod(1) on other systems.
Note since chmod(1) currently ignores symlinks by default,
and -h is primarily a mechanism to avoid following symlinks, rather than
for operating on the symlink itself, we make -h try to chmod a symlink,
but ignore ENOTSUP. In that way we're consistent with chown(1)
where it also ignores ENOTSUP for symlinks, and we don't fail when
trying to be extra secure with command line params.
* doc/coreutils.texi (chmod invocation): Reference the -H,-L,-P
descriptions, and adjust the corresponding macros to say
the default is -H or -P as appropriate.
Add --dereference and -h,--no-dereference descriptions.
* man/chmod.x: Adjust discussion of symlink handling.
* src/chmod.c (main): Accept new options and set
fts flags appropriately.
(process_file): Process / dereference symlinks as necessary.
* src/system.h (emit_symlink_recurse_options): A new function
refactored from chown.c and chmod.c usage().
* tests/chmod/symlinks.sh: New test for the new options.
* tests/local.mk: Reference the new test.
* NEWS: Mention the new feature.
Since this functionality is recently available
in the exch(1) utility from util-linux,
it was thought best not to complicate mv with it.
This reverts commit 6cd2d5e533
renameat2() syscall allows atomically swapping 2 paths on one
file system. Expose this ability to the user with --swap.
* doc/coreutils.texi: Describe mv --swap option.
* src/mv.c (main): Support --swap.
* tests/mv/mv-swap.sh: Add test for mv -x.
* tests/local.mk: Reference new test.
* NEWS: Mention the new option.
When recursively copying files into OS trees, it often happens that
some subdirectory of the source directory is a symlink in the target
directory. Currently, cp will fail in that scenario with the error:
"cannot overwrite non-directory %s with directory %s"
However, we'd like cp in this scenario to follow the destination
directory symlink and copy the files into the symlinked directory
instead. Let's support this by adding a new option
--keep-directory-symlink that makes cp follow destination directory
symlinks.
We name the option --keep-directory-symlink to keep consistent with
tar which has the same option with the same effect.
* doc/coreutils.texi (cp invocation): Describe the new option.
* src/copy.h: Add the new setting.
* src/copy.h: Adjust to follow symlinks if setting enabled.
* src/cp.c (usage): Describe the new option.
(main): Accept the new option.
* tests/cp/keep-directory-symlink.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the new feature.
Update to latest gnulib with new copyright year.
Run "make update-copyright" and then...
* gnulib: Update included in this commit as copyright years
are the only change from the previous gnulib commit.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Manually update copyright year,
until we fully sync with gnulib at a later stage.
* tests/sample-test: Adjust to use the single most recent year.
* doc/coreutils.texi (chown invocation): Convert --from option
description to a macro and call from ...
(chgrp description): ... here.
* src/chown-core.h (emit_from_option_description): A new function
refactored from ...
* src/chown.c (usage): ... here, and called from ...
* src/chgrp.c (usage): ... here.
(main): Accept the --from option as chown(1) does.
* po/POTFILES.in: Add chown-core.h as now translated.
* tests/chown/basic.sh: Decouple the root user from id 0.
* tests/chgrp/from.sh: A new test largely based on chown/basic.sh.
* tests/local.mk: Reference the new test.
* NEWS: Mention the new feature.
Suggested by Ed Neville.
* src/tail.c (file_lines): Ensure we use a buffer size >= PAGE_SIZE when
searching backwards to avoid seeking within a file,
which on sysfs files is accepted but also returns no data.
* tests/tail/tail-sysfs.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/67490
* src/ls.c (print_long_format): Use correct column width,
introduced due to a copy/paste error in commit v9.4-2-gcbb6dfec5
* tests/ls/size-align.sh: Add a test.
* tests/local.mk: Reference the new test.
Fixes https://bugs.gnu.org/66919
* NEWS: Mention this.
* bootstrap.conf (gnulib_modules): Remove cu-ctype, as this module
is now more trouble than it’s worth. All uses removed.
Add skipchars.
* gl/lib/cu-ctype.c, gl/lib/cu-ctype.h, gl/modules/cu-ctype:
Remove.
* gl/lib/skipchars.c, gl/lib/skipchars.h, gl/modules/skipchars:
* tests/misc/join-utf8.sh:
New files.
* src/join.c: Include skipchars.h and mcel.h instead of cu-ctype.h.
(tab): Now mcel_t, not int. All uses changed.
(output_separator, output_seplen): New static vars.
(eq_tab, newline_or_blank, comma_or_blank): New functions.
(xfields, prfields, prjoin, add_field_list, main):
Support multi-byte characters.
* src/numfmt.c: Include ctype.h, skipchars.h.
Do not include cu-ctype.h.
(newline_or_blank): New function.
(next_field): Support multi-byte characters.
* src/sort.c: Include ctype.h instead of cu-ctype.h.
(inittables): Open-code field_sep since it no longer exists.
‘sort’ is not multi-byte safe yet, but when it is this code
will need revamping anyway.
* src/uniq.c: Include mcel.h and skipchars.h instead of cu-ctype.h.
(newline_or_blank): New function.
(find_field): Support multi-byte characters.
* tests/local.mk (all_tests): Add tests/misc/join-utf8.sh
Most of this just affects commentary and documentations. The only
significant behavior change is translating author names via
proper_name_lite rather than proper_name_utf8, or not translating
them at all. proper_name_lite is good enough for coreutils and
avoids the bloat that had coreutils not using Gnulib proper_name.
* bootstrap.conf (gnulib_modules): Use propername-lite instead
of propername.
(XGETTEXT_OPTIONS): Look for proper_name_lite instead of for
proper_name_utf8.
* cfg.mk (local-checks-to-skip): Remove
sc_proper_name_utf8_requires_ICONV, since we no longer use
proper_name_utf8.
(old_NEWS_hash): Update.
(sc_check-I18N-AUTHORS): Remove; no longer needed.
* tests/split/l-chunk.sh: Move the "expensive" portion to ...
* tests/split/l-chunk-root.sh: .. A new test split from l-chunk.sh
which uses an isolated TMPDIR, rather than exhausting /tmp,
as that gives false positive failures with some other coreutils tests
like tac-2-nonseekable.sh and shuf-reservoir.sh at least.
* tests/local.mk: Reference the new test.
* src/ls.c (gobble_file): Ensure we lstat() a symlink
specified on the command line, if we receive ELOOP from stat().
* tests/ls/symlink-loop.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/63931
