* cfg.mk (sc_some_programs_must_avoid_exit_failure): Adjust to
avoid false positive.
(sc_prohibit_exit_write_error): A new syntax check to prohibit
open coding error(..., "write error"); instead directing to use...
* src/system.h (write_error): ... a new function to clear stdout errors
before we explicitly diagnose a write error and exit.
* src/basenc.c: Use write_error() to ensure no repeated diagnostics.
* src/cat.c: Likewise.
* src/expand.c: Likewise.
* src/factor.c: Likewise.
* src/paste.c: Likewise.
* src/seq.c: Likewise.
* src/shuf.c: Likewise.
* src/split.c: Likewise.
* src/tail.c: Likewise.
* src/tr.c: Likewise.
* src/unexpand.c: Likewise.
* tests/misc/write-errors.sh: Remove TODOs for the fixed utilities:
expand, factor, paste, shuf, tr, unexpand.
* src/digest.c (problematic_chars): This recently introduced
function does not modify state so is pure, even though GCC 13.1 at least
did not warn about that attribute being appropriate.
* src/digest.c (digest_check): Also escape in the case that the
file name contains '\'.
* tests/cksum/md5sum-bsd.sh: Add a test case.
* doc/coreutils.texi (md5um invocation): Clarify escaping operation.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/64392
Support -b, --binary, and -t, --text
to allow full emulation of older utilities with:
exec cksum -a $algo --untagged "$@"
Note this would diverge from OpenBSD's support of cksum -b.
* src/digest.c: Change -b to mean --binary, not --base64 in all cases.
Accept -b and -t in all cases. Keep --binary and --text undocumented
for cksum.
* tests/cksum/cksum-base64.pl: s/-b/--base64/.
* tests/cksum/cksum-a.sh: Ensure cksum supports -b and -t appropriately.
* NEWS: Mention the change in behavior.
* bootstrap.conf (gnulib_modules): Add stdckdint.
Also, in C source code, prefer C23 macros like ckd_add
to their Gnulib near-equivalents like INT_ADD_WRAPV.
Include <stdckdint.h> as needed.
* src/who.c (idle_string): Avoid signed integer overflow
if the superuser messes with the clock in bizarre ways.
Remove an ‘assume’ that wasn’t correct under this scenario.
* src/cut.c (cut_file):
* src/nl.c (nl_file): Pacify GCC Bug#109613 in a better way, by
narrowing the coverage of the ‘assume’ so that bugs in the
no-longer-covered part are not masked.
* src/fmt.c (get_paragraph):
* src/stty.c (display_changed, display_all): Omit calls to
‘assume’ that are present only to pacify false positives by Parfait
<https://labs.oracle.com/pls/apex/f?p=94065:12:17236785746387:13>,
which went in-house in 2012 and never came back.
Now that Gnulib’s ‘error’ module does proper static checking
for not returning, we need no longer use the ‘die’ macro.
This makes code easier to read for people that are used to ‘error’.
* cfg.mk (error_fns, exclude_file_name_regexp): Remove ‘die’.
(sc_die_EXIT_FAILURE): Remove.
* src/die.h: Remove. All includes removed. All calls to ‘die’
changed back to calls to ‘error’.
* src/install.c (get_ids): Use quoteaf (problem found with
make syntax-check).
* src/system.h: Include error.h, since some of our macros call ‘error’.
Stop including error.h elsewhere.
This modernizes the source code somewhat, to take advantage
of advances in GCC over the years, and Gnulib’s ‘assure’ module.
Include assure.h in files that now need it.
Do not include assert.h directly; it’s no longer needed.
* bootstrap.conf (gnulib_modules): Add ‘assure’.
* gl/lib/randread.c (randread_error):
* src/chmod.c (describe_change):
* src/chown-core.c (describe_change):
* src/cp.c (decode_preserve_arg):
* src/head.c (diagnose_copy_fd_failure):
* src/ls.c (parse_ls_color):
* src/od.c (decode_one_format):
* src/split.c (main):
* src/test.c (binary_operator, posixtest):
Prefer affirm to abort, since it has better diagnostics in the
normal case and better performance with -DNDEBUG.
* gl/lib/xdectoint.c, src/die.h: Include stddef.h, for unreachable.
* gl/lib/xdectoint.c: Do not include verify.h; no longer needed.
* gl/lib/xdectoint.c (__xnumtoint):
* src/die.h (die):
Prefer C23 unreachable () to assume (false).
* gl/lib/xfts.c (xfts_open):
* src/basenc.c (base32hex_encode):
* src/copy.c (abandon_move, copy_internal, valid_options):
* src/cut.c (cut_fields):
* src/df.c (alloc_field, decode_output_arg, get_dev):
* src/du.c (process_file, main):
* src/echo.c (usage):
* src/factor.c (udiv_qrnnd, mod2, gcd2_odd, factor_insert_large)
(mulredc2, factor_using_pollard_rho, isqrt2, div_smallq)
(factor_using_squfof):
* src/iopoll.c (iopoll_internal, fwrite_wait):
* src/join.c (add_field):
* src/ls.c (dev_ino_pop, main, gobble_file, sort_files):
* src/mv.c (do_move):
* src/od.c (decode_format_string, read_block, dump, main):
* src/remove.c (rm):
* src/rm.c (main):
* src/sort.c (stream_open):
* src/split.c (next_file_name, lines_chunk_split):
* src/stdbuf.c (main):
* src/stty.c (set_speed):
* src/tac-pipe.c (line_ptr_decrement, line_ptr_increment):
* src/touch.c (touch):
* src/tr.c (find_bracketed_repeat, get_next)
(validate_case_classes, get_spec_stats, string2_extend, main):
* src/tsort.c (search_item, tsort):
* src/wc.c (main):
Prefer affirm to assert, as it allows for better static
checking when compiling with -DNDEBUG.
* src/chown-core.c (change_file_owner):
* src/df.c (get_field_list):
* src/expr.c (printv, null, tostring, toarith, eval2):
* src/ls.c (time_type_to_statx, calc_req_mask, get_funky_string)
(print_long_format):
* src/numfmt.c (simple_strtod_fatal):
* src/od.c (decode_one_format):
* src/stty.c (mode_type_flag):
* src/tail.c (xlseek):
* src/tr.c (is_char_class_member, get_next, get_spec_stats)
(string2_extend):
Prefer unreachable () to abort () or assert (false) when merely
pacifying the compiler, e.g., in a switch statement on an enum
where all cases are covered.
* src/copy.c (valid_options): Now returns void; the bool was useless.
Caller no longer needs to assert.
* src/csplit.c (find_line):
* src/expand-common.c (next_file):
* src/shred.c (incname):
* src/sort.c (main):
* src/tr.c (append_normal_char, append_range, append_char_class)
(append_repeated_char, append_equiv_class):
* src/tsort.c (search_item):
Omit assert, since the hardware will check for us.
* src/df.c (header_mode): Now the enum type it should have been.
* src/du.c (process_file):
* src/ls.c (assert_matching_dev_ino):
* src/tail.c (valid_file_spec):
* src/tr.c (validate_case_classes):
Mark defns with MAYBE_UNUSED if they’re not used when -DNDEBUG.
* src/factor.c (prime_p, prime2_p, mp_prime_p): Now ATTRIBUTE_PURE.
Prefer affirm to error+abort. No need to translate this diagnostic.
* src/fmt.c (get_paragraph):
* src/stty.c (display_changed, display_all, sane_mode):
* src/who.c (idle_string):
Prefer assume to assert, since the goal is merely pacification
and assert doesn’t pacify anyway if -DNDEBUG is used.
* src/join.c (decode_field_spec):
Omit unreachable abort.
* src/ls.c (assert_matching_dev_ino, main):
* src/tr.c (get_next):
Prefer assure to assert, since the check is relatively expensive
and won’t help static analysis.
* src/ls.c (main):
Prefer static_assert to assert of a constant expression.
(format_inode): Redo to make it clear that buflen doesn’t matter,
and that buf must have a certain number of bytes. All callers changed.
This pacifies -Wformat-overflow.
* src/od.c (decode_one_format):
Omit an assert that tested for obviously undefined behavior,
as the compiler could optimize it away anyway.
* src/od.c (decode_one_format, decode_format_string):
Prefer ATTRIBUTE_NONNULL to runtime checking.
* src/stat.c: Do not include <stddef.h> since system.h does that now.
* src/sync.c (sync_arg):
Prefer unreachable () to assert (true), which was a typo.
* src/system.h: Include stddef.h, for unreachable.
* src/tail.c (xlseek): Simplify by relying on ‘error’ to exit.
* src/digest.c (split_3): Reinstate the check for whitespace after the
digest portion of the line, so that we exit early before inspecting
the file name which would be outside the passed buffer in the case
where the input does not contain a newline.
* tests/cksum/b2sum.sh: Add a test case.
* NEWS: Mention the bug fix.
* THANKS.in: Add Frank Busse who has reported multiple bugs using KLEE.
Fixes https://bugs.gnu.org/64229
* NEWS: Mention the error message to aid those searching
for solutions to the issue, and mention cksum also
as that was confirmed to fix the error with the adjusted
cpu feature detection, as discussed at https://bugs.debian.org/1037264
* src/cksum.c: Cleanup syntax-check failure from previous commit.
* src/cksum.c (cksum_pclmul) [!CRCTAB && !USE_PCLMUL_CRC32]:
Remove macro.
(cksum_fp): No longer file-scope.
(pclmul_supported): Define only if USE_PCLMUL_CRC32.
This omits the debug output "using generic hardware support"
for simplicity and consistency with wc’s output.
(crc_sum_stream) [!USE_PCLMUL_32]: No need for static function pointer.
* src/wc.c (wc_lines_p) [USE_AVX2_WC_LINECOUNT]: No longer file-scope.
(wc) [USE_AVX2_WC_LINECOUNT]: Check for avx2 support at most once,
which was surely the code’s original intent.
(wc) [!USE_AVX2_WC_LINECOUNT]: No need for static function pointer.
This fixes a typo in the previous patch.
Problem reported by Pádraig Brady <https://bugs.gnu.org/64058#11>.
* src/cksum.c (pclmul_supported): Also require AVX support
to use cksum_pclmul.
Problem reported by Dave Hansen <https://bugs.gnu.org/64058>.
Apply similar change to cksum and pclmul, too.
* NEWS: Mention wc fix.
* configure.ac (cpuid_exists, get_cpuid_count_exists):
Remove. All uses removed, since we no longer use __get_cpuid or
__get_cpuid_count.
(pclmul_intrinsic_exists, avx2_intrinsic_exists): Set to no if
__builtin_cpu_supports calls cannot be compiled.
(HAVE_PCLMUL_INTRINSIC, HAVE_AVX2_INTRINSIC): Remove; unused.
Simplify surrounding code because of this.
* src/cksum.c (pclmul_supported):
* src/wc.c (avx2_supported):
Use __builtin_cpu_supports instead of doing it by hand.
Simplify surrounding code because of this.
* src/dircolors.hin: Sort backup section by extension.
Treat .dpkg-new and .dpkg-tmp as backup files.
Treat .crdownload (Chromium based browsers' partial download)
as a backup file.
* src/dd.c (parse_integer): Use recursion to support more than two
multipliers. Also protect suffix[-1] access to ensure we don't
inspect before the passed string.
* tests/dd/bytes.sh: Add test cases.
* doc/coreutils.texi (dd invocation): Note the support for specifying
many multipliers in a number.
* NEWS: Mention the bug fix.
Fixes https://bugs.debian.org/1037275
* doc/coreutils.texi (od invocation): Remove mention of ASCII,
as all printable characters in unibyte locales are output.
* src/od.c (usage): Clarify that only NUL terminated strings
are displayed, and that it's printable chars, not only graphic chars
that are output. I.e., spaces are output also if part of the string.
Reported at https://bugs.ddebian.org/1037217
* src/ls.c (gobble_file): Ensure we lstat() a symlink
specified on the command line, if we receive ELOOP from stat().
* tests/ls/symlink-loop.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/63931
* src/ls.c (gobble_file): stat() symlinks directly,
rather than their targets. This will be more consistent
with how symlinks are generally accessed.
(make_link_name): Remove no longer used function.
Addresses https://bugs.gnu.org/63931
Note mktemp --suffix has the same inconsistency,
but mktemp -d does support creating dirs
so probably best to leave that as is.
* src/split.c (main): Check for trailing /.
* tests/split/additional-suffix.sh: Augment the test.
Reported in https://bugs.debian.org/1036827
* src/dd.c: Don't include no longer used error.h.
Use quoteaf() rather than quote() to quote appropriate for the shell
and to avoid the syntax-check failure,
* src/stty.c: Use quoteaf() rather than quotef()
to have more consistent quoting of the invalid arg.
* src/dd.c (_GL_NO_INLINE_ERROR): Remove; no longer needed.
(diagnose): Rename from nl_error and omit first arg since it is
always zero. All uses changed.
(error): Remove macro.
This reverts commit 800c86d5, as that was deemed too invasive.
We do keep the change to tee.c to allow using -O3 without warnings.
For other optimization options like -O0, -Og, -O1, -Os,
one can use WERROR_CFLAGS= to stop warnings inducing a build failure.
Allow easily building a debug build for example with:
make CFLAGS='-O0 -ggdb'
False -Wmaybe-uninitialized warnings hit in different
places depending on the compiler passes used.
These changes were tested with gcc 10.2.1, 12.2.1, and 13.1.1 like:
for o in g s z fast 0 1 2 3; do
make clean && make -j$(nproc) CFLAGS="-O$o" || break
done
* src/digest.c: Disable -Wmaybe-uninitialized that gives
false positive here at -O0.
* src/ln.c: Avoid -Wmaybe-uninitialized that gives
false positive here at -O1.
* src/pr.c: Likewise.
* src/sort.c: Likewise.
* src/tee.c: Avoid -Wmaybe-uninitialized that gives
false positive here at -O3 on gcc 13.1.1 at least.
* src/cp.c: Avoid -Wmaybe-uninitialized that gives
false positive here at -Os on gcc 13.1.1 at least.
* src/copy.c: Avoid -Wmaybe-uninitialized that gives
false positive here at -Og on gcc 13.1.1 at least.
* src/head.c: Likewise.
* src/paste.c: Likewise.
As split is often dealing with large files,
ensure we indicate to the kernel our sequential access pattern.
This was seen to operate 5% faster when reading from SSD,
as tested with:
dd bs=1M count=2K if=/dev/urandom of=big.in
for split in split.orig split; do
# Ensure big file is not cached
dd of=big.in oflag=nocache conv=notrunc,fdatasync count=0 status=none
# Test read efficiency
CWD=$PWD; (cd /dev/shm && time $CWD/src/$split -n2 $CWD/big.in)
done
real 0m9.039s
user 0m0.055s
sys 0m3.510s
real 0m8.568s
user 0m0.056s
sys 0m3.752s
* src/split.c (main): Use fdadvise to help the kernel
choose a more appropriate readahead buffer.
* NEWS: Mention the improvement.
Before:
$ pr --expand-tabs=
pr: '-e' extra characters or invalid number in the argument:
‘SHELL=/bin/bash’: Value too large for defined data type
After:
$ pr --expand-tabs=
pr: '-e': Invalid argument: ‘’
* src/pr.c (getoptarg): Ensure we don't parse beyond the
end of an empty argument, thus outputting arbitrary stack
info in subsequent error messages.
Addresses https://bugs.debian.org/1035596
This doesn’t change behavior; it just clarifies the code a bit.
* src/cp.c (re_protect): New arg DST_SRC_NAME, for clarity, and so
that we need to skip '/'s only once. Caller changed.
Rename a couple of local variables to try to make things clearer.
* src/cp.c (re_protect): Ensure copy_acl() is passed an absolute path.
* tests/cp/cp-parents.sh: Add a test case.
* NEWS: Mention the bug.
Fixes https://bugs.gnu.org/63245
* src/csplit.c, src/fmt.c, src/make-prime-list.c, src/nohup.c:
Add pragmas to pacify GCC 13 when coreutils is configured
with --enable-gcc-warnings='expensive'.
* src/chmod.c (main): Use xpalloc instead of X2REALLOC,
and make the corresponding variables signed instead of unsigned.
When reallocating the buffer, this grows it by a factor of 1.5, not 2.
This also pacifies gcc -Wanalyzer-null-dereference.
* src/csplit.c (load_buffer): Refactor for clarity.
This also xpacifies gcc -Wanalyzer-use-of-uninitialized-value.
When reallocating the buffer, grow it by a factor of 1.5, not 2.
* src/pr.c (init_parameters): Ensure we avoid a 0 lines_per_body
which was possible when adjusting for double spacing.
That caused print_page() to always return true,
causing an infinite loop.
* tests/pr/pr-tests.pl: Add a test case.
* NEWS: Mention the fix.
Fixes https://bugs.debian.org/1034808
Since skipping of files is central to the operation of -i and -u,
and with -u one may be updating few files out of many,
reinstate the verbosity of this functionality as it was before 9.3.
* src/copy.c (copy_internal): Only output "skipped" message
with --debug. Also adjust so message never changes with --debug.
* tests/cp/cp-i.sh: Adjust accordingly.
* tests/mv/mv-n.sh: Likewise.
* tests/cp/debug.sh: Add explicit test case for message.
* NEWS: Mention the change in behavior.
