* tests/misc/numfmt.pl: Verify that printf field width specs
count characters and not bytes before enabling locale tests.
This was seen on FreeBSD 14.0 and Solaris 11 OpenIndiana.
Reported by Bruno Haible
* tests/misc/numfmt.pl: Some systems with the fr_FR.UTF-8
locale installed, do not have a thousands grouping character defined.
In this case we skip the locale tests which depend on a non empty
grouping character.
Some signals with values less that the max signal number for the system
do not have defined names. For example, currently on amd64 Linux,
signals 32 and 33 do not have defined names, and Android has a wider
gap of undefined names where it reserves some realtime signals.
Previously the signal listing in env ended up reusing the name
of the last printed valid signal (the repeated HUP below):
$ env --list-signal-handling true
HUP ( 1): IGNORE
HUP (32): BLOCK
HUP (38): IGNORE
..and the corresponding signal numbers were rejected as operands for the
env, kill, and timeout commands.
This patch removes the requirement that sig2str returns 0 for a signal
number associated with an operand. This allows unnamed signals to be in
the sets `env' attempts to manipulate when a --*-signal option is used
with no argument, and kill(1) and timeout(1) to send such unnamed
signals.
* src/operand2sig.c (operand2sig): Drop signame argument, accept all
signal numbers <= SIGNUM_BOUND. All callers updated.
* src/env.c (parse_signal_action_params, reset_signal_handlers)
(parse_block_signal_params, set_signal_proc_mask)
(list_signal_handling): Accept all signal numbers <= SIGNUM_BOUND,
use SIG%d for printing if necessary.
* src/kill.c (list_signals, main): Likewise.
(send_signals): Check errno from kill(3) for bad signo.
* src/timeout.c (main): Update operand2sig call.
* tests/misc/kill.sh: Test listing all signal numbers.
* NEWS: Mention the improvement.
* src/mktemp.c (main): When --suffix is specified, TEMPLATE
points to the meraged buffer DEST_NAME. As X's in the suffix are
not significant to the generated random characters, the diagnostic
for too few X's should only refer to the template portion.
* tests/misc/mktemp.pl: Adjust accordingly.
* NEWS: Mention the bug fix.
* tests/df/problematic-chars.sh: Rely on gnulib setting
this to "none" where not usable.
* tests/misc/sleep.sh: Likewise.
* tests/printf/printf-mb.sh: Likewise.
* tests/printf/printf-quote.sh: Likewise.
* tests/sort/sort-debug-keys.sh: Likewise.
Update to latest gnulib with new copyright year.
Run "make update-copyright" and then...
* gnulib: Update included in this commit as copyright years
are the only change from the previous gnulib commit.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Manually update copyright year,
until we fully sync with gnulib at a later stage.
* tests/sample-test: Adjust to use the single most recent year.
* tests/misc/numfmt.pl: Following on from v9.4-86-g615167cc4,
adjust this test accordingly. This test was being skipped
on some systems, and so only noticed now.
Reported by Jim Meyering.
* tests/misc/numfmt.pl: Determine the thousands grouping character
in use, rather than skipping locale tests when it's not a space.
For example fr_FR.UTF-8 uses "NARROW NO-BREAK SPACE" as the grouping
char on modern glibc systems at least.
* tests/sort/sort-h-thousands-sep.sh: Likewise.
For consistency with the "SI" standard, and with other coreutils
which output a lowercase 'k' in "SI" mode.
* src/numfmt.c (suffix_power): Treat 'k' like 'K' on input.
(double_to_human): Output lowercase 'k' in SI mode.
(usage): Adjust accordingly.
* doc/coreutils.texi: Mention 'k' accepted, and printed in SI mode.
* tests/misc/numfmt.pl: Adjust accordingly.
* NEWS: Mention the change in behavior.
Fixes https://bugs.gnu.org/47103
* src/join.c (xfields): Simplify and fix bug with fields
that start with a NUL byte when -t is not used.
* tests/misc/join-utf8.sh: Also test when -t is not used,
and when a field starts with NUL.
* NEWS: Mention this.
* bootstrap.conf (gnulib_modules): Remove cu-ctype, as this module
is now more trouble than it’s worth. All uses removed.
Add skipchars.
* gl/lib/cu-ctype.c, gl/lib/cu-ctype.h, gl/modules/cu-ctype:
Remove.
* gl/lib/skipchars.c, gl/lib/skipchars.h, gl/modules/skipchars:
* tests/misc/join-utf8.sh:
New files.
* src/join.c: Include skipchars.h and mcel.h instead of cu-ctype.h.
(tab): Now mcel_t, not int. All uses changed.
(output_separator, output_seplen): New static vars.
(eq_tab, newline_or_blank, comma_or_blank): New functions.
(xfields, prfields, prjoin, add_field_list, main):
Support multi-byte characters.
* src/numfmt.c: Include ctype.h, skipchars.h.
Do not include cu-ctype.h.
(newline_or_blank): New function.
(next_field): Support multi-byte characters.
* src/sort.c: Include ctype.h instead of cu-ctype.h.
(inittables): Open-code field_sep since it no longer exists.
‘sort’ is not multi-byte safe yet, but when it is this code
will need revamping anyway.
* src/uniq.c: Include mcel.h and skipchars.h instead of cu-ctype.h.
(newline_or_blank): New function.
(find_field): Support multi-byte characters.
* tests/local.mk (all_tests): Add tests/misc/join-utf8.sh
Padding of encoded data is useful in cases where
base64 encoded data is concatenated / streamed.
I.e. where there are padding chars _within_ the stream.
In other cases padding is optional and can be inferred.
Note we continue to treat partial padding as invalid,
as that would be indicative of truncation.
* src/basenc.c (do_decode): Auto pad the end of the input.
* NEWS: Mention the change in behavior.
* tests/misc/base64.pl: Adjust to not fail for missing padding.
Addresses https://bugs.gnu.org/66265
This simplifies memory allocation a bit, and removes an arbitrary
limitation from numfmt, which formerly limited cell output to 127
bytes.
* bootstrap.conf (gnulib_modules): Remove mbsalign, strncat.
Add strnlen (the code already used strnlen directly, and we were
saved only because Gnulib used the module indirectly)
* gl/lib/mbsalign.c, gl/lib/mbsalign.h, gl/modules/mbsalign:
* gl/modules/mbsalign-tests, gl/tests/test-mbsalign.c: Remove.
* src/df.c, src/ls.c: Do not include mbsalign.h.
(MBSWIDTH_FLAGS): New constant, now used for all
mbswidth calls. All callers changed to check for -1 return.
* src/df.c (struct field_data_t): ‘width’ is now int not size_t,
since mbswidth can’t do widths greater than INT_MAX anyway.
Replace ‘align’ with ‘align_right’. All uses changed.
(print_table): Redo to avoid the need for ambsalign.
(get_header, get_dev): mbswidth returns int, not size_t.
* src/ls.c (MAX_MON_WIDTH): Remove; no longer used.
(abmon_init): Use strnlen to cheaply discard too-long month names.
Align by hand instead of using mbsalign.
* src/numfmt.c: Include stdckdint.h, mbswidth.h.
Do not include mbsalign.h.
(padding_buffer_size): Now idx_t. All uses changed.
(padding_width): Now intmax_t, since it’s no longer an object
size. Its sign now records alignment. All uses changed.
(zero_padding_width): Now int, since it’s given to sprintf.
All uses changed.
(padding_alignment): Remove; it’s now taken from padding_width’s sign.
(double_to_human): Return string length. BUF_SIZE arg is now idx_t.
Include suffix in output. All callers changed. Simplify by not
calling strncat or stpcpy. Calculate fmt size bound more carefully.
(setup_padding_buffer): Remove. All uses removed.
(parse_format_string): Use intmax_t, not long, for pad.
On overflow, set widths to large values that cause later code
to do the right thing, rather than separately checking for
overflow here.
(prepare_padded_number): Return bool, not int 0/1. New arg
PADDING. All uses changed. Do not limit padded output to 127
bytes; instead, use xpalloc to expand the output buffer.
(print_padded_number): New arg PADDING. All uses changed.
(process_suffixed_number): Simplify.
(main): Take extremum if xstrtoimax overflows, as this does
the right thing.
* tests/misc/numfmt.pl: New test suf-20 to test for truncation bug.
Remove tests pad-3.2, fmt-err-7, as they’re no longer invalid but
are quite expensive.
Most of this just affects commentary and documentations. The only
significant behavior change is translating author names via
proper_name_lite rather than proper_name_utf8, or not translating
them at all. proper_name_lite is good enough for coreutils and
avoids the bloat that had coreutils not using Gnulib proper_name.
* bootstrap.conf (gnulib_modules): Use propername-lite instead
of propername.
(XGETTEXT_OPTIONS): Look for proper_name_lite instead of for
proper_name_utf8.
* cfg.mk (local-checks-to-skip): Remove
sc_proper_name_utf8_requires_ICONV, since we no longer use
proper_name_utf8.
(old_NEWS_hash): Update.
(sc_check-I18N-AUTHORS): Remove; no longer needed.
* src/join.c (prjoin): Check for write errors after each line.
* tests/misc/write-errors.sh: enable the test for join.
* NEWS: Mention the improvement.
* src/comm.c (writeline): Simplify by removing the unneeded STREAM
parameter. Call write_error() upon ferror().
(compare_files): Adjust to simplified writeline().
* tests/misc/write-errors.sh: Enable comm test.
* NEWS: Mention the improvement.
* src/cut.c (cut_bytes): Diagnose errors from fwrite() and putchar().
(cut_fields): Likewise.
* tests/misc/write-errors.sh: Enable the test for cut,
and augment to cover both cut_bytes() and cut_fields().
* NEWS: Mention the improvement.
* src/uniq.c (write_line): Check the output from fwrite() immediately.
(check_file): Likewise.
* tests/misc/write-errors.sh: Enable the test case.
* NEWS: Mention the improvement.
* src/od.c (dump): Check for write errors after each block written,
to exit early even with large / unbounded inputs.
* tests/misc/write-errors.sh: enable od check.
* NEWS: Mention the improvement.
Fixes https://bugs.gnu.org/64540
* cfg.mk (sc_some_programs_must_avoid_exit_failure): Adjust to
avoid false positive.
(sc_prohibit_exit_write_error): A new syntax check to prohibit
open coding error(..., "write error"); instead directing to use...
* src/system.h (write_error): ... a new function to clear stdout errors
before we explicitly diagnose a write error and exit.
* src/basenc.c: Use write_error() to ensure no repeated diagnostics.
* src/cat.c: Likewise.
* src/expand.c: Likewise.
* src/factor.c: Likewise.
* src/paste.c: Likewise.
* src/seq.c: Likewise.
* src/shuf.c: Likewise.
* src/split.c: Likewise.
* src/tail.c: Likewise.
* src/tr.c: Likewise.
* src/unexpand.c: Likewise.
* tests/misc/write-errors.sh: Remove TODOs for the fixed utilities:
expand, factor, paste, shuf, tr, unexpand.
* tests/misc/read-errors.sh: Exercise more modes of
various utilities for better read error coverage.
* tests/split/fail.sh: Remove part refactored into the above test.
* tests/misc/tty-eof.pl: Ensure we don't erroneously
skip commands with parameters.
Comment as to why cut(1) is treated differently.
Adjust expect calls to not wait needlessly for cut output.
* src/wc.c (wc): Update the offset when not reading,
and do read if we can't update the offset.
* tests/misc/wc-proc.sh: Add a test case.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/61300
* tests/misc/tee.sh: Call cleanup_ in all cases to ensure
there are no overlapping interactions on the fifo that
might impact later parts of the test. This was seen to
cause issue with dash on musl libc.
Addresses https://bugs.gnu.org/62542
* tests/misc/csplit-heap.sh: More memory is required to avoid
a false failure on some systems. Noticed with musl libc
with bash as the shell. This is confirmed to still easily
trigger with the original memory leak being tested.
Addresses https://bugs.gnu.org/62542
* src/wc.c (wc): Use INT_ADD_WRAPV() to detect overflow.
(main): Upon overflow, saturate the total, print a diagnostic,
and set exit status.
* tests/misc/wc-total.sh: Add a test case, which operates
on BTRFS and 64 bit systems at least.
Reported at https://bugs.debian.org/1027100
* NEWS: Mention the fix.
* src/dircolors.c: Fail upon read error from getline().
* tests/misc/dircolors.sh: Add a new test.
* tests/local.mk: Reference the new test.
This applies to all checksumming utilities,
where we incorrectly report all subsequent files as checking 'OK'
once any file has passed a digest check.
The exit status was not impacted, only the printed status.
* src/digest.c (digest_check): Use the correct state variable
to determine if the _current_ file has passed or not.
* tests/misc/md5sum.pl: Add a test case.
Fixes https://bugs.gnu.org/62403
Similarly to the fix to tests/rmdir/ignore.sh in c0e5f8c59,
tee should not be expected to fail when run with read-only outputs
when run as root.
* tests/misc/tee.sh: Add uid_is_privileged_ guard around test for
read-only outputs.
* tests/misc/tee.sh: AIX doesn't support detecting
closed outputs either with poll() or select() so avoid
testing that functionality.
* tests/tail-2/pipe-f.sh: Likewise.
Non blocking outputs can be seen for example
when piping telnet through tee to a terminal.
In that case telnet sets its input to nonblocking mode,
which results in tee's output being nonblocking,
in which case in may receive an EAGAIN error upon write().
The same issue was seen with mpirun.
The following can be used to reproduce this
locally at a terminal (in most invocations):
$ { dd iflag=nonblock count=0 status=none;
dd bs=10K count=10 if=/dev/zero status=none; } |
tee || echo fail >/dev/tty
* src/iopoll.c (iopoll_internal): A new function refactored from
iopoll(), to also support a mode where we check the output
descriptor is writeable.
(iopoll): Now refactored to just call iopoll_internal().
(fwait_for_nonblocking_write): A new internal function which
uses iopoll_internal() to wait for writeable output
if an EAGAIN or EWOULDBLOCK was received.
(fwrite_nonblock): An fwrite() wrapper which uses
fwait_for_nonblocking_write() to handle EAGAIN.
(fclose_nonblock): Likewise.
src/iopoll.h: Add fclose_nonblock, fwrite_nonblock.
src/tee.c: Call fclose_nonblock() and fwrite_nonblock wrappers,
instead of the standard functions.
* tests/misc/tee.sh: Add a test case.
* NEWS: Mention the improvement.
The idea was suggested by Kamil Dudka in
https://bugzilla.redhat.com/1615467
This was introduced recently with commit v9.1-166-g6b12e62d9
* src/tee.c (tee_files): Check the return from fopen()
before passing to fileno() etc.
* tests/misc/tee.sh: Add a test case.
