Note we don't support the --algorithm option of the
b2sum command in the external BLAKE2 project, as that
was deemed too confusing for users.
"BLAKE2b" was chosen as the default algorithm to use,
which is single threaded but performs well on 64 bit.
* src/blake2: CC0 source copied from external project.
* cfg.mk[VC_LIST_ALWAYS_EXCLUDE_REGEX]: Exclude blake2/
from syntax checks, make update-copyright, etc.
* src/local.mk: Reference the sources for b2sum,
and set the compilation flags.
* doc/coreutils.texi (b2sum invocation): Reference the
md5sum invocation node, and add descriptions of -l.
* tests/misc/b2sum.sh: Add new test.
* tests/local.mk: Reference new test.
* AUTHORS: Add new binary.
* README: Likewise.
* build-aux/gen-lists-of-programs.sh: Likewise.
* man/.gitignore: Likewise.
* scripts/git-hooks/commit-msg: Likewise.
* man/b2sum.x: New man page template.
* man/local.mk: Reference new template.
* src/.gitignore: Ignore new binaries.
* src/blake2/.gitignore: Ignore new build atrifacts.
* src/md5sum.c (usage): Describe the new -l option.
* NEWS: Mention the new program.
* src/copy.h (cp_options): Add a new flag for install(1).
* src/copy.c (copy_internal): For cp, also output remediation
advice which also indicates why directories aren't copied by default.
The message is unchanged for install(1).
* src/cp.c (cp_option_init): Init install_mode to false.
* src/mv.c (cp_option_init): Likewise.
* src/install.c (cp_option_init): Init install_mode to true.
* tests/install/basic-1.sh: Add a test case.
* tests/cp/link-deref.sh: Adjust test case.
Fixes http://bugs.gnu.org/24958
* tests/misc/sort-h-thousands-sep.sh: The sv_SE locale
may be available while sv_SE.utf8 is not. Therefore
test and use the same locale, now definfed in a variable.
Reported by Jim Meyering.
The previous "returns_"-using change failed to convert many
uses of "$?". Convert all but two of the remaining ones.
* tests/ls/stat-vs-dirent.sh: Likewise.
* tests/misc/head-write-error.sh: Likewise.
* tests/misc/nice.sh: Likewise.
* tests/misc/nohup.sh: Likewise.
* tests/misc/stdbuf.sh: Likewise.
* tests/misc/sync.sh: Likewise.
* tests/tail-2/pid.sh: Likewise.
* tests/tail-2/wait.sh: Likewise.
Thanks to Bernhard Volker for spotting this.
* src/tail.c (tail_file): On failure to open a file,
set ignore=true when --retry is not specified.
* tests/tail-2/assert-2.sh: Adjust to the new behavior.
* tests/tail-2/retry.sh: Add a test case. Also change
from `tail ... && fail=1` to the more robust `returns_ 1 ...`
construct which detects segfaults etc.
* NEWS: Document the fix.
* src/tail.c (tail_forever): The BLOCKING optimization is only
enabled for non regular files (which can't be truncated), so ensure
we don't enable that unless we've a valid file descriptor.
* tests/tail-2/retry.sh: Add a test case.
* NEWS: Mention the bug fix.
* src/tail.c (ignore_pipe_or_fifo): Mark the descriptor as -1
for pipes so that any_live_files() detects correctly that
the entry is no longer live.
* tests/tail-2/pipe-f.sh: Add a test case.
* NEWS: Mention the fix.
Fixes http://bugs.gnu.org/24903 which was detected
using Symbolic Execution techniques developed in
the course of the SYMBIOSYS research project at
COMSYS, RWTH Aachen University.
* doc/coreutils.texi(cut invocation): Give a more accurate description
of cut field handling limitations.
* tests/misc/stat-fmt.sh: Fix the test header to be more general.
* tests/tail-2/retry.sh: Spellings.
Incorporating these coreutils specific changes:
backupfile: initialize default suffix within the implementation
strftime,strptime: support %q to represent the quarter
Also sync these copies with gnulib:
* bootstrap: Now supports AIX.
* gnulib: Update to the latest.
* tests/init.sh: dash(1) is now disabled.
* doc/coreutils.texi (stat invocation): Describe the
QUOTING_STYLE values now supported.
* src/stat.c (getenv_quoting_style): A new function called
from main, that sets the default quoting style for quotearg.
(main): Call getenv_quoting_style() when %N specified.
* tests/misc/stat-fmt.sh: Add a test case.
* NEWS: Mention the improvement.
Fixes http://bugs.gnu.org/23422
* src/dd.c (parse_integer): Suggest to use "00x" instead of "0x",
which is significant for the "count", "seek", and "skip" operands.
* tests/dd/misc.sh: Add a test case.
Fixes http://bugs.gnu.org/24874
* NEWS: Mention the fix.
* src/md5sum.c (digest_file): Add a new MISSING parameter to
return whether the file was missing, separately from the digest.
* tests/misc/md5sum.pl: Add a test case.
Fixes http://bugs.gnu.org/24795
* src/rm.c (main): Ensure the full --no-preserve-root
option is specified, rather than allowing --n etc.
* tests/rm/r-root.sh: Add a test case.
* NEWS: Mention the change in behavior.
Improved by Jim Meyering.
Fixes http://bugs.gnu.org/24604
which was not the case when inotify was not available.
* src/tail.c (any_live_files): Simplify, since the IGNORE
flag is now only set when a file should be ignored indefinitely.
(recheck): Only output the "giving up on name" message
when that's actually the case. Only set the IGNORE flag
when ignoring a file indefinitely.
(tail_file): Likewise.
* tests/tail-2/retry.sh: Add a test case. Also run
all existing test cases with and without inotify.
NEWS: Mention the fix.
THANKS.in: Add the reporter.
Fixes http://bugs.gnu.org/24495 which was detected
using Symbolic Execution techniques developed in
the course of the SYMBIOSYS research project at
COMSYS, RWTH Aachen University.
The old code mishandled --time-spec='+%%b', as it misinterpreted
the '%b' as being the month abbreviation. Also, it mishandled
the extremely unlikely case of a month abbreviation containing '%'.
The performance part of this patch sped up 'ls' by about 1% on my
little benchmark of 'ls -lR' on the source directory in the
en_US.UTF-8 locale (Fedora 24 x86-64).
* NEWS: Document the bug fix.
* src/ls.c (first_percent_b, abformat_init): New static functions.
(ABFORMAT_SIZE): New constant.
(use_abformat): New static var.
(abmon, required_mon_width): Remove these static vars.
(abmon_init): Now accepts a pointer to abmon, and returns a boolean.
All callers changed. Reject month abbrs containing '%', as these
would mess up strftime. Simplify mbsalign result checking,
since (size_t) -1 exceeds ABFORMAT_SIZE.
(abformat_init, align_nstrftime): Precompute all 24 formats at
startup, rather than computing a format for each time stamp.
(decode_switches): Call abformat_init instead of abmon_init.
(align_nstrftime): Accept recentness bool instead of format.
All callers changed.
* tests/misc/time-style.sh: Test for format with '%%b'.
* src/cp.c (make_dir_parents_private): Use default permissions for
created directories when --no-preserve=mode is specified.
* tests/cp/cp-parents.sh: Add a test case.
* NEWS: Mention the fix.
Fixes http://bugs.gnu.org/24251
* src/expand.c, src/unexpand.c: Move global variables from here...
* src/expand-common.h, src/expand-common.c: ... to here.
* src/expand.c, src/unexpand.c: (parse_tab_stops, validate_tab_stops,
next_file): Move identical functions to new module.
(add_tab_stop): Move to new module, including additional code from
'unexpand' (keeping max_column_width) which will have no effect in when
used in 'expand'. Refactor common next-column calculation code
into a new function 'get_next_tab_column'.
* src/local.mk: (src_expand_SOURCES, src_unexpand_SOURCES): Add
'expand-common.c'; (noinst_HEADERS): Add 'expand-common.h'.
* po/POTFILES.in: Add 'expand-common.c'.
* tests/misc/expand.pl: Add more tests.
* tests/misc/unexpand.pl: Likewise.
* TODO: Move conclusions to above test after investigation.
* src/sort.c (traverse_raw_number): Accept thousands separator only
if it is immediately followed by a digit.
* tests/misc/sort-h-thousands-sep.sh: Cover the fix for this bug.
Suggested by Pádraig Brady in http://bugs.gnu.org/24015
* src/sort.c (traverse_raw_number): Allow to skip only one occurrence
of thousands_sep to avoid finding the unit in the next column in case
thousands_sep matches as blank and is used as column delimiter.
* tests/misc/sort-h-thousands-sep.sh: Add regression test for this bug.
* tests/local.mk: Reference the test.
* NEWS: Mention the bug fix.
Reported at https://bugzilla.redhat.com/1355780
Fixes http://bugs.gnu.org/24015
* doc/coreutils.texi (install invocation): Update -Z documentation.
* src/install.c (make_ancestor): Set default security context before
calling mkdir() if the -Z option is given.
(process_dir): Call restorecon() on the destination directory if the
-Z option is given.
(usage): Update -Z documentation.
* tests/install/install-Z-selinux.sh: A new test for 'install -Z -D'
and 'install -Z -d' based on tests/mkdir/restorecon.sh.
* tests/local.mk: Reference the test.
* NEWS: Mention the improvement.
Reported at https://bugzilla.redhat.com/1339135
Fixes http://bugs.gnu.org/23868
* tests/rm/rm-readdir-fail.sh: Since we use the returned dirent
from the readdir wrapper it must be the correct type and not
just cast. Therefore setup so that we only have to define a
wrapper for readdir(), which works appropriately on 32 and 64 bit.
Issue reported by Bernhard Voelker, where rm was seen to invoke
rmdir() on invalid file names.
Follow-up to commit c92585b1: improve epipe test script, with
suggestions from Bernhard Voelker and Pádraig Brady.
see: http://lists.gnu.org/archive/html/coreutils/2016-04/msg00067.html .
* tests/misc/seq-epipe.sh: Avoid too many subshells,
stricter validation of errors and output from seq,
skip if trapping SIGPIPE is not supported.
* tests/rm/rm-readdir-fail.sh: A new test to simulate readdir()
failing immediately or after returning a few entries, and verifying
that rm does the appropriate thing.
This was initially reported at:
http://bugzilla.opensuse.org/show_bug.cgi?id=984910
where it was mentioned that readdir() may fail
when an NFS server has a poor readdir cookie implementation.
* gnulib: Update to latest.
* NEWS: Specifically mention the fts readdir() fix
and reindent to standard indentation.
* tests/init.sh: Update from gnulib.
Print warning regardless of locale, avoid warning if key is zero width.
Problem reported by Karl Berry in http://bugs.gnu.org/23665 .
* src/sort.c: (key_warnings): change conditions for 'leading spaces'
warning.
* tests/misc/sort-debug-warn.sh: adjust tests accordingly.
Easier troubleshooting of individual 'sort --debug' messages.
Discussed in http://bugs.gnu.org/23665 .
* tests/misc/sort-debug-warn.sh: add progress number before each sort
invocation.
Ensure I/O errors are detected (and terminate seq), preventing seq
from infloop (or running for long time with a large
range) upon write errors or ignored SIGPIPE. Examples:
seq 1 inf > /dev/full (seq_fast)
seq 1.1 0.1 inf >/dev/full (print_numbers)
* src/seq.c (io_error): A new function to diagnose appropriate
stdio errors and exit the program with failure status.
(seq_fast, print_numbers): Explicitly check for write errors
and terminate the program with diagnostic.
* tests/misc/seq-io-errors.sh: Test error detection with /dev/full.
* tests/misc/seq-epipe.sh: Test error detection with broken pipes.
* tests/local.mk: Add new tests.
* NEWS: Mention the fix.
* src/sleep.c (main): Allow ERANGE since we allow "inf" values.
* src/timeout.c (parse_duration): Likewise.
* tests/misc/sleep.sh: New file. Tests for sleep(1).
* tests/misc/timeout-parameters.sh: Add case for newly allowed
$LDBL_MAX. Also use returns_ throughout the file. Also avoid
small timeout values which might give false failures under load.
* cfg.mk (sc_prohibit_expr_unsigned): expr uses intmax_t internally
when GMP is not available, so flag any uses of unsigned limits.
* tests/misc/timeout-parameters.sh: Remove the overflow related
to UINT_MAX as it's handled by the following case. Change the
following case from 99... to $TIME_T_OFLOW to be more expressive
in what it's actually testing. Directly check that commands succeed,
rather than inspecting $? afterwards.
* tests/dd/skip-seek-past-dev.sh: Guard against large device sizes.
* tests/id/setgid.sh: Protect: Guard against large group IDs.
* tests/misc/cut-huge-range.sh: Tweak comment to avoid syntax check.
It's tempting to simplify to just skip the test if expr fails,
but SIZE_MAX > INTMAX_MAX is the common case.
* tests/cp/parent-perm-race.sh: This new race introduced in
commit v8.25-5-g632eda5 is quite hard to hit, but is due to
`ls > fifo` doing write()/close()/exit() once `cp` has
open() the source fifo. Then the subsequent comparison of the
destination file may fail due to the file being missing or empty.
Previously `ls` generated output that was independent of `cp`.
Now we must wait for `cp` to finish before inspecting the
destination file that it wrote.
* src/seq.c (isnan): Define macro.
(scan_arg): Add check if the given argument is NaN, and exit with
a proper error diagnostic in such a case.
(usage): Document it.
* tests/misc/seq.pl: Add tests.
* doc/coreutils.texi (seq invocation): Document the change.
* NEWS (Changes in behavior): Mention the change.
* src/seq.c (main): Exit with an error diagnostic when the given
step value is Zero.
(usage): Document it.
* doc/coreutils.texi (seq invocation): Likewise.
* tests/misc/seq.pl: Add tests.
* NEWS (Changes in behavior): Mention the change.
Reported by Маренков Евгений in:
http://bugs.gnu.org/23110
The problematic code computed a struct tm in one time zone, and
then printed it or converted it to a string in another. To be
portable the same time zone needs to be used for both operations.
On GNU platforms this is not an issue, but incorrect output can be
generated on System V style platforms like AIX where time zone
abbreviations are available only in the 'tzname' global variable.
Problem reported by Assaf Gordon in: http://bugs.gnu.org/23035
* NEWS: Document the bug.
* src/date.c (show_date):
* src/ls.c (long_time_expected_width, print_long_format):
* src/pr.c (init_header):
* src/stat.c (human_time): Use localtime_rz instead of localtime,
so that the time zone information is consistent for both localtime
and time-formatting functions like fprintftime and nstrftime. For
'stat' this change is mostly just a code cleanup but it also
causes stat to also print nanoseconds when printing time stamps
that are out of localtime range, as this is more consistent with
what other programs do. For programs other than 'stat' this fixes
bugs with time zone formats that use %Z.
* src/du.c, src/pr.c (localtz): New static var.
(main): Initialize it.
* src/du.c (show_date): New time zone argument, so that localtime
and fprintftime use the same time zone information. All callers
changed.
* tests/misc/time-style.sh: New file.
* tests/local.mk (all_tests): Add it.
* tests/misc/date.pl: Test alphabetic time zone abbreviations.
* tests/split/filter.sh: Use OFF_T_MAX-1 rather than OFF_T_MAX
as the size of a test file, to avoid false failure on an XFS file
system (or any file system permitting a file of size OFF_T_MAX).
Reported as http://bugs.gnu.org/22931
sort, tail, and uniq now support traditional usage like 'sort +2'
and 'tail +10' on systems conforming to POSIX 1003.1-2008 and later.
* NEWS: Document this.
* doc/coreutils.texi (Standards conformance, tail invocation)
(sort invocation, uniq invocation, touch invocation):
Document new behavior, or behavior's dependence on POSIX 1003.1-2001.
* src/sort.c (struct keyfield.traditional_used):
Rename from obsolete_used, since implementations are now allowed
to support it. All uses changed.
(main): Allow traditional usage if _POSIX2_VERSION is 200809.
* src/tail.c (parse_obsolete_option): Distinguish between
traditional usage (which POSIX 2008 and later allows) and obsolete
(which it still does not).
* src/uniq.c (strict_posix2): New function.
(main): Allow traditional usage if _POSIX2_VERSION is 200809.
* tests/misc/tail.pl: Test for new behavior.
* src/split.c (lines_rr): Reinstate the conditional
setting of the WROTE boolean, as otherwise split -n r/1 would
consume all input when all --filter commands are stopped.
There was a test in place to check for this, but
it was incorrect as detailed below.
(input_file_size): Immediately disallow --number with
non seekable inputs, as such an invocation is not currently
generally supported and will fail as the data overflows
the internal buffer.
* tests/split/l-chunk.sh: Adjust to again disallow -n /dev/zero.
Also change all '&& fail=1' checks to use the 'returns_ 1' form.
* tests/split/filter.sh: Change the no longer supported /dev/zero
case to a regular $OFF_T_MAX file (supported on XFS for example).
Also fix the timeout(1) commands so they're not subject to
pipefail issues.
* tests/cp/parent-perm-race.sh: Add timeouts so that the test does
not wait forever on GNU/Hurd. This does not fix the underlying
bug but at least lets the tests make progress.
Problem reported by Nelson H.F. Beebe in: http://bugs.gnu.org/22624
Also, check that 'cp' actually copies the data; this is a better
test anyway, and simplifies the test code.
Also, avoid unlikely race if 'pid' is set in the environment.
Problem reported by Nelson H.F. Beebe in: http://bugs.gnu.org/22624
Other problems also fixed: basically, the code got confused because
GNU/Linux reports that /dev/zero has size zero.
* src/split.c (input_file_size): Now takes struct stat *, not just
size. Always store the first buffer. All callers changed. Treat
/dev/zero as an infinitely-large file, both on GNU/Linux where
fstat and lseek say its size is zero, and on GNU/Hurd where they
say the size is OFF_T_MAX.
(cwrite): Return true on success.
(bytes_split): Don't try to read past EOF, and stop if a write fails.
(lines_rr): Omit stray check for ignorable errno.
(main): Get file size only when n_units > 1, since that's the only
time it is needed. Defer most of the work to input_file_size.
* tests/split/l-chunk.sh: Adjust tests to match new behavior
on oddball inputs.
* tests/tail-2/wait.sh: Restrict inotify specific test portion
to local file systems and also not with ---disable-inotify specified.
Failure noticed on NFS.
* tests/rm/rm1.sh: Also remove the group write bit which
was required on one NFS setup at least. Note u-w was
enough to deny file creation, g-w was also required to
deny file removal.
* tests/rm/cycle.sh: Likewise.
* tests/mv/perm-1.sh: Likewise.
