Problem reported by Paul Wise for Debian, in:
https://bugs.debian.org/851934
This is fallout from the fix for GNU Bug#23035.
* src/date.c (batch_convert): New args TZ and TZSTRING.
All uses changed.
(batch_convert, main): Adjust to parse_datetime2 API change.
(main): Allocate time zone object.
* tests/misc/date-debug.sh: Fix incorrect test case,
caught by the fix.
* tests/misc/date.pl: Test the fix.
A lot of this is converting http: to https:.
Also, gmane went away, so remove URLs that no longer work and
are not easy to figure out what they were.
Some of this stuff is so old that it no longer matters anyway.
* src/stty.c (apply_settings): A new function refactored
from main() that is used to both check and apply options.
(main): Call apply_settings before we open the device,
so all validation is done before interacting with a device.
* NEWS: Mention the improvement.
* tests/misc/stty.sh: Add a test case.
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
* tests/chgrp/basic.sh: On some NFS setups a user is
not allowed to set a group on a file even if a member
of that group. Therefore skip this test on remote file systems.
* tests/chgrp/default-no-deref.sh: Likewise.
* tests/chgrp/no-x.sh: Likewise.
* tests/chgrp/posix-H.sh: Likewise.
* tests/chgrp/recurse.sh: Likewise.
* tests/tail-2/inotify-rotate-resources.sh: Change to skipping
on remote file systems in the standard way.
* src/wc.c (wc): Avoid reading the end of the file
when the size is not a multiple of PAGE_SIZE,
as the special case handling for files in /proc and /sys
is only required when st_size is 0 or a multiple of PAGE_SIZE.
* tests/misc/wc-proc.sh: Add a test case.
While st_size would have been incorrect for subsequent
files since v7.1, it was only used since v8.24.
* tests/misc/wc-files0.sh: s/7.1/8.24/
* NEWS: Likewise.
Reported by Bernhard Voelker
* src/wc.c (main): Reset fstatus[0].failed between files when reusing
the fstatus[0] entry in --files-from0 mode. This ensures a stat() is
done for each file, avoiding incorrect counts and redundant reading.
* NEWS: Mention the bug fix.
* tests/misc/wc-files0.sh: Add a test case.
Fixes http://bugs.gnu.org/23073
When running "make check" on a Linux system running SELinux with a
non-MLS policy, tests/mkdir/restorecon.sh test fails with:
chcon: invalid context: root:object_r:tmp_t:s0: Invalid argument
Indeed in such a configuration, contexts cannot have ":s0" suffix.
* init.cfg (get_selinux_type): Refactor this function to here
from various tests. Update to work with a non-MLS policy.
(mls_enabled_): A new function to detect if MLS is enabled.
(skip_if_mcstransd_is_running_): Update to not skip when
MLS is not enabled.
* tests/mkdir/restorecon.sh: Use a valid non-MLS context when needed.
* tests/install/install-Z-selinux.sh: Likewise.
* tests/cp/cp-a-selinux.sh: Likewise.
* tests/misc/selinux.sh: Likewise.
* tests/misc/chcon.sh: Skip if non-MLS as --range used throughout.
Fixes http://bugs.gnu.org/22631
* tests/misc/ls-time.sh: Skip the test rather than ERROR
when `touch -m -d ...` fails (Hurd).
* tests/tail-2/follow-stdin.sh: Avoid false FAILs by ignoring
the variances in sterror output.
* tests/rm/rm-readdir-fail.sh: Likewise. Also avoid ERRORs
on systems that don't define _D_EXACT_NAMELEN.
This was detected with ASAN, but can also be seen without ASAN with:
$ tac - - <&-
tac: standard input: read error: Bad file descriptor
*** Error in `tac': malloc(): memory corruption: 0x...
* src/tac.c (copy_to_temp): Don't close our output stream on
(possibly transient) output error, or on input error.
(temp_stream): clearerr() on the stream about to be reused,
to ensure future stream use is not impacted by transient errors.
* tests/misc/tac-2-nonseekable.sh: Add a test case.
* NEWS: Mention the bug fix.
Fixes http://bugs.gnu.org/25041
Reproduced under UBSAN with `tail -f <&-` giving:
tail.c:2220:18: runtime error: load of value 190,
which is not a valid value for type ‘_Bool'
* src/tail.c (tail_file): Ensure f->ignore is initialized
in all cases where we can't tail the specified file.
* tests/tail-2/follow-stdin.sh: Add a test case which
checks stderr has no UBSAN warnings.
Fixes http://bugs.gnu.org/25041
* NEWS: Mention in improvements about the workaround for
the glibc issue with closed stdin, and the new supported file systems.
* tests/misc/b2sum.sh: Spelling fix.
* tests/ls/quote-align.sh: Remove "total" line
which can vary per file system depending on allocation.
Reported by Assaf Gordon on OpenSolaris (5.11/5.10).
This was detected with:
echo a > a; pr "-S$(printf "\t\t\t")" a -m a > /dev/null
Resulting in ASAN triggering:
====================================================
ERROR: AddressSanitizer: global-buffer-overflow
READ of size 1 at 0x00000041b622 thread T0
#0 0x40506a in print_sep_string ../src/pr.c:2241
#1 0x407ec4 in read_line ../src/pr.c:2493
#2 0x40985c in print_page ../src/pr.c:1802
#3 0x40985c in print_files ../src/pr.c:1618
#4 0x4036e0 in main ../src/pr.c:1136
* src/pr.c (init_parameters): Ensure we only override the
specified separator when it's a single tab, thus matching
the calculated separator length.
* tests/pr/pr-tests.pl: Add a test case.
* NEWS: Mention the fix.
* src/ptx.c (fix_output_parameters): Ensure line_width doesn't
go negative, which can happen when the --width is less
than the --gap-size.
* tests/misc/ptx-overrun.sh: Add a test case that triggers
with ASAN. (Note the longer filename is needed to trigger).
Fixes http://bugs.gnu.org/25011
* tests/rm/rm-readdir-fail.sh: ASAN correctly indicated
that fts was writing to freed memory. This was because
we reused a single dirent in our readdir() test wrapper.
Since fts was deallocating those dirents, we now get
a new dirent for each call to our readdir wrapper.
ASAN reported this error for: split -n2/3 /dev/null
ERROR: AddressSanitizer: negative-size-param: (size=-1)
#0 0x7f0d4c36951d in __asan_memmove (/lib64/libasan.so.2+0x8d51d)
#1 0x404e06 in memmove /usr/include/bits/string3.h:59
#2 0x404e06 in bytes_chunk_extract src/split.c:988
#3 0x404e06 in main src/split.c:1626
Specifically there would be invalid memory access
and subsequent processing if the chunk to be extracted
was beyond the initial amount read from file (which is
currently capped at 128KiB). This issue is not in a
released version, only being introduced in commit v8.25-4-g62e7af0
* src/split.c (bytes_chunk_extract): The initial_read != SIZE_MAX
should have been combined with && rather than ||, but also this
condition is always true in this function so remove entirely.
* tests/split/b-chunk.sh: Add a test case.
Fixes http://bugs.gnu.org/25003
* src/comm.c (total_option): Add bool variable for the new option.
(TOTAL_OPTION): Add enum value.
(long_options): Add array element for the new option.
(usage): Document the new option here.
(compare_files): Count the lines in total[3], and output the summary at
the end.
(main): Accept the new option.
* doc/coreutils.texi (comm invocation): Document it.
* tests/misc/comm.pl: Test it. While at it, improve the test data
to have 1 unique line in the first file, 2 unique lines in the second
file, and 3 common lines.
* NEWS (New Features): Mention the new option.
Fixes http://bugs.gnu.org/24929
* tests/misc/printf-quote.sh: FreeBSD 11 was seen to treat
\u0378 as a printable character. Therefore change to
using the \u0081 C1 control character. We use the UTF-8
representation because our printf implementation explicitly
disallows \u0081 as input.
Reported by Assaf Gordon
On BSD /bin/sh it was seen that unexported env vars passed to
returns_() would not be propagated to the wrapped command.
* cfg.mk (sc_prohibit_env_returns): Add a syntax check to disallow.
* tests/misc/csplit-io-err.sh: Rearrange to export vars in a subshell.
* tests/rm/rm-readdir-fail.sh: Likewise.
* tests/misc/nohup.sh: Export and unset vars around returns_.
* tests/misc/printenv.sh: Likewise.
Reported by Assaf Gordon
This provides better alignment when some names are quoted,
which also provides better indication that quotes are not
part of the name.
* src/ls.c (align_variable_outer_quotes): A new variable
set when ls is aligning columns (not using -m, non-zero -w),
and has a variable quoting style (shell, shell-escape, c-maybe).
(quote_name_buf): Writes to buffer rather than FILE,
taking care to avoid data copying if possible. Refactored from...
(quote_name): ...here. This now manages the buffer passed
to quote_name_buf() and outputs the padding, colors and name
in the appropriate order, while managing the --dired offsets.
(get_color_indicator): A new function to return the color sequence,
refactored from...
(print_color_indicator): ...here. This now simply outputs.
(print_dir): Refactor common parts to quote_name().
(clear_files): Reset the flag indicating at least one
file is quoted in the current directory.
(needs_quoting): A new function to indicate at the scan stage
whether a name needs quoting. Called from...
(gobble_file): ...here, until we find the first quoted file.
(print_name_with_quoting): Mostly refactored to quote_name().
* tests/ls/quote-align.sh: A new test for various output formats.
* tests/local.mk: Reference the new test.
* NEWS: Mention the improvement.
Note we don't support the --algorithm option of the
b2sum command in the external BLAKE2 project, as that
was deemed too confusing for users.
"BLAKE2b" was chosen as the default algorithm to use,
which is single threaded but performs well on 64 bit.
* src/blake2: CC0 source copied from external project.
* cfg.mk[VC_LIST_ALWAYS_EXCLUDE_REGEX]: Exclude blake2/
from syntax checks, make update-copyright, etc.
* src/local.mk: Reference the sources for b2sum,
and set the compilation flags.
* doc/coreutils.texi (b2sum invocation): Reference the
md5sum invocation node, and add descriptions of -l.
* tests/misc/b2sum.sh: Add new test.
* tests/local.mk: Reference new test.
* AUTHORS: Add new binary.
* README: Likewise.
* build-aux/gen-lists-of-programs.sh: Likewise.
* man/.gitignore: Likewise.
* scripts/git-hooks/commit-msg: Likewise.
* man/b2sum.x: New man page template.
* man/local.mk: Reference new template.
* src/.gitignore: Ignore new binaries.
* src/blake2/.gitignore: Ignore new build atrifacts.
* src/md5sum.c (usage): Describe the new -l option.
* NEWS: Mention the new program.
* src/copy.h (cp_options): Add a new flag for install(1).
* src/copy.c (copy_internal): For cp, also output remediation
advice which also indicates why directories aren't copied by default.
The message is unchanged for install(1).
* src/cp.c (cp_option_init): Init install_mode to false.
* src/mv.c (cp_option_init): Likewise.
* src/install.c (cp_option_init): Init install_mode to true.
* tests/install/basic-1.sh: Add a test case.
* tests/cp/link-deref.sh: Adjust test case.
Fixes http://bugs.gnu.org/24958
* tests/misc/sort-h-thousands-sep.sh: The sv_SE locale
may be available while sv_SE.utf8 is not. Therefore
test and use the same locale, now definfed in a variable.
Reported by Jim Meyering.
The previous "returns_"-using change failed to convert many
uses of "$?". Convert all but two of the remaining ones.
* tests/ls/stat-vs-dirent.sh: Likewise.
* tests/misc/head-write-error.sh: Likewise.
* tests/misc/nice.sh: Likewise.
* tests/misc/nohup.sh: Likewise.
* tests/misc/stdbuf.sh: Likewise.
* tests/misc/sync.sh: Likewise.
* tests/tail-2/pid.sh: Likewise.
* tests/tail-2/wait.sh: Likewise.
Thanks to Bernhard Volker for spotting this.
* src/tail.c (tail_file): On failure to open a file,
set ignore=true when --retry is not specified.
* tests/tail-2/assert-2.sh: Adjust to the new behavior.
* tests/tail-2/retry.sh: Add a test case. Also change
from `tail ... && fail=1` to the more robust `returns_ 1 ...`
construct which detects segfaults etc.
* NEWS: Document the fix.
* src/tail.c (tail_forever): The BLOCKING optimization is only
enabled for non regular files (which can't be truncated), so ensure
we don't enable that unless we've a valid file descriptor.
* tests/tail-2/retry.sh: Add a test case.
* NEWS: Mention the bug fix.
* src/tail.c (ignore_pipe_or_fifo): Mark the descriptor as -1
for pipes so that any_live_files() detects correctly that
the entry is no longer live.
* tests/tail-2/pipe-f.sh: Add a test case.
* NEWS: Mention the fix.
Fixes http://bugs.gnu.org/24903 which was detected
using Symbolic Execution techniques developed in
the course of the SYMBIOSYS research project at
COMSYS, RWTH Aachen University.
* doc/coreutils.texi(cut invocation): Give a more accurate description
of cut field handling limitations.
* tests/misc/stat-fmt.sh: Fix the test header to be more general.
* tests/tail-2/retry.sh: Spellings.
Incorporating these coreutils specific changes:
backupfile: initialize default suffix within the implementation
strftime,strptime: support %q to represent the quarter
Also sync these copies with gnulib:
* bootstrap: Now supports AIX.
* gnulib: Update to the latest.
* tests/init.sh: dash(1) is now disabled.
* doc/coreutils.texi (stat invocation): Describe the
QUOTING_STYLE values now supported.
* src/stat.c (getenv_quoting_style): A new function called
from main, that sets the default quoting style for quotearg.
(main): Call getenv_quoting_style() when %N specified.
* tests/misc/stat-fmt.sh: Add a test case.
* NEWS: Mention the improvement.
Fixes http://bugs.gnu.org/23422
* src/dd.c (parse_integer): Suggest to use "00x" instead of "0x",
which is significant for the "count", "seek", and "skip" operands.
* tests/dd/misc.sh: Add a test case.
Fixes http://bugs.gnu.org/24874
* NEWS: Mention the fix.
* src/md5sum.c (digest_file): Add a new MISSING parameter to
return whether the file was missing, separately from the digest.
* tests/misc/md5sum.pl: Add a test case.
Fixes http://bugs.gnu.org/24795
* src/rm.c (main): Ensure the full --no-preserve-root
option is specified, rather than allowing --n etc.
* tests/rm/r-root.sh: Add a test case.
* NEWS: Mention the change in behavior.
Improved by Jim Meyering.
Fixes http://bugs.gnu.org/24604
which was not the case when inotify was not available.
* src/tail.c (any_live_files): Simplify, since the IGNORE
flag is now only set when a file should be ignored indefinitely.
(recheck): Only output the "giving up on name" message
when that's actually the case. Only set the IGNORE flag
when ignoring a file indefinitely.
(tail_file): Likewise.
* tests/tail-2/retry.sh: Add a test case. Also run
all existing test cases with and without inotify.
NEWS: Mention the fix.
THANKS.in: Add the reporter.
Fixes http://bugs.gnu.org/24495 which was detected
using Symbolic Execution techniques developed in
the course of the SYMBIOSYS research project at
COMSYS, RWTH Aachen University.
The old code mishandled --time-spec='+%%b', as it misinterpreted
the '%b' as being the month abbreviation. Also, it mishandled
the extremely unlikely case of a month abbreviation containing '%'.
The performance part of this patch sped up 'ls' by about 1% on my
little benchmark of 'ls -lR' on the source directory in the
en_US.UTF-8 locale (Fedora 24 x86-64).
* NEWS: Document the bug fix.
* src/ls.c (first_percent_b, abformat_init): New static functions.
(ABFORMAT_SIZE): New constant.
(use_abformat): New static var.
(abmon, required_mon_width): Remove these static vars.
(abmon_init): Now accepts a pointer to abmon, and returns a boolean.
All callers changed. Reject month abbrs containing '%', as these
would mess up strftime. Simplify mbsalign result checking,
since (size_t) -1 exceeds ABFORMAT_SIZE.
(abformat_init, align_nstrftime): Precompute all 24 formats at
startup, rather than computing a format for each time stamp.
(decode_switches): Call abformat_init instead of abmon_init.
(align_nstrftime): Accept recentness bool instead of format.
All callers changed.
* tests/misc/time-style.sh: Test for format with '%%b'.
