* src/longlong.h: Sync from gmp repo incorporating:
Protect umul_ppmm with do ... while (0)
Replace obsolete ARC asm 'J' constraints with 'Cal'
Provide umul_ppmm for riscv64
glibc has changed the public define
from _STRING_ARCH_unaligned to _STRING_INLINE_unaligned as per
https://sourceware.org/bugzilla/show_bug.cgi?id=19462
* gl/lib/rand-isaac.c: Cater for both defines.
* gl/lib/randread.c: Likewise.
* src/system.h: Update commented out code.
This was detected with:
echo a > a; pr "-S$(printf "\t\t\t")" a -m a > /dev/null
Resulting in ASAN triggering:
====================================================
ERROR: AddressSanitizer: global-buffer-overflow
READ of size 1 at 0x00000041b622 thread T0
#0 0x40506a in print_sep_string ../src/pr.c:2241
#1 0x407ec4 in read_line ../src/pr.c:2493
#2 0x40985c in print_page ../src/pr.c:1802
#3 0x40985c in print_files ../src/pr.c:1618
#4 0x4036e0 in main ../src/pr.c:1136
* src/pr.c (init_parameters): Ensure we only override the
specified separator when it's a single tab, thus matching
the calculated separator length.
* tests/pr/pr-tests.pl: Add a test case.
* NEWS: Mention the fix.
* src/ptx.c (fix_output_parameters): Ensure line_width doesn't
go negative, which can happen when the --width is less
than the --gap-size.
* tests/misc/ptx-overrun.sh: Add a test case that triggers
with ASAN. (Note the longer filename is needed to trigger).
Fixes http://bugs.gnu.org/25011
ASAN reported this error for: split -n2/3 /dev/null
ERROR: AddressSanitizer: negative-size-param: (size=-1)
#0 0x7f0d4c36951d in __asan_memmove (/lib64/libasan.so.2+0x8d51d)
#1 0x404e06 in memmove /usr/include/bits/string3.h:59
#2 0x404e06 in bytes_chunk_extract src/split.c:988
#3 0x404e06 in main src/split.c:1626
Specifically there would be invalid memory access
and subsequent processing if the chunk to be extracted
was beyond the initial amount read from file (which is
currently capped at 128KiB). This issue is not in a
released version, only being introduced in commit v8.25-4-g62e7af0
* src/split.c (bytes_chunk_extract): The initial_read != SIZE_MAX
should have been combined with && rather than ||, but also this
condition is always true in this function so remove entirely.
* tests/split/b-chunk.sh: Add a test case.
Fixes http://bugs.gnu.org/25003
In recent commit v8.25-93-g7fc7206 we used the f->remote flag
which wasn't set in all cases. This was detected with
ASAN giving this error when reading f->remote;
runtime error: load of value 190,
which is not a valid value for type '_Bool'
* src/tail.c (fremote): Query the system even without inotify.
(recheck): Always set f->fremote for valid files.
Installing with -j2 or greater could result in a failure like
/bin/sh: line 29: /P/bin/install: Permission denied
when /P/bin/install specifies your PATH-selected install program.
This would arise because we're using "install" to install all
man/*.1 files, and that command would run concurrently with the one
that installs "/P/bin/install" itself. We would run this command:
"src/ginstall src/ginstall /P/bin/install", and it would result
in intervals during which the destination file does not exist,
is empty or incomplete and not executable. We addressed this
problem long ago for installation of actual binaries by telling the
installation rules to use our just-built bin/ginstall (only when not
cross-compiling) rather than the PATH-resolved "install" program.
This change is to do the same for those .1 files.
* src/local.mk (INSTALL): Override automake's default of something
like "INSTALL = /P/bin/install -c".
(INSTALL_PROGRAM): Now that we set INSTALL, there is no longer any
need to set this derived variable. Its default definition,
"INSTALL_PROGRAM = ${INSTALL}" does what we require.
Improved by Eric Blake.
* src/comm.c (total_option): Add bool variable for the new option.
(TOTAL_OPTION): Add enum value.
(long_options): Add array element for the new option.
(usage): Document the new option here.
(compare_files): Count the lines in total[3], and output the summary at
the end.
(main): Accept the new option.
* doc/coreutils.texi (comm invocation): Document it.
* tests/misc/comm.pl: Test it. While at it, improve the test data
to have 1 unique line in the first file, 2 unique lines in the second
file, and 3 common lines.
* NEWS (New Features): Mention the new option.
Fixes http://bugs.gnu.org/24929
This provides better alignment when some names are quoted,
which also provides better indication that quotes are not
part of the name.
* src/ls.c (align_variable_outer_quotes): A new variable
set when ls is aligning columns (not using -m, non-zero -w),
and has a variable quoting style (shell, shell-escape, c-maybe).
(quote_name_buf): Writes to buffer rather than FILE,
taking care to avoid data copying if possible. Refactored from...
(quote_name): ...here. This now manages the buffer passed
to quote_name_buf() and outputs the padding, colors and name
in the appropriate order, while managing the --dired offsets.
(get_color_indicator): A new function to return the color sequence,
refactored from...
(print_color_indicator): ...here. This now simply outputs.
(print_dir): Refactor common parts to quote_name().
(clear_files): Reset the flag indicating at least one
file is quoted in the current directory.
(needs_quoting): A new function to indicate at the scan stage
whether a name needs quoting. Called from...
(gobble_file): ...here, until we find the first quoted file.
(print_name_with_quoting): Mostly refactored to quote_name().
* tests/ls/quote-align.sh: A new test for various output formats.
* tests/local.mk: Reference the new test.
* NEWS: Mention the improvement.
Note we don't support the --algorithm option of the
b2sum command in the external BLAKE2 project, as that
was deemed too confusing for users.
"BLAKE2b" was chosen as the default algorithm to use,
which is single threaded but performs well on 64 bit.
* src/blake2: CC0 source copied from external project.
* cfg.mk[VC_LIST_ALWAYS_EXCLUDE_REGEX]: Exclude blake2/
from syntax checks, make update-copyright, etc.
* src/local.mk: Reference the sources for b2sum,
and set the compilation flags.
* doc/coreutils.texi (b2sum invocation): Reference the
md5sum invocation node, and add descriptions of -l.
* tests/misc/b2sum.sh: Add new test.
* tests/local.mk: Reference new test.
* AUTHORS: Add new binary.
* README: Likewise.
* build-aux/gen-lists-of-programs.sh: Likewise.
* man/.gitignore: Likewise.
* scripts/git-hooks/commit-msg: Likewise.
* man/b2sum.x: New man page template.
* man/local.mk: Reference new template.
* src/.gitignore: Ignore new binaries.
* src/blake2/.gitignore: Ignore new build atrifacts.
* src/md5sum.c (usage): Describe the new -l option.
* NEWS: Mention the new program.
* src/copy.h (cp_options): Add a new flag for install(1).
* src/copy.c (copy_internal): For cp, also output remediation
advice which also indicates why directories aren't copied by default.
The message is unchanged for install(1).
* src/cp.c (cp_option_init): Init install_mode to false.
* src/mv.c (cp_option_init): Likewise.
* src/install.c (cp_option_init): Init install_mode to true.
* tests/install/basic-1.sh: Add a test case.
* tests/cp/link-deref.sh: Adjust test case.
Fixes http://bugs.gnu.org/24958
This reduces a standard coreutils install size by about 160K.
* src/cat.c: Change to proper_name() which removes about 18K text.
* src/cp.c: Likewise.
* src/df.c: Likewise.
* src/du.c: Likewise.
* src/getlimits.c: Likewise.
* src/realpath.c: Likewise.
* src/split.c: Likewise.
* src/stdbuf.c: Likewise.
* src/timeout.c: Likewise.
* src/truncate.c: Likewise.
* src/local.mk: Remove -llibiconv from the above programs.
* cfg.mk (sc_check-AUTHORS): Adjust to use factor(1).
* AUTHORS: Adjust to use ASCII to satisfy sc_check-AUTHORS.
* src/tail.c (tail_file): On failure to open a file,
set ignore=true when --retry is not specified.
* tests/tail-2/assert-2.sh: Adjust to the new behavior.
* tests/tail-2/retry.sh: Add a test case. Also change
from `tail ... && fail=1` to the more robust `returns_ 1 ...`
construct which detects segfaults etc.
* NEWS: Document the fix.
* src/tail.c (tail_forever): The BLOCKING optimization is only
enabled for non regular files (which can't be truncated), so ensure
we don't enable that unless we've a valid file descriptor.
* tests/tail-2/retry.sh: Add a test case.
* NEWS: Mention the bug fix.
* src/tail.c (tail_forever): Only read up to st_size on network
file systems to avoid the issue with a stale attribute cache
returning a smaller st_size than we have already read().
The was seen with glusterfs at least and caused the complete
file to be repeatedly output due to assuming the file was
truncated in this case.
* NEWS: Mention the fix.
* src/tail.c (ignore_pipe_or_fifo): Mark the descriptor as -1
for pipes so that any_live_files() detects correctly that
the entry is no longer live.
* tests/tail-2/pipe-f.sh: Add a test case.
* NEWS: Mention the fix.
Fixes http://bugs.gnu.org/24903 which was detected
using Symbolic Execution techniques developed in
the course of the SYMBIOSYS research project at
COMSYS, RWTH Aachen University.
* src/system.h (emit_backup_suffix_note): A new function to
output the backup suffix info. The strings are unchanged,
so translations are not impacted.
* src/cp.c (usage): Use the new function.
* src/ln.c (usage): Likewise.
* src/mv.c (usage): Likewise.
* src/install.c (usage): Likewise.
* src/cp.c (main): Avoid the getenv("SIMPLE_BACKUP_SUFFIX") call,
which is now done if needed in the gnulib backupfile module.
Also avoid the redundant strdup, as we don't modify this suffix.
* src/install.c (main): Likewise.
* src/ln.c (main): Likewise.
* src/mv.c (main): Likewise.
Fixes http://bugs.gnu.org/23153
* doc/coreutils.texi (stat invocation): Describe the
QUOTING_STYLE values now supported.
* src/stat.c (getenv_quoting_style): A new function called
from main, that sets the default quoting style for quotearg.
(main): Call getenv_quoting_style() when %N specified.
* tests/misc/stat-fmt.sh: Add a test case.
* NEWS: Mention the improvement.
Fixes http://bugs.gnu.org/23422
* src/dd.c (parse_integer): Suggest to use "00x" instead of "0x",
which is significant for the "count", "seek", and "skip" operands.
* tests/dd/misc.sh: Add a test case.
Fixes http://bugs.gnu.org/24874
* NEWS: Mention the fix.
* src/md5sum.c (digest_file): Add a new MISSING parameter to
return whether the file was missing, separately from the digest.
* tests/misc/md5sum.pl: Add a test case.
Fixes http://bugs.gnu.org/24795
* src/expr.c (integer_overflow): Remove an abort-after-die.
* src/paste.c (write_error): Likewise.
* src/sort.c (badfieldspec): Likewise.
* src/tr.c (string2_extend): Likewise. Also remove a few
break statements, each after an abort.
die() has the advantage of being apparent to the compiler
that it doesn't return, which will avoid warnings in some cases,
and possibly generate better code.
* cfg.mk (sc_die_EXIT_FAILURE): A new syntax check rule to
catch any new uses of error (CONSTANT, ...);
* src/die.h (die): New file/function from grep.
Note: we expect this file to migrate to gnulib.
* src/csplit.c: Include die.h.
(check_format_conv_type): Use die in place of error-nonzero;break;
* src/install.c (strip): Likewise.
* src/nl.c (proc_text): Likewise. This also suppresses a new warning
from GCC 7's -Werror=strict-overflow.
* src/tail.c (parse_options): Likewise.
* src/basename.c (main): Adjust "fall through" comment
so that GCC 7's -Wimplicit-fallthrough honors it.
* src/cp.c (main): Add a "fall through" comment.
* src/ls.c (gobble_file): Likewise.
(get_funky_string): Adjust a "fall through" comment so it is
recognized.
* cfg.mk (exclude_file_name_regexp--sc_system_h_headers): Add die.h
to this list of exempt src/*.h files.
* src/pr.c (main): Avoid this warning from GCC 7:
src/pr.c:1119:6: error: assuming signed overflow does not occur when \
simplifying conditional to constant [-Werror=strict-overflow]
if (n_files == 0)
* src/rm.c (main): Ensure the full --no-preserve-root
option is specified, rather than allowing --n etc.
* tests/rm/r-root.sh: Add a test case.
* NEWS: Mention the change in behavior.
Improved by Jim Meyering.
Fixes http://bugs.gnu.org/24604
which was not the case when inotify was not available.
* src/tail.c (any_live_files): Simplify, since the IGNORE
flag is now only set when a file should be ignored indefinitely.
(recheck): Only output the "giving up on name" message
when that's actually the case. Only set the IGNORE flag
when ignoring a file indefinitely.
(tail_file): Likewise.
* tests/tail-2/retry.sh: Add a test case. Also run
all existing test cases with and without inotify.
NEWS: Mention the fix.
THANKS.in: Add the reporter.
Fixes http://bugs.gnu.org/24495 which was detected
using Symbolic Execution techniques developed in
the course of the SYMBIOSYS research project at
COMSYS, RWTH Aachen University.
* src/who.c (idle_string): This function would fail to compile
with -Werror and today's built-from-git gcc due to this warning:
src/who.c: In function 'print_user':
src/who.c:201:36: error: may write format character ':' at offset 4 \
past the end of the destination [-Werror=format-length=]
sprintf (idle_hhmm, "%02d:%02d",
^~~~~
The fix is to use an assertion to inform gcc of the existing
invariant that guarantees the number of hours is less than 24.
The old code mishandled --time-spec='+%%b', as it misinterpreted
the '%b' as being the month abbreviation. Also, it mishandled
the extremely unlikely case of a month abbreviation containing '%'.
The performance part of this patch sped up 'ls' by about 1% on my
little benchmark of 'ls -lR' on the source directory in the
en_US.UTF-8 locale (Fedora 24 x86-64).
* NEWS: Document the bug fix.
* src/ls.c (first_percent_b, abformat_init): New static functions.
(ABFORMAT_SIZE): New constant.
(use_abformat): New static var.
(abmon, required_mon_width): Remove these static vars.
(abmon_init): Now accepts a pointer to abmon, and returns a boolean.
All callers changed. Reject month abbrs containing '%', as these
would mess up strftime. Simplify mbsalign result checking,
since (size_t) -1 exceeds ABFORMAT_SIZE.
(abformat_init, align_nstrftime): Precompute all 24 formats at
startup, rather than computing a format for each time stamp.
(decode_switches): Call abformat_init instead of abmon_init.
(align_nstrftime): Accept recentness bool instead of format.
All callers changed.
* tests/misc/time-style.sh: Test for format with '%%b'.
Given that `ttyname` is already conditionally declared in src/system.h,
other declarations are redundant and problematic for example in ChromeOS
which has a new FORTIFY implementation that, as an artifact of how it's
implemented, causes the compiler to complain if certain standard library
functions are redeclared without special compiler-specific attributes.
* src/pinky.c: Remove declaration (which was unused anyway).
* src/who.c: Remove declaration.
* src/system.h (getlogin, getuid, geteuid, getgrgid, getpwuid, ttyname):
Add the parameter type to the declarations to avoid warnings when these
backup declarations are used.
This doesn't affect the generated code on my system; it's merely
a cleaner way to use the recently-introduced Linux-specific API.
* m4/jm-macros.m4 (gl_CHECK_ALL_HEADERS): Check for linux/fs.h.
* src/copy.c: Include <linux.fs.h> if available.
(FICLONE) [__linux__]: Define if not already defined.
(clone_file): Use FICLONE instead of the older BTRFS_IOC_CLONE,
as this ioctl is no longer btrfs-specific.
* bootstrap.conf (gnulib_modules): Add flexmember.
* src/sort.c: Include flexmember.h.
(struct tempnode): Make the last member flexible.
(create_temp_file): Port to strict C11/C99 rules for
allocation alignment with flexible array members.
* src/factor.c (lbuf_putc): Only buffer more than a line
when not using the tool interactively.
* NEWS: Mention the bug fix.
Fixes http://pad.lv/1620139
Postpone installation of signal handlers until they're needed.
That is right before the first escape sequence is printed.
* src/ls.c (signal_setup): A new function refactored from main()
to set and restore signal handlers.
(main): Move signal handler setup to put_indicator()
so that the default signal handling is untouched as long as possible.
Adjusted condition for restoring signal handlers to reflect the change.
(put_indicator): Install signal handlers if called for the very first
time. It uses the same code that was in main() prior to this commit.
* NEWS: Mention the improvement.
See https://bugzilla.redhat.com/1365933
Fixes http://bugs.gnu.org/24232
* src/tr.c (unquote): Use indices of type "unsigned int", rather
than a mix or "size_t" and "int", presuming that no command-line-
specified string will have length longer than UINT_MAX.
* src/ptx.c (fix_output_parameters): Switch to an unsigned type that
matches the OCCURS.file_index type. This avoids the following error
from gcc-7.0.0 20160829 (experimental):
src/ptx.c:1220:14: error: assuming signed overflow does not occur \
when simplifying conditional to constant [-Werror=strict-overflow]
if (file_index > 0)
* src/cp.c (make_dir_parents_private): Use default permissions for
created directories when --no-preserve=mode is specified.
* tests/cp/cp-parents.sh: Add a test case.
* NEWS: Mention the fix.
Fixes http://bugs.gnu.org/24251
* src/expand.c, src/unexpand.c: Move global variables from here...
* src/expand-common.h, src/expand-common.c: ... to here.
* src/expand.c, src/unexpand.c: (parse_tab_stops, validate_tab_stops,
next_file): Move identical functions to new module.
(add_tab_stop): Move to new module, including additional code from
'unexpand' (keeping max_column_width) which will have no effect in when
used in 'expand'. Refactor common next-column calculation code
into a new function 'get_next_tab_column'.
* src/local.mk: (src_expand_SOURCES, src_unexpand_SOURCES): Add
'expand-common.c'; (noinst_HEADERS): Add 'expand-common.h'.
* po/POTFILES.in: Add 'expand-common.c'.
* tests/misc/expand.pl: Add more tests.
* tests/misc/unexpand.pl: Likewise.
* TODO: Move conclusions to above test after investigation.
