cksum --check is often the first interaction
users have with possibly untrusted downloads, so we should try
to be as defensive as possible when processing it.
Specifically we currently only escape \n characters in file names
presented in checksum files being parsed with cksum --check.
This gives some possibilty of dumping arbitrary data to the terminal
when checking downloads from an untrusted source.
This change gives these advantages:
1. Avoids dumping arbitrary data to vulnerable terminals
2. Avoids visual deception with ansi codes hiding checksum failures
3. More secure if users copy and paste file names from --check output
4. Simplifies programmatic parsing
Note this changes programmatic parsing, but given the original
format was so awkward to parse, I expect that's extremely rare.
I was not able to find example in the wild at least.
To parse the new format from from shell, you can do something like:
cksum -c checksums | while IFS= read -r line; do
case $line in
*': FAILED')
filename=$(eval "printf '%s' ${line%: FAILED}")
cp -v "$filename" /quarantine
;;
esac
done
This change also slightly reduces the size of the sum(1) utility.
This change also apples to md5sum, sha*sum, and b2sum.
* src/cksum.c (digest_check): Call quotef() instead of
cksum(1) specific quoting.
* tests/cksum/md5sum-bsd.sh: Adjust accordingly.
* doc/coreutils.texi (cksum general options): Describe the
shell quoting used for problematic file names.
* NEWS: Mention the change in behavior.
Reported by: Aaron Rainbolt
Cleartext signatures have many gotchas. Therefore, the use of detached
signatures is recommended where possible. See:
<https://gnupg.org/blog/20251226-cleartext-signatures.html>.
* doc/coreutils.texi (tee invocation): Adjust gpg invocation to produce
a detached signature.
Here is an example of the performance improvement:
$ yes abcdefghijklmnopqrstuvwxyz | head -n 100000000 > input
$ time ./src/wc-prev -l < input
100000000
real 0m0.793s
user 0m0.630s
sys 0m0.162s
$ time ./src/wc -l < input
100000000
real 0m0.230s
user 0m0.065s
sys 0m0.164s
* NEWS: Mention the performance improvement.
* gnulib: Update to the latest commit.
* configure.ac: Check the the necessary intrinsics and functions.
* src/local.mk (noinst_LIBRARIES) [USE_NEON_WC_LINECOUNT]: Add
src/libwc_neon.a.
(src_libwc_neon_a_SOURCES, wc_neon_ldadd, src_libwc_neon_a_CFLAGS)
[USE_NEON_WC_LINECOUNT]: New variables.
(src_wc_LDADD) [USE_NEON_WC_LINECOUNT]: Add $(wc_neon_ldadd).
* src/wc.c [USE_NEON_WC_LINECOUNT]: Include sys/auxv.h and asm/hwcap.h.
(neon_supported) [USE_NEON_WC_LINECOUNT]: New function.
(wc_lines) [USE_NEON_WC_LINECOUNT]: Use neon_supported and
wc_lines_neon.
* src/wc.h (wc_lines_neon): Add declaration.
* src/wc_neon.c: New file.
* doc/coreutils.texi (Hardware Acceleration): Document the "-ASIMD"
hwcap and the variable used in ./configure to override detection of Neon
instructions.
* tests/wc/wc-cpu.sh: Also add "-ASIMD" to disable the use of Neon
instructions.
A user that had an environment with SHELL=/usr/bin/fish
reported the following when trying to build info docs:
build-aux/missing (line 4): Unsupported use of '='. In fish
* doc/local.mk: Propagate make's SHELL to our
makeinfo wrapper, so that the substituted ${SHELL}
uses a compatible shell from the Makefile.
Font problem reported by Michael Aramini via Alejandro Colomar
<https://bugs.gnu.org/80258>. This patch also fixes some
longstanding confusion with date synopses.
* src/date.c (usage): Do not imply that only -u can be used with
MMDDhhmm..., and do not put misleading brackets around the latter.
This is a more standard mechanism to disable markup.
* src/system.h (oputs_): Logic change to honor TERM=dumb,
rather than HELP_NO_MARKUP=something.
* doc/coreutils.texi: Adjust the description for --help.
* man/local.mk: Ensure TERM is set to something,
so that man pages have links included.
* man/viewman: Just honor users $TERM.
* tests/misc/getopt_vs_usage.sh: Remove env var complication,
as TERM is unset automatically.
* tests/misc/usage_vs_refs.sh: Likewise.
* NEWS: Adjust the change in behavior note.
* src/cp (usage): The -HLP options are close
in functionality and close alphabetically, so describe together.
* doc/coreutils.texi (cp invocation): Likewise.
* doc/coreutils.texi: Add missing anchors.
* src/pr.c (Usage): Adjust to use -COLS, to avoid a clash
with the additional anchor added to the manual.
Also markup the --columns option as done for other options.
* tests/split/line-bytes.sh: Also fix --lines-bytes typo here.
* src/ls.c (oputs): A new function that wraps puts(),
but also highlights the --option-text portion, and
adds links to the appropriate part of the online manual.
(usage): Call oputs() rather than puts().
* doc/coreutils.texi (--help): Document new HELP_NO_MARKUP env var,
which can be used in the edge case one wants to suppress ansi escapes.
* tests/misc/getopt_vs_usage.sh: Use HELP_NO_MARKUP to ensure the
test continues to pass.
* .gitignore (/build-aux/makeinfo-wrapper.sh):
* Makefile.am (EXTRA_DIST): Remove build-aux/makeinfo-wrapper.sh.
* build-aux/makeinfo-wrapper.sh.in: New file, renamed from
build-aux/makeinfo-wrapper.sh and modified to use @MAKEINFO@ instead of
makeinfo.
* cfg.mk (MAKEINFO): Remove variable.
* configure.ac: Add build-aux/makeinfo-wrapper.sh as a configure file.
* doc/local.mk (MAKEINFO): Define to
$(abs_top_builddir)/build-aux/makeinfo-wrapper.sh.
(AM_MAKEINFOFLAGS): Move over some values from the previous MAKEINFO
definition.
* src/paste.c (usage): Mention how lines are processed
with and without the -s option. Also mention that -d
supports backslash escapes.
* doc/coreutils.texi (paste invocation): Likewise.
Also detail the backslash escapes, noting which are non-POSIX.
Align the -t implementation with the Heirloom project.
* src/ptx.c (usage): Describe -t, and also mention
the default width is 72 when not used.
* doc/coreutils.texi (ptx invocation): Likewise.
(main): Override the default width if -t is specified.
* tests/ptx/ptx.pl: Add test cases.
* NEWS: Mention the change in behavior.
* doc/coreutils.texi (Introduction): Use ç instead of @,{c}.
(Character arrays): Use ö instead of @"o. Use Ł instead of @L{}.
(Formatting file timestamps): Use ä instead of @"a.
Following commit v9.3-92-g1b86b70dd
$TMPDIR is part of the interface and an important behavioral
characteristic of a command, which should be documented.
* doc/coreutils.texi (split invocation): Mention $TMPDIR is honored.
(tac invocation): Likewise.
* src/split.c (usage): Likewise.
* src/tac.c (usage): Likewise.
* doc/coreutils.texi (dd invocation): Document the behavior of 'dd' on
multibyte characters and some unspecified behavior that will be
documented in a future POSIX release [1].
[1] https://austingroupbugs.net/view.php?id=1959
* doc/coreutils.texi (quotingStyles): Expand on the advantages
of "shell-escape" quoting, and mention it's the default when
outputting to a tty. Also mention how it's also useful with
LC_ALL=C to further disambiguate output. Also reference the
separate page detailing various considerations and options
for file name quoting. Also move the mention of the default
quoting style to the top of the page where it's more obvious.
Previously the html-local make rule only worked for `make html`.
Instead add support for `make doc/coreutils.html` or `make web-manual`
through the use of a makeinfo wrapper.
* doc/local.mk: Move post processing from here to ...
* build-aux/makeinfo-wrapper.sh: ... here.
* cfg.mk: Ensure our wrapper is called with MAKEINFO.
Also pass --no-node-files so redirection html files
are not created for each anchor.
* doc/local.mk (html-local): Add html files to prerequisites so they are
generated before 'sed' is invoked. Also interate over the already
populated $(HTMLS) to be more general (to multiple html files),
and more portable (to non GNU make).
* doc/coreutils.texi: Explicitly supply empty arguments to macros,
as dvi (a required prerequisite to pdf) is more strict in its
handling of macro arguments.
* cfg.mk (sc_texi_ensure_empty_option_args): Add a syntax check,
since this is not verified in the default build.
Reported by Collin Funk.
* doc/coreutils.texi (optAnchor): A new macro to output a
referencable anchor, called from ...
(optItem): ... here; a new macro to output all index entries
for each option item.
(optZero,optZeroTerminated): Show an example of the adjustment
done to each option description.
* doc/local.mk (html-local): Post-process the texinfo generated HTML
(`make html`) to remove our "-option" tag, and replace all
escaped _002d with a standard hyphen, which is fine in URLs.
* doc/coreutils.texi (du invocation): Mention that --threshold has no
effect on the grand total calculated when --total is used.
Addresses https://bugs.gnu.org/71309
The --apparent-size option to 'du' was added in
coreutils-4.5.8 (2003). FreeBSD 8.0 (2009) added the same functionality
under the short option -A. This long option previously had no short
option, so this patch adds -A to be compatible with FreeBSD.
* NEWS: Mention the new short option.
* doc/coreutils.texi: Document the short option.
* src/du.c (usage): Likewise.
(APPARENT_SIZE_OPTION): Remove definition.
(EXCLUDE_OPTION): Define to CHAR_MAX + 1.
(long_options): Use the -A short option for --apparent-size.
(main): Likewise.
* tests/du/apparent.sh: Test that '-b', '-A -B 1', and
'--apparent-size --block-size 1' function the same.
* doc/coreutils.texi (tail invocation): Describe --debug.
* src/tail.c (tail_forever, tail_forever_inotify): Output
which --follow implementation is being used.
* tests/tail/debug.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the new feature.
* doc/coreutils.texi (shred invocation): Give more details on what
--verbose outputs, to give some indication of its utility.
* src/shred.c (usage): Likewise. Also since we're changing the string,
split out translations to give translators more granular translations.
Make the documentation more accurate following v9.3-80-g5e1e0993b
* doc/coreutils.texi (cksum common options): State that --text and
--binary are actually supported, but only to support emulation
of the legacy utils.
Many people are used to seeing ISO 8601 dates using a period separating
seconds and nanoseconds. This behavior seems to be worth documenting
given the bug reports:
https://bugs.gnu.org/63119https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1118970
* doc/coreutils.texi (Options for date): Mention that
'date --iso-8601=ns' uses a comma as a separator, following the
preference of ISO 8601. Give an example of how to get an ISO 8601 date
with a period separator.
* NEWS: Mention the improvement. Mention that 'sort' will continue
without compressing temporary files if the program specified by
--compress-program cannot be executed.
* doc/coreutils.texi (sort invocation): Document the behavior when the
program specified by --compress-program cannot be executed.
* src/sort.c: Include spawn.h.
(MAX_FORK_TRIES_COMPRESS, MAX_FORK_TRIES_DECOMPRESS): Remove definition.
(MAX_TRIES_COMPRESS, MAX_TRIES_DECOMPRESS): New definitions based on
MAX_FORK_TRIES_COMPRESS and MAX_FORK_TRIES_DECOMPRESS.
(async_safe_die): Remove function.
(posix_spawn_file_actions_move_fd): New function.
(pipe_fork): Remove function.
(pipe_child): New function based on pipe_fork. Return an error number
instead of a pid. Use posix_spawnp instead of calling fork and expecting
the caller to exec.
(maybe_create_temp): Call pipe_child instead of pipe_fork. Print a
warning to standard error if --compress-program cannot be executed and
the error is different than the previous call. Remove code for the child
process.
(open_temp): Remove code for the child process. Improve error message.
* tests/sort/sort-compress.sh: Add a test case for when the program
specified by --compress-program does not exist.
Output, accept, or disallow a string between the number and unit
as recommended in <https://physics.nist.gov/cuu/Units/checklist.html>
I.e. support outputting numbers of the form: "1234 M"
* src/numfmt.c (simple_strtod_human): Skip unit separator if present,
or disallow a unit separator if empty.
(double_to_human): Output unit separator if specified.
(main): Accept --unit-separator.
* tests/misc/numfmt.pl: Add test cases.
* doc/coreutils.texi: Describe the new option,
giving examples of interaction with --delimiter.
* NEWS: Mention the new feature.
* THANKS.in: Add Johannes Schauer Marin Rodrigues,
who provided a preliminary patch.
* src/numfmt.c (simple_strtod_human): Accept (multi-byte)
non-breaking space character between number and unit.
Note we restrict this to a single character between number
and unit, to allow less ambiguous parsing if multiple blanks
are used to delimit fields.
* tests/misc/numfmt.pl: Add test cases.
* doc/coreutils.texi (numfmt invocation): Fix stale description
--delimiter skipping whitespace.
* NEWS: Mention the improvement.
Pádraig Brady
Collin Funk
Paul Eggert
Egmont Koblinger