Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
Encodes/decodes data in various common formats:
base64,base64url,base32,base32,base16,base2,z85.
Discussed here:
https://lists.gnu.org/r/coreutils/2018-11/msg00014.htmlhttps://lists.gnu.org/r/coreutils/2018-12/msg00019.html
* AUTHORS: Add basenc.
* README: Reference the new program.
* NEWS: Mention the new program.
* build-aux/gen-lists-of-programs.sh: Add basenc.
* doc/coreutils.texi: (basenc invocation): Document the new command.
* man/.gitignore: Ignore the generated man page.
* man/basenc.x: A new template, with few examples.
* man/local.mk: Reference the new man page.
* scripts/git-hooks/commit-msg: Allow basenc as program prefix.
* src/.gitignore: Ignore the new binary.
* src/basenc.c:
(usage): Mention new options.
(main): Handle new options.
(isbase*, base*_length, base*_encode, base*_decode_ctx): Implement new
encoding/decoding formats.
* src/local.mk: Add new program.
* tests/local.mk: Add new test.
* tests/misc/basenc.pl: New tests.
* tests/misc/help-version.sh (basenc_setup): use '--version' for default
invocation (basenc errors with no parameters).
* src/echo.c (main): Always enable backslash processing if
POSIXLY_CORRECT is set.
* tests/misc/echo.sh: Add (the first) test for the echo command.
* tests/local.mk: Reference the new test.
* tests/misc/printf.sh: Update a stale comment.
* doc/coreutils.texi (echo invocation). Mention that POSIXLY_CORRECT
now always enables backslash processing.
* NEWS: Mention the change in behavior.
Fixes https://bugs.gnu.org/32703
Issue identified by Eric Blake.
Bash knows 'test -N FILE'. Add it to GNU 'test' as well.
* src/test.c (unary_operator): Add a case for 'N'.
(usage): Document it.
* doc/coreutils.texi (node File characteristic tests): Likewise.
* NEWS (New features): Likewise.
* tests/misc/test-N.sh: Add a test.
* tests/local.mk (all_tests): Reference it.
$ id root nobody
uid=0(root) gid=0(root) groups=0(root)
uid=99(nobody) gid=99(nobody) groups=99(nobody)
* src/id.c (main): Make variables opt_zero, just_group_list,
just_group, use_real, just_user global to be used in a new
function.
(print_stuff): New function that will print user and group
information for the specified USER.
When using -G option delimit each record with two NULs.
Restructure the code in the file to have global variables
followed by functions.
* tests/id/zero.sh: Add test cases to check the usage
of -z option with multiple users.
* tests/id/uid.sh: Add a test case to ensure all users
are queried in the presence of errors.
* doc/coreutils.texi: Document the interface changes.
* NEWS: Mention the new feature.
* doc/coreutils.texi (csplit invocation): Detail the behavior
with regexp patterns and negative offsets, which differs from
line number patterns, to avoid looping on the input. For example:
$ seq 50 | csplit -s - /15/-5 /12/
csplit: ‘/12/’: match not found
* doc/coreutils.texi (csplit invocation): Clarify that
portions of the input may be skipped and thus the input
may not be reproducible by just concatenating the output files.
Fixes https://bugs.gnu.org/32317
* src/remove.c (rm_fts): With the --preserve-root=all extension,
reject command line arguments that are mount points.
* src/remove.h (rm_options): Add preserve_all_root to store config.
* src/mv.c (rm_option_init): Init preserve_all_root to false.
* src/rm.c (main): Init preserve_all_root as per option.
(usage): Describe the new option.
* src/remove.c (rm_fts): Lookup the parent device id,
and reject the cli argument if a separate file system.
* tests/rm/one-file-system.sh: Add a test case.
* NEWS: Mention the new feature.
This mode is currently the default, but most if not all users of
reflink-capable filesystems want --reflink=auto, which is often
encapsulated into an alias. Adding --reflink=never allows overriding
such an alias.
* doc/coreutils.texi (cp invocation): Describe the new option.
* src/cp.c: Support --reflink=never.
* tests/cp/reflink-auto.sh: Add a test case.
* NEWS: Mention the new feature.
Adopted from FreeBSD's env(1), useful for specifing multiple
parameters on a shebang (#!) script line, e.g:
#!/usr/bin/env -S perl -w -T
Discussed in https://lists.gnu.org/r/coreutils/2018-04/msg00011.html
* src/env.c (valid_escape_sequence,escape_char,scan_varname,
extract_varname,validate_split_str,build_argv,
parse_split_string): New functions.
(main): Process new option and call parse_split_string.
(usage): Mention new option.
* tests/misc/env-S.pl: Test new option from the command line.
* tests/misc/env-S-script.sh: Test new option from shebang scripts.
* tests/local.mk (all_tests): Add new tests.
* man/env.x (OPTIONS): Show a brief example of -S usage and point to
the full documentation for more information.
* doc/coreutils.texi (env invocation): Detail usage of -S/--split-string
option.
* NEWS: Mention new option.
* doc/coreutils.texi (md5sum invocation): Describe the new option,
and how it's not supported by --check, and how it disables escaping.
* src/md5sum.c (delim): A new global to parmeterize the out delimiter.
(main): Don't enable file name escaping with -z, and output '\0'.
* tests/misc/md5sum-newline.pl: Add a test case.
* NEWS: Mention the new feature.
* src/copy.c (copy_internal): Don't fail immediately upon
getting ELOOP when running stat() on the destination,
rather proceeding if -f specified, allowing the link
to be removed. If the loop is not in the final component
of the destination path, we still fail but at the
subsequent unlink() stage.
* doc/coreutils.texi (cp invocation): Adjust wording to say
that --force doesn't work with dangling links, rather than
all links that can't be traversed.
* tests/cp/thru-dangling.sh: Add a test case.
* NEWS: Mention the change in behavior.
Discussed in https://bugs.gnu.org/31335
* src/cp.c (target_directory_operand): Allow through inaccessible
arguments with -f or --remove.
* doc/coreutils.texi (cp invocation): Clarify that -f doesn't directly
impact the removal of non-traversable symlinks.
* tests/cp/dir-rm-dest.sh: Test the new behavior.
* tests/cp/thru-dangling.sh: Enforce -f behavior wrt symlinks.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/31335
* doc/coreutils.texi: Use @example consistently
as we don't need the smaller or fixed width representation.
This is especially true for the synopsis of commands.
@smallexample is rendered left aligned for HTML
which is awkward to read with the center aligned main content.
In both chown and chgrp (which shares its code with chown), operating
on symlinks recursively has a window of vulnerability where the
destination user or group can change the target of the operation.
Warn about combining the --dereference, --recursive, and -L flags.
* doc/coreutils.texi (warnOptDerefWithRec): Add macro.
(node chown invocation): Add it to --dereference and -L.
(node chgrp invocation): Likewise.
See also: CVE-2017-18018
* NEWS: Mention these fixes.
* doc/coreutils.texi (cp invocation, mv invocation):
Mention that -n is silent, and that it overrides -u.
* src/cp.c, src/mv.c (main): -n overrides -u.
* doc/coreutils.texi: the documentation for the --dereference
flag of chown/chgrp states that it is the default mode of
operation. Document that this is only the case when operating
non-recursively.
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
* doc/coreutils.texi (cp invocation): The language used
to describe recreating the file was a little confusing
as it mentioned opening a removed file.
Fixes https://bugs.gnu.org/29315
This is useful as handling in shell is complicated
with the varying exit status in the --kill-after case.
* src/timeout.c (main): Handle '-v' and store
COMMAND for the diagnostic.
(cleanup): Diagnose the signal name before sending.
(usage): Document -v, --verbose.
* doc/coreutils.texi (timeout invocation): Likewise.
* tests/misc/timeout.sh: Add a test case.
* NEWS: Mention the new feature
Fixes https://bugs.gnu.org/21760
Older versions of 'makeinfo' choke on a missing reference:
./doc/coreutils.texi:14177: `Realpath usage examples' has no Up field\
(perhaps incorrect sectioning?).
makeinfo: Removing output file `doc/coreutils.info' due to errors; \
use --force to preserve.
* doc/coreutils.texi (realpath invocation): Add a menu referencing
the usage examples - introduced in v8.27-91-g7449f0d.
Previously with oflag=direct the call to invalidate_cache()
was not passed to the kernel, as it was less than a page size,
and a subsequent call was not made to invalidate the pending space.
Similarly with oflag=nocache the pending space at EOF was
not invalidated. Even though these amount to only a single page
in the page cache it can be significant. For example on
XFS before kernel patch v4.9-rc1-4-g0ee7a3f, O_DIRECT files
would have been read inefficiently if any pages were cached,
even if they were already synced to storage.
* src/dd.c (i_nocache_eof, o_nocache_eof): New bools used
to control when we want invalidate_cache(,0) to clear to EOF.
(cache_round): Use IO_BUFSIZE (currently 132KiB) to minimize
calls to the relatively expensive advise function, rather
than page_size. This also makes it clear that while the
kernel function operates on pages, this size is chosen for
performance reasons.
(invalidate_cache): Refactor to share more code between
input and output paths. Use i_nocache_eof and o_nocache_eof
rather than proxying off max_records. Ensure we
invalidate full pages when clearing to EOF as the kernel
will ignore any non complete pages. Fix the offset used
for the output path.
(dd_copy): Invalidate the cache of the input after the
offset is updated, for consistency and so we don't try to
invalidate before the start of the file. When we read
EOF on input, set flags so that we invalidate to EOF.
(main): Invalidate to EOF in more cases, by depending
on the i_nocache_eof and o_nocache_eof flags.
* doc/coreutils.texi (dd invocation): Clarify the alignment
and persisted caveats on the example applying "nocache"
to part of a file.
* tests/dd/nocache_eof.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
Issue reported by Eric Bergen.
* doc/coreutils.texi: Remove doubled word.
* src/targetdir.c: Explicitly mark exported function.
* tests/local.mk: This is not a root only test.
* tests/mv/vulnerable-target.sh: Use returns_.
Introduced in v8.28-3-g44ccd1c
* NEWS, doc/coreutils.texi (Target directory): Document this.
* src/cp.c, src/install.c, src/ln.c, src/mv.c: Include targetdir.h.
(target_directory_operand): Use the new targetdir_operand_type
function to check for vulnerable target directories.
* src/cp.c (stat_target_operand): New function.
(target_directory_operand, do_copy): Use it.
* src/local.mk (noinst_HEADERS): Add src/targetdir.h.
(src_ginstall_SOURCES, src_cp_SOURCES, src_ln_SOURCES)
(src_mv_SOURCES): Add src/targetdir.c.
* src/targetdir.c, src/targetdir.h: New files.
* tests/mv/vulnerable-target.sh: New test.
* tests/local.mk (all_root_tests): Add it.
* src/tty.c (main): Don't distinguish ENOTTY from other errors,
because isatty() doesn't portably distinguish errors.
Solaris returns ENOENT for all input errors for example.
Musl also returns ENOENT, and ENODEV may be returned as disscussed at:
http://openwall.com/lists/musl/2017/04/06/6
* tests/misc/tty.sh: Adjust accordingly.
Terminals such as iTerm2 and VTE based terminals
(as of version 0.49.1), support hyperlinks when
passed terminals codes as described at:
https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda
* src/ls.c (gobble_file): Allocate an absolute file name to output.
(quote_name): Output the absolute name with the appropriate codes.
(file_escape): A new function to encode file names as per rfc8089.
(main): Handle the new option and call the file_escape_init() helper.
Disable --dired when --hyperlink is specified.
(print_dir): Get the absolute file name here too, so that the
directory name can be linkified.
* NEWS: Mention the new feature.
* tests/ls/hyperlink.sh: Add a new test.
* tests/local.mk: Reference the new test.
* doc/coreutils.texi (ls invocation): Describe --hyperlink.
This is useful when chaining with other commands that run commands in a
different context, while avoiding using the shell to cd, and thus
having to consider shell quoting the chained command.
* NEWS (New features): Document the new option.
* doc/coreutils.texi (env invocation): Likewise.
* src/env.c (usage): Likewise.
(main): Implement the new option.
* tests/misc/env.sh: Test the new option.
The 'readlink' node has '@findex realpath' in it. This results in
info doc/coreutils.info realpath
incorrectly jumping to the 'readlink' node (instead of the 'realpath'
node). Change it to @cindex instead.
* doc/coreutils.texi (readlink): Change '@findex realpath' to @cindex.
* src/realpath.c (usage): Explicitly say 'DIR' instead of 'FILE' for
--relative-{to,base} parameters, to avoid giving the impression
that regular files can be used as relative base.
* doc/coreutils.texi (realpath): Same.
Suggested by Kamil Dudka in:
http://lists.gnu.org/archive/html/coreutils/2017-07/msg00072.html
* NEWS: Document the changed nature of the fix.
* doc/coreutils.texi, tests/cp/backup-is-src.sh:
* tests/mv/backup-is-src.sh: Revert previous change.
* src/copy.c (source_is_dst_backup): New function.
(copy_internal): Use it. Fail instead of falling back on numbered
backups when it looks like the backup will overwrite the source.
Although this reintroduces a race, it's more compatible with
previous behavior.
* NEWS, doc/coreutils.texi (Backup options): Document the change.
* bootstrap.conf (gnulib_modules): Add backup-rename.
* src/copy.c (copy_internal): Silently switch to numbered backups
if a simple backup might lose data. Use backup_file_rename
to avoid races with numbered backups.
* tests/cp/backup-is-src.sh, tests/mv/backup-is-src.sh:
Adjust to match new behavior.
* doc/coreutils.texi (Options for date): Capitalize a sentence.
* tests/misc/date-debug.sh: Adjust --debug output to match
recent changes to Gnulib’s parse-datetime module.
On some platforms, isatty succeeds but ttyname fails.
POSIX does not seem to allow this, but there it is.
Problem reported by Christian Brauner (Bug#26371).
While we’re at it, check for errors more carefully and return a
new exit status 4 if stdin is closed or a similar error occurs.
* doc/coreutils.texi (tty invocation): Document new behavior.
* init.cfg (stderr_fileno_):
Don't assume have_input_tty is not in the environment.
* src/tty.c (TTY_STDIN_ERROR): New constant.
(main): Exit with nonzero status if there is a usage error,
like other coreutils programs.
Check for error in getting stdin type.
* tests/misc/tty.sh: New file.
* tests/local.mk (all_tests): Add it.
Support --tabs="1,+8" which is equivalent to --tabs="1,9,17,..."
useful for viewing unified diff output with its 1 character
gutter for example.
* doc/coreutils.texi ({expand,unexpand} invocation): Document,
using diff processing as the example.
* src/expand-common.c (set_increment_size): Update the new
increment_size global.
(parse_tab_stops): Handle the new '+' prefix.
(finalize_tab_stops): Verify both '+' and '/' prefixes
are not used together.
* tests/misc/expand.pl: Add test cases.
* NEWS: Mention the new feature.
* doc/coreutils.texi (split invocation): Document the new option.
* src/split.c (usage): Likewise.
(main): Process the new option much like --numeric-suffixes,
but with an adjusted alphabet.
* tests/split/numeric.sh: Refactor to support --hex mode.
* NEWS: Mention the new feature.
