* doc/coreutils.texi (chown invocation): Mention the system dependent
restrictions on setting groups.
(chgrp invocation): Likewise. Reference the 'chown' superset.
* man/chgrp.x: Cross reference chown(1) which is the superset interface,
and also chown(2) which gives details of the platform restrictions.
Fixes http://bugs.gnu.org/17495
* src/install.c (install_file_in_file_parents): Factor out the
creation of any parent directories into ...
(mkancesdirs_safe_wd): ... this new function.
(install_file_in_dir): Add the parameter 'mkdir_and_install', and
call the above new function if it evaluates to true.
(main): During parsing of the -t option, move the check whether
the target_directory exists down after the option parsing loop,
and do not complain about stat(optarg,...) failing if -D was given.
Pass 'mkdir_and_install' to install_file_in_dir().
* doc/coreutils.texi (install invocation): Remove the (false)
restriction that -D would be ignored together with -t. Instead,
clarify install's new bahavior.
Fix the node's reference in the top-level @direntry for consistency.
* src/install/basic-1.sh: Add tests for the now-allowed combination
of the -D and -t options.
* NEWS: Mention the improvement.
* src/join.c (usage): Reword to avoid implication that
the NUL byte is only generated as the output delimeter.
* src/sort.c (usage): Likewise.
* src/shuf.c (usage): Likewise. Also since we're changing the
translation string take the opportunity to separate out
the description to a separate string to reduce translation overhead.
* src/uniq.c (usage): Likewise.
* src/stty.c (usage): s/null/NUL/ for consistency.
* src/basename.c (usage): Reword for accuracy/consistency.
* src/dirname.c (usage): Likewise.
* src/du.c (usage): Likewise.
* src/env.c (usage): Likewise.
* src/printenv.c (usage): Likewise.
* src/readlink.c (usage): Likewise.
* src/realpath.c (usage): Likewise.
* doc/coreutils.texi: Consolidate/share the descriptions of
--null, --zero and --zero-terminated.
It's dangerous and confusing to leave root's supplemental
groups in place when specifying other users with --userspec.
In the edge case that that is desired one can explicitly
specify --groups.
Also we implicitly set the system defined supplemental groups
for a user. The existing mechanism where supplemental groups
needed to be explicitly specified is confusing and not general
when the lookup needs to be done within the chroot.
Also we extend the --groups syntax slightly to allow clearing
the set of supplementary groups using --groups=''.
* src/chroot.c (setgroups): On systems without supplemental groups,
clearing then is a noop and so should return success.
(main): Lookup the primary GID with getpwuid() when just a numeric
uid is specified, and also infer the USERNAME from this call,
needed when we're later looking up the supplemental groups for a user.
Support clearing supplemental groups, either implicitly for
unknown users, or explicitly when --groups='' is specified.
* tests/misc/chroot-credentials.sh: Various new test cases
* doc/coreutils.texi (chroot invocation): Adjust for the new behavior.
* NEWS: Mention the change in behavior.
Problem reported by Don Baggett in <http:/bugs.gnu.org/17422>.
* NEWS:
* doc/coreutils.texi (dd invocation): Document this.
* src/dd.c (conversions): conv=ascii implies conv=unblock.
conv=ebcdic and conv=ibm imply conv=block.
(ascii_to_ebcdic, ebcdic_to_ascii): Correct to match
POSIX 1003.1-2013.
* tests/dd/ascii.sh: New file.
* tests/local.mk (all_tests): Add it.
* src/numfmt.c (setup_padding_buffer): Simplify the code by not
explicitly dealing with heap exhaustion.
(parse_format_string): Likewise. Handle multiple grouping
modifiers as does the standard printf. Handle the new leading
zero --format modifier.
(double_to_human): Use more defensive coding against overwriting
stack buffers. Honor the leading zeros width.
(usage): Mention the leading zero --format modifier.
(main): Allow --padding in combo with a --format (width),
as the number of leading zeros are useful independent of
the main field width.
* doc/coreutils.texi (numfmt invocation): Likewise.
* tests/misc/numfmt.pl: Add new test cases.
* NEWS: Mention the improvement.
* doc/coreutils.texi (pr invocation): Clarify that -w or -W
will be rounded down so that each column has the same width.
Adjust the wording for -W, to avoid the implication that the
width of -S is insignificant to the page width.
* src/pr.c (usage): Add a period to avoid ambiguity in
the man page output.
* doc/coreutils.texi (shred invocation): Mention some reasons
why clearing slack space might be useful.
* src/shred.c (do_wipefd): Add initial writes for each pass
for small regular files in case the storage for those is
in the inode, and thus a larger write up to a block size would
bypass that. Move the direct I/O control to...
(dopass): ... here so we can avoid enabling it for these small
initial writes. It's better to retry direct I/O for each pass
anyway to handle the case where direct I/O is disabled for only
the last portion of a file when the size is not a multiple of
the block size. Note we don't avoid the sync for the initial
write as it will be small but more importantly could be on a
different part of the disk and so worth doing independently
to ensure the write is not discarded.
* tests/misc/shred-exact.sh: Check some more direct I/O cases.
* NEWS: Mention the improvements.
The inode storage issue was mentioned by Paul Eggert.
The description of -u was inaccurate when combined with -D:
$ printf '%s\n' '1 a' '2 a' | uniq -uD -f1
1 a
* doc/coreutils.texi (uniq invocation): Clarify that it's
the last repeated line that is suppressed from the output.
Fixes http://bugs.gnu.org/17022
- Support arbitrary numbers in --groups, consistent with
what is already done for --userspec
- Avoid look-ups entirely for --groups items with a leading '+'
- Support names that are actually numbers in --groups
- Ignore an empty --groups="" option for consistency with --userspec
- Look up both inside and outside the chroot with inside taking
precedence. The look-up outside may load required libraries
to complete the look-up inside the chroot. This can happen for
example with a 32 bit chroot on a 64 bit system, where the
32 bit NSS plugins within the chroot fail to load.
* src/chroot.c (parse_additional_groups): A new function refactored
from set_addition_groups(), to just do the parsing. The actual
setgroups() call is separated out for calling from the chroot later.
(main): Call parse_user_spec() and parse_additional_groups()
both outside and inside the chroot for the reasons outlined above.
* tests/misc/chroot-credentials.sh: Ensure arbitrary numeric IDs
can be specified without causing look-up errors.
* NEWS: Mention the improvements.
* THANKS.in: Add Norihiro Kamae who initially reported the issue
with a proposed patch.
Also thanks to Dmitry V. Levin for his diagnosis and sample patch.
* doc/coreutils.texi (dd invocation): Add an example for how to call
dd to save data from a failing disk. Mention GNU 'ddrescue' as one
of the more specialized tools in such a case.
* src/od.c (main): Handle the new --endian option,
taking "little" and "big" as parameters.
(usage): Describe the new option.
(PRINT_FIELDS): Adjust to swap bytes if required.
* tests/misc/od-endian.sh: A new test to verify
the byte swapping operations for hex (ints) and floats
for all sizes between 1 and 16 inclusive.
* test/local.mk: Reference the new test.
* doc/coreutils.texi (od invocation): Describe the new option.
* NEWS: Mention the new feature.
* src/ls.c: Remove all mention of SELinux since ls
should treat all security context labels equally.
* doc/coreutils.texi (ls invocation): Likewise.
(id invocation): Clarify that -Z outputs the context
inherited by the process, rather than one specific to a user.
Note for SMACK this can be set instead by the SMACK64EXEC label,
in the unusual case where this is set on the id executable.
* src/id.c (usage): Likewise.
* src/mkdir.c (usage): Clarify that -Z is specific to SELinux,
while --context=CTX is also supported for SMACK.
* src/mkfifo.c (usage): Likewise.
* src/mknod.c (usage): Likewise.
Run "make update-copyright", but then also run this,
perl -pi -e 's/2\d\d\d-//' tests/sample-test
to make that one script use the single most recent year number.
Original problem reported by Philipp Thomas in
<http://bugs.gnu.org/16061>.
* NEWS: shuf --repeat, not shuf --repetitions.
* doc/coreutils.texi (shuf invocation):
* src/shuf.c (usage, long_opts, main):
* tests/misc/shuf.sh:
Likewise. Also, the default head-count is infinity.
Since v8.21-172-g33660b4, df not only treats symbolic link arguments
differently, as stated there, but now generally processes special file
arguments in a non-canonicalized form correctly:
$ cd /dev && df-old sdb
Filesystem 1K-blocks Used Available Use% Mounted on
devtmpfs 1014572 48 1014524 1% /dev
$ cd /dev && df-new sdb
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdb 10190136 6039532 3609932 63% /home
Document df's new behavior.
* doc/coreutils.texi (df invocation): In the paragraph describing
df's behavior regarding special file arguments, relax the condition
for such special files from "... is an absolute name of ..." to
"... resolves to ...".
* NEWS (Bug fixes): Mention the new behavior also here.
* doc/coreutils.texi (ls invocation): Remove the note about
OSX terminals not aligning properly as this is no longer the case.
Tested by: Philipp Thomas
cp, mv, install, mkdir, mkfifo, mknod are adjusted so that:
-Z no longer accepts an argument.
-Z or --context without an argument do not warn without SELinux.
--context with an argument will warn without SELinux.
* src/local.mk: Reference the new selinux module where required.
* src/system.h: Make the argument to --context optional.
* src/mkdir.c: Likewise. Also handle the SMACK case for --context.
Note we currently silently ignore -Z with SMACK.
* src/mkfifo.c: Likewise.
* src/mknod.c: Likewise.
* src/install.c: Likewise. Note install(1) by default already
set the context for target files to their system default,
albeit with an older method. Use the -Z option to select between
the old and new context restoration behavior, and document
the differences and details for how context restoration
is done in new and old methods, with a view disabling the
old method entirely in future.
* src/cp.c: Make the argument to --context optional.
Note -Z implies --no-preserve=context. I.E. -Z overrides
that aspect of -a no matter what order specified.
(struct cp_options): Document the context handling options.
(main): Check/adjust option combinations after all
options are processed, to both simplify processing
and to make handling independent of order of options
on the command line. Also improve the diagnostics
from a failed call to setfscreatecon().
(set_process_security_ctx): A new function,
refactored to set the default context from the source file,
or with the type adjusted as per the system default for
the destination path.
(set_file_security_ctx): A new function refactored to
set the security context of an existing file, either based on
the process context or the default system context for a path.
(copy_internal): Use the refactored functions to simplify
error handling and consistently fail or warn as needed.
(copy_reg): Likewise.
(copy_internal): With --preserve=context, also copy
context from non regular files. Note for directories this may
impact the copying of subsequent files to that directory?
(copy_attr): If we're handling SELinux explicitly,
then exclude to avoid the redudant copy with --preserve=context,
and the problematic copy with -Z. Note SELinux attribute exclusion
also now honors cp -a --no-preserve=context. Note there was a
very small window over 10 years ago, where attr_copy_file was
available, while attr_copy_check_permissions was not, so we
don't bother adding an explicit m4 check for the latter function.
* src/mv.c: Support --context, but don't allow specifying an argument.
* src/chcon.c: Adjust a comment to be specific to SELinux.
* src/runcon.c: Likewise.
* src/copy.c: Honor the context settings to "restorecon" as appropriate.
* src/copy.h: Add a new setting to select "restorecon" functionality.
* tests/mkdir/selinux.sh: s/-Z/--context=/
* tests/cp/cp-a-selinux.sh: Augment this test with cases
testing basic -Z functionality, and also test the various
invalid option combinations and option precedence.
* tests/mkdir/restorecon.sh: Add a new test for the
more involved mkdir -Z handling, since the directory changing
and non existent directories need to be specially handled.
Also check the similar but simpler handling of -Z by mk{nod,fifo}.
* tests/local.mk: Reference the new test.
* doc/coreutils.texi (cp invocation): Update as per interface changes.
(mv invocation): Likewise.
(install invocation): Likewise.
(mkfifo invocation): Likewise.
(mknod invocation): Likewise.
(mkdir invocation): Likewise.
* NEWS: Mention the new feature and change in behavior.
A sync operation is very often expensive. For illustration
I timed the following python script which indicated that
each ext4 dir sync was taking about 2ms and 12ms, on an
SSD and traditional disk respectively.
import os
d=os.open(".", os.O_DIRECTORY|os.O_RDONLY)
for i in range(1000):
os.fdatasync(d)
So syncing for each character for each file can result
in significant delays. Often this overhead is redundant,
as only the data is sensitive and not the file name.
Even if the names are sensitive, your file system may
employ synchronous metadata updates, which also makes
explicit syncing redundant.
* tests/misc/shred-remove.sh: Ensure all the new parameters
actually unlink the file.
* doc/coreutils.texi (shred invocation): Describe the new
parameters to the --remove option.
* src/shred.c (Usage): Likewise.
(main): Parse the new options.
(wipename): Inspect the new enum to see which of
the now optional tasks to perform.
* NEWS: Mention the new feature.
* THANKS.in: Add reporter Joseph D. Wagner
* src/df.c (usage): Document the new 'file' --output field.
(get_dev): Add a new parameter to pass the specified
argument from the command line through. Use '-' if a
command line parameter is not being used.
* doc/coreutils.texi (df invocation): Describe the new 'file' field.
* tests/df/df-output.sh: Adjust all fields test, and
add a specific test for --output=file.
* NEWS: Mention the new feature.
* src/ls.c (usage): Mention -k only changes the display
for disk usage (directory total, and ls -s), and imply
that it can be overridden (by --block-size, and -h).
* doc/coreutils.texi (block size): Mention that ls -k
handling is different to other utilities.
Addresses http://bugs.gnu.org/14525
The error diagnostic
"rm: cannot remove directory: '.'"
does not give the user a hint for the reason.
Issue a clearer error message.
* src/remove.c (rm_fts): Enhance the error diagnostic in the above
case to emphasize that skipping is done deliberately.
In the corresponding comment, mention that POSIX mandates this
behavior. Likewise in the subsequent comment for skipping "/".
* doc/coreutils.texi (rm invocation): In the paragraph describing
the above behavior, mention that POSIX mandates it.
This adds support for using a constant or "stick" parity bit.
* src/stty.c (usage): Mention the new flag.
* tests/misc/stty.sh: Adjust for the new flag.
* NEWS: Mention the improvement.
* docs/coreutils.texi (stty invocation): Mention the new flag.
* src/copy.c (create_hard_link): Add a bool 'dereference' parameter,
and pass AT_SYMLINK_FOLLOW as 'flags' to linkat() when dereference
is true.
(should_dereference): Add new 'bool' function to determine if a
file should be dereferenced or not.
(copy_internal): Use the above new should_dereference() and remember
its return value in a new local bool 'dereference' variable. Use that
in all three calls to create_hard_link().
* src/cp.c (main): after parsing the options, if x.dereference is
still DEFEF_UNDEFINED and the x.recursive is true, then only set
x.dereference to DEREF_NEVER iff --link was not specified.
* doc/coreutils.texi (cp invocation): Mention that cp(1) does not
follow symbolic links in the source when --link is specified.
Likewise in the description of the -R option when used together with
that option.
* tests/cp/same-file.sh: Adapt the expected results for the -fl,
the -bl and the -bfl tests.
* tests/cp/link-deref.sh: Add a new test.
* tests/local.mk (all_tests): Reference the above new test.
* NEWS (Changes in behavior): Mention the change.
This fixes http://bugs.gnu.org/15173
Co-authored-by: Bernhard Voelker <mail@bernhard-voelker.de>
* doc/coreutils.texi (cp invocation): Give more detail about what's
happening in the example, explicitly calling out the --no-dereference
option required to make the -H and -L options significant.
Also mention the option order significance of the -H and -L options.
Fixes http://bugs.gnu.org/15579
* doc/coreutils.texi (paste invocation): Move the synopsis to the top.
Provide examples for the different type of operations possible.
Add a specific common example to join consecutive lines with a space.
* src/id.c (usage): Remove 'name' from the synopsis,
implying that one can also specify by user ID.
(main): Like chown(1), call parse_user_spec() to implement
user name or ID lookup with appropriate precedence.
* doc/coreutils.texi (id invocation): Mention that
a user ID is supported and how '+' affects lookup order.
* tests/misc/id-groups.sh: Remove test now subsumed into...
* tests/misc/id-uid.sh: New test covering new interface.
* tests/local.mk: Rename the test.
* NEWS: Mention the new feature.
Addresses http://bugs.gnu.org/15421
* src/group-list.h (print_group_list): Add a parameter for the
delimiter of type char.
* src/group-list.c (print_group_list): Likewise, and use it instead
of a white space character to delimit the group entries.
* src/groups.c (main): Pass white space character to print_group_list().
* src/id.c (longopts): Add array element for the new long option.
(usage): Document the new option. While at it, fix the alignment
of the descriptions to match that of HELP_OPTION_DESCRIPTION.
(main): Define the bool flag opt_zero indicating the use of the
new option. In the getopt_long loop, handle it.
Output an error diagnostic in the case the --zero option has been
specified together with the default format.
In the case of -gG, pass either a NUL or a white space character to
print_group_list() - depending on the above new flag.
Likewise change the printing of the final newline character: output
a NUL instead if the --zero option has been specified.
* doc/coreutils.texi (id invocation): Document the new option.
While at it, move the @exitstatus macro down after the macro
@primaryAndSupplementaryGroups in order to be consistent with
other texinfo documents.
(groups invocation): Move @exitstatus down after the macro
@primaryAndSupplementaryGroups here, too.
* tests/misc/id-zero.sh: Add new test exercising the new option.
* tests/local.mk (all_tests): Reference it.
* NEWS (New features): Mention the new option.
Fixes http://bugs.gnu.org/9987
* doc/coreutils.texi (seq invocation): Add a sentence clarifying
that seq terminates when LAST becomes smaller than the current number
plus INCREMENT.
* src/seq.c (usage): Likewise.
Fixes http://bugs.gnu.org/15068
* doc/coreutils.texi (df invocation): In the example list of common
file system types, exchange the entries which are not so common
anymore (4.2, ufs, efs, hsfs, pcfs) by far more prominent ones
(ext2, ext3, ext4, xfs, btrfs, iso9660, ntfs, fat).
* doc/coreutils.texi (df invocation): s/pseude/pseudo/
* THANKS.in (Filipus Klutiero): Remove entry, now that it will be
automatically included in the generated THANKS file.
Fixes http://bugs.gnu.org/15041
* doc/coreutils.texi (cp invocation): Mention explicitly what
happens to permissions of existing files when -p is not specified.
Fixes http://bugs.gnu.org/14972
This new option can be used to find directories with a huge
amount of files. The GNU find utility has the printf format
"%h" which prints the number of entries in a directory, but
this is non-cumulative and doesn't handle hard links.
* src/du.c (struct duinfo): Add new member for counting inodes.
(duinfo_init): Initialize inodes member with Zero.
(duinfo_set): Set inodes counter to 1.
(duinfo_add): Sum up the 2 given inodes counters.
(opt_inodes): Add new boolean flag to remember if the --inodes
option has been specified.
(INODES_OPTION): Add new enum value to be used ...
(long_options): ... here.
(usage): Add description of the new option.
(print_size): Pass inodes counter or size to print_only_size,
depending on the inodes mode.
(process_file): Adapt threshold handling: with --inodes, print or
elide the entries according to the struct member inodes.
(main): Add a case for accepting the new INODES_OPTION.
Print a warning diagnostic when --inodes is used together with the
option --apparent-size or -b.
Reset the output_block_size to 1 ... and thus ignoring the
options -m and -k.
* tests/du/inodes.sh: Add a new test.
* tests/local.mk (all_tests): Mention it.
* doc/coreutils.texi (du invocation): Document the new option.
* NEWS: Mention the new option.
* src/dd.c (STATUS_NONE): Simplify the enum so that
it's more general than just suppressing transfer counts.
Then test this in all locations where non fatal diagnostics
are output.
* tests/dd/misc.sh: Ensure the diagnostic about
being unable to skip past the end of input is suppressed.
* NEWS: Mention the change in behavior.
Fixes http://bugs.gnu.org/14897
main(): Process new option. Replace input_numbers_option_used()
with a local variable. Re-organize argument processing.
usage(): Describe the new option.
(write_random_numbers): A new function to generate a
permutation of the specified input range with repetition.
(write_random_lines): Likewise for stdin and --echo.
(write_permuted_numbers): New function refactored from
write_permuted_output().
(write_permuted_lines): Likewise.
* tests/misc/shuf.sh: Add tests for --repetitions option.
* doc/coreutils.texi: Mention --repetitions, add examples.
* TODO: Mention an optimization to avoid needing to
read all of the input into memory with --repetitions.
* NEWS: Mention new shuf option.
* man/df.x: Don't say that a dev node is always on the root file system.
* doc/coreutils.texi (df invocation): Likewise. Also state that the
device node to mounted file system interpretation is only done when
passed absolute paths to device nodes.
* src/truncate.c (usage): Mention that --size is in bytes which
is by far the most common usage.
* doc/coreutils.texi (truncate invocation): Likewise. Also cross
reference the --io-blocks option.
Reported in http://bugs.gnu.org/14686
* src/du.c (usage): Clarify that --separate-dirs doesn't exclude
all directories.
* doc/coreutils.texi (du invocation): Avoid implying that -S
excludes the size of any non directory entries for a directory.
Also don't mention st_size as it's dependent on --apparent-size.
Reported by C de-Avillez in <https://launchpad.net/bugs/1187044>
The --retry option is indeed useful for both following modes
by name and by file descriptor. The difference is that in the
latter case, it is effective only during the initial open.
As a regression of the implementation of the inotify support,
tail -f --retry would immediately exit if the given file is
inaccessible.
* src/tail.c (usage): Change the description of the --retry option:
remove the note that this option would mainly be useful when
following by name.
(main): Change diagnosing dubios uses of --retry option:
when the --retry option is used without following, then issue
a warning that this option is ignored; when it is used together
with --follow=descriptor, then issue a warning that it is only
effective for the initial open.
Disable inotify also in the case when the initial open in tail_file()
failed (which is the actual bug fix).
* init.cfg (retry_delay_): Pass excess arguments to the test function.
* tests/tail-2/retry.sh: Add new tests.
* tests/local.mk (all_tests): Mention it.
* doc/coreutils.texi (tail invocation): Enhance the documentation
of the --retry option. Clarify the difference in tail's behavior
regarding the --retry option when combined with the following modes
name versus descriptor.
* NEWS (Bug fixes): Mention the fix.
Reported by Noel Morrison in:
http://lists.gnu.org/archive/html/coreutils/2013-04/msg00003.html
* doc/coreutils.texi (install invocation): Mention that install(1) may
not correctly determine the default user or permissions for installed
files, and so is best used with options specifying these attributes.
With --suppress-matched, the lines that match the pattern will not be
printed in the output files. I.E. the first line from the second
and subsequent splits will be suppressed.
* src/csplit.c: process_regexp(),process_line_count(): Don't output the
matched lines. Since csplit includes "up to but not including" matched
lines in each split, the first line (in the next group) is the matched
line - so just skip it.
main(): Handle new option.
usage(): Mention new option.
* doc/coreutils.texi (csplit invocation): Mention new option, examples.
* tests/misc/csplit-suppress-matched.pl: New test script.
* tests/local.mk: Reference the new test.
* NEWS: Mention new feature.
Pádraig Brady
Bernhard Voelker
Paul Eggert
Assaf Gordon
Edward Welbourne
Aleksej Serdjukov
Niels Möller
Chengwei Yang
Tiger Lee
Colin Leitner
Gian Piero Carrubba
Karl Berry
Filipus Klutiero
D. Hugh Redelmeier