1
0
mirror of git://git.sv.gnu.org/coreutils.git synced 2026-04-20 18:56:39 +02:00
Commit Graph

4206 Commits

Author SHA1 Message Date
Pádraig Brady
7958a4a4fe selinux: adjust utils to run restorecon with -Z
cp, mv, install, mkdir, mkfifo, mknod are adjusted so that:
 -Z no longer accepts an argument.
 -Z or --context without an argument do not warn without SELinux.
 --context with an argument will warn without SELinux.

* src/local.mk: Reference the new selinux module where required.
* src/system.h: Make the argument to --context optional.
* src/mkdir.c: Likewise.  Also handle the SMACK case for --context.
Note we currently silently ignore -Z with SMACK.
* src/mkfifo.c: Likewise.
* src/mknod.c: Likewise.
* src/install.c: Likewise.  Note install(1) by default already
set the context for target files to their system default,
albeit with an older method.  Use the -Z option to select between
the old and new context restoration behavior, and document
the differences and details for how context restoration
is done in new and old methods, with a view disabling the
old method entirely in future.
* src/cp.c: Make the argument to --context optional.
Note -Z implies --no-preserve=context.  I.E. -Z overrides
that aspect of -a no matter what order specified.
(struct cp_options): Document the context handling options.
(main): Check/adjust option combinations after all
options are processed, to both simplify processing
and to make handling independent of order of options
on the command line.  Also improve the diagnostics
from a failed call to setfscreatecon().
(set_process_security_ctx): A new function,
refactored to set the default context from the source file,
or with the type adjusted as per the system default for
the destination path.
(set_file_security_ctx): A new function refactored to
set the security context of an existing file, either based on
the process context or the default system context for a path.
(copy_internal): Use the refactored functions to simplify
error handling and consistently fail or warn as needed.
(copy_reg): Likewise.
(copy_internal): With --preserve=context, also copy
context from non regular files.  Note for directories this may
impact the copying of subsequent files to that directory?
(copy_attr): If we're handling SELinux explicitly,
then exclude to avoid the redudant copy with --preserve=context,
and the problematic copy with -Z.  Note SELinux attribute exclusion
also now honors cp -a --no-preserve=context.  Note there was a
very small window over 10 years ago, where attr_copy_file was
available, while attr_copy_check_permissions was not, so we
don't bother adding an explicit m4 check for the latter function.
* src/mv.c: Support --context, but don't allow specifying an argument.
* src/chcon.c: Adjust a comment to be specific to SELinux.
* src/runcon.c: Likewise.
* src/copy.c: Honor the context settings to "restorecon" as appropriate.
* src/copy.h: Add a new setting to select "restorecon" functionality.
* tests/mkdir/selinux.sh: s/-Z/--context=/
* tests/cp/cp-a-selinux.sh: Augment this test with cases
testing basic -Z functionality, and also test the various
invalid option combinations and option precedence.
* tests/mkdir/restorecon.sh: Add a new test for the
more involved mkdir -Z handling, since the directory changing
and non existent directories need to be specially handled.
Also check the similar but simpler handling of -Z by mk{nod,fifo}.
* tests/local.mk: Reference the new test.
* doc/coreutils.texi (cp invocation): Update as per interface changes.
(mv invocation): Likewise.
(install invocation): Likewise.
(mkfifo invocation): Likewise.
(mknod invocation): Likewise.
(mkdir invocation): Likewise.
* NEWS: Mention the new feature and change in behavior.
2013-11-27 14:19:32 +00:00
Pádraig Brady
569b4edd18 shred: provide --remove methods to avoid excessive syncing
A sync operation is very often expensive.  For illustration
I timed the following python script which indicated that
each ext4 dir sync was taking about 2ms and 12ms, on an
SSD and traditional disk respectively.

  import os
  d=os.open(".", os.O_DIRECTORY|os.O_RDONLY)
  for i in range(1000):
     os.fdatasync(d)

So syncing for each character for each file can result
in significant delays.  Often this overhead is redundant,
as only the data is sensitive and not the file name.
Even if the names are sensitive, your file system may
employ synchronous metadata updates, which also makes
explicit syncing redundant.

* tests/misc/shred-remove.sh: Ensure all the new parameters
actually unlink the file.
* doc/coreutils.texi (shred invocation): Describe the new
parameters to the --remove option.
* src/shred.c (Usage): Likewise.
(main): Parse the new options.
(wipename): Inspect the new enum to see which of
the now optional tasks to perform.
* NEWS: Mention the new feature.
* THANKS.in: Add reporter Joseph D. Wagner
2013-11-27 01:43:12 +00:00
Pádraig Brady
ba6582e95c tail: improve inotify handling of symlinks
Previous behavior failed to read contents of a (re)appearing file,
when symlinked by tail's watched file.  Also we now diagnose other
edge cases when running in inotify mode, where an initially
missing or regular file changes to a symlink.

* src/tail.c (main): If any arg is a symlink, use polling mode.
(recheck): Diagnose the edge case where a symlink appears during
inotify processing.
* tests/tail-2/symlink.sh: Test the fix. Mention the edge cases.
* tests/local.mk: Reference the new test.
* NEWS: Mention the fix.
Reported by: Ondrej Oprala
2013-11-27 01:40:08 +00:00
Pádraig Brady
799e10f361 df: add --output=file to directly output specified arguments
* src/df.c (usage): Document the new 'file' --output field.
(get_dev): Add a new parameter to pass the specified
argument from the command line through.  Use '-' if a
command line parameter is not being used.
* doc/coreutils.texi (df invocation): Describe the new 'file' field.
* tests/df/df-output.sh: Adjust all fields test, and
add a specific test for --output=file.
* NEWS: Mention the new feature.
2013-11-27 01:39:11 +00:00
Pádraig Brady
6b91f64ed9 tests: ensure factor tests can be regenerated
* tests/local.mk (factor-tests:) Add -f to the mv command that
replaces any existing generated tests.  This is required to avoid
prompts when root initially generates the tests, and they subsequently
need to be regenerated by a non root user.
2013-11-27 01:28:20 +00:00
Bernhard Voelker
2fa074ebd4 tests: enhance rm test regarding "." and ".."
Recent commit 2da7009d changed the error diagnostic of rm(1) trying
to remove "." or "..".  Enhance the corresponding test.

* tests/rm/r-4.sh: Ensure rm(1) outputs the expected error diagnostic.
2013-11-22 15:54:06 +01:00
Pádraig Brady
65f8a39d90 tests: fix false rm -I test failure when run as root
Fix a recent regression introduced in commit v8.21-127-g5ee7d8f

Also related to this is the recent query about root run `rm -I`
ignoring the mode bits of a file: https://bugzilla.redhat.com/1013171

* tests/rm/interactive-once.sh: Avoid the messages and
corresponding file presence checks with write protected files
when running as root.
2013-11-18 01:30:19 +00:00
Colin Leitner
4e37fc642f stty: add support for mark/space parity
This adds support for using a constant or "stick" parity bit.

* src/stty.c (usage): Mention the new flag.
* tests/misc/stty.sh: Adjust for the new flag.
* NEWS: Mention the improvement.
* docs/coreutils.texi (stty invocation): Mention the new flag.
2013-11-17 12:26:12 +00:00
Pádraig Brady
90181a5926 shred: avoid data write pass with --size=0
* src/shred.c (dopass): Exit early to avoid redundant heap
allocation, and more importantly avoiding a file sync
when we're writting no data, as this can have side effects.
Also with --verbose, this avoids printing of "pass status"
which could be confusing as to whether data was actually written.
* tests/misc/shred-passes.sh: Ensure the status for data
passes are not written when not doing any data writes.
2013-11-08 13:39:35 +00:00
Pádraig Brady
c93d5b4aa7 shred: fix direct I/O failures for last write to file
Since direct I/O is now enabled with commit v8.21-139-gebaf961
we must handle the case where we write an odd size at the
end of a file (with --exact), or we specify an odd --size that
is larger than 64KiB, or in the very unlikely case of a device
with an odd size.  This issue was present since direct I/O
support was first added in v5.3.0, but latent since v6.0.
Theoretically this could have also been an issue after that on
systems which didn't have alignment constraints, but did have
size constraints for direct I/O.

* src/shred.c (dopass): On the first pass for a file, always
retry a write that fails with EINVAL, so we handle direct I/O
failure at either the start or end of the file.  Adjust the comment
as the original case is out of date and implicitly handled
by this more general fix.
* tests/misc/shred-exact.sh: Add a test case.
* NEWS: Add a "bug fix" entry for shred since there are
two related issues now fixed.
2013-11-08 13:33:50 +00:00
Gian Piero Carrubba
bf6bf52dce cp: fix --link regarding the dereferencing of symbolic links
* src/copy.c (create_hard_link): Add a bool 'dereference' parameter,
and pass AT_SYMLINK_FOLLOW as 'flags' to linkat() when dereference
is true.
(should_dereference): Add new 'bool' function to determine if a
file should be dereferenced or not.
(copy_internal): Use the above new should_dereference() and remember
its return value in a new local bool 'dereference' variable.  Use that
in all three calls to create_hard_link().
* src/cp.c (main): after parsing the options, if x.dereference is
still DEFEF_UNDEFINED and the x.recursive is true, then only set
x.dereference to DEREF_NEVER iff --link was not specified.
* doc/coreutils.texi (cp invocation): Mention that cp(1) does not
follow symbolic links in the source when --link is specified.
Likewise in the description of the -R option when used together with
that option.
* tests/cp/same-file.sh: Adapt the expected results for the -fl,
the -bl and the -bfl tests.
* tests/cp/link-deref.sh: Add a new test.
* tests/local.mk (all_tests): Reference the above new test.
* NEWS (Changes in behavior): Mention the change.

This fixes http://bugs.gnu.org/15173

Co-authored-by: Bernhard Voelker <mail@bernhard-voelker.de>
2013-11-07 23:45:22 +01:00
Pádraig Brady
4d94e65d06 maint: simplify printing of md5sum file names
* src/md5sum.c (main): Add a comment as to why we continue
to escape names that do not have '\n' but do have '\\' chars.
(print_filename): Use the predetermined boolean to decide
whether to escape or not, so that in the common case we
can output the file name directly, rather than inspecting each char.
* tests/misc/md5sum.pl: Add case to show '\\' chars cause escaping.
* tests/misc/sha1sum.pl: Likewise.
2013-11-01 22:34:24 +00:00
Pádraig Brady
0c1d7917f1 mktemp: with --quiet, only suppress I/O errors
The reason for having a --quiet option is to
suppress only some subset of possible errors.
The most useful separation here is with usage/internal errors,
and errors due to file creation etc. (i.e. I/O errors).

* src/mktemp.c (main): Match the --help and info docs and
only suppress the file/dir creation error messages.
* tests/misc/mktemp.pl: Adjust accordingly.
2013-10-06 17:59:44 +01:00
Sergio Durigan Junior
5ee7d8f55b rm: with -I, prompt before deleting a write protected file
This regression was introduced in commit v6.7-71-g0928c24

* src/rm.c (main): Make the -I option behave like --interactive=once.
* tests/rm/interactive-once.sh: Add cases for single and multiple files.
* NEWS: Mention the bug fix.
Fixes http://bugs.gnu.org/9308
2013-09-26 11:25:05 +01:00
Pádraig Brady
3414301f59 maint: move all id(1) tests to the same directory
* tests/misc/id-context.sh -> tests/id/context.sh
* tests/misc/id-setgid.sh -> tests/id/setgid.sh
* tests/misc/id-uid.sh -> tests/id/uid.sh
* tests/misc/id-zero.sh -> tests/id/zero.sh
* tests/local.mk: Reference the renamed tests
2013-09-22 04:15:34 +01:00
Pádraig Brady
587cc684dc tests: parameterize the "root" username
* tests/misc/chroot-credentials.sh: Don't assume uid 0
has the "root" name, nor any name for that matter.
2013-09-22 03:59:58 +01:00
Pádraig Brady
7415d8d9f8 id: support specifying the user by user ID
* src/id.c (usage): Remove 'name' from the synopsis,
implying that one can also specify by user ID.
(main): Like chown(1), call parse_user_spec() to implement
user name or ID lookup with appropriate precedence.
* doc/coreutils.texi (id invocation): Mention that
a user ID is supported and how '+' affects lookup order.
* tests/misc/id-groups.sh: Remove test now subsumed into...
* tests/misc/id-uid.sh: New test covering new interface.
* tests/local.mk: Rename the test.
* NEWS: Mention the new feature.
Addresses http://bugs.gnu.org/15421
2013-09-22 03:27:17 +01:00
Pádraig Brady
f4f4568d8c tests: avoid a failure when there isn't a name for all user IDs
* tests/misc/id-zero.sh: Don't check exit status when in -n mode.

Prompted by the continuous integration build failure at:
http://hydra.nixos.org/build/6196762
2013-09-22 02:51:37 +01:00
Bernhard Voelker
2a0d241f69 id: add -z, --zero option
* src/group-list.h (print_group_list): Add a parameter for the
delimiter of type char.
* src/group-list.c (print_group_list): Likewise, and use it instead
of a white space character to delimit the group entries.
* src/groups.c (main): Pass white space character to print_group_list().
* src/id.c (longopts):  Add array element for the new long option.
(usage): Document the new option.  While at it, fix the alignment
of the descriptions to match that of HELP_OPTION_DESCRIPTION.
(main): Define the bool flag opt_zero indicating the use of the
new option.  In the getopt_long loop, handle it.
Output an error diagnostic in the case the --zero option has been
specified together with the default format.
In the case of -gG, pass either a NUL or a white space character to
print_group_list() - depending on the above new flag.
Likewise change the printing of the final newline character: output
a NUL instead if the --zero option has been specified.
* doc/coreutils.texi (id invocation): Document the new option.
While at it, move the @exitstatus macro down after the macro
@primaryAndSupplementaryGroups in order to be consistent with
other texinfo documents.
(groups invocation): Move @exitstatus down after the macro
@primaryAndSupplementaryGroups here, too.
* tests/misc/id-zero.sh: Add new test exercising the new option.
* tests/local.mk (all_tests): Reference it.
* NEWS (New features): Mention the new option.
Fixes http://bugs.gnu.org/9987
2013-09-21 14:15:45 +02:00
Bernhard Voelker
dffaa8db2d maint: update copyright year number ranges of numfmt sources
Run "make update-copyright".

* src/numfmt.c: Update copyright year number range.  This file has
obviously been added to coreutils after and without the annual update.
* tests/misc/numfmt.pl: Likewise.
2013-08-02 08:31:36 +02:00
Bernhard Voelker
eea81b46cd tests: fix another test failure in test/du/inodes.sh
Prompted by the continuous integration build failure at:
http://hydra.nixos.org/build/5584121

du(1) uses the first file object of the two test files linked to the
same inode, 'd/f' and 'd/h', whatever the system returns first.
Use 'd/f' in both the expected and the actual output.

* test/du/inodes.sh: Change the expected output as described above
when returning the --all directory entries (without -l).  Also replace
the name of the hardlink 'd/h' by 'd/f' in the actual output.
2013-07-29 01:51:55 +02:00
Bernhard Voelker
9e52ebac57 maint: cleanup compare calls in test/du/inodes.sh
* test/du/inodes.sh: In the cases where compare() fails, that function
would show the unified diff automatically.  Therefore, remove the
excess "cat out".
In the cases where expecting an empty file, use compare() again
rather than the simpler "test -s" because possible error reports
will then include the file's content for the same reason as above.
2013-07-28 18:32:22 +02:00
Pádraig Brady
012fbb6371 tests: fix intermittent failure in test/du/inodes.sh
Prompted by the continuous integration build failure at:
http://hydra.nixos.org/build/5582213

* test/du/inodes.sh: Due to undefined order in returned directory
entries, the expected output might not match, so sort both expected
and actual output when returning --all directory entries.
Also use a simpler test for ensuring no errors are output.
2013-07-28 00:26:57 +01:00
Bernhard Voelker
333dc83d52 du: add --inodes option
This new option can be used to find directories with a huge
amount of files.  The GNU find utility has the printf format
"%h" which prints the number of entries in a directory, but
this is non-cumulative and doesn't handle hard links.

* src/du.c (struct duinfo): Add new member for counting inodes.
(duinfo_init): Initialize inodes member with Zero.
(duinfo_set): Set inodes counter to 1.
(duinfo_add): Sum up the 2 given inodes counters.
(opt_inodes): Add new boolean flag to remember if the --inodes
option has been specified.
(INODES_OPTION): Add new enum value to be used ...
(long_options): ... here.
(usage): Add description of the new option.
(print_size): Pass inodes counter or size to print_only_size,
depending on the inodes mode.
(process_file): Adapt threshold handling: with --inodes, print or
elide the entries according to the struct member inodes.
(main): Add a case for accepting the new INODES_OPTION.
Print a warning diagnostic when --inodes is used together with the
option --apparent-size or -b.
Reset the output_block_size to 1 ... and thus ignoring the
options -m and -k.
* tests/du/inodes.sh: Add a new test.
* tests/local.mk (all_tests): Mention it.
* doc/coreutils.texi (du invocation): Document the new option.
* NEWS: Mention the new option.
2013-07-27 14:25:28 +02:00
Ken Booth
2bdb74ec1a mv: replace empty directories in cross file system move
src/copy.c (copy_internal): Use rmdir() rather than unlink()
when the source is a directory, so that empty directories
are replaced in the destination as per POSIX.
* tests/mv/part-rename.sh: Augment with various combinations.
* NEWS: Mention the bug fix.
Fixes http://bugs.gnu.org/14763
2013-07-25 16:45:02 +01:00
Bernhard Voelker
0c452fe97a maint: make some shell and perl scripts executable in 'tests/'
Some newer test scripts - partially ones from me - are not executable.
It does not seem to be a problem, but for consistency and to avoid
future problems on unusual platforms or shells change the permissions
by adding the executable bit.

* cfg.mk (sc_tests_executable): Add new syntax-check rule to ensure
that all test scripts are executable.
* tests/df/df-output.sh: Change file mode from 644 to 755.
* tests/du/threshold.sh: Likewise.
* tests/factor/run.sh: Likewise.
* tests/init.sh: Likewise.
* tests/misc/csplit-suppress-matched.pl: Likewise.
* tests/misc/numfmt.pl: Likewise.
* tests/tail-2/retry.sh: Likewise.
2013-07-22 01:07:52 +02:00
Rasmus Villemoes
c490a6ff2b rm: output number of arguments at the interactive prompt
Include the number of arguments which rm received in the "Remove all
arguments?" prompt.  This is useful in the, presumably, common case
where the arguments were not provided by hand, but instead were the
result of various shell expansions.  A simple, if somewhat contrived,
example (assuming rm is aliased to rm -I) could be:

  rm * .o

where the prompt "Remove 120 arguments?" is more likely to make
the user catch the problem.

* src/rm.c (main): Include correctly pluralized n_files
in the output message.  Also remove the now redudant "all".
* tests/rm/interactive-always.sh: Adjust to the new prompt.
* tests/rm/interactive-once.sh: Likewise.
2013-07-19 12:27:09 +01:00
Pádraig Brady
e6fc265b75 dd: make status=none suppress all diagnostics
* src/dd.c (STATUS_NONE): Simplify the enum so that
it's more general than just suppressing transfer counts.
Then test this in all locations where non fatal diagnostics
are output.
* tests/dd/misc.sh: Ensure the diagnostic about
being unable to skip past the end of input is suppressed.
* NEWS: Mention the change in behavior.
Fixes http://bugs.gnu.org/14897
2013-07-19 11:23:12 +01:00
Assaf Gordon
3a84293987 shuf: add --repetition to support repetition in output
main(): Process new option.  Replace input_numbers_option_used()
with a local variable.  Re-organize argument processing.
usage(): Describe the new option.
(write_random_numbers): A new function to generate a
permutation of the specified input range with repetition.
(write_random_lines): Likewise for stdin and --echo.
(write_permuted_numbers):  New function refactored from
write_permuted_output().
(write_permuted_lines): Likewise.
* tests/misc/shuf.sh: Add tests for --repetitions option.
* doc/coreutils.texi: Mention --repetitions, add examples.
* TODO: Mention an optimization to avoid needing to
read all of the input into memory with --repetitions.
* NEWS: Mention new shuf option.
2013-07-11 01:17:31 +01:00
Assaf Gordon
f3fa3b2990 tests: add more tests for shuf option combinations
* test/misc/shuf.sh: Add tests for erroneous conditions
like multiple '-o' and '--random-source'.
2013-07-11 01:17:30 +01:00
Pádraig Brady
616fa9c7b3 df: fix mount list processing with unstatable mount dirs
* src/df.c (filter_mount_list): Initialize devlist->dev_num correctly
when unable to stat() a mount point.  This will avoid possible invalid
deduplication done on the list due to use of uninitialized memory.
* tests/df/skip-duplicates.sh: Ensure this code path is exercised.
Also refactor the test to be table driven.
* NEWS: Mention the bug fix.
2013-07-09 02:36:27 +01:00
Pádraig Brady
37c9ce40f3 tests: fix "shuf unreadable" test to work when run as root
* tests/misc/shuf.sh: Restrict the test to the significant
case where we can't in fact read the "unreadable" file.
2013-07-05 02:43:22 +01:00
Pádraig Brady
bc70095df2 stdbuf: make it mandatory to specify a buffering option
This is consistent with the documented interface and
avoids any ambiguity in a user thinking that stdbuf without options
might reset to a "standard" buffering setup.

* src/stdbuf.c (set_libstdbuf_options): Indicate with the return value
whether any env variables were actually set.
(main): Fail unless some env variables were set.
* tests/misc/stdbuf.sh: Ensure this constraint is enforced.
* NEWS: Mention the small change in behavior.
2013-06-22 04:02:19 +01:00
Pádraig Brady
eadf835794 tests: avoid a race in tail --retry testing
Prompted by the continuous integration build failure at:
http://hydra.nixos.org/build/5221053

* tests/tail-2/retry.sh: Ensure the 'out' file is truncated,
as it's used to arbitrate the run order of commands.
Relying on the truncation in the background tail command
is racy because the truncation can occur after the fork
of the background shell and thus wait4lines would not wait
for output to occur in 'out', which would mean that the
'missing' file could be populated by the time tail(1)
gets to process it.
2013-06-03 14:31:37 +01:00
Jim Meyering
51a8f70713 od: -wN, N>64K, avoid misbehavior on systems with 32-bit size_t
* src/od.c (PRINT_FIELDS): Declare "i" to be of type uintmax_t, so that
the numerator in the expression for "next_pad" does not overflow.
(print_named_ascii): Likewise.
(print_ascii): Likewise.
Bug introduced via commit v6.12-42-g20c0b87.
* tests/misc/od.pl: Exercise each of the three affected code paths.
* NEWS (Bug fixes): Mention it.
Reported by Rich Burridge.
2013-06-02 09:25:38 -07:00
Jim Meyering
aeaeb87c13 tests: head-c: avoid spurious failure with a 32-bit size_t
* tests/misc/head-c.sh: Don't try to elide 1 exabytes, since on
32-bit systems, that number is not representable as a size_t.
This command would fail on 32-bit systems, where SIZE_MAX < 1E:
  head --bytes=-E < /dev/null
Instead of "E", use $SSIZE_MAX.
For discussion, see http://bugs.gnu.org/13530
2013-06-01 20:44:47 -07:00
Jim Meyering
5a7f03c210 tests: avoid spurious failure when SIZE_MAX is 2^32-1
* tests/split/line-bytes.sh: Since we've limited virtual memory to
20MB, choose a smaller size, 1GiB (which is <= SIZE_MAX) rather than
1EiB, which is larger than SIZE_MAX on 32-bit systems.
I confirmed that this test still fails when the split.c-modifying
part of v8.21-58-gfec363c is backed out.
2013-05-27 15:02:50 -07:00
Jim Meyering
909a0561de tests: fix an ls test not to fail when user or group name contains SP
* tests/ls/block-size.sh (size_etc): The sed expression through which
we filtered the output of "ls -l ..." assumed that the user and group
name components of each line would not contain spaces.  Avoid the
problem by using -og instead of -l, thus not printing either of those
fields.  Adjust the sed expression accordingly.
2013-05-26 19:06:55 -07:00
Stefano Lattarini
ba60445f73 tests: numfmt: use the printf program, not the shell builtin
* tests/misc/numfmt.pl: Avoid a spurious failure when
/bin/sh is dash (as can happen on Debian systems).
2013-05-27 00:24:00 +01:00
Pádraig Brady
fec363cebf split: with --line-bytes only allocate memory as needed
* src/split.c (line_bytes_split): Rewrite to only buffer
when necessary.  I.E. only increase the buffer when we've
already lines output in a split and we encounter a line
larger than the input buffer size, in which case a hold
buffer will be increased in increments of the input buffer size.
(lines_rr): Use the more abstract xalloc_die() just like
we did in line_bytes_split(), rather than explicitly
printing the "memory exhausted" message and exiting.
* tests/split/line-bytes.sh: Add a new test for this
function which previously had no test coverage.
* tests/local.mk: Reference the new test.
* NEWS: Mention the improvement.
Fixes http://bugs.gnu.org/13537
2013-05-23 11:13:24 +01:00
Paul Eggert
c704573729 tests: don't assume expr was built with GMP
* tests/misc/cut-huge-range.sh (subtract_one): New string.
(CUT_MAX): Don't pass a too-large integer to 'expr'.
2013-05-18 16:29:57 -07:00
Paul Eggert
e7198a67b8 mkdir: don't assume umask equals POSIX default ACL mask
This fixes Bug#14371, reported by Killer Bassist.
* NEWS: Document this.
* src/mkdir.c (struct mkdir_options): Remove member ancestor_mode.
New member umask_value.  All uses changed.
* src/mkdir.c (make_ancestor): Fix umask assumption.
* src/mkdir.c, src/mkfifo.c, src/mknod.c (main):
Leave umask alone.  This requires invoking lchmod after creating
the file, which introduces a race condition, but this can't be
avoided on hosts with "POSIX" default ACLs, and there's no easy
way with network file systems to tell what kind of host the
directory is on.
* tests/local.mk (all_tests): Add tests/mkdir/p-acl.sh.
* tests/mkdir/p-acl.sh: New file.
2013-05-11 19:22:54 -07:00
Cojocaru Alexandru
465f9512b7 cut: improve performance, especially with --output-delimiter
Use a sentinel value that's checked implicitly, rather than
a bit array, to determine if an item should be output.

Benchmark results for this change are:

$ yes abcdfeg | head -n1MB > big-file

$ for c in orig sentinel; do
    src/cut-$c 2>/dev/null
    echo -ne "\n== $c =="
    time src/cut-$c -b1,3 big-file > /dev/null
  done
== orig ==
real    0m0.049s
user    0m0.044s
sys     0m0.005s

== sentinel ==
real    0m0.035s
user    0m0.032s
sys     0m0.002s

 ## Again with --output-delimiter ##
$ for c in orig sentinel; do
    src/cut-$c 2>/dev/null
    echo -ne "\n== $c =="
    time src/cut-$c -b1,3 --output-delimiter=: big-file > /dev/null
  done
== orig ==
real    0m0.106s
user    0m0.103s
sys     0m0.002s

== sentinel ==
real    0m0.055s
user    0m0.052s
sys     0m0.003s

eol_range_start: Removed. 'n-' is no longer treated specially,
and instead SIZE_MAX is set for the 'hi' limit, and tested implicitly.
complement_rp: Used to complement 'rp' when '--complement' is specified.
ADD_RANGE_PAIR: Macro renamed to 'add_range_pair' function.
* tests/misc/cut-huge-range.sh: Adjust to the SENTINEL value.
Also remove the overlapping range test as this is no longer
dependent on large ranges and also is already handled with
the EOL-subsumed-3 test in cut.pl.
2013-05-08 11:51:37 +01:00
Cojocaru Alexandru
b54b47f954 cut: fix handling of overlapping ranges
This issue was introduced in commit v8.21-43-g3e466ad

* src/cut.c (set_fields): Process all range pairs when merging.
* tests/misc/cut-huge-range.sh: Add a test for this edge case.
Also fix an issue where we could miss reported errors due
to truncation of the 'err' file.
2013-05-07 13:07:02 +01:00
Cojocaru Alexandru
ef9db5735a cut: reduce CPU overhead in determining item to output
print_kth() is the central function of cut used to
determine if an item is to be output or not,
so simplify it by moving some logic outside.
Benchmark results for this change are:

$ yes abcdfeg | head -n1MB > big-file
$ for c in orig split; do
    src/cut-$c 2>/dev/null
    echo -ne "\n== $c =="
    time src/cut-$c -b1,3 big-file > /dev/null
  done

== orig ==
real    0m0.111s
user    0m0.108s
sys     0m0.002s

== split ==
real    0m0.088s
user    0m0.081s
sys     0m0.007s

* src/cut.c (print_kth): Refactor a branch to outside the function.
Related to http://bugs.gnu.org/13127
2013-04-29 17:54:39 +01:00
Cojocaru Alexandru
3e466ad051 cut: make memory allocation independent of range width
The current implementation of cut, uses a bit array,
an array of `struct range_pair's, and (when --output-delimiter
is specified) a hash_table.  The new implementation will use
only an array of `struct range_pair's.
The old implementation is memory inefficient because:
 1. When -b with a big num is specified, it allocates a lot of
    memory for `printable_field'.
 2. When --output-delimiter is specified, it will allocate 31 buckets.
    Even if only a few ranges are specified.

Note CPU overhead is increased to determine if an item is to be printed,
as shown by:

$ yes abcdfeg | head -n1MB > big-file
$ for c in with-bitarray without-bitarray; do
    src/cut-$c 2>/dev/null
    echo -ne "\n== $c =="
    time src/cut-$c -b1,3 big-file > /dev/null
  done

== with-bitarray ==
real    0m0.084s
user    0m0.078s
sys     0m0.006s

== without-bitarray ==
real    0m0.111s
user    0m0.108s
sys     0m0.002s

Subsequent patches will reduce this overhead.

* src/cut.c (set_fields): Set and initialize RP
instead of printable_field.
* src/cut.c (is_range_start_index): Use CURRENT_RP rather than a hash.
* tests/misc/cut.pl: Check if `eol_range_start' is set correctly.
* tests/misc/cut-huge-range.sh: Rename from cut-huge-to-eol-range.sh,
and add a test to verify large amounts of mem aren't allocated.
Fixes http://bugs.gnu.org/13127
2013-04-29 17:54:27 +01:00
Pádraig Brady
94f4f658f0 tests: fix usage of require_ulimit_
* init.cfg (require_ulimit_v_): Renamed from require_ulimit_
as this only checks for ulimit -v support.  Other uses of
ulimit -t and ulimit -n in tests shouldn't cause false failures
if not supported.
* cfg.mk (sc_prohibit_test_ulimit_without_require_): A new syntax check
to ensure that require_ulimit_v_() is used iff required.
* tests/misc/head-c.sh: Add missing call to require_ulimit_v_.
* tests/rm/many-dir-entries-vs-OOM.sh: Likewise.
* tests/split/r-chunk.sh: Remove non mandatory require_ulimit_ call.
* tests/misc/sort-merge-fdlimit.sh: Likewise.
* tests/cp/link-heap.sh: Adjust to renamed require_ulimit_v_.
* tests/dd/no-allocate.sh: Likewise.
* tests/misc/csplit-heap.sh: Likewise.
* tests/misc/cut-huge-to-eol-range.sh: Likewise.
* tests/misc/printf-surprise.sh: Likewise.
2013-04-25 11:13:48 +01:00
Bernhard Voelker
8f97679836 tail: exit following by descriptor when no tailable file left
As a side effect of the previous commit which fixes 'tail -f --retry'
to wait for a file to appear, tail would not exit when the last file
appears untailable and gives up on this file.
This can happen, for example, when the argument file name appears
as directory.  Tail sets the 'ignore' flag of this file to true,
but instead of exiting the program, tail would continue the loop.

* src/tail.c (any_live_files): Change the function to return true
if any of the files is still tailable or if tail should continue to
try to check again.
(tail_forever): Change the condition to break the loop in the
"no files remaining" case, because now any_live_files() will care
about it, as mentioned above.
(parse_options): When --retry is used without any follow mode,
then reset reopen_inaccessible_files to false.
* tests/tail-2/retry.sh: Add test case.
2013-04-20 16:34:23 +02:00
Bernhard Voelker
d461bfd274 tail: let -f --retry wait for inaccessible files
The --retry option is indeed useful for both following modes
by name and by file descriptor.  The difference is that in the
latter case, it is effective only during the initial open.

As a regression of the implementation of the inotify support,
tail -f --retry would immediately exit if the given file is
inaccessible.

* src/tail.c (usage): Change the description of the --retry option:
remove the note that this option would mainly be useful when
following by name.
(main): Change diagnosing dubios uses of --retry option:
when the --retry option is used without following, then issue
a warning that this option is ignored; when it is used together
with --follow=descriptor, then issue a warning that it is only
effective for the initial open.
Disable inotify also in the case when the initial open in tail_file()
failed (which is the actual bug fix).
* init.cfg (retry_delay_): Pass excess arguments to the test function.
* tests/tail-2/retry.sh: Add new tests.
* tests/local.mk (all_tests): Mention it.
* doc/coreutils.texi (tail invocation): Enhance the documentation
of the --retry option.  Clarify the difference in tail's behavior
regarding the --retry option when combined with the following modes
name versus descriptor.
* NEWS (Bug fixes): Mention the fix.

Reported by Noel Morrison in:
http://lists.gnu.org/archive/html/coreutils/2013-04/msg00003.html
2013-04-20 16:33:06 +02:00
Pádraig Brady
1dd8a33169 tests: avoid false failures with non default groups
On OS X it was seen that the group ID used for new files,
are set to a that of the directory rather than the current user.
It's not currently understood when this happens, but it was confirmed
that ACLs, extended attributes and setgid bits are _not_ involved.

* init.cfg (skip_if_nondefault_group_): A new function to detect
and avoid this situation.  Document with links to the discussions
for hopefully future clarification.
* tests/install/install-C-root.sh: Use the new function.
* tests/install/install-C-selinux.sh: Likewise.
* tests/install/install-C.sh: Likewise.
2013-04-14 01:03:56 +01:00