Problem reported by Vincent Lefevre in: http://bugs.gnu.org/18449
* src/cat.c (main): Allow copying an empty file to itself.
* tests/misc/cat-self.sh: New test.
* tests/local.mk (all_tests): Add it.
Since commit v8.22-94-g99960ee, chroot(1) skips the chroot(2) syscall
for "/" arguments (and synonyms). The problem is that it also skips
the following chdir("/") call in that case. The latter breaks existing
scripts which expect "/" to be the working directory inside the chroot.
While the first part of the change - i.e., skipping chroot("/") - is
okay for consistency with systems where it might succeed for a non-root
user, the second part might be malicious, e.g.
cd /home/user && chroot '/' bin/foo
In the "best" case, chroot(1) could not execute 'bin/foo' with ENOENT,
but in the worst case, chroot(1) would execute '/home/user/bin/foo' in
the case that exists - instead of '/bin/foo'.
Revert that second part of the patch, i.e., perform the chdir("/)
in the common case again - unless the new --skip-chdir option is
specified. Restrict this new option to the case of "/" arguments.
* src/chroot.c (SKIP_CHDIR): Add enum.
(long_opts): Add entry for the new --skip-chdir option.
(usage): Add --skip-chdir option, and while at it, move the other
to options into alphabetical order.
(main): Accept the above new option, allowing it only in the case
when NEWROOT is the old "/".
Move down the chdir() call after the if-clause to ensure it is
run in any case - unless --skip-chdir is specified.
Add a 'newroot' variable for the new root directory as it is used
in a couple of places now.
* tests/misc/chroot-fail.sh: Invert the last tests which check the
working directory of the execvp()ed program when a "/"-like
argument was passed: now expect it to be "/" - unless --skip-chdir
is given.
* doc/coreutils.texi (chroot invocation): Document the new option.
Document that chroot(1) usually calls chdir("/") unless the new
--skip-chdir option is specified. Sort options.
* NEWS (Changes in behavior): Mention the fix.
(New features): Mention the new option.
* init.cfg (nonroot_has_perm_): Add chroot's new --skip-chdir option.
* tests/cp/preserve-gid.sh (t1): Likewise.
* tests/cp/special-bits.sh: Likewise.
* tests/id/setgid.sh: Likewise.
* tests/misc/truncate-owned-by-other.sh: Likewise.
* tests/mv/sticky-to-xpart.sh: Likewise.
* tests/rm/fail-2eperm.sh: Likewise.
* tests/rm/no-give-up.sh: Likewise.
* tests/touch/now-owned-by-other.sh: Likewise.
Reported by Andreas Schwab in http://bugs.gnu.org/18062
* tests/misc/env.sh: Skip if we can't execute the generated shebang,
which would be the case if there are spaces in the directory hierarchy.
This is triggered by `make distcheck`
Add the --enable-single-binary option to the configure file.
When enabled, this option builds a single binary file containing
the selected tools. Which tool gets executed depends on the value
of argv[0] which can be set implicitly through symlinks to the
single program.
This setup reduces significantly the size of a complete coreutils
install, since code from lib/libcoreutils.a is not duplicated in
every one of the more than 100 binaries. Runtime overhead is
increased due to more dynamic libraries being loaded, and extra
initialization being performed for all utils. Also initially
a larger binary is loaded from storage, though this is usually
alleviated due to caching and lazy mmaping of unused blocks,
and in fact the single binary should have better caching
characteristics.
Comparing the size of the individual versus single binary on x86_64:
$ cd src
$ size coreutils
$ size -t $(../build-aux/gen-lists-of-programs.sh --list-progs |
grep -Ev '(coreutils|libstdbuf)') | tail -n1
text data bss dec hex filename
1097416 5388 88432 1191236 122d44 src/coreutils
4901010 124964 163768 5189742 4f306e (TOTALS)
Storage requirements are reduced similarly:
$ cd src
$ du -h coreutils
$ du -ch $(../build-aux/gen-lists-of-programs.sh --list-progs |
grep -Ev '(coreutils|libstdbuf)') | tail -n1
1.2M coreutils
5.3M total
When installing, the makefile will create either symlinks or
shebangs based on the --enable-single-binary setting, for
each configured tool. In this way, all the tools are still
callable individually, but they are all implemented by the same
"coreutils" binary installed on the same directory.
* .gitignore: Add new generated files.
* Makefile.am: New rules to generate build-aux/gen-single-binary.sh
and install symlinks.
* NEWS: Mention the new feature.
* README: Add "coreutils" to the list of utils.
* bootstrap.conf: Regenerate src/single-binary.mk
* build-aux/gen-lists-of-programs.sh: New --list-progs option.
* build-aux/gen-single-binary.sh: Regenerate
* configure.ac: New --enable-single-binary option and other variables.
Disallow --enable-single-binary=symlinks with --program-prefix et. al.
* man/coreutils.x: Manpage hook.
* man/local.mk: Add manpage hook and fix dependencies.
* src/coreutils.c: Multicall implementation.
* src/local.mk: New rules for the single binary option.
* tests/local.mk: Add $single_binary_progs to support
require_built_() from init.cfg
* tests/misc/env.sh: Avoid the use of symlink to echo.
* tests/misc/help-version.sh: Add exception for coreutils.
* tests/install/basic-1.sh: Really avoid using ginstall strip
functionality if there is an issue with the independent strip command.
* src/kill.c: Changes to call exit() in main.
* src/readlink.c: Likewise.
* src/shuf.c: Likewise.
* src/timeout.c: Likewise.
* src/truncate.c: Likewise.
Avoid complicated and error-prone parsing of df's output via
sed(1), cut(1), etc., and instead use df's more modern --output
option.
* src/ioblksize.h (in a comment): Simplify the extraction of the
device name of the mounted file system from df's output.
* tests/dd/skip-seek-past-dev.sh: Likewise.
* tests/du/2g.sh: Likewise for the 'avail' column here.
Also avoid the deprecated use of "tail -NUM".
* tests/misc/stat-mount.sh: While at it, remove the determination
of the mount point of "." via df(1) plus sed(1) as it is unused
since commit v8.5-159-gf57cb37 anyway. Instead, improve this test
by verifying that the output of "stat -c%m ." at least starts with
a slash '/'.
Revert commit v8.22-131-g3e89d5b as even though POSIX
states that the default mode should be -L,
common practice for stand-alone pwd implementations
is to default to -P.
* src/pwd.c (usage): Retain mention of the default mode of operation.
Suggested by Bob Proulx
This works for any program that might also be a shell built-in.
It is more concise. In addition, it makes output more reproducible:
some diagnostics include argv[0], which will now be just the program
name, rather than the full absolute name of the executable.
* tests/misc/sort-compress.sh: Use env kill, rather than absolute name.
* tests/install/trap.sh: Likewise.
* tests/misc/timeout.sh: Likewise.
* tests/touch/no-dereference.sh: Do the same for two uses of "test".
* tests/touch/no-create-missing.sh: Likewise.
* tests/misc/pwd-option.sh (base): Initialize with -P,
now that -L is the default, to accommodate an initial
working directory with a symlink component.
* tests/misc/readlink-fp-loop.sh: Use $(env pwd -P) to get the
absolute working directory. Using "env" ensures we do not invoke
any shell built-in, and PATH ensures we invoke the one from coreutils.
* tests/readlink/can-e.sh: Likewise.
* tests/readlink/can-f.sh: Likewise.
* tests/readlink/can-m.sh: Likewise.
* src/pwd.c (main): Adjust default mode to be "logical"
and independent of the POSIXLY_CORRECT env var.
(usage): Mention the default mode of operation.
* doc/coreutils.texi (pwd invocation): Adjust accordingly.
* tests/misc/pwd-option.sh: Likewise.
* NEWS: Mention the change in behavior.
* HACKING: GID is more useful in tests than group name, so rename
input param from NON_ROOT_GROUP to NON_ROOT_GID to make it obvious
that only a group ID is now acceptable, thus allowing GID lookups
to be avoided throughout the tests.
* init.cfg (require_root_): Likewise.
* tests/misc/truncate-owned-by-other.sh: Avoid looking up the GID.
* tests/touch/now-owned-by-other.sh: Likewise.
* tests/misc/chroot-credentials.sh: Likewise. Also fix an instance
of comparison against NON_ROOT_GROUP which would have given a false
failure if a non numeric value was passed in.
* tests/id/setgid.sh: Use previously looked up gid as a more
accurate base for the subsequent adjustment, and move
the uid lookup within chroot, rather than having the overhead
of a separate `id` invocation.
* src/seq.c (main): Avoid seq_fast() with a start or end of -0.
* tests/misc/seq.pl: Add test cases.
* NEWS: Mention the fix.
Fixes http://bugs.gnu.org/17800
* src/true.c (main): Add a comment about the possibility
of true returning EXIT_FAILURE due to write failure.
* tests/misc/false-status.sh: Fix so we're testing
the tool and not the shell builtin. Add a case for true(1).
* tests/misc/help-version.sh: Skip /dev/full test
for true as well as false since the exit status is tested separately.
Also remove the iterations for different LC_MESSAGES, as this was only
applied for false(1). Translations are not honored in the test dir
and so would need separate handling in any case.
commits v8.20-98-g51ce0bf and v8.20-99-gd302aed changed cut(1)
to process each line independently and thus promptly output
each line without buffering. As part of those changes we removed
the special handling of --delimiter=$'\n' --fields=... which
could be used to select arbitrary (ranges of) lines, so as to
simplify and optimize the implementation while also matching the
behavior of different cut(1) implementations.
However that GNU behavior was in place for a long time, and
could be useful in certain cases like making a separated list like
`seq 10 | cut -f1- -d$'\n' --output-delimiter=,` although other tools
like head(1) and paste(1) are more suited to this operation.
This patch reinstates that functionality but restricts the
"line behind" buffering behavior to only the -d$'\n' case.
We also fix the following related edge case to be more consistent:
before> printf "\n" | cut -s -d$'\n' -f1- | wc -l
2
before> printf "\n" | cut -d$'\n' -f1- | wc -l
1
after > printf "\n" | cut -s -d$'\n' -f1- | wc -l
1
after > printf "\n" | cut -d$'\n' -f1- | wc -l
1
* src/cut.c (cut_fields): Adjust as discussed above.
* tests/misc/cut.pl: Likewise.
* NEWS: Mention the change in behavior both for v8.21
and this effective revert.
* cfg.mk (old_NEWS_hash): Adjust for originally omitted v8.21 entry.
* src/paste.c: s/delimeter/delimiter/ comment typo fix.
* src/chroot.c (is_root): A new helper function to
determine if the passed argument is the root directory
based on inode comparison.
(main): Use the new helper rather than comparing strings.
* tests/misc/chroot-fail.sh: Add cases for alternative root paths.
It's dangerous and confusing to leave root's supplemental
groups in place when specifying other users with --userspec.
In the edge case that that is desired one can explicitly
specify --groups.
Also we implicitly set the system defined supplemental groups
for a user. The existing mechanism where supplemental groups
needed to be explicitly specified is confusing and not general
when the lookup needs to be done within the chroot.
Also we extend the --groups syntax slightly to allow clearing
the set of supplementary groups using --groups=''.
* src/chroot.c (setgroups): On systems without supplemental groups,
clearing then is a noop and so should return success.
(main): Lookup the primary GID with getpwuid() when just a numeric
uid is specified, and also infer the USERNAME from this call,
needed when we're later looking up the supplemental groups for a user.
Support clearing supplemental groups, either implicitly for
unknown users, or explicitly when --groups='' is specified.
* tests/misc/chroot-credentials.sh: Various new test cases
* doc/coreutils.texi (chroot invocation): Adjust for the new behavior.
* NEWS: Mention the change in behavior.
This allows chroot to be used as a light weight tool
to change user identification for a command,
while not changing the current working directory.
It also makes `chroot / true` consistently succeed on
all platforms for non root users.
* src/chroot.c (main): If the same root is specified. i.e. '/'
then don't change the current working directory, and avoid the
overhead of the other redundant calls.
* tests/misc/chroot-fail.sh: Remove failure guard previously
needed on some systems. Also add an explicit case to ensure
we don't change directory.
* NEWS: Mention the change in behavior.
* src/shred.c (main): With the preceding change, shred -s-2 FILE
would write 64KB blocks forever -- or until disk full. This change
makes shred reject a negative size.
* tests/misc/shred-negative.sh: New file.
* tests/local.mk (all_tests): Add it.
* src/numfmt.c (setup_padding_buffer): Simplify the code by not
explicitly dealing with heap exhaustion.
(parse_format_string): Likewise. Handle multiple grouping
modifiers as does the standard printf. Handle the new leading
zero --format modifier.
(double_to_human): Use more defensive coding against overwriting
stack buffers. Honor the leading zeros width.
(usage): Mention the leading zero --format modifier.
(main): Allow --padding in combo with a --format (width),
as the number of leading zeros are useful independent of
the main field width.
* doc/coreutils.texi (numfmt invocation): Likewise.
* tests/misc/numfmt.pl: Add new test cases.
* NEWS: Mention the improvement.
This issue was identified by running the test suite with
http://code.google.com/p/address-sanitizer/
which is included in GCC 4.8 and enabled with -fsanitize=address
This was checked on Fedora 20 with GCC 4.8 as follows:
$ yum install libasan # http://bugzilla.redhat.com/991003
$ rm -f src/ptx.o
$ make check AM_CFLAGS='-fsanitize=address' SUBDIRS=. VERBOSE=yes
$ failure identified in tests/test-suite.log
To see this particular failure triggered with multiple files:
$ src/ptx <(echo a) <(echo a) 2>&1 | asan_symbolize.py -d
=================================================================
==32178==ERROR: AddressSanitizer: heap-buffer-overflow on address
0x60200000e74f at pc 0x435442 bp 0x7fffe8a1b290 sp 0x7fffe8a1b288
READ of size 1 at 0x60200000e74f thread T0
#0 0x435441 in define_all_fields coreutils/src/ptx.c:1425
#1 0x7fa206d31d64 in __libc_start_main ??:?
#2 0x42f77c in _start ??:?
0x60200000e74f is located 1 bytes to the left of 3-byte region
[0x60200000e750,0x60200000e753) allocated by thread T0 here:
#0 0x421809 in realloc ??:?
#1 0x439b4e in fread_file coreutils/lib/read-file.c:97
Shadow bytes around the buggy address:
0x0c047fff9c90: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9ca0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9cb0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9cc0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9cd0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fd fd
=>0x0c047fff9ce0: fa fa 03 fa fa fa fd fd fa[fa]03 fa fa fa 00 00
0x0c047fff9cf0: fa fa 04 fa fa fa 04 fa fa fa fd fa fa fa fd fa
0x0c047fff9d00: fa fa 00 fa fa fa fd fa fa fa 00 fa fa fa 00 fa
0x0c047fff9d10: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa
0x0c047fff9d20: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa
0x0c047fff9d30: fa fa fd fa fa fa 00 fa fa fa 00 fa fa fa 00 fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Heap right redzone: fb
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack partial redzone: f4
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
ASan internal: fe
==32178==ABORTING
The initial report and high level analysis were from Jim Meyering...
"The underlying problem is that swallow_file_in_memory()
is setting the contents of the global text_buffer for the first file,
then updating it (clobbering old value) for the second file.
Yet, some pointers to the initial buffer have been squirreled away
and later, one of them (keyafter) is presumed to point into
the new "text_buffer", which it does not. The subsequent
SKIP_WHITE_BACKWARDS use backs up "cursor" and goes out of bounds."
* src/ptx.c (text_buffers): Maintain references for the limits of each
buffer corresponding to each file, rather than just the last processed.
(struct OCCURS): Add a member to map back to the corresponding file.
Note normally this could be computed from the "reference" member
rather than needing the extra storage, however this is not possible
when in --references mode.
(find_occurs_in_text): Reference the array rather than a single entry.
(define_all_fields): Likewise. Also avoid computing the file index
since this is now stored directly.
(main): Update text_buffers[] array rather than a single text_buffer.
* tests/misc/ptx-overrun.sh: Even though this issue is already triggered
with AddressSanitizer, add a new case to demonstrate the whitespace
trimming issue, and to trigger without AddressSanitizer.
Fixes https://bugs.gnu.org/16171
Since the recent commit v8.22-68-g08783f1, ls coloring
is now dependent on the COLORTERM environment variable.
* tests/envvar-check: Unset COLORTERM from test environment.
* tests/ls/color-dtype-dir.sh: Ensure coloring is used.
* tests/misc/ls-misc.pl: Likewise.
Prompted by the continuous integration build failure at:
http://hydra.nixos.org/build/10397646
* doc/coreutils.texi (shred invocation): Mention some reasons
why clearing slack space might be useful.
* src/shred.c (do_wipefd): Add initial writes for each pass
for small regular files in case the storage for those is
in the inode, and thus a larger write up to a block size would
bypass that. Move the direct I/O control to...
(dopass): ... here so we can avoid enabling it for these small
initial writes. It's better to retry direct I/O for each pass
anyway to handle the case where direct I/O is disabled for only
the last portion of a file when the size is not a multiple of
the block size. Note we don't avoid the sync for the initial
write as it will be small but more importantly could be on a
different part of the disk and so worth doing independently
to ensure the write is not discarded.
* tests/misc/shred-exact.sh: Check some more direct I/O cases.
* NEWS: Mention the improvements.
The inode storage issue was mentioned by Paul Eggert.
* src/shred.c (do_wipefd): Don't increase the size written
for an empty file up to a full block. Also increase the size
to OFF_T_MAX in the edge case where we do overflow.
* NEWS: Mention the shred improvements from recent changes.
* tests/misc/shred-passes.sh: Adjust as we no longer
write a BLKSIZE of data for empty files.
* src/ptx.c (main): Add a 'break' after the --format handling case.
Otherwise it would fall through into the usage case.
* tests/misc/ptx.pl: Add test cases for --format=tex and --format=roff.
* NEWS (Bug fixes): Mention the fix.
Bug introduced in 1999-04-04 commit, SH-UTILS-1_16f-269-gd815c15.
Spotted by coverity (MISSING_BREAK).
- Support arbitrary numbers in --groups, consistent with
what is already done for --userspec
- Avoid look-ups entirely for --groups items with a leading '+'
- Support names that are actually numbers in --groups
- Ignore an empty --groups="" option for consistency with --userspec
- Look up both inside and outside the chroot with inside taking
precedence. The look-up outside may load required libraries
to complete the look-up inside the chroot. This can happen for
example with a 32 bit chroot on a 64 bit system, where the
32 bit NSS plugins within the chroot fail to load.
* src/chroot.c (parse_additional_groups): A new function refactored
from set_addition_groups(), to just do the parsing. The actual
setgroups() call is separated out for calling from the chroot later.
(main): Call parse_user_spec() and parse_additional_groups()
both outside and inside the chroot for the reasons outlined above.
* tests/misc/chroot-credentials.sh: Ensure arbitrary numeric IDs
can be specified without causing look-up errors.
* NEWS: Mention the improvements.
* THANKS.in: Add Norihiro Kamae who initially reported the issue
with a proposed patch.
Also thanks to Dmitry V. Levin for his diagnosis and sample patch.
On most shells `:>file || framework_failure_` will not evaluate
the framework_failure_ even if there was an error writing the file.
shells which do evaluate the failure are ksh 93u+ and bash 4.2,
while shells wich don't include bash 4.3, solaris, freebsd, dash.
Furthermore this construct is problematic on Solaris 10 sh,
which will try to optimize away a `:' command in a loop
after the first iteration, even if it is redirected.
* tests/cp/link-deref.sh: Remove the leading colon on redirections.
* tests/cp/reflink-perm.sh: Likewise.
* tests/id/zero.sh: Likewise.
* tests/install/install-C.sh: Likewise.
* tests/misc/env.sh: Likewise.
* tests/misc/md5sum-bsd.sh: Likewise.
* tests/misc/runcon-no-reorder.sh: Likewise.
* tests/mv/partition-perm.sh: Likewise.
* tests/rm/r-root.sh: Likewise.
* tests/split/l-chunk.sh: Likewise.
* tests/split/line-bytes.sh: Likewise.
* tests/tail-2/inotify-rotate.sh: Likewise.
* tests/tail-2/retry.sh: Likewise.
* tests/tail-2/symlink.sh: Likewise.
* tests/tail-2/wait.sh: Likewise.
* tests/touch/read-only.sh: Likewise.
+ cfg.mk (sc_prohibit_colon_redirection): A new syntax check
to avoid further instances of this creeping in.
* tests/misc/nohup.sh: When running tests without a controlling tty,
an exec failure is triggered in a subshell, which causes POSIX
shells to immediately exit the subshell. This was brought
to notice by the newly conforming bash 4.3.
Fixes http:/bugs.gnu.org/16940
* NEWS: Mention the fix.
* gnulib: Update to incorporate the fix.
This is the only change in this gnulib update.
* tests/misc/date.pl: Add a test for this case.
Fixes http://bugs.gnu.org/16872
Problem reported by valiant xiao in <http://bugs.gnu.org/16855>.
* NEWS: Document this.
* src/shuf.c (main): With -r, report an error if the input is empty.
* tests/misc/shuf.sh: Test for the bug.
If we can't output more data, we should immediately
diagnose the issue and exit rather than consuming all
of input (in some cases).
* src/tail.c (xwrite_stdout): Also diagnose the case where
only some data is written. Also clearerr() to avoid the
redundant less specific error from atexit (close_stdout);
* src/head.c (xwrite_stdout): Copy this new function from tail,
and use it to write all output.
* tests/misc/head-write-error.sh: A new test to ensure we
exit immediately on write error.
* tests/local.mk: Reference the new test.
* src/head.c (elide_tail_lines_pipe): Just output all input in
this case to avoid the issue and also avoid redundant '\n' processing.
(elide_tail_lines_seekable): Likewise.
* tests/misc/head-elide-tail.pl: Add tests for no '\n' at EOF.
* NEWS: Mention the fix.
Fixes http://bugs.gnu.org/16329
* src/od.c (main): Handle the new --endian option,
taking "little" and "big" as parameters.
(usage): Describe the new option.
(PRINT_FIELDS): Adjust to swap bytes if required.
* tests/misc/od-endian.sh: A new test to verify
the byte swapping operations for hex (ints) and floats
for all sizes between 1 and 16 inclusive.
* test/local.mk: Reference the new test.
* doc/coreutils.texi (od invocation): Describe the new option.
* NEWS: Mention the new feature.
Run "make update-copyright", but then also run this,
perl -pi -e 's/2\d\d\d-//' tests/sample-test
to make that one script use the single most recent year number.
* tests/misc/shuf-reservoir.sh: Restrict the valgrind
"exit on leak" behavior to developer environments where
specific "lint" code is enabled to avoid inconsequential leaks.
Previously, the test triggered another error diagnostic:
shuf: invalid input range ‘-e’
and therefore eclipsed the expected one:
shuf: cannot combine -e and -i options
While at it, reindent a line with more than 80 characters, present
since the previous commit to silence sc_long_lines.
* tests/misc/shuf.sh: Pass a valid range to the -i option.
Original problem reported by Philipp Thomas in
<http://bugs.gnu.org/16061>.
* NEWS: shuf --repeat, not shuf --repetitions.
* doc/coreutils.texi (shuf invocation):
* src/shuf.c (usage, long_opts, main):
* tests/misc/shuf.sh:
Likewise. Also, the default head-count is infinity.
* src/selinux.h (ignorable_ctx_err): A new function used
to determine if a warning should be given after a call
to defaultcon() or restorecon().
* src/cp.c (main): Fix the setfscreatecon() call to use
the argument passed by the user.
* src/mkdir.c (make_ancestor): Show all but "ignoreable" errors
from defaultcon() and restorecon().
* tests/misc/selinux.sh: Add a test run as root in selinux enforcing
mode, to ensure cp --context=invalid is honored and fails immediately.
A sync operation is very often expensive. For illustration
I timed the following python script which indicated that
each ext4 dir sync was taking about 2ms and 12ms, on an
SSD and traditional disk respectively.
import os
d=os.open(".", os.O_DIRECTORY|os.O_RDONLY)
for i in range(1000):
os.fdatasync(d)
So syncing for each character for each file can result
in significant delays. Often this overhead is redundant,
as only the data is sensitive and not the file name.
Even if the names are sensitive, your file system may
employ synchronous metadata updates, which also makes
explicit syncing redundant.
* tests/misc/shred-remove.sh: Ensure all the new parameters
actually unlink the file.
* doc/coreutils.texi (shred invocation): Describe the new
parameters to the --remove option.
* src/shred.c (Usage): Likewise.
(main): Parse the new options.
(wipename): Inspect the new enum to see which of
the now optional tasks to perform.
* NEWS: Mention the new feature.
* THANKS.in: Add reporter Joseph D. Wagner
This adds support for using a constant or "stick" parity bit.
* src/stty.c (usage): Mention the new flag.
* tests/misc/stty.sh: Adjust for the new flag.
* NEWS: Mention the improvement.
* docs/coreutils.texi (stty invocation): Mention the new flag.
* src/shred.c (dopass): Exit early to avoid redundant heap
allocation, and more importantly avoiding a file sync
when we're writting no data, as this can have side effects.
Also with --verbose, this avoids printing of "pass status"
which could be confusing as to whether data was actually written.
* tests/misc/shred-passes.sh: Ensure the status for data
passes are not written when not doing any data writes.
Since direct I/O is now enabled with commit v8.21-139-gebaf961
we must handle the case where we write an odd size at the
end of a file (with --exact), or we specify an odd --size that
is larger than 64KiB, or in the very unlikely case of a device
with an odd size. This issue was present since direct I/O
support was first added in v5.3.0, but latent since v6.0.
Theoretically this could have also been an issue after that on
systems which didn't have alignment constraints, but did have
size constraints for direct I/O.
* src/shred.c (dopass): On the first pass for a file, always
retry a write that fails with EINVAL, so we handle direct I/O
failure at either the start or end of the file. Adjust the comment
as the original case is out of date and implicitly handled
by this more general fix.
* tests/misc/shred-exact.sh: Add a test case.
* NEWS: Add a "bug fix" entry for shred since there are
two related issues now fixed.
* src/md5sum.c (main): Add a comment as to why we continue
to escape names that do not have '\n' but do have '\\' chars.
(print_filename): Use the predetermined boolean to decide
whether to escape or not, so that in the common case we
can output the file name directly, rather than inspecting each char.
* tests/misc/md5sum.pl: Add case to show '\\' chars cause escaping.
* tests/misc/sha1sum.pl: Likewise.
The reason for having a --quiet option is to
suppress only some subset of possible errors.
The most useful separation here is with usage/internal errors,
and errors due to file creation etc. (i.e. I/O errors).
* src/mktemp.c (main): Match the --help and info docs and
only suppress the file/dir creation error messages.
* tests/misc/mktemp.pl: Adjust accordingly.
