This also fixes a free-memory-read (FMR) bug: when fillbuf's realloc
of buf->buf frees the buffer into which saved_line.text points,
the processing of that just-read longer line includes comparison
against the saved line in freed memory.
* src/sort.c (overlap): Remove.
(fillbuf): Do not try to copy saved lines, as that is too risky
in the presence of parallelism, reallocated buffers, etc.
(sort): Invalidate any saved line before sorting a new batch.
sort -u could omit one or more lines of expected output.
This bug arose because sort recorded the most recently printed line via
reference, and if you were unlucky, the storage for that line would be
reused (overwritten) as additional input was read into memory. If you
were doubly unlucky, the new value of the "saved" line would not only
match the very next line, but if that next line were also the first in
a series of identical, not-yet-printed lines, then the corrupted "saved"
line value would result in the omission of all matching lines.
* src/sort.c (saved_line): New static/global, renamed and moved from...
(write_unique): ...here. Old name was "saved", which was too generic
for its new role as file-scoped global.
(fillbuf): With --unique, when we're about to read into a buffer that
overlaps the saved "preceding" line (saved_line), copy the line's .text
member to a realloc'd-as-needed temporary buffer and adjust the line's
key-defining members if they're set.
(overlap): New function.
* tests/misc/sort: New tests.
* NEWS (Bug fixes): Mention it.
* THANKS.in: Update.
Bug introduced via commit v8.5-89-g9face83.
Reported by Rasmus Borup Hansen in
http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/23173/focus=24647
* tests/Coreutils.pm (_compare_files): Reverse diff arguments so
that we invoke diff -c $expected $actual, which is consistent with
how init.sh-using tests invoke "compare exp out".
Add new option to rm (-d/--dir), which allows removal of
empty directories, while still safely disallowing removal
of non-empty ones.
This improves compatibility with Mac OS X and BSD systems,
which honor the -d option.
* src/remove.c (rm_fts): Remove empty directories when requested.
* src/remove.h (rm_options) [remove_empty_directories]: New member.
* src/rm.c (long_opts, usage, main): Update usage and option parsing.
(rm_option_init): Initialize the new member.
* src/mv.c (rm_option_init): Initialize the new member.
* tests/rm/d-1: New test case - successfully delete empty dir.
* tests/rm/d-2: New test case - refuse to delete nonempty dir.
* tests/Makefile.am (TESTS): Add them.
* src/df.c (main): Add conditions to fail when the mount list cannot
be read: this includes the cases when a file name argument is given
and any of -a, -l, -t or -x is used.
* doc/coreutils.texi: Document the additional error conditions.
* tests/df/no-mtab-status: Add a new test.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the fix.
* tests/init.cfg (require_mount_list_): A new function
to ensure we can read the list of file systems.
(require_local_dir_): Call the above function, as otherwise
the check is invalid.
* tests/df/total-unprocessed: Ensure df can read the
list of mounted file systems so that --local can be honored.
* src/Makefile.am (sort_LDADD): Sort uses euidaccess, which may require
whatever library configure deemed necessary to resolve the eaccess
function, but no one told sort to link with that library.
(sort_LDADD): Add $(LIB_EACCESS).
When the combination of the file system options with given files or
devices does not lead to output, "df --total" would exit successfully
although it should not.
Examples:
$ df --total --type=xfs / # when / is not an XFS file system
$ df --total --local -t nfs DIR # nfs is remote per se ...
$ df --total -t qwerty /dev/sdb5 # typo in file system type
Furthermore, "df --total" would not print the error message "no file
systems processed" when the file argument does not exist or is otherwise
not accessible.
Example:
$ df --total __not_exist__
These 2 bugs are present since --total was added by commit
v6.12-166-gea2887b.
* src/df.c (get_dev): Do not set file_systems_processed to true when
force_fsu is true, i.e. when the row for the "total" line is processed.
(main): Don't print totals unless we've processed a file system.
Also only print the "no FS processed" message if there was no
preceding diagnostic.
* tests/df/total-unprocessed: Add a new test.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the fix.
Improved-by: Jim Meyering
* src/truncate.c (main): For a user who makes the mistake of
using a non-seekable file as a reference for the desired length,
truncate would open that file, attempt to seek to its end, but
upon seek failure would neglect to close the file descriptor.
Close the file descriptor even when lseek fails.
In addition, ignore failure to close that reference FD, since as
long as the lseek succeeds, a close failure doesn't matter.
Coverity spotted the potential FD leak.
Improved-by: Pádraig Brady.
* tests/init.cfg (require_ulimit_): Raise VM limit from 10MiB to
20MiB, to accommodate overhead of a valgrind-wrapped date program.
Also declare this function's local variables "local".
* src/split.c (lines_rr) [IF_LINT]: Plug a harmless leak.
(main) [IF_LINT]: Free a usually-small (~70KB) buffer
just before exit, mainly to take this off the radar of
leak-detecting tools.
Improved-by: Pádraig Brady.
We carry local adjustments for a few gnulib modules via the
patches in gl/. Nearly all of those patches had become stale
due to evolution of the originals in gnulib.
To refresh them, first make sure you have no local changes in gl/
or in the gnulib submodule, then run "make refresh-gnulib-patches".
* src/tail.c (check_fspec): Save fstat-induced errno *before*
calling close_fd, not after. Otherwise, the close could well
clobber the global errno, making tail print an invalid diagnostic.
This could happen only with tail -f, and even then, only when
a valid file descriptor were to provoke fstat failure.
Add a test and NEWS entry for a bug inadvertently fixed in
a refactoring in commit v8.9-32-gd4db0cb
* tests/misc/join (v2-format): Add a new test.
* THANKS.in: Add the reporter.
* NEWS: Mention the old bug.
* cfg.mk (old_NEWS_hash): Update.
Reported-by: Jean-Pierre Tosoni
* cfg.mk (exclude_file_name_regexp--sc_file_system): Sync this
exemption regexp to match renamed tests/df/df-P. This avoids
a "make syntax-check" failure.
* src/remove.c (cache_statted, is_dir_lstat): Remove unused
static-inlined functions.
* THANKS.in: Remove my name from this list, now that (with this
commit) it is included automatically.
Copyright-paperwork-exempt: Yes
* tests/cp/fiemap-perf: Skip the test on ext2 file systems,
as we do for ext3. Also skip the test if we can't create
a 1TiB file, which might not be supported on certain file systems.
Signed-off-by: Pádraig Brady <P@draigBrady.com>
It's awkward to read and problematic for scripts when
control characters like '\n' are output.
Note other fields are already handled with mbsalign,
which converts non printable chars to the replacement char.
A caveat to note with that, is the replacement char takes
a place in the field and so possibly truncates the field
if it was the widest field in the records.
Note a more general replacement function, that
handles all printable, or non white space characters,
would require more sophisticated support for various
encodings, and the complexity vs benefit was not
deemed beneficial enough at present.
Perhaps in future a more general replacement function
could be shared between the various utilities.
Note <space> is unaffected in any field,
which could impact scripts processing the output.
However any of the number fields at least could have
spaces considering `LANG=fr_FR df -B\'1`, so it's
probably best to leave spaces, which also allows
scripts to handle mount points with spaces without change.
* src/df.c (hide_problematic_chars): Replace control chars with '?'.
* tests/df/problematic-chars: Add a new root only test.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the fix.
* src/system.h (stzncpy): New function.
* src/pinky.c (print_entry): Use stzncpy, not stpncpy.
The latter does not NUL-terminate. I assumed that strncpy was
the only function with such a horrible API. Today I learned that
stpncpy also may not NUL-terminate its result.
The bugs were introduced in commit v8.17-48-gf79263d.
* src/who.c (print_user): Likewise.
Thanks to Erik Auerswald for spotting my error.
Or rather, with the development version 4.13.90, which will eventually
become Texinfo 5.0.
* doc/coreutils.texi: Use '@item' instead of '@itemx' in several places,
as Texinfo 5 refuses to process an '@itemx' that is not preceded by an
'@item'. Ensure that node extended names in menus and sectioning are
consistent, and that ordering and presence of nodes in menus and in the
actual text are consistent as well.
Fixes http://bugs.gnu.org/11828
* src/df.c (MEGABYTES_OPTION): Add enum and mark it for removal
in August 2013.
(long_options): Use MEGABYTES_OPTION for --megabytes option.
(main): Add a case for it and issue a deprecation warning if
the long form is used. Document the short -m option to
exist only for BSD compatibility.
* doc/coreutils.texi (touch invocation, Time conversion specifiers)
(Options for date, Examples of date): Index "leap seconds" and
improve their documentation a bit.
date -d "$(printf '\xb0')" would print 00:00:00 with today's date
rather than diagnosing the invalid input. Now it reports this:
date: invalid date '\260'
* gnulib: Update submodule to latest for fixed parse-datetime.y.
* tests/misc/date [invalid-high-bit-set]: New test.
* NEWS (Bug fixes): Mention it.
* bootstrap, tests/init.sh: Also update to latest.
Reported by Peter Evans in http://bugs.gnu.org/11843
* cfg.mk (sc_some_programs_must_avoid_exit_failure): New rule,
to help us avoid using EXIT_FAILURE in programs like sort, ls, nohup,
timeout, env, etc. that use different exit codes in many cases.
* src/sort.c (outfd): Remove. All uses replaced by STDOUT_FILENO.
(stream_open): When writing, use stdout rather than fdopen.
(move_fd_or_die): Renamed from dup2_or_die, with the added functionality
of closing its first argument. All uses changed.
(avoid_trashing_input): Special case for !outfile no longer needed.
(check_output): Arrange for standard output to go to the file,
rather than storing the fd in outfd.
* src/sort.c (check_inputs): A new function to verify all inputs
are accessible before further processing.
(check_output): A new function to open or create a specified
output file, before futher processing.
(stream_open): Adjust to truncating the previously opened
output file rather than opening directly.
(avoid_trashing_input): Optimize to stat the output file
descriptor, rather than the file name.
(main): Call the new functions to check accessibility of
inputs and output, before processing starts.
* tests/misc/sort: Adjust to the changed error message.
* tests/misc/sort-merge-fdlimit: Account for the earlier opened
file descriptor of the specified output file.
* tests/misc/sort-exit-early: A new test to exercise the improvements.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the improvement.
Suggested-by: Bernhard Voelker
* src/stty.c (main): Mark speed_was_set as possibly unused,
as is the case when CIBAUD is undefined (on ppc64 GNU/Linux
for example).
Reported-by: Stefano Lattarini
