This is following on from this change:
[02c3dc9d 2008-03-06 cat: use larger buffer sizes ...]
which increased the IO block size used by cat by 8 times,
but also capped it at 32KiB.
* NEWS: Mention the change in behavior.
* src/system.h: Add a new io_blksize() function that
returns the max of ST_BLKSIZE or 32KiB, as this was
seen as a good value for a minimum block size to use
to get good performance while minimizing system call overhead.
* src/cat.c: Use it.
* src/copy.c: ditto
* src/split.c: ditto
* src/pr.c (print_header): Honor chars_per_margin also for the header.
Inspired by a patch from C de-Avillez.
This change also adjusts two other %*s directives to be %*.*s instead,
(for the padding on either side of "file_text") to correct a bug that
could make pr print one space of padding instead of none.
* NEWS (Bug fixes): Mention it.
* tests/pr/o3Jml24f-lm-lo: Adjust for new expected output.
* tests/pr/o3a3Sl24f-tn: Likewise.
* tests/pr/o3a3Snl24f-tn: Likewise.
* tests/pr/o3a3l24f-tn: Likewise.
* tests/pr/o3b3Sl24f-tn: Likewise.
* tests/pr/o3b3Snl24f-tn: Likewise.
* tests/pr/o3b3l24f-tn: Likewise.
* tests/pr/o3mSl24f-bl-tn: Likewise.
* tests/pr/o3mSnl24fbltn: Likewise.
* tests/pr/o3ml24f-bl-tn: Likewise.
* src/cat.c (max): Remove definition. Use MAX from system.h instead.
(compute_buffer_size): New function to compute the input and output
buffer sizes, which are now set at 8 times st_blksize with a maximum
of 32KiB. Previously the typical block sizes used were 1KiB for pipes
and 4KiB for files, and now will be 8KiB and 32KiB respectively.
(main): Use it.
This change can double throughput on modern systems. For timings,
see http://article.gmane.org/gmane.comp.gnu.core-utils.bugs/16040
Suggestion from Tzvi Rotshtein.
Running "mkdir dir; cp -rl dir dir" would create dir/dir/dir/...
rather than diagnosing the "copy-into-self" failure.
The easy fix would have been to revert this part of the change
[3ece0355 2008-11-09 cp: use far less memory in some cases]
that introduced the bug:
- remember_copied (dst_name, dst_sb.st_ino, dst_sb.st_dev);
+ if (!x->hard_link)
+ remember_copied (dst_name, dst_sb.st_ino, dst_sb.st_dev);
However, that would have induced the failure of the new cp/link-heap
test, due to the added memory pressure of recording 10k dev/ino pairs.
And besides, I liked that improvement and wanted to keep it.
Now that it's obvious recording the just-created-directory dev/ino
needn't depend on the setting of hard_link, I realized it is necessary
to record the pair only for the first directory created for each
source command-line argument.
I made that change, then noticed the new test, cp -rl a d d, would pass
when run once, yet output the into-self diagnostic twice. Also note
the side effect: it creates d/a and d/d. However, running that same
command a second time, now with the modified directory, would fail.
That turned out to be due to the fact that although the first into-self
failure was detected in copy_dir, that function would continue copying
other entries regardless -- and that would make it fail (eventually)
with the unwanted recursion.
* src/copy.c (copy_internal): This function needed an indicator of
whether, for a give command line argument, it had already created its
first directory. If so, no more need to record dev/ino pairs. If this
is the first, then do record its pair. Hence, the new parameter.
(copy_dir, copy): Update callers.
(copy_dir): Upon any into-self failure, break out of the loop.
* tests/cp/into-self: Test for the above.
Reported by Mikael Magnusson.
* src/sort.c: When no specific number of chars to skip
is specified for the end field, always skip the whole field.
Also never include leading spaces from next field.
* tests/misc/sort: Add 2 new tests for these cases.
* NEWS: Mention this bug fix.
* THANKS: Add bug reporter.
Reported by Davide Canova.
* src/ls.c (usage): Make the description of -s mention "allocated size",
so that it's less likely to be confused with an apparent byte-count.
Suggested by Vito Caputo.
* src/dd.c (usage): Document the default block size.
* doc/coreutils.texi (dd invocation): Document that the default
block size (bs, ibs, obs) is 512 bytes.
Reported by Petr Uzel.
* copy.c (copy_reg): Reduce SELinux context diagnostics for 'cp -a'.
(copy_internal): Likewise
* copy.h (cp_options): Add boolean reduce_diagnostics.
* cp.c (usage): Say that --archive (-a) behaves like -dR --preserve=all.
(cp_option_init): Initialize added reduce_diagnostics.
(main): Add reduce_diagnostics for the -a option, and preserve SELinux
context, if possible.
* mv.c (cp_options_init): Initialize new cp_options booleans.
* install.c (cp_option_init): Likewise.
* NEWS: Mention those behaviour changes.
* doc/coreutils.texi: Document --preserve=context, document that
diagnostics are not shown for failures of non-mandatory attributes
(just SELinux at the moment).
* tests/cp/cp-a-selinux: Check not only failures, but succesful use
of preserving SELinux context in cp.
* src/install.c (have_same_content): New function to compare files
content.
(extra_mode): New function checking for non-permission bits in mode.
(need_copy): New function to check if copy is necessary.
(main): Handle new option --compare (-C).
(copy_file): Skip file copying if not necessary.
(usage): Show new option --compare (-C) in --help.
* tests/install/install-C: Basic tests for install --compare (-C).
* tests/install/install-C-root: Tests requiring root privileges.
* tests/install/install-C-selinux: Tests requiring SELinux.
* tests/Makefile.am: Add new tests for install --compare (-C).
* doc/coreutils.texi: Document new install option --compare (-C).
* NEWS: Mention the change.
* src/id.c (usage): Improve description, based on a suggestion
from Brian M. Carlson in http://bugs.debian.org/514675
* man/id.x: Use a better one-liner, based on the one at top of id.c.
* src/du.c (main): Use FTS_DEFER_STAT, for better locality of inode
reference. Important when traversing file systems with fake inodes.
* src/chgrp.c (main): Likewise.
* src/chmod.c (main): Likewise.
* src/chown.c (main): Likewise.
The only remaining fts client, chcon, doesn't need this, since it goes
further and uses FTS_NOSTAT, which suppresses all non- directory
stat calls.
This change addresses a relatively unusual case: ls --color, with
a highlighted name being printed initially in the last row of a
terminal emulator (possibly followed by other lines of output) such
that it is wrapped onto the following line, as the terminal emulator
scrolls the output. That would cause the entire following line to
be highlighted, even if the name happened to use only one position.
The least-invasive patch would have made colorized output larger for
all uses. The approach taken below is more invasive, but limits the
increase in overhead to only those lines that are expected to wrap.
* src/ls.c (enum indicator_no): Add C_CLR_TO_EOL.
(indicator_name): Add "cl".
(color_indicator): Add default escape codes for "cl".
(print_long_format): Propagate width to print_name_with_quoting.
(print_name_with_quoting): Print new C_CLR_TO_EOL string if needed.
Return the width of what we're printing.
(print_file_name_and_frills): Propagate width.
(print_type_indicator): Return bool (aka width).
(print_many_per_line): Pass column position to print_* function.
(print_current_files): Likewise.
(print_horizontal): Likewise.
(print_with_commas): Likewise.
* src/dircolors.c (slack_codes): Add "CLRTOEOL".
(ls_codes): Add "cl".
* tests/ls/color-clear-to-eol: New file. Test for this fix.
* tests/Makefile.am (TESTS): Add ls/color-clear-to-eol.
* THANKS: Update.
Reported by Alexander V. Lukyanov. See thread for details:
http://thread.gmane.org/gmane.linux.kernel/740021/focus=14824
Thanks to Jan Engelhardt for helping me reproduce the problem.
Demonstrate with this in an 80-column xterm:
seq 200 # to start in the "bottom" row
touch zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz.foo
env LS_COLORS='*.foo=0;31;42' ls -og --color=always
Before the fix, you'd see something like this:
(where the file name is printed in red on a green background,
and each "=" denotes a space on a green background)
...
-rw-r--r-- 1 0 Feb 5 11:31 zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz\
zzzzzzzzzzzzzz.foo===================================================
After the patch, the trailing green spaces are gone:
-rw-r--r-- 1 0 Feb 5 11:31 zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz\
zzzzzzzzzzzzzz.foo
This patch was originally written by Andreas Grünbacher, nowadays
available at
http://www.suse.de/~agruen/coreutils/5.91/coreutils-xattr.diff
* bootstrap.conf: Add gnulib module verror.
* po/POTFILES.in: Add lib/verror.c.
* m4/xattr.m4: Check for libattr availability, new configure option
--disable-xattr.
* m4/prereq.m4: Require gl_FUNC_XATTR.
* src/Makefile.am: Link cp, mv and ginstall with libattr.
* src/copy.h: Add preserve_xattr and require_preserve_xattr to
cp_options.
* src/copy.c (copy_attr_error): New function to handle errors during
xattr copying.
(copy_attr_quote): New function to quote file name in error messages
printed by libattr.
(copy_attr_free): Empty function requested by libattr to free quoted
string.
(copy_attr_by_fd): New fd-oriented function to copy xattr.
(copy_attr_by_name): New name-oriented function to copy xattr.
(copy_reg, copy_internal): Call copy_extended_attributes function.
* src/cp.c (usage): Mention new --preserve=xattr option.
(decode_preserve_arg): Handle new --preserve=xattr option.
* src/mv.c: Always attempt to preserve xattr.
* src/install.c: Never attempt to preserve xattr.
* tests/misc/xattr: New test for xattr support in cp, mv and install.
* tests/Makefile.am: Add the new test to list.
* doc/coreutils.texi: Mention xattr support, new --preserve=xattr
option.
* NEWS: Mention the change.
Following are the before and after operations for seekable files,
for the various erroneous offsets handled by this patch:
skip beyond end of file
before: immediately exit(0);
after : immediately printf("cannot skip to specified offset"); exit(0);
skip > max file size
before: read whole file and exit(0);
after : immediately printf("cannot skip: Invalid argument"); exit(1);
seek > max file size
before: immediately printf("truncate error: EFBIG"); exit(1);
after : immediately printf("truncate error: EFBIG"); exit(1);
skip > OFF_T_MAX
before: read whole device/file and exit(0);
after : immediately printf("cannot skip:"); exit(1);
seek > OFF_T_MAX
before: immediately printf("truncate error: offset too large"); exit(1);
after : immediately printf("truncate error: offset too large"); exit(1);
skip > device size
before: read whole device and exit(0);
after : immediately printf("cannot skip: Invalid argument"); exit(1);
seek > device size
before: read whole device and printf("write error: ENOSPC"); exit(1);
after : immediately printf("cannot seek: Invalid argument"); exit(1);
* NEWS: Summarize this change in behavior.
* src/dd.c (skip): Add error checking for large seek/skip offsets on
seekable files, rather than deferring to using read() to advance offset.
(dd_copy): Print a warning if skip past EOF, as per FIXME comment.
* test/Makefile.am: Add 2 new tests.
* tests/dd/seek-skip-past-file: Add tests for first 3 cases above.
* tests/dd/seek-skip-past-dev: Add root only test for last case above.
* src/system.h (emit_bug_reporting_address): End each "sentence"
with period, mark URLs with <...>.
Based on a suggestion from Eric Blake.
Use fputs on a string without %s, not printf.
* src/system.h (emit_bug_reporting_address): Define away,
now, gnulib's version-etc.h also declares this function.
In this package, we choose to include each program's name in
the diagnostic. Define away the conflicting declaration.
(emit_bug_reporting_address): Also emit home page and
"General help..." links, like the new function does.
Reported by Bob Proulx.
* src/shred.c: The concensus is that a default of 3
passes is appropriate for current drive technologies.
* src/TODO: Reference Paul Eggert's suggestion
of enhancing shred to conform to DoD 5220 rules.
* src/shred.c: Use already assigned signed variable sizeof_r,
rather than the unsigned sizeof(r). Don't use signed integer
overflow check that contemporary compilers may remove anyway.
* gl/lib/mgetgroups.c: Avoid -Wsign-compare warning by using unsigned
types for the parameters of the new function realloc_groupbuf().
mgetgroups() was refactored to use this function rather than
explicitly allocating and copying from automatic storage itself.
* src/group-list.c: Use int rather than size_t as variable is
used in signed comparisons.
* src/id.c: ditto.
* src/cp.c (usage): Show new option -n in --help.
(main): Handle new option -n.
* src/mv.c (usage): Show new option -n in --help.
(main): Handle new option -n.
* doc/coreutils.texi: Document new cp/mv option -n.
* tests/cp/cp-i: Add tests for -f, -i and -n options.
* tests/mv/mv-n: New test for mv -n.
* tests/Makefile.am: Add test mv/mv-n to the list.
* NEWS: Mention the change.
* doc/coreutils.texi (pathchk invocation): Mention pathchk
checks validity (for current system) as well as portability.
Say messages go to stderr, and reorder description of checks
done for the -p option, to match what's done in code.
* src/pathchk.c (usage): Mention pathchk checks name validity.
Suggested clarifications were from Dan Jacobson.
* bootstrap.conf (gnulib_modules): Include mbrtowc explicitly.
* src/ls.c (quote_name): Don't test HAVE_MBRTOWC, now that we're
guaranteed to have the function.
* src/wc.c (wc): Likewise.
* src/dd.c (O_FULLBLOCK): Use a more uniform initializer, that makes
it easier to extract all O_ symbol names.
* maint.mk (syntax-check-rules): Also search for sc_ rules in cfg.mk.
(sc_root_tests): Ensure that this rule sets $diff.
* cfg.mk (sc_dd_O_FLAGS): New rule.
Pádraig Brady
Bruno Haible
Jim Meyering
Ondřej Vašík
Eric Blake
Kamil Dudka
Michael Meskes