* gl/modules/mbbuf: New file.
* gl/lib/mbbuf.c: Likewise.
* gl/lib/mbbuf.h: Likewise.
* gl/local.mk (EXTRA_DIST): Add the new files.
* bootstrap.conf (gnulib_modules): Add mbbuf.
* src/fold.c: Include mbbuf.h.
(fold_file): Use the mbbuf functions instead of calling fread and
handling the input buffer ourselves.
* cfg.mk (exclude_file_name_regexp--sc_preprocessor_indentation)
(exclude_file_name_regexp--sc_GPL_version): Match gl/lib/mbbuf.c and
gl/lib/mbbuf.h.
Also, change sleep and tail to not sleep less than requested.
* bootstrap.conf (gnulib_modules): Add dtimespec-bound.
* gl/lib/dtimespec-bound.c, gl/lib/dtimespec-bound.h:
* gl/modules/dtimespec-bound: New files.
* src/sleep.c, src/tail.c, src/timeout.c: Include dtimespec-bound.h.
* src/sleep.c, src/tail.c: Don’t include xstrtod.h.
* src/sleep.c (apply_suffix, main):
* src/tail.c (parse_options):
* src/timeout.c (apply_time_suffix):
Don’t sleep less than the true number of seconds.
* src/timeout.c: Don’t include ctype.h.
(is_negative): Remove; no longer needed.
(parse_duration): Use a slightly looser bound on the timeout, one
that doesn’t need -lm on GNU/Linux. Clear errno before calling
cl_strtod.
* gl/lib/strnumcmp-in.h (ISDIGIT):
* src/system.h (ISDIGIT): Remove. All uses replaced by c_isdigit,
with appropriate inclusions of c-ctype.h. This is more regular,
and is more portable to existing (but unlikely) platforms where
INT_MAX == UINT_MAX.
* gl/lib/heap.c (struct heap, heap_alloc, heap_insert)
(heapify_down, heapify_up): Prefer idx_t to size_t for sizes.
(heap_insert): Use xpalloc instead of x2nrealloc.
(heapify_down): Return void since no caller cares about value.
* gl/modules/heap: Depend on idx.
<stdbit.h> is in C23 and should be more portable in the long run,
now that Gnulib supports it.
* bootstrap.conf (gnulib_modules): Remove count-leading-zeros.
Add stdc_leading_zeros.
* gl/lib/randperm.c, src/ioblksize.h:
Include stdbit.h instead of count-leading-zeros.h.
* gl/lib/randperm.c (floor_lg): Remove; no longer needed.
(randperm_bound): Use stdc_bit_width instead of floor_lg;
* gl/modules/randperm (Depends-on): Remove count-leading-zeros.
Add stdc_bit_width.
* src/ioblksize.h (io_blksize): Use stdc_leading_zeros_ull
instead of count_leading_zeros_ll.
* NEWS: Mention this.
* bootstrap.conf (gnulib_modules): Remove cu-ctype, as this module
is now more trouble than it’s worth. All uses removed.
Add skipchars.
* gl/lib/cu-ctype.c, gl/lib/cu-ctype.h, gl/modules/cu-ctype:
Remove.
* gl/lib/skipchars.c, gl/lib/skipchars.h, gl/modules/skipchars:
* tests/misc/join-utf8.sh:
New files.
* src/join.c: Include skipchars.h and mcel.h instead of cu-ctype.h.
(tab): Now mcel_t, not int. All uses changed.
(output_separator, output_seplen): New static vars.
(eq_tab, newline_or_blank, comma_or_blank): New functions.
(xfields, prfields, prjoin, add_field_list, main):
Support multi-byte characters.
* src/numfmt.c: Include ctype.h, skipchars.h.
Do not include cu-ctype.h.
(newline_or_blank): New function.
(next_field): Support multi-byte characters.
* src/sort.c: Include ctype.h instead of cu-ctype.h.
(inittables): Open-code field_sep since it no longer exists.
‘sort’ is not multi-byte safe yet, but when it is this code
will need revamping anyway.
* src/uniq.c: Include mcel.h and skipchars.h instead of cu-ctype.h.
(newline_or_blank): New function.
(find_field): Support multi-byte characters.
* tests/local.mk (all_tests): Add tests/misc/join-utf8.sh
This is so that we don’t need to have every source file
include ctype.h.
* bootstrap.conf (gnulib_modules): Add cu-ctype.
* gl/lib/cu-ctype.c, gl/lib/cu-ctype.h, gl/modules/cu-ctype:
New files.
* src/join.c, src/numfmt.c, src/sort.c, src/uniq.c:
Include cu-ctype.h, for field_sep.
* src/system.h (field_sep): Remove; now supplied by cu-ctype.
This simplifies memory allocation a bit, and removes an arbitrary
limitation from numfmt, which formerly limited cell output to 127
bytes.
* bootstrap.conf (gnulib_modules): Remove mbsalign, strncat.
Add strnlen (the code already used strnlen directly, and we were
saved only because Gnulib used the module indirectly)
* gl/lib/mbsalign.c, gl/lib/mbsalign.h, gl/modules/mbsalign:
* gl/modules/mbsalign-tests, gl/tests/test-mbsalign.c: Remove.
* src/df.c, src/ls.c: Do not include mbsalign.h.
(MBSWIDTH_FLAGS): New constant, now used for all
mbswidth calls. All callers changed to check for -1 return.
* src/df.c (struct field_data_t): ‘width’ is now int not size_t,
since mbswidth can’t do widths greater than INT_MAX anyway.
Replace ‘align’ with ‘align_right’. All uses changed.
(print_table): Redo to avoid the need for ambsalign.
(get_header, get_dev): mbswidth returns int, not size_t.
* src/ls.c (MAX_MON_WIDTH): Remove; no longer used.
(abmon_init): Use strnlen to cheaply discard too-long month names.
Align by hand instead of using mbsalign.
* src/numfmt.c: Include stdckdint.h, mbswidth.h.
Do not include mbsalign.h.
(padding_buffer_size): Now idx_t. All uses changed.
(padding_width): Now intmax_t, since it’s no longer an object
size. Its sign now records alignment. All uses changed.
(zero_padding_width): Now int, since it’s given to sprintf.
All uses changed.
(padding_alignment): Remove; it’s now taken from padding_width’s sign.
(double_to_human): Return string length. BUF_SIZE arg is now idx_t.
Include suffix in output. All callers changed. Simplify by not
calling strncat or stpcpy. Calculate fmt size bound more carefully.
(setup_padding_buffer): Remove. All uses removed.
(parse_format_string): Use intmax_t, not long, for pad.
On overflow, set widths to large values that cause later code
to do the right thing, rather than separately checking for
overflow here.
(prepare_padded_number): Return bool, not int 0/1. New arg
PADDING. All uses changed. Do not limit padded output to 127
bytes; instead, use xpalloc to expand the output buffer.
(print_padded_number): New arg PADDING. All uses changed.
(process_suffixed_number): Simplify.
(main): Take extremum if xstrtoimax overflows, as this does
the right thing.
* tests/misc/numfmt.pl: New test suf-20 to test for truncation bug.
Remove tests pad-3.2, fmt-err-7, as they’re no longer invalid but
are quite expensive.
Move target directory code out of system.h to a new targetdir module.
This doesn’t change functionality.
* bootstrap.conf (gnulib_modules): Add targetdir.
* src/cp.c, src/install.c, src/mv.c: Include targetdir.h.
* src/system.h (must_be_working_directory, target_directory_operand)
(targetdir_dirfd_valid): Move from here ...
* gl/lib/targetdir.c, gl/lib/targetdir.h, gl/modules/targetdir:
... to these new files.
* configure.ac: Use AC_PROG_CC, not AC_PROG_CC_STDC.
* gl/modules/smack (configure.ac):
* m4/jm-macros.m4 (coreutils_MACROS):
* m4/xattr.m4 (gl_FUNC_XATTR):
Use AS_HELP_STRING, not AC_HELP_STRING.
* m4/check-decl.m4 (gl_CHECK_DECLS):
Do not require AC_HEADER_TIME; we no longer care about it directly.
* m4/jm-macros.m4 (coreutils_MACROS):
Do not require AC_ISC_POSIX, which became obsolete in 2006.
Use AC_LINK_IFELSE instead of AC_TRY_LINK.
This makes for one Gnulib module less, and at runtime there’s
typically just one getrandom syscall instead of several for large
nonces.
* gl/lib/randread.c: Include sys/random.h instead of sys/time.h
and unistd.h.
(get_nonce): Use getrandom, not getentropy.
* gl/modules/randread (Depends-on):
Depend on getrandom, not getentropy.
* src/shred.c (main):
* src/shuf.c (main):
* src/sort.c (random_md5_state_init):
Say "getrandom" rather than "getentropy" in (unlikely) diagnostic.
Update gnulib submodule to latest and use its new features.
Gnulib’s new getentropy module means coreutils can now assume
getentropy instead of approximating it, badly in some cases.
Gnulib’s improvements to the tempname module mean coreutils no
longer needs to maintain private patches.
* bootstrap.conf (gnulib_modules): Remove gettimeofday.
* gl/lib/randread.c (NAME_OF_NONCE_DEVICE): Remove.
(get_nonce): Return success indicator. Remove bytes_bound arg.
All callers changed. Rewrite by using getentropy instead of
reading the nonce device and falling back on gettimeofday.
Fail if getentropy fails.
(randread_new): Return NULL (setting errno) if get_nonce fails.
All callers changed.
* gl/lib/tempname.c.diff, gl/lib/tempname.h.diff:
* gl/modules/tempname.diff: Remove.
* gl/modules/randread (Depends-on):
Depend on getentropy, not gettimeofday.
* src/ptx.c (swallow_file_in_memory):
* src/shuf.c (read_input):
Adjust to read_file changes in Gnulib.
* src/shred.c (main):
* src/shuf.c (main):
* src/sort.c (random_md5_state_init):
Diagnose the new form of randread_new failures: randread_new can
fail now when !random_source, meaning getentropy failed.
* gl/lib/randperm.c: Include randperm.h first, since it’s the API.
Include stdint.h, count-leading-zeros.h, verify.h.
(floor_lg): Rename from ceil_log (which was not actually
implementing the ceiling!) and implement the floor using
count_leading_zeros.
(randperm_bound): Use floor_lg, not ceil_log. Use uintmax_t
instead of size_t in case the size gets large on a 32-bit host.
* gl/modules/randperm (Depends-on): Add count-leading-zeros, stdint.
These commands now accept floating-point numbers in the
current locale, as well as in the C locale.
Compatibility problem reported by Robert Elz.
* NEWS: Document this.
* bootstrap.conf (gnulib_modules): Add cl-strtod, cl-strtold.
Remove c-strtold.
* doc/coreutils.texi (Floating point, tail invocation)
(printf invocation, timeout invocation, sleep invocation)
(seq invocation): Document this.
* gl/lib/cl-strtod.c, gl/lib/cl-strtod.h, gl/lib/cl-strtold.c:
* gl/modules/cl-strtod, gl/modules/cl-strtold: New files.
* src/printf.c, src/seq.c, src/sleep.c, src/tail.c, src/timeout.c:
Include cl-strtod.h instead of c-strtod.
* src/printf.c (vstrtold):
* src/seq.c (scan_arg, print_numbers):
* src/sleep.c (main):
* src/tail.c (parse_options):
* src/timeout.c (parse_duration):
Use cl_strtold instead of c_strtold.
Related to commit v8.24-61-g6796698 this provides
more consistent quoting, as quotearg_colon() defaults
to "literal" quoting by default, while quote()
provides appropriate quoting for diagnostics by default.
* gl/modules/randread: Depend on quote module rather than quotearg.
* gl/lib/randread.c: Used quote() not quotearg_colon().
* src/: Likewise.
* src/shred.c: Likewise. Also avoid unnecessary quoting
introducing overhead when wiping names.
* cfg.mk: Relax the matching expression to allow
"qname" variables as used in shred.c to satisfy the check.
* tests/: Adjust accordingly.
This includes a change to require --with-libmount
to be used with configure, due to the many libmount dependencies.
* bootstrap: Sync with gnulib to exit early on gnulib-tool error.
* gl/lib/tempname.c.diff: Adjust for gnulib changes.
* gl/lib/tempname.h.diff: Likewise.
* gl/modules/tempname: Likewise.
* doc/.gitignore: Add new gendocs_template_min gnulib script.
Following on from commit v8.23-82-gaddae94, consistently diagnose
numbers that are too large, so as to distinguish from other errors,
and make the limits obvious.
* gl/modules/xdectoint: A new module implementing xdecto[iu]max(),
which handles the common case of parsing a bounded integer and
exiting with a diagnostic on error.
* gl/lib/xdectoimax.c: The signed variant.
* gl/lib/xdectoint.c: The parameterized implementation.
* gl/lib/xdectoint.h: The interface.
* gl/lib/xdectoumax.c: The unsigned variant.
* bootstrap.conf: Reference the new module.
* cfg.mk (exclude_file_name_regexp--sc_require_config_h_first):
Exclude the parameterized templates.
* src/csplit.c: Output EOVERFLOW or ERANGE errors if appropriate.
* src/fmt.c: Likewise.
* src/fold.c: Likewise.
* src/head.c: Likewise.
* src/ls.c: Likewise.
* src/nl.c: Likewise.
* src/nproc.c: Likewise.
* src/shred.c: Likewise.
* src/shuf.c: Likewise.
* src/stdbuf.c: Likewise.
* src/stty.c: Likewise.
* src/tail.c: Likewise.
* src/truncate.c: Likewise.
* src/split.c: Likewise.
* src/pr.c: Likewise.
* tests/pr/pr-tests.pl: Adjust to avoid matching errno diagnostic.
* tests/fmt/base.pl: Likewise.
* tests/split/l-chunk.sh: Likewise.
* tests/misc/shred-negative.sh: Likewise.
* tests/misc/tail.pl: Likewise. Also remove the redundant
existing ERR_SUBST from test err-6.
* tests/ls/hex-option.sh: Check HEX/OCT options.
* tests/misc/shred-size.sh: Likewise.
* tests/misc/stty-row-col.sh: Likewise.
The patch was corrupted in commit v8.21-50-g7b65f8e
* gl/modules/tempname.diff: Fix the offsets so that the patch
applies cleanly. Note that this was only apparent with patch < 2.6.
With patch >= 2.6, patch will not indicate an error applying
the second hunk of the patch and silently ignore it.
I double checked that all patches now apply cleanly by adjusting
gnulib-tool to run patch with --fuzz=0 which might be advisable
going forward, even on a per project basis.
The silent ignoring of hunks by newer patch(1) has been reported.
Fixes http://bugs.gnu.org/15255
Consolidate all smack routines and checks in a module.
We replace and wrap the most commonly used smack routines,
which allows removing ifdefs throughout the code.
* gl/lib/smack.h: A new header containing the implementation
of the wrapped and replacement routines. Note the is_smack_enabled()
routine should be optimized out at compile time when compiled
on a system without libsmack.
* gl/modules/smack: Describe the new module and move the
configure time code here from ...
* m4/jm-macros.m4: ... here.
* bootstrap.conf: Reference the new module.
* src/id.c: Use the routines without ifdefs where possible.
* src/ls.c: Likewise.
* src/mkdir.c: Likewise.
* src/mkfifo.c: Likewise.
* src/mknod.c: Likewise.
I've moved the non-recursive-gnulib-prefix-hack module to
gnulib, with two small improvements, so remove it from here
and update gnulib to the latest.
* gl/build-aux/prefix-gnulib-mk: Remove file.
* gl/m4/non-recursive-gnulib-prefix-hack.m4: Remove file.
* gl/modules/non-recursive-gnulib-prefix-hack: Remove file.
* gnulib: Update to latest.
* gl/modules/non-recursive-gnulib-prefix-hack: New module.
* gl/m4/non-recursive-gnulib-prefix-hack.m4:
(gl_NON_RECURSIVE_GNULIB_PREFIX_HACK): This is the snippet
that this module inserts near the end of configure.
* gl/build-aux/prefix-gnulib-mk: New script, from bison.
Changes from the code in bison:
(prefix_assignment): Split a long line.
(prefix): Add trailing slashes to avoid a single false match.
Prefix imaxtostr.c and the other *tostr.c file names manually.
Also, use $prefix in place of hard-coded "lib/".
Convert the few remaining coreutils-specific files in lib/ to
gnulib-style modules under gl/, removing their corresponding .m4
files, since the information recorded in those files is better
stored in module-description file in gl/modules/.
* bootstrap.conf (gnulib_modules): Add new modules:
fd-reopen, buffer-lcm, xfts, strnumcmp.
* gl/lib/buffer-lcm.c: Renamed from the file in lib/.
* gl/lib/buffer-lcm.h: Likewise.
* gl/lib/fd-reopen.c: Likewise.
* gl/lib/fd-reopen.h: Likewise.
* gl/lib/strintcmp.c: Likewise.
* gl/lib/strnumcmp-in.h: Likewise.
* gl/lib/strnumcmp.c: Likewise.
* gl/lib/strnumcmp.h: Likewise.
* gl/lib/xfts.c: Likewise.
* gl/lib/xfts.h: Likewise.
* gl/modules/buffer-lcm: New module-description file.
* gl/modules/fd-reopen: Likewise.
* gl/modules/strnumcmp: Likewise.
* gl/modules/xfts: Likewise.
* m4/fd-reopen.m4: Remove, no longer needed.
* m4/strnumcmp.m4: Likewise.
* m4/xfts.m4: Likewise.
* m4/prereq.m4: Do not AC_REQUIRE the m4 functions from
our just-removed m4/*.m4 files.
We can get the same effect using the modules file.
* gl/m4/root-dev-ino.m4: Remove file.
* gl/modules/root-dev-ino (Depends-on): Add lstat.
(Files): Remove m4/root-dev-ino.m4.
(Makefile.am) [lib_SOURCES]: Add root-dev-ino.c and root-dev-ino.h.
(configure.ac): Remove reference to gl_ROOT_DEV_INO.
We carry local adjustments for a few gnulib modules via the
patches in gl/. Nearly all of those patches had become stale
due to evolution of the originals in gnulib.
To refresh them, first make sure you have no local changes in gl/
or in the gnulib submodule, then run "make refresh-gnulib-patches".
* gl/modules/randread (Depends-on): Add stdalign.
* gl/lib/randread.c: Include <stdalign.h>, so we can ...
[!_STRING_ARCH_unaligned]: remove definition of stdalign.
* gl/modules/tempname.diff: Regenerate to correct bogus offsets
and adjust for 1-line offset. Eric Blake reported that this
patch failed to apply when using patch-2.5.8.
Note that it applies fine using patch-2.6.1.
Using a .diff is much more maintainable. Otherwise, changes in
the gnulib module description file may not be noticed and merged
promptly and may even result in subtle errors. Luckily, this time,
the failure to propagate gnulib's changes to modules/tempname resulted
only in an obvious link failure.
* gl/modules/tempname: Remove file.
* gl/modules/tempname.diff: Use a .diff file instead.
* gnulib: Update submodule to latest.
* gl/lib/randperm.c (randperm_new): When the number of items
to return H, is much smaller than the total number of items N,
use a hash to represent the sparse permutations of the set N.
This is currently enabled for N > 128K and N/H > 32.
* tests/misc/shuf: Ensure shuf can quickly return 2 numbers
from a large range.
* gl/modules/randperm: Depend on hash.
* NEWS: Mention the change.
* gl/modules/fadvise-tests: Add the module previously missed
in commit 63b5e816, 2010-07-14, "fadvise: new module ...".
* gl/tests/test-fadvise.c: Add a comment as to why we don't
check return values.
The implementation of variable-precision time stamps relied
on heuristics that made the output subtly nondeterministic,
or at least hard to reproduce:
http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/21531/focus=21538
So, for now at least, we're removing that feature.
* bootstrap.conf (gnulib_modules): Remove fstimeprec.
* gl/lib/fstimeprec.c, gl/lib/fstimeprec.h: Remove files.
* gl/modules/fstimeprec, gl/modules/fstimeprec-tests: Likewise.
* gl/tests/test-fstimeprec.c: Remove file.
* NEWS: Document this.
* doc/coreutils.texi (stat invocation): Likewise.
* gl/lib/fstimeprec.c, gl/lib/fstimeprec.h, gl/modules/fstimeprec:
* gl/modules/fstimeprec-tests, gl/tests/test-fstimeprec.c:
New files.
* bootstrap.conf (gnulib_modules): Add fstimeprec.
* src/stat.c: Include fstimeprec.h. Don't include xstrtol.h.
(decimal_point, decimal_point_len): New static vars.
(main): Initialize them.
(epoch_sec, out_ns): Remove.
(out_int, out_uint): Now returns whatever printf returned.
(out_minus_zero, out_epoch_secs): New functions.
(print_stat): Use out_epoch_sec instead of out_ns and epoch_sec.
(print_stat, print_it, usage): Remove the %:X-style formats.
* tests/misc/stat-nanoseconds: Set TZ=UTC0 to avoid problems
with weird time zones. Use a time stamp near the Epoch so that we
don't have to worry about leap seconds. Redo test cases to match
new behavior.
* tests/touch/60-seconds: Change %Y.%:Y to %.9Y, to adjust to
new behavior.
* gl/lib/mbsalign.c: Include <stdint.h> for SIZE_MAX.
Use SIZE_MAX rather than (size_t) -1, to avoid warning for
"size_t x = -1" on Solaris 10 cc.
* gl/modules/mbsalign (Depends-on): Add stdint.
* gl/lib/rand-isaac.c: Remove the I/O; this belongs elsewhere.
Add support for ISAAC64. Port to hosts with padding bits.
Add self to author list. Include <limits.h>, for CHAR_BIT.
Don't include string.h, sys/time.h, unistd.h.
(min, just): New functions.
(IF32): New macros.
(ind, ISAAC_STEP, isaac_refill, mix, isaac_init, isaac_seed):
Add support for ISAAC64. Port to hosts with padding bits.
(ind): Now an inline function rather than a macro; no need for it
to be a macro with modern compilers.
(ISAAC_STEP): Renamed from isaac_step, since it's not function-like.
Don't bother to pass args that are always the same. All uses changed.
(ISAAC_STEP, ISAAC_SEED): Move to inside the only function body
that can use it.
(ISAAC_MIX): Renamed from isaac_mix, since it's now a macro and is
no longer function-like. Don't bother saving and restoring state;
no longer needed now that we're not a function. All uses changed.
(isaac_seed_start, isaac_seed_data, isaac_seed_finish): Remove.
(isaac_seed): Take just the one arg; the caller now sets s->m.
* gl/lib/rand-isaac.h: Use _GL_RAND_ISAAC_H to protect, instead
of RAND_ISAAC_H. Try out " #" rather than "# " for indenting.
(ISAAC_BITS_LOG, ISAAC_BITS): New macros.
(ISAAC_WORDS_LOG): Renamed from ISAAC_LOG.
(isaac_word): New type. All uses of uint32_t changed to isaac_word,
to support ISAAC64.
(struct isaac_state): Rename member MM to M, and make it public.
(isaac_seed, isaac_refill): Adjust to new API.
* gl/lib/randread.c: Include sys/time.h.
(get_nonce): New function, containing the nonce stuff that used
to be in rand-isaac.c but better belongs here.
(randread_new): Use it.
* gl/modules/randread (Depends-on): Add inline.
* gl/modules/randread-tests: New file.
* gl/tests/test-rand-isaac.c: New file.
* bootstrap.conf: Include the new module
* gl/lib/fadvise.c: Provide a simpler interface to posix_fadvise.
(fadvise): Provide hint to the whole file associated with a stream.
(fdadvise): Provide hint to the specific portion of a file
associated with a file descriptor.
* gl/lib/fadvise.h: Redefine POSIX_FADV_* to FADVISE_* enums.
* gl/modules/fadvise: New file.
* m4/jm-macros.m4: Remove the no longer needed posix_fadvise check.
* .x-sc_program_name: Exclude test-fadvise.c from this check.
* gl/tests/test-fadvise (main): New test program.
* gl/modules/fadvise-testss: A new index to reference the tests.
* src/sort.c (stream_open): Use the new interface.
* src/dd.c (iwrite): Likewise.
