* tests/cp/cp-mv-enotsup-xattr.sh: The nouser_xattr option
is not supported by ext2 since about 2022. Instead use
ramfs to ensure xattrs are not supported. We also remove
part of the test that relied on partial xattr support.
Fixes https://github.com/coreutils/coreutils/issues/135
* src/copy-file-data.c (infer_scantype): Fall back to a plain copy
if SEEK_HOLE indicates non-sparse, as zero copy avoids copy offload.
This was seen with transparently compressed files on OpenZFS.
* tests/cp/sparse-perf.sh: Add a test case even though it might
only trigger on compressed file systems that don't support reflink.
* NEWS: Mention the bug fix.
Addresses https://github.com/coreutils/coreutils/issues/122
Related to commits v9.1-109-g879d2180d and v9.7-248-g306de6c26
* tests/cp/sparse-perf.sh: This edge case was missed a couple of times,
so add a test to ensure we attempt copy offload.
These tests were seen to hang on PPC Mac OS X 10.4.11
* tests/cp/existing-perm-race.sh: Protect fifo interactions
with a 1 minute timeout.
* tests/cp/file-perm-race.sh: Likewise.
Reported in https://bugs.gnu.org/78509
* cfg.mk (sc_prohibit_bare_set): A new syntax check to
ensure we protect use of set with '--', so that args
beginning with '-' are not interpreted as options,
and if no args are present, all existing args are cleared.
* tests/cp/symlink-slash.sh: Add -- to unprotected use of set.
* tests/ls/ls-time.sh: Likewise.
* tests/ls/symlink-slash.sh: Likewise.
* tests/mkdir/perm.sh: Likewise.
* tests/mkdir/selinux.sh: Likewise.
* tests/mkdir/smack-no-root.sh: Likewise.
* tests/mkdir/smack-root.sh: Likewise.
* tests/mv/part-hardlink.sh: Likewise.
* tests/nice/nice.sh: Likewise.
* tests/stty/stty-row-col.sh: Likewise.
* src/copy.h: Change update member from bool to enum.
* src/copy.c: s/interactive == I_ALWAYS_NO/update == UPDATE_NONE_FAIL/;
s/interactive == I_ALWAYS_SKIP/update == UPDATE_NONE/;
s/update/update == UPDATE_OLDER/;
* src/install.c: Init with UPDATE_ALL, rather than false.
* src/cp.c: Likewise. Simply parse -f,-i,-n to x.interactive,
and parse --update to x.update.
* src/mv.c: Likewise.
* tests/cp/cp-i.sh: Add a test case where -n --update -i
honors the --update option, which would previously have been
ignored due to the preceding -n.
Since coreutils 9.3 we had --update={all,older} override -i.
In coreutils 9.5 this was expanded to -u
(to make it consistent with --update=older).
This patch reinstates things so that -i combines with -u instead.
I.e. have -i be protective, rather than selective (like -u).
The -f option of mv is similarly adjusted in this patch,
so now --update does not override any of -f,-i,-n.
* NEWS: Mention the bug fix.
* src/cp.c (main): Don't have -u disable prompting.
* src/mv.c (main): Likewise.
* tests/cp/cp-i.sh: Add a test case for -i.
* tests/mv/update.sh: Likewise.
* tests/mv/i-3.sh. Add a test case for -f.
Fixes https://bugs.gnu.org/70887
Update to latest gnulib with new copyright year.
Run "make update-copyright" and then...
* gnulib: Update included in this commit as copyright years
are the only change from the previous gnulib commit.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
* src/cp.c (main): Add support for --update=none-fail to provide the
functionality of diagnosing files in the destination,
and exiting with failure status.
(usage): Mark -n as deprecated.
* src/mv.c: Likewise.
* src/copy.h: Add UPDATE_NONE_FAIL definition.
* src/system.h (emit_update_parameters_note): Add --update=none-fail
description.
* doc/coreutils.texi (cp invocation): Likewise.
Also mention why -n is deprecated.
* tests/mv/update.sh: Add a test case, including precedence
with -n and other --update options.
* tests/cp/cp-i.sh: Verify that --backup and --update=none{,-fail}
are mutually exclusive.
* tests/mv/mv-n.sh: Likewise.
* NEWS: Mention the new feature.
Addresses https://bugs.gnu.org/62572
When recursively copying files into OS trees, it often happens that
some subdirectory of the source directory is a symlink in the target
directory. Currently, cp will fail in that scenario with the error:
"cannot overwrite non-directory %s with directory %s"
However, we'd like cp in this scenario to follow the destination
directory symlink and copy the files into the symlinked directory
instead. Let's support this by adding a new option
--keep-directory-symlink that makes cp follow destination directory
symlinks.
We name the option --keep-directory-symlink to keep consistent with
tar which has the same option with the same effect.
* doc/coreutils.texi (cp invocation): Describe the new option.
* src/copy.h: Add the new setting.
* src/copy.h: Adjust to follow symlinks if setting enabled.
* src/cp.c (usage): Describe the new option.
(main): Accept the new option.
* tests/cp/keep-directory-symlink.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the new feature.
ulimit -v is generally not supported with ASAN, giving errors like:
"ReserveShadowMemoryRange failed while trying to map 0x... bytes.
Perhaps you're using ulimit -v"
* tests/cp/link-heap.sh: Mention ASAN as a possible reason for skipping.
* tests/csplit/csplit-heap.sh: Likewise.
* tests/cut/cut-huge-range.sh: Likewise.
* tests/dd/no-allocate.sh: Likewise.
* tests/printf/printf-surprise.sh: Likewise.
* tests/rm/many-dir-entries-vs-OOM.sh: Likewise.
* tests/head/head-c.sh: Only skip the part of the test needing ulimit.
* tests/split/line-bytes.sh: Likewise.
Update to latest gnulib with new copyright year.
Run "make update-copyright" and then...
* gnulib: Update included in this commit as copyright years
are the only change from the previous gnulib commit.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Manually update copyright year,
until we fully sync with gnulib at a later stage.
* tests/sample-test: Adjust to use the single most recent year.
Do not perform SELinux context translation for operations not involving
user input or output. Context translation converts MCS/MLS labels into
human readable form, which is useful for user facing applications like
ls(1) or the --context=CTX argument of cp(1).
* src/copy.c (set_process_security_ctx): Use raw selinux variants.
* src/install.c (need_copy): Likewise.
(setdefaultfilecon): Likewise.
* src/selinux.c (computecon): Likewise.
(defaultcon): Likewise.
* tests/cp/no-ctx.sh: Add raw variants to preload lib.
* NEWS: Mention the improvement.
Most of this just affects commentary and documentations. The only
significant behavior change is translating author names via
proper_name_lite rather than proper_name_utf8, or not translating
them at all. proper_name_lite is good enough for coreutils and
avoids the bloat that had coreutils not using Gnulib proper_name.
* bootstrap.conf (gnulib_modules): Use propername-lite instead
of propername.
(XGETTEXT_OPTIONS): Look for proper_name_lite instead of for
proper_name_utf8.
* cfg.mk (local-checks-to-skip): Remove
sc_proper_name_utf8_requires_ICONV, since we no longer use
proper_name_utf8.
(old_NEWS_hash): Update.
(sc_check-I18N-AUTHORS): Remove; no longer needed.
* src/cp.c (main): Set default reflink mode appropriately
with --sparse=never.
* src/copy.c (infer_scantype): Add a comment to related code.
* tests/cp/sparse-2.sh: Add a test case.
* NEWS: Mention the bug.
* src/cp.c (re_protect): Ensure copy_acl() is passed an absolute path.
* tests/cp/cp-parents.sh: Add a test case.
* NEWS: Mention the bug.
Fixes https://bugs.gnu.org/63245
Since skipping of files is central to the operation of -i and -u,
and with -u one may be updating few files out of many,
reinstate the verbosity of this functionality as it was before 9.3.
* src/copy.c (copy_internal): Only output "skipped" message
with --debug. Also adjust so message never changes with --debug.
* tests/cp/cp-i.sh: Adjust accordingly.
* tests/mv/mv-n.sh: Likewise.
* tests/cp/debug.sh: Add explicit test case for message.
* NEWS: Mention the change in behavior.
When run under QEmu emulation emulated /proc files have
unstable inode numbers.
* tests/cp/proc-short-read.sh: Skip if unstable inode numbers detected.
* tests/cp/sparse-2.sh: Don't depend on the copy taking
<= allocation of the source. Instead leverage --debug
to check that zero detection is being enabled.
* NEWS: Mention the change in behavior to issue a "not replaced"
error diagnostic with -n, and the "skipped" message with -v.
* src/copy.c (copy_internal): Adjust to output the "skipped" messages
depending on -i, -n, -u.
* tests/cp/cp-i.sh: Adjust accordingly.
* tests/mv/mv-n.sh: Likewise.
* gnulib: Reference the latest gnulib including the
fix to the backupfile module in commit 94496522.
* tests/cp/backup-dir.sh: Add a test to ensure
we rename appropriately when backing up through subdirs.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/62607
Upcomming gnulib changes may disable SEEK_HOLE
even if the system supports it, so dynamically
check if we've SEEK_HOLE enabled.
* init.cfg (seek_data_capable_): SEEK_DATA may be disabled in the build
if the system support is deemed insufficient, so also use `cp --debug`
to determine if it's enabled.
* tests/cp/sparse-2.sh: Adjust to a more general diagnostic.
* tests/cp/sparse-extents-2.sh: Likewise.
* tests/cp/sparse-extents.sh: Likewise.
* tests/cp/sparse-perf.sh: Likewise.
How a file is copied is dependent on the sparseness of the file,
what file system it is on, what file system the destination is on,
the attributes of the file, and whether they're being copied or not.
Also the --reflink and --sparse options directly impact the operation.
Given it's hard to reason about the combination of all of the above,
the --debug option is useful for users to directly identify if
copy offloading, reflinking, or sparse detection are being used.
It will also be useful for tests to directly query if
these operations are supported.
The new output looks as follows:
$ src/cp --debug src/cp file.sparse
'src/cp' -> 'file.sparse'
copy offload: yes, reflink: unsupported, sparse detection: no
$ truncate -s+1M file.sparse
$ src/cp --debug file.sparse file.sparse.cp
'file.sparse' -> 'file.sparse.cp'
copy offload: yes, reflink: unsupported, sparse detection: SEEK_HOLE
$ src/cp --reflink=never --debug file.sparse file.sparse.cp
'file.sparse' -> 'file.sparse.cp'
copy offload: avoided, reflink: no, sparse detection: SEEK_HOLE
* doc/coreutils.texi (cp invocation): Describe the --debug option.
(mv invocation): Likewise.
(install invocation): Likewise.
* src/copy.h: Add a new DEBUG member to cp_options, to control
whether to output debug info or not.
* src/copy.c (copy_debug): A new global structure to
unconditionally store debug into from the last copy_reg operations.
(copy_debug_string, emit_debug): New functions to print debug info.
* src/cp.c: if ("--debug") x->debug=true;
* src/install.c: Likewise.
* src/mv.c: Likewise.
* tests/cp/debug.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the new feature.
This reverts the previous change, so that when a file
is skipped due to -u, this is not considered a failure.
* doc/coreutils.texi: Document this.
* src/copy.c (copy_internal): If --update says to skip,
treat this as success instead of failure.
* tests/mv/update.sh, tests/cp/slink-2-slink.sh:
Revert previous change, to match reverted behavior.
* NEWS, doc/coreutils.texi: Document this.
* src/copy.c (copy_internal):
* src/ln.c (do_link): Return false when skipping action due to
--interactive or --no-clobber.
* tests/cp/cp-i.sh, tests/cp/preserve-link.sh:
* tests/cp/slink-2-slink.sh, tests/mv/i-1.pl, tests/mv/i-5.sh:
* tests/mv/mv-n.sh, tests/mv/update.sh:
Adjust expectations of exit status to match revised behavior.
Update to latest gnulib with new copyright year.
Run "make update-copyright" and then...
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Manually update copyright year,
until we fully sync with gnulib at a later stage.
* tests/sample-test: Adjust to use the single most recent year.
* tests/cp/proc-short-read.sh: Kernel on ARMv7 Processor rev 3 (v7l)
spells it "BogoMIPS", so allow any capitalization. Patch from
Zach van Rijn in <https://bugs.gnu.org/60339>.
commit v9.0-66-ge2daa8f79 introduced an issue, for example
where cp could hang when overwriting a destination fifo,
when it would try to open() the fifo on systems
like Solaris 10 that didn't support the O_DIRECTORY flag.
This is still racy on such systems, but only in the
case where a directory is replaced by a fifo in
the small window between stat() and open().
* src/system.h (target_directory_operand): On systems without
O_DIRECTORY, ensure the file is a directory before attempting to open().
* tests/cp/special-f.sh: Protect cp with timeout(1),
as cp was seen to hang when trying to overwrite an existing fifo.
* NEWS: Mention the bug fix.
When copying to a directory, use functions like openat to access
the destination files, when such functions are available. This
should be more efficient and should avoid some race conditions.
* bootstrap.conf (gnulib_modules): Add areadlinkat-with-size,
fchmodat, fchownat, mkdirat, mkfifoat, utimensat.
* src/copy.c (lchown) [!HAVE_LCHOWN]:
* src/copy.c, src/system.h (rpl_mkfifo, mkfifo) [!HAVE_MKFIFO]:
Remove. All uses removed.
(utimens_symlink): Remove; we shouldn’t have to worry about
those obsolete systems any more. All uses replaced by utimensat.
* src/copy.c (copy_dir, set_owner, fchmod_or_lchmod, copy_reg)
(same_file_ok, writable_destination, overwrite_ok, abandon_move)
(create_hard_link, src_is_dst_backup, copy_internal, copy):
* src/cp.c (make_dir_parents_private, re_protect):
New args for directory-relative names. All uses changed.
Continue to pass full names as needed, for diagnostics and for
lower-level functions like qset_acl that do not support
directory-relative names.
* src/copy.c (copy_reg): Prefer readlinkat to lstatat for merely
checking whether a file is a symlink, to avoid EOVERFLOW issues.
(subst_suffix): New function.
(create_hard_link): Accept a null SRC_NAME as meaning that if it
is needed it needs to be constructed from SRC_RELNAME, DST_NAME,
and DST_RELNAME.
(source_is_dst_backup): Use subst_suffix instead of doing it by hand.
(copy_internal): Remember and use directory-relative names instead
of full names.
* src/cp.c (lchown) [!HAVE_LCHOWN]: Remove. All uses removed.
(must_be_working_directory): New function.
(target_directory_operand): Simply take file name as arg,
and return a file descriptor or negative number on failure;
open with O_DIRECTORY to obtain any file descriptor.
All uses changed.
(target_dirfd_valid): New function.
(do_copy): Use these new functions to obtain a file descriptor
for any target directory, and use directory-relative names
for that directory.
(main): Omit no-longer-needed stat when --target-directory,
as do_copy now does this.
* src/ln.c (O_PATHSEARCH): Move from here ...
* src/system.h: ... to here.
* tests/cp/fail-perm.sh: Adjust to change in diagnostic wording,
and add a test for --no-target-directory.
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
This fixes a bug that I introduced in
2006-12-06T19:44:08Z!eggert@cs.ucla.edu.
* src/copy.c (USE_XATTR): New macro.
(copy_reg): Use it to help the compiler. Prefer open u+w to a
later chmod u=rw; u+r isn’t needed for xattr. For the later u-r,
do only one (or zero) chmod calls instead of two (or one).
In the last chmod, respect the umask instead of ignoring it.
* tests/cp/preserve-mode.sh: Test for the bug.
* init.cfg (seek_data_capable_): Add a timeout to ensure failure for
slow lseek(...SEEK_DATA) calls (even if that syscall isn't interrupted).
* tests/cp/sparse-perf.sh: Run the SEEK_DATA check on the
1TiB empty file to exclude both FreeBSD 9.1 which takes 35s,
and ZFS which requires a delay of about 5s between file creation
and use of SEEK_DATA to correctly determine it's empty (return ENXIO).
Also remove the stat size checks as they invalidate the test
due to cp never writing data due to it being always zeros,
and thus converted to holes in the output.
* tests/cp/sparse-perf.sh: Avoid the case where
we saw SEEK_DATA take 35s to return a result
against a 1TB sparse file. This happened on
a FreeBSD 9.1 VM at least.
Reported by Nelson H. F. Beebe.
This better tests the SEEK_HOLE logic which
replaced the original fiemap hole identification logic.
Also it avoids a false failure in sparse-2.sh
on reflink supporting file systems, where we
try to correlate the file sizes produced by cp and dd.
* tests/cp/sparse-2.sh: s/cp/cp --reflink=never/
* tests/cp/sparse-extents-2.sh: Likewise.
* tests/cp/sparse-extents.sh: Likewise.
* tests/cp/sparse-perf.sh: Likewise.
* tests/cp/sparse.sh: Likewise.
Fixes https://github.com/coreutils/coreutils/issues/54
In documentation and comments, don’t assume that secondary storage
devices are disk devices. Similarly, don’t assume that main memory
uses magnetic cores, which became obsolete in the 1970s.
* src/du.c (usage):
* src/ls.c (usage):
* src/shred.c (usage): Reword to avoid “disk” in usage messages.
* tests/cp/sparse-2.sh: Double check cp --sparse=always,
with dd conv=sparse, in the case where the former didn't
create a sparse file. Now that this test is being newly run
on macos, we're seeing a failure due to seek() not creating
holes on apfs unless the size is >= 16MiB.
fiemap is no longer the default copy implementation,
so check for SEEK_DATA support instead as that's preferred.
This will ensure better test coverage on systems without fiemap.
* init.cfg: Replace fiemap_capable_ with seek_data_capable_.
This is best supported with python 3 so prefer that.
* tests/seek-data-capable: A new test script checking for
SEEK_DATA support on the passed file name,
called from seek_data_capable_.
* tests/fiemap-capable: Remove no longer used probing script.
* tests/cp/fiemap-perf.sh: Renamed to tests/cp/sparse-perf.sh
* tests/cp/fiemap-2.sh: Renamed to tests/cp/sparse-2.sh
* tests/cp/fiemap-extents.sh: Renamed to tests/cp/sparse-extents.sh
* tests/cp/sparse-fiemap.sh: Renamed to tests/cp/sparse-extents-2.sh
* tests/cp/fiemap-FMR.sh: Renamed to tests/cp/copy-FMR.sh
* tests/local.mk: Reference the renamed tests.
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
* tests/cp/fiemap-FMR.sh: Avoid FICLONE ioctl,
which would avoid the point of the test (fiemap testing).
Also it avoids a valgrind bug with this ioctl:
https://bugs.kde.org/show_bug.cgi?id=397605
* tests/cp/proc-short-read.sh: Switch to using /proc/cpuinfo,
rather than /proc/kallsyms which was seen to vary in some cases.
Fixes https://bugs.gnu.org/39357
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
