When copying to a directory, use functions like openat to access
the destination files, when such functions are available. This
should be more efficient and should avoid some race conditions.
* bootstrap.conf (gnulib_modules): Add areadlinkat-with-size,
fchmodat, fchownat, mkdirat, mkfifoat, utimensat.
* src/copy.c (lchown) [!HAVE_LCHOWN]:
* src/copy.c, src/system.h (rpl_mkfifo, mkfifo) [!HAVE_MKFIFO]:
Remove. All uses removed.
(utimens_symlink): Remove; we shouldn’t have to worry about
those obsolete systems any more. All uses replaced by utimensat.
* src/copy.c (copy_dir, set_owner, fchmod_or_lchmod, copy_reg)
(same_file_ok, writable_destination, overwrite_ok, abandon_move)
(create_hard_link, src_is_dst_backup, copy_internal, copy):
* src/cp.c (make_dir_parents_private, re_protect):
New args for directory-relative names. All uses changed.
Continue to pass full names as needed, for diagnostics and for
lower-level functions like qset_acl that do not support
directory-relative names.
* src/copy.c (copy_reg): Prefer readlinkat to lstatat for merely
checking whether a file is a symlink, to avoid EOVERFLOW issues.
(subst_suffix): New function.
(create_hard_link): Accept a null SRC_NAME as meaning that if it
is needed it needs to be constructed from SRC_RELNAME, DST_NAME,
and DST_RELNAME.
(source_is_dst_backup): Use subst_suffix instead of doing it by hand.
(copy_internal): Remember and use directory-relative names instead
of full names.
* src/cp.c (lchown) [!HAVE_LCHOWN]: Remove. All uses removed.
(must_be_working_directory): New function.
(target_directory_operand): Simply take file name as arg,
and return a file descriptor or negative number on failure;
open with O_DIRECTORY to obtain any file descriptor.
All uses changed.
(target_dirfd_valid): New function.
(do_copy): Use these new functions to obtain a file descriptor
for any target directory, and use directory-relative names
for that directory.
(main): Omit no-longer-needed stat when --target-directory,
as do_copy now does this.
* src/ln.c (O_PATHSEARCH): Move from here ...
* src/system.h: ... to here.
* tests/cp/fail-perm.sh: Adjust to change in diagnostic wording,
and add a test for --no-target-directory.
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
This fixes a bug that I introduced in
2006-12-06T19:44:08Z!eggert@cs.ucla.edu.
* src/copy.c (USE_XATTR): New macro.
(copy_reg): Use it to help the compiler. Prefer open u+w to a
later chmod u=rw; u+r isn’t needed for xattr. For the later u-r,
do only one (or zero) chmod calls instead of two (or one).
In the last chmod, respect the umask instead of ignoring it.
* tests/cp/preserve-mode.sh: Test for the bug.
* init.cfg (seek_data_capable_): Add a timeout to ensure failure for
slow lseek(...SEEK_DATA) calls (even if that syscall isn't interrupted).
* tests/cp/sparse-perf.sh: Run the SEEK_DATA check on the
1TiB empty file to exclude both FreeBSD 9.1 which takes 35s,
and ZFS which requires a delay of about 5s between file creation
and use of SEEK_DATA to correctly determine it's empty (return ENXIO).
Also remove the stat size checks as they invalidate the test
due to cp never writing data due to it being always zeros,
and thus converted to holes in the output.
* tests/cp/sparse-perf.sh: Avoid the case where
we saw SEEK_DATA take 35s to return a result
against a 1TB sparse file. This happened on
a FreeBSD 9.1 VM at least.
Reported by Nelson H. F. Beebe.
This better tests the SEEK_HOLE logic which
replaced the original fiemap hole identification logic.
Also it avoids a false failure in sparse-2.sh
on reflink supporting file systems, where we
try to correlate the file sizes produced by cp and dd.
* tests/cp/sparse-2.sh: s/cp/cp --reflink=never/
* tests/cp/sparse-extents-2.sh: Likewise.
* tests/cp/sparse-extents.sh: Likewise.
* tests/cp/sparse-perf.sh: Likewise.
* tests/cp/sparse.sh: Likewise.
Fixes https://github.com/coreutils/coreutils/issues/54
In documentation and comments, don’t assume that secondary storage
devices are disk devices. Similarly, don’t assume that main memory
uses magnetic cores, which became obsolete in the 1970s.
* src/du.c (usage):
* src/ls.c (usage):
* src/shred.c (usage): Reword to avoid “disk” in usage messages.
* tests/cp/sparse-2.sh: Double check cp --sparse=always,
with dd conv=sparse, in the case where the former didn't
create a sparse file. Now that this test is being newly run
on macos, we're seeing a failure due to seek() not creating
holes on apfs unless the size is >= 16MiB.
fiemap is no longer the default copy implementation,
so check for SEEK_DATA support instead as that's preferred.
This will ensure better test coverage on systems without fiemap.
* init.cfg: Replace fiemap_capable_ with seek_data_capable_.
This is best supported with python 3 so prefer that.
* tests/seek-data-capable: A new test script checking for
SEEK_DATA support on the passed file name,
called from seek_data_capable_.
* tests/fiemap-capable: Remove no longer used probing script.
* tests/cp/fiemap-perf.sh: Renamed to tests/cp/sparse-perf.sh
* tests/cp/fiemap-2.sh: Renamed to tests/cp/sparse-2.sh
* tests/cp/fiemap-extents.sh: Renamed to tests/cp/sparse-extents.sh
* tests/cp/sparse-fiemap.sh: Renamed to tests/cp/sparse-extents-2.sh
* tests/cp/fiemap-FMR.sh: Renamed to tests/cp/copy-FMR.sh
* tests/local.mk: Reference the renamed tests.
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
* tests/cp/fiemap-FMR.sh: Avoid FICLONE ioctl,
which would avoid the point of the test (fiemap testing).
Also it avoids a valgrind bug with this ioctl:
https://bugs.kde.org/show_bug.cgi?id=397605
* tests/cp/proc-short-read.sh: Switch to using /proc/cpuinfo,
rather than /proc/kallsyms which was seen to vary in some cases.
Fixes https://bugs.gnu.org/39357
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
* tests/cp/cp-a-selinux.sh: Use 'skip_' rather than the probably
undefined 'skip'.
* tests/du/2g.sh: Likewise.
* tests/install/install-Z-selinux.sh: Likewise.
* tests/misc/chcon.sh: Likewise.
* tests/misc/selinux.sh: Likewise.
* tests/mkdir/restorecon.sh: Likewise.
* cfg.mk (sc_prohibit-skip): A new syntax check to catch the issue.
This mode is currently the default, but most if not all users of
reflink-capable filesystems want --reflink=auto, which is often
encapsulated into an alias. Adding --reflink=never allows overriding
such an alias.
* doc/coreutils.texi (cp invocation): Describe the new option.
* src/cp.c: Support --reflink=never.
* tests/cp/reflink-auto.sh: Add a test case.
* NEWS: Mention the new feature.
This issue was introduced in commit v8.19-145-g24ebca6
* src/copy.c (copy_internal): With --no-preserve=mode,
only reset permissions for newly created files.
(copy_reg): Likewise.
* NEWS: Mention the fix.
* tests/cp/preserve-mode.sh: Add a test case.
Fixes https://bugs.gnu.org/31675
* src/copy.c (copy_internal): Don't fail immediately upon
getting ELOOP when running stat() on the destination,
rather proceeding if -f specified, allowing the link
to be removed. If the loop is not in the final component
of the destination path, we still fail but at the
subsequent unlink() stage.
* doc/coreutils.texi (cp invocation): Adjust wording to say
that --force doesn't work with dangling links, rather than
all links that can't be traversed.
* tests/cp/thru-dangling.sh: Add a test case.
* NEWS: Mention the change in behavior.
Discussed in https://bugs.gnu.org/31335
Ensure this _does_ recreate the symlink
Given "path1" and "path2" are on different devices.
$ touch "path1/file"
$ cd path2/; ln -s path1/file
$ cp -dsf path1/file .
Ensure this does _not_ overwrite file
$ touch file
$ ln -s file l1
$ cp -sf l1 file
* src/copy.c (same_file_ok): Remove device ids from consideration,
instead deferring to future EXDEV with --link or allowing
the first case above to work.
Also ensure that we do not exist this function too early,
when the destination file is not a symlink, which protects
against the second case.
* tests/cp/cross-dev-symlink.sh: Add a test for the first case.
* tests/cp/same-file.sh: Add a test for the second case above.
* NEWS: Mention the bug fixes.
* THANKS.in: Mention the reporters who also analyzed the code.
Fixes https://bugs.gnu.org/31364
* src/cp.c (target_directory_operand): Allow through inaccessible
arguments with -f or --remove.
* doc/coreutils.texi (cp invocation): Clarify that -f doesn't directly
impact the removal of non-traversable symlinks.
* tests/cp/dir-rm-dest.sh: Test the new behavior.
* tests/cp/thru-dangling.sh: Enforce -f behavior wrt symlinks.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/31335
This issue was introduced in commit v8.19-145-g24ebca6
* src/copy.c (copy_internal): When setting default permissions
to use with --no-preserve=mode, only set executable bits for
directories or sockets.
* NEWS: Mention the fix.
* tests/cp/preserve-mode.sh: Add a test case.
Fixes https://bugs.gnu.org/30534
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
* tests/cp/preserve-mode.sh: This was the only use of awk,
which may not be available on the system resulting
in an ineffective test. Also the permissions bits for
directories were not being checked at all.
On some setups the root:object_r:tmp_t context is invalid.
This does indicate a limitation in the test framework,
but for now we'll relax this to skipping the tests.
The tests still run on a Fedora 25 system for example.
* tests/cp/cp-a-selinux.sh: Upon chcon error, skip rather than ERROR.
* tests/install/install-Z-selinux.sh: Likewise.
* tests/misc/chcon.sh: Likewise.
* tests/misc/runcon-no-reorder.sh: Likewise.
* tests/misc/selinux.sh: Likewise.
* tests/mkdir/restorecon.sh: Likewise.
Suggested by Kamil Dudka in:
http://lists.gnu.org/archive/html/coreutils/2017-07/msg00072.html
* NEWS: Document the changed nature of the fix.
* doc/coreutils.texi, tests/cp/backup-is-src.sh:
* tests/mv/backup-is-src.sh: Revert previous change.
* src/copy.c (source_is_dst_backup): New function.
(copy_internal): Use it. Fail instead of falling back on numbered
backups when it looks like the backup will overwrite the source.
Although this reintroduces a race, it's more compatible with
previous behavior.
* NEWS, doc/coreutils.texi (Backup options): Document the change.
* bootstrap.conf (gnulib_modules): Add backup-rename.
* src/copy.c (copy_internal): Silently switch to numbered backups
if a simple backup might lose data. Use backup_file_rename
to avoid races with numbered backups.
* tests/cp/backup-is-src.sh, tests/mv/backup-is-src.sh:
Adjust to match new behavior.
* src/copy.c (set_process_security_ctx, set_file_security_ctx):
Export for use in cp.c.
* src/copy.h: Likewise.
* src/cp.c (make_dir_parents_private): Call the exported functions
to set the security context for new and updated directories.
* tests/cp/cp-a-selinux.sh: Add a test case.
Fixes http://bugs.gnu.org/25378
If the file B already exists, commands like 'ln -f A B' and
'cp -fl A B' no longer remove B before creating the new link.
Instead, they arrange for the new link to replace B atomically.
This should fix a race condition reported by Mike Crowe (Bug#25680).
* NEWS, doc/coreutils.texi (cp invocation, ln invocation):
Document this.
* bootstrap.conf (gnulib_modules): Add symlinkat.
* src/copy.c, src/ln.c: Include force-link.h.
* src/copy.c (same_file_ok): It's also OK to remove a destination
symlink when creating symbolic links, or when the source and
destination are on the same file system and when creating hard links.
* src/copy.c (create_hard_link, copy_internal):
* src/ln.c (do_link):
Rewrite using force_linkat and force_symlinkat, to close a window
where the destination temporarily does not exist.
* src/cp.c (main): Do not set x.unlink_dest_before_opening
merely because we are in link-creation mode.
* src/force-link.c, src/force-link.h: New files.
* src/local.mk (copy_sources, src_ln_SOURCES): Add them.
* tests/cp/same-file.sh: Adjust test case to match fixed behavior.
A lot of this is converting http: to https:.
Also, gmane went away, so remove URLs that no longer work and
are not easy to figure out what they were.
Some of this stuff is so old that it no longer matters anyway.
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
When running "make check" on a Linux system running SELinux with a
non-MLS policy, tests/mkdir/restorecon.sh test fails with:
chcon: invalid context: root:object_r:tmp_t:s0: Invalid argument
Indeed in such a configuration, contexts cannot have ":s0" suffix.
* init.cfg (get_selinux_type): Refactor this function to here
from various tests. Update to work with a non-MLS policy.
(mls_enabled_): A new function to detect if MLS is enabled.
(skip_if_mcstransd_is_running_): Update to not skip when
MLS is not enabled.
* tests/mkdir/restorecon.sh: Use a valid non-MLS context when needed.
* tests/install/install-Z-selinux.sh: Likewise.
* tests/cp/cp-a-selinux.sh: Likewise.
* tests/misc/selinux.sh: Likewise.
* tests/misc/chcon.sh: Skip if non-MLS as --range used throughout.
Fixes http://bugs.gnu.org/22631
* src/copy.h (cp_options): Add a new flag for install(1).
* src/copy.c (copy_internal): For cp, also output remediation
advice which also indicates why directories aren't copied by default.
The message is unchanged for install(1).
* src/cp.c (cp_option_init): Init install_mode to false.
* src/mv.c (cp_option_init): Likewise.
* src/install.c (cp_option_init): Init install_mode to true.
* tests/install/basic-1.sh: Add a test case.
* tests/cp/link-deref.sh: Adjust test case.
Fixes http://bugs.gnu.org/24958
* src/cp.c (make_dir_parents_private): Use default permissions for
created directories when --no-preserve=mode is specified.
* tests/cp/cp-parents.sh: Add a test case.
* NEWS: Mention the fix.
Fixes http://bugs.gnu.org/24251
* tests/cp/parent-perm-race.sh: This new race introduced in
commit v8.25-5-g632eda5 is quite hard to hit, but is due to
`ls > fifo` doing write()/close()/exit() once `cp` has
open() the source fifo. Then the subsequent comparison of the
destination file may fail due to the file being missing or empty.
Previously `ls` generated output that was independent of `cp`.
Now we must wait for `cp` to finish before inspecting the
destination file that it wrote.
* tests/cp/parent-perm-race.sh: Add timeouts so that the test does
not wait forever on GNU/Hurd. This does not fix the underlying
bug but at least lets the tests make progress.
Problem reported by Nelson H.F. Beebe in: http://bugs.gnu.org/22624
Also, check that 'cp' actually copies the data; this is a better
test anyway, and simplifies the test code.
Also, avoid unlikely race if 'pid' is set in the environment.
* tests/cp/preserve-slink-time.sh: Add a delay between the
ln and the cp so that there is enough difference between
the timestamps so GPFS won't discard the update.
Reported by Assaf Gordon.
* tests/cp/fiemap-extents.sh: Support RHEL6 fallocate
which doesn't support IEC suffixes like "MiB" on numbers.
Also add some extra framework_failure_ protections.
Reported by Assaf Gordon.
commit v8.23-31-g90aa291 failed to consider this case,
where the previous rename has failed, thus causing the
following to remove the specified directory:
mv dir dir dir
* src/copy.c (copy_internal): Assume this rename attempt has
succeeded, as a previous failure will already have been handled,
and we don't want to remove the source directory in this case.
* tests/cp/duplicate-sources.sh: Consolidate this test file to...
* tests/mv/dup-source.sh: ...here. Add test cases for same
source and dest.
* tests/local.mk: Remove the consolidated test.
* NEWS: Mention the bug fix.
Reported at https://bugzilla.redhat.com/1297464
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
fallocate can allocate extents beyond EOF via FALLOC_FL_KEEP_SIZE.
Where there is a gap (hole) between the extents, and EOF is within
that gap, the final hole wasn't reproduced, resulting in silent
data corruption in the copied file (size too small).
* src/copy.c (extent_copy): Ensure we don't process extents
beyond the apparent file size, since processing and allocating
those is not currently supported.
* tests/cp/fiemap-extents.sh: Renamed from tests/cp/fiemap-empty.sh
and re-enable parts checking the extents at and beyond EOF.
* tests/local.mk: Reference the renamed test.
* NEWS: Mention the bug fix.
Fixes http://bugs.gnu.org/21790
programs may not be built due to missing system dependencies,
or any program can be excluded at configure time with
--enable-no-install-program. So ensure we're not testing the
system versions in these cases.
* init.cfg (print_ver_): Call require_built_ first.
* tests/misc/tty-eof.pl: Skip programs not built.
* tests/Coreutils.pm (run_tests): Likewise.
* tests/misc/ls-misc.pl: Use 'env test' rather than abs path.
* tests/misc/test-diag.pl: Likewise.
* tests/local.mk: Adjust include order for dependencies.
* tests/misc/arch.sh: Remove redundant calls to require_built_.
* tests/misc/chroot-fail.sh: Likewise.
* tests/misc/groups-dash.sh: Likewise.
* tests/misc/groups-version.sh: Likewise.
* tests/misc/stdbuf.sh: Likewise.
* tests/cp/acl.sh: Remove problematic call to print_ver_ [gs]etfacl.
* tests/mv/acl.sh: Likewise.
* cfg.mk (sc_env_test_dependencies): A new syntax check to enforce
specifying dependencies with print_ver_ for programs
specified through the env command.
* du/bigtime.sh: Add new print_ver_ dependencies.
* du/max-depth.sh: Likewise.
* dd/ascii.sh: Likewise.
* tests/ls/capability.sh: Likewise.
* tests/ls/root-rel-symlink-color.sh: Likewise.
* tests/misc/chroot-fail.sh: Likewise.
* tests/misc/readlink-fp-loop.sh: Likewise.
* tests/misc/sort-debug-keys.sh: Likewise.
* tests/readlink/can-e.sh: Likewise.
* tests/readlink/can-f.sh: Likewise.
* tests/readlink/can-m.sh: Likewise.
* tests/tail-2/inotify-race.sh: Likewise.
* tests/tail-2/inotify-race2.sh: Likewise.
* tests/touch/no-create-missing.sh: Likewise.
* tests/touch/no-dereference.sh: Likewise.
* tests/misc/printenv.sh: Tweak to avoid syntax check trigger.
* tests/misc/help-version.sh: Likewise.
* tests/misc/yes.sh: Likewise.
* tests/misc/printf-quote.sh: Use previously unused $prog.
* configure.ac (EXTRA_MANS): Add $gl_no_install_prog to the list
so that check-x-vs-1 syntax check is satisfied.
