Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #222102 > unrolled thread

unexpected behavior of cp and mv

Back to article view | Back to linux.debian.user

Contents

  unexpected behavior of cp and mv Alberto Sentieri <22t@tripolho.com> - 2020-04-29 19:10 +0200
    Re: unexpected behavior of cp and mv Roberto C. Sánchez <roberto@debian.org> - 2020-04-29 19:20 +0200
      Re: Re: unexpected behavior of cp and mv Alberto Sentieri <22t@tripolho.com> - 2020-04-29 20:50 +0200
      Re: Re: unexpected behavior of cp and mv Alberto Sentieri <22t@tripolho.com> - 2020-04-29 21:20 +0200
        Re: unexpected behavior of cp and mv David Wright <deblis@lionunicorn.co.uk> - 2020-04-30 03:50 +0200
          Re: unexpected behavior of cp and mv Alberto Sentieri <22t@tripolho.com> - 2020-04-30 06:20 +0200
    Re: unexpected behavior of cp and mv Charles Curley <charlescurley@charlescurley.com> - 2020-04-29 21:20 +0200
      Re: unexpected behavior of cp and mv Alberto Sentieri <22t@tripolho.com> - 2020-04-29 22:10 +0200
        Re: unexpected behavior of cp and mv "Thomas Schmitt" <scdbackup@gmx.net> - 2020-04-29 22:20 +0200
          Re: unexpected behavior of cp and mv Alberto Sentieri <22t@tripolho.com> - 2020-04-29 22:30 +0200
          Re: unexpected behavior of cp and mv Alberto Sentieri <22t@tripolho.com> - 2020-04-29 23:00 +0200
            Re: unexpected behavior of cp and mv "Thomas Schmitt" <scdbackup@gmx.net> - 2020-04-29 23:30 +0200
              Re: unexpected behavior of cp and mv Alberto Sentieri <22t@tripolho.com> - 2020-04-30 06:40 +0200
                Re: unexpected behavior of cp and mv "Thomas Schmitt" <scdbackup@gmx.net> - 2020-04-30 12:40 +0200
                  Re: unexpected behavior of cp and mv Alberto Sentieri <22t@tripolho.com> - 2020-04-30 16:20 +0200
              Re: unexpected behavior of cp and mv Alberto Sentieri <22t@tripolho.com> - 2020-04-30 18:50 +0200
                Re: unexpected behavior of cp and mv "Thomas Schmitt" <scdbackup@gmx.net> - 2020-04-30 19:40 +0200
                  Re: unexpected behavior of cp and mv Alberto Sentieri <22t@tripolho.com> - 2020-04-30 20:10 +0200
                  Re: unexpected behavior of cp and mv Alberto Sentieri <22t@tripolho.com> - 2020-04-30 21:50 +0200
                    Re: unexpected behavior of cp and mv The Wanderer <wanderer@fastmail.fm> - 2020-04-30 22:00 +0200
                      Re: unexpected behavior of cp and mv Alberto Sentieri <22t@tripolho.com> - 2020-04-30 22:10 +0200
                      Re: unexpected behavior of cp and mv Alberto Sentieri <22t@tripolho.com> - 2020-04-30 23:40 +0200
                Re: unexpected behavior of cp and mv David Wright <deblis@lionunicorn.co.uk> - 2020-04-30 20:00 +0200
            Re: unexpected behavior of cp and mv songbird <songbird@anthive.com> - 2020-04-30 00:00 +0200
              Re: unexpected behavior of cp and mv Alberto Sentieri <22t@tripolho.com> - 2020-04-30 06:10 +0200

Page 1 of 2  [1] 2  Next page →

#222102 — unexpected behavior of cp and mv

cp and mv are not preserving the file timestamps when copying from a 
ext4 file system to a smb file system.

I am running cp and mv on:
$ lsb_release -a
No LSB modules are available.
Distributor ID:    Debian
Description:    Debian GNU/Linux 10 (buster)
Release:    10
Codename:    buster

SMB is running on:
$ lsb_release -a
No LSB modules are available.
Distributor ID:    Debian
Description:    Debian GNU/Linux 9.12 (stretch)
Release:    9.12
Codename:    stretch

I am using "cp -pi" to copy the files and "mv -i" to move them.

If I run cp or mv on a Debian stretch, time stamps will be correct on 
the SMB file system.

Buster gnome3 file manager will copy or move the files without problems, 
I mean, they will preserve time stamps.

This started happening since I upgraded my workstation to buster, so the 
problem seems to be in buster. I am mounting the file system on buster 
with the following line added to fstab, which, by the way, is the same I 
was using when my workstation was running stretch:

//192.168.1.2/alberto /mnt/u1/rw cifs 
credentials=/root/pass/alberto.pass,uid=alberto,gid=alberto,rw 0 0

Does anyone knows which package is causing this behavior?

Alberto

[toc] | [next] | [standalone]

#222107

On Wed, Apr 29, 2020 at 12:22:41PM -0400, Alberto Sentieri wrote:
> cp and mv are not preserving the file timestamps when copying from a ext4
> file system to a smb file system.
> 
> I am running cp and mv on:
> $ lsb_release -a
> No LSB modules are available.
> Distributor ID:    Debian
> Description:    Debian GNU/Linux 10 (buster)
> Release:    10
> Codename:    buster
> 
> SMB is running on:
> $ lsb_release -a
> No LSB modules are available.
> Distributor ID:    Debian
> Description:    Debian GNU/Linux 9.12 (stretch)
> Release:    9.12
> Codename:    stretch
> 
> I am using "cp -pi" to copy the files and "mv -i" to move them.
> 
> If I run cp or mv on a Debian stretch, time stamps will be correct on the
> SMB file system.
> 
> Buster gnome3 file manager will copy or move the files without problems, I
> mean, they will preserve time stamps.
> 
> This started happening since I upgraded my workstation to buster, so the
> problem seems to be in buster. I am mounting the file system on buster with
> the following line added to fstab, which, by the way, is the same I was
> using when my workstation was running stretch:
> 
> //192.168.1.2/alberto /mnt/u1/rw cifs
> credentials=/root/pass/alberto.pass,uid=alberto,gid=alberto,rw 0 0
> 
> Does anyone knows which package is causing this behavior?
> 
First, do you see the same behavior if you use /bin/cp and /bin/mv
instead of simply cp and mv?

Regards,

-Roberto

-- 
Roberto C. Sánchez

[toc] | [prev] | [next] | [standalone]

#222112

[Multipart message — attachments visible in raw view] — view raw

Exactly the same behavior for /bin/cp and /bin/mv. I do not have any other cp or mv in my path.

$ sha1sum  /bin/cp /bin/mv
220687a082fb9d0dbb48e9a2b1093cbb4e9de55a  /bin/cp
46e71d67df7eb1c41f8f8c9039f401e242cce94a  /bin/mv


On Wed, Apr 29, 2020 at 12:22:41PM -0400, Alberto Sentieri wrote:
> cp and mv are not preserving the file timestamps when copying from a ext4
> file system to a smb file system.
> 
> I am running cp and mv on:
> $ lsb_release -a
> No LSB modules are available.
> Distributor ID:    Debian
> Description:    Debian GNU/Linux 10 (buster)
> Release:    10
> Codename:    buster
> 
> SMB is running on:
> $ lsb_release -a
> No LSB modules are available.
> Distributor ID:    Debian
> Description:    Debian GNU/Linux 9.12 (stretch)
> Release:    9.12
> Codename:    stretch
> 
> I am using "cp -pi" to copy the files and "mv -i" to move them.
> 
> If I run cp or mv on a Debian stretch, time stamps will be correct on the
> SMB file system.
> 
> Buster gnome3 file manager will copy or move the files without problems, I
> mean, they will preserve time stamps.
> 
> This started happening since I upgraded my workstation to buster, so the
> problem seems to be in buster. I am mounting the file system on buster with
> the following line added to fstab, which, by the way, is the same I was
> using when my workstation was running stretch:
> 
> //192.168.1.2/alberto /mnt/u1/rw cifs
> credentials=/root/pass/alberto.pass,uid=alberto,gid=alberto,rw 0 0
> 
> Does anyone knows which package is causing this behavior?
> 
First, do you see the same behavior if you use /bin/cp and /bin/mv
instead of simply cp and mv?

Regards,

-Roberto

-- 
Roberto C. Sánchez

[toc] | [prev] | [next] | [standalone]

#222116

[Multipart message — attachments visible in raw view] — view raw

When I tried strace with /bin/cp -pi I can see on both commands 
something like this:

utimensat(4, NULL, [{tv_sec=1588174263, tv_nsec=908624390} /* 
2020-04-29T11:31:03.908624390-0400 */, {tv_sec=1486350336, 
tv_nsec=481422339} /* 2017-02-05T22:05:36.481422339-0500 */], 0) = 0

The utimensat is exactly the same for both cp -pi to a ext4 file system 
or to a smb filesystem. So, cp seems to be working as expected. Probably 
the same is true to /bin/mv.

-----------------------------------------------------------------

Exactly the same behavior for /bin/cp and /bin/mv. I do not have any other cp or mv in my path.

$ sha1sum  /bin/cp /bin/mv
220687a082fb9d0dbb48e9a2b1093cbb4e9de55a  /bin/cp
46e71d67df7eb1c41f8f8c9039f401e242cce94a  /bin/mv


On Wed, Apr 29, 2020 at 12:22:41PM -0400, Alberto Sentieri wrote:
> cp and mv are not preserving the file timestamps when copying from a ext4
> file system to a smb file system.
> 
> I am running cp and mv on:
> $ lsb_release -a
> No LSB modules are available.
> Distributor ID:    Debian
> Description:    Debian GNU/Linux 10 (buster)
> Release:    10
> Codename:    buster
> 
> SMB is running on:
> $ lsb_release -a
> No LSB modules are available.
> Distributor ID:    Debian
> Description:    Debian GNU/Linux 9.12 (stretch)
> Release:    9.12
> Codename:    stretch
> 
> I am using "cp -pi" to copy the files and "mv -i" to move them.
> 
> If I run cp or mv on a Debian stretch, time stamps will be correct on the
> SMB file system.
> 
> Buster gnome3 file manager will copy or move the files without problems, I
> mean, they will preserve time stamps.
> 
> This started happening since I upgraded my workstation to buster, so the
> problem seems to be in buster. I am mounting the file system on buster with
> the following line added to fstab, which, by the way, is the same I was
> using when my workstation was running stretch:
> 
> //192.168.1.2/alberto /mnt/u1/rw cifs
> credentials=/root/pass/alberto.pass,uid=alberto,gid=alberto,rw 0 0
> 
> Does anyone knows which package is causing this behavior?
> 
First, do you see the same behavior if you use /bin/cp and /bin/mv
instead of simply cp and mv?

Regards,

-Roberto

-- 
Roberto C. Sánchez

[toc] | [prev] | [next] | [standalone]

#222135

On Wed 29 Apr 2020 at 15:07:52 (-0400), Alberto Sentieri wrote:
> When I tried strace with /bin/cp -pi I can see on both commands
> something like this:
> 
> utimensat(4, NULL, [{tv_sec=1588174263, tv_nsec=908624390} /*
> 2020-04-29T11:31:03.908624390-0400 */, {tv_sec=1486350336,
> tv_nsec=481422339} /* 2017-02-05T22:05:36.481422339-0500 */], 0) = 0
> 
> The utimensat is exactly the same for both cp -pi to a ext4 file
> system or to a smb filesystem. So, cp seems to be working as expected.
> Probably the same is true to /bin/mv.

There's no possibility, is there, that you're reading the file status
time rather than the file modification time? cp -ip/mv will update the
former though not the latter. All three times will be set by touch in
the absence of any options (c and m).

I'm not familiar with cifs/smb protocols and filesystems.

> -----------------------------------------------------------------
> 
> Exactly the same behavior for /bin/cp and /bin/mv. I do not have any other cp or mv in my path.
> 
> $ sha1sum  /bin/cp /bin/mv
> 220687a082fb9d0dbb48e9a2b1093cbb4e9de55a  /bin/cp
> 46e71d67df7eb1c41f8f8c9039f401e242cce94a  /bin/mv
> 
> 
> On Wed, Apr 29, 2020 at 12:22:41PM -0400, Alberto Sentieri wrote:
> > cp and mv are not preserving the file timestamps when copying from a ext4
> > file system to a smb file system.
> > 
> > I am running cp and mv on:
> > $ lsb_release -a
> > No LSB modules are available.
> > Distributor ID:    Debian
> > Description:    Debian GNU/Linux 10 (buster)
> > Release:    10
> > Codename:    buster
> > 
> > SMB is running on:
> > $ lsb_release -a
> > No LSB modules are available.
> > Distributor ID:    Debian
> > Description:    Debian GNU/Linux 9.12 (stretch)
> > Release:    9.12
> > Codename:    stretch
> > 
> > I am using "cp -pi" to copy the files and "mv -i" to move them.
> > 
> > If I run cp or mv on a Debian stretch, time stamps will be correct on the
> > SMB file system.
> > 
> > Buster gnome3 file manager will copy or move the files without problems, I
> > mean, they will preserve time stamps.
> > 
> > This started happening since I upgraded my workstation to buster, so the
> > problem seems to be in buster. I am mounting the file system on buster with
> > the following line added to fstab, which, by the way, is the same I was
> > using when my workstation was running stretch:
> > 
> > //192.168.1.2/alberto /mnt/u1/rw cifs
> > credentials=/root/pass/alberto.pass,uid=alberto,gid=alberto,rw 0 0
> > 
> > Does anyone knows which package is causing this behavior?
> > 
> First, do you see the same behavior if you use /bin/cp and /bin/mv
> instead of simply cp and mv?

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#222138

I am using ls -ls to check the data.

As captured from my screen:

$ rm /mnt/u1/rw/receipt/u1.crontab
$ ls -ls /mnt/1g/home/u1/data/u1.crontab /mnt/u1/rw/receipt/u1.crontab
ls: cannot access '/mnt/u1/rw/receipt/u1.crontab': No such file or directory
4 -rw-r--r-- 1 u1 u1 54 Feb  5  2017 /mnt/1g/home/u1/data/u1.crontab
$ cp -pi /mnt/1g/home/u1/data/u1.crontab /mnt/u1/rw/receipt/
$ ls -ls /mnt/1g/home/u1/data/u1.crontab /mnt/u1/rw/receipt/u1.crontab
    4 -rw-r--r-- 1 u1 u1 54 Feb  5  2017 /mnt/1g/home/u1/data/u1.crontab
1024 -rwxr-xr-x 1 u1 u1 54 Apr 30 00:00 /mnt/u1/rw/receipt/u1.crontab

Other messages have more details.

On 4/29/20 9:47 PM, David Wright wrote:
> On Wed 29 Apr 2020 at 15:07:52 (-0400), Alberto Sentieri wrote:
>> When I tried strace with /bin/cp -pi I can see on both commands
>> something like this:
>>
>> utimensat(4, NULL, [{tv_sec=1588174263, tv_nsec=908624390} /*
>> 2020-04-29T11:31:03.908624390-0400 */, {tv_sec=1486350336,
>> tv_nsec=481422339} /* 2017-02-05T22:05:36.481422339-0500 */], 0) = 0
>>
>> The utimensat is exactly the same for both cp -pi to a ext4 file
>> system or to a smb filesystem. So, cp seems to be working as expected.
>> Probably the same is true to /bin/mv.
> There's no possibility, is there, that you're reading the file status
> time rather than the file modification time? cp -ip/mv will update the
> former though not the latter. All three times will be set by touch in
> the absence of any options (c and m).
>
> I'm not familiar with cifs/smb protocols and filesystems.
>
>> -----------------------------------------------------------------
>>
>> Exactly the same behavior for /bin/cp and /bin/mv. I do not have any other cp or mv in my path.
>>
>> $ sha1sum  /bin/cp /bin/mv
>> 220687a082fb9d0dbb48e9a2b1093cbb4e9de55a  /bin/cp
>> 46e71d67df7eb1c41f8f8c9039f401e242cce94a  /bin/mv
>>
>>
>> On Wed, Apr 29, 2020 at 12:22:41PM -0400, Alberto Sentieri wrote:
>>> cp and mv are not preserving the file timestamps when copying from a ext4
>>> file system to a smb file system.
>>>
>>> I am running cp and mv on:
>>> $ lsb_release -a
>>> No LSB modules are available.
>>> Distributor ID:    Debian
>>> Description:    Debian GNU/Linux 10 (buster)
>>> Release:    10
>>> Codename:    buster
>>>
>>> SMB is running on:
>>> $ lsb_release -a
>>> No LSB modules are available.
>>> Distributor ID:    Debian
>>> Description:    Debian GNU/Linux 9.12 (stretch)
>>> Release:    9.12
>>> Codename:    stretch
>>>
>>> I am using "cp -pi" to copy the files and "mv -i" to move them.
>>>
>>> If I run cp or mv on a Debian stretch, time stamps will be correct on the
>>> SMB file system.
>>>
>>> Buster gnome3 file manager will copy or move the files without problems, I
>>> mean, they will preserve time stamps.
>>>
>>> This started happening since I upgraded my workstation to buster, so the
>>> problem seems to be in buster. I am mounting the file system on buster with
>>> the following line added to fstab, which, by the way, is the same I was
>>> using when my workstation was running stretch:
>>>
>>> //192.168.1.2/alberto /mnt/u1/rw cifs
>>> credentials=/root/pass/alberto.pass,uid=alberto,gid=alberto,rw 0 0
>>>
>>> Does anyone knows which package is causing this behavior?
>>>
>> First, do you see the same behavior if you use /bin/cp and /bin/mv
>> instead of simply cp and mv?
> Cheers,
> David.
>

[toc] | [prev] | [next] | [standalone]

#222115

On Wed, 29 Apr 2020 12:22:41 -0400
Alberto Sentieri <22t@tripolho.com> wrote:

> cp and mv are not preserving the file timestamps when copying from a 
> ext4 file system to a smb file system.

What I see is:

* mv does preserve the time of the file, regardless of copying to an
  SMB share or not.

* cp indeed does not preserve the time of the original file, regardless
  of copying to an SMB share or not.

mv simply moves a directory entry, including the inode, from here to
there. cp creates a new directory entry at the destination. So these
results make sense.

To have cp preserve the time (i.e. copy the original file's time to the
copy), use its -p option.

Also, check for aliases that might include options you don't want.

-- 
Does anybody read signatures any more?

https://charlescurley.com
https://charlescurley.com/blog/

[toc] | [prev] | [next] | [standalone]

#222123

Charles,

Please read the whole thread, which starts at 
https://lists.debian.org/debian-user/2020/04/msg01361.html. It is clear 
there that -p has no effect on that particular case of smb destinations. 
A similar problem is happening with mv.

I resurrect my stretch workstation and the behavior is different from 
buster, when copying a file with -pi from ext4 to smb.


On 4/29/20 3:16 PM, Charles Curley wrote:
> On Wed, 29 Apr 2020 12:22:41 -0400
> Alberto Sentieri <22t@tripolho.com> wrote:
>
>> cp and mv are not preserving the file timestamps when copying from a
>> ext4 file system to a smb file system.
> What I see is:
>
> * mv does preserve the time of the file, regardless of copying to an
>    SMB share or not.
>
> * cp indeed does not preserve the time of the original file, regardless
>    of copying to an SMB share or not.
>
> mv simply moves a directory entry, including the inode, from here to
> there. cp creates a new directory entry at the destination. So these
> results make sense.
>
> To have cp preserve the time (i.e. copy the original file's time to the
> copy), use its -p option.
>
> Also, check for aliases that might include options you don't want.
>

[toc] | [prev] | [next] | [standalone]

#222126

Hi,

Alberto Sentieri wrote:
> It is clear
> there that -p has no effect on that particular case of smb destinations. A
> similar problem is happening with mv.

Maybe its not the file copying operation but the subsequent adjustment
of the timestamps.
Did you already try whether you can change timestamps on SMB by e.g.
touch(1) ?


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#222127

The content file is copied correctly. And touch works as expected on smb 
files. The command below produced the expected results:

touch -t 201901011300  /mnt/u1/rw/receipt/u1.crontab


On 4/29/20 4:13 PM, Thomas Schmitt wrote:
> Hi,
>
> Alberto Sentieri wrote:
>> It is clear
>> there that -p has no effect on that particular case of smb destinations. A
>> similar problem is happening with mv.
> Maybe its not the file copying operation but the subsequent adjustment
> of the timestamps.
> Did you already try whether you can change timestamps on SMB by e.g.
> touch(1) ?
>
>
> Have a nice day :)
>
> Thomas
>

[toc] | [prev] | [next] | [standalone]

#222128

Just to document what I've seen so far:

These are some straces which may help finding the problem:

1) cp -p  from ext4 to smb, debian buster, which failed:

utimensat(4, NULL, [{tv_sec=1588174263, tv_nsec=908624390} /* 
2020-04-29T11:31:03.908624390-0400 */, {tv_sec=1486350336, 
tv_nsec=481422339} /* 2017-02-05T22:05:36.481422339-0500 */], 0) = 0
fgetxattr(3, "system.posix_acl_access", 0x7ffd302bd820, 132) = -1 
ENODATA (No data available)
fstat(3, {st_mode=S_IFREG|0644, st_size=54, ...}) = 0
fsetxattr(4, "system.posix_acl_access", 
"\2\0\0\0\1\0\6\0\377\377\377\377\4\0\4\0\377\377\377\377 
\0\4\0\377\377\377\377", 28, 0) = -1 EOPNOTSUPP (Operation not supported)
fchmod(4, 0100644)                      = 0
close(4)                                = 0
close(3)                                = 0
munmap(0x7fda0caa2000, 139264)          = 0
lseek(0, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)
close(0)                                = 0
close(1)                                = 0
close(2)                                = 0
exit_group(0)                           = ?
+++ exited with 0 +++

2) cp -p from ext4 to ext4, debian buster, which worked correctly:

utimensat(4, NULL, [{tv_sec=1588174263, tv_nsec=908624390} /* 
2020-04-29T11:31:03.908624390-0400 */, {tv_sec=1486350336, 
tv_nsec=481422339} /* 2017-02-05T22:05:36.481422339-0500 */], 0) = 0
fgetxattr(3, "system.posix_acl_access", 0x7fffea95d4d0, 132) = -1 
ENODATA (No data available)
fstat(3, {st_mode=S_IFREG|0644, st_size=54, ...}) = 0
fsetxattr(4, "system.posix_acl_access", 
"\2\0\0\0\1\0\6\0\377\377\377\377\4\0\4\0\377\377\377\377 
\0\4\0\377\377\377\377", 28, 0) = 0
close(4)                                = 0
close(3)                                = 0
munmap(0x7f13495c9000, 139264)          = 0
lseek(0, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)
close(0)                                = 0
close(1)                                = 0
close(2)                                = 0
exit_group(0)                           = ?

3) touch smb file, debian buster (command touch -t 202001011700 
u1.crontab), which works correctly:

openat(AT_FDCWD, "u1.crontab", O_WRONLY|O_CREAT|O_NOCTTY|O_NONBLOCK, 
0666) = 3
dup2(3, 0)                              = 0
close(3)                                = 0
utimensat(0, NULL, [{tv_sec=1577916000, tv_nsec=0} /* 
2020-01-01T17:00:00-0500 */, {tv_sec=1577916000, tv_nsec=0} /* 
2020-01-01T17:00:00-0500 */], 0) = 0
close(0)                                = 0
close(1)                                = 0
close(2)                                = 0
exit_group(0)                           = ?

4) cp -p from ext4 to smb on debian stretch, which works correctly:

utimensat(4, NULL, [{tv_sec=1588174263, tv_nsec=908624390}, 
{tv_sec=1486350336, tv_nsec=481422339}], 0) = 0
fgetxattr(3, "system.posix_acl_access", 0x7ffdd4465000, 132) = -1 
ENODATA (No data available)
fstat(3, {st_mode=S_IFREG|0644, st_size=54, ...}) = 0
fsetxattr(4, "system.posix_acl_access", 
"\2\0\0\0\1\0\6\0\377\377\377\377\4\0\4\0\377\377\377\377 
\0\4\0\377\377\377\377", 28, 0) = -1 EOPNOTSUPP (Operation not supported)
fchmod(4, 0100644)                      = 0
close(4)                                = 0
close(3)                                = 0
munmap(0x7f8ab2aaf000, 139264)          = 0
lseek(0, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)
close(0)                                = 0
close(1)                                = 0
close(2)                                = 0
exit_group(0)                           = ?

So, the cp behavior on debian stretch and buster seems to be the same.

touch does the trick of dup2 and close, before calling utimensat.

Alberto

On 4/29/20 4:13 PM, Thomas Schmitt wrote:
> Hi,
>
> Alberto Sentieri wrote:
>> It is clear
>> there that -p has no effect on that particular case of smb destinations. A
>> similar problem is happening with mv.
> Maybe its not the file copying operation but the subsequent adjustment
> of the timestamps.
> Did you already try whether you can change timestamps on SMB by e.g.
> touch(1) ?
>
>
> Have a nice day :)
>
> Thomas
>

[toc] | [prev] | [next] | [standalone]

#222129

Hi,

assumed that the success of "touch" indicates that utimensat(2) works
fine, i would pick the failed fsetxattr(2) as next suspect.

Does this set the timestamps despite failing ?

  setfattr -n user.test_name -v test_value /mnt/u1/rw/receipt/u1.crontab

(I expect an "Operation not supported" error as in strace.
If it succeeds against our will, try -n "test_name", without prefix
"user.".)


Alberto Sentieri wrote:
> So, the cp behavior on debian stretch and buster seems to be the same.

But the implementation of the system calls is not.


> touch does the trick of dup2 and close, before calling utimensat.

Hm. To verify suchtheories you will have to create a C program which
uses the traced system calls and by which you can test variations.

Quite interesting would be to inquire the file timestamps immediately
after utimensat() in order to learn whether it gets into effect at
least for a short time.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#222139

I tried setfattr as you suggested with "user" and without "user". Both 
failed with "Operation not supported" and none of them changed the 
timestamp.

On 4/29/20 5:31 PM, Thomas Schmitt wrote:
> Hi,
>
> assumed that the success of "touch" indicates that utimensat(2) works
> fine, i would pick the failed fsetxattr(2) as next suspect.
>
> Does this set the timestamps despite failing ?
>
>    setfattr -n user.test_name -v test_value /mnt/u1/rw/receipt/u1.crontab
>
> (I expect an "Operation not supported" error as in strace.
> If it succeeds against our will, try -n "test_name", without prefix
> "user.".)
>
>
> Alberto Sentieri wrote:
>> So, the cp behavior on debian stretch and buster seems to be the same.
> But the implementation of the system calls is not.
>
>
>> touch does the trick of dup2 and close, before calling utimensat.
> Hm. To verify suchtheories you will have to create a C program which
> uses the traced system calls and by which you can test variations.
>
> Quite interesting would be to inquire the file timestamps immediately
> after utimensat() in order to learn whether it gets into effect at
> least for a short time.
>
>
> Have a nice day :)
>
> Thomas
>

[toc] | [prev] | [next] | [standalone]

#222143

Hi,

Alberto Sentieri wrote:
> I tried setfattr as you suggested with "user" and without "user". Both
> failed with "Operation not supported" and none of them changed the
> timestamp.

This only leaves the idea to mimick the strace of cp -p in an own
C program to see whether utimensat() has the desired effect and
whether following calls spoil it.
Maybe it is necessary to really create a new file and to write
some realistic amount of data to it in order to set up the conditions
for the problem.

The code of cp.c and copy.c in
  https://sources.debian.org/src/coreutils/8.30-3/src/
looks not overly self-contained. So it might be difficult to get a
debuggable version of cp, if not Debian package development magic can
produce it.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#222147

Thanks.

By any chance is there a cifs specialist watching this thread?

Apparently the timestamp is initially correct but it changes after a 
while. Does anyone knows why? Is there a worker finishing the file copy, 
which could be creating this effect?

I wrote the following script to repeat my tests:

#!/bin/bash
rm -f /mnt/u1/rw/receipt/u1.crontab
/bin/stat /mnt/u1/rw/receipt/u1.crontab
/bin/cp -pi /mnt/1g/home/u1/data/u1.crontab /mnt/u1/rw/receipt/
ls -ls /mnt/1g/home/u1/data/u1.crontab /mnt/u1/rw/receipt/u1.crontab
/bin/stat /mnt/1g/home/u1/data/u1.crontab /mnt/u1/rw/receipt/u1.crontab
sleep 1
ls -ls /mnt/1g/home/u1/data/u1.crontab /mnt/u1/rw/receipt/u1.crontab
/bin/stat /mnt/1g/home/u1/data/u1.crontab /mnt/u1/rw/receipt/u1.crontab

The result is this:

$ ./do_it.sh
/bin/stat: cannot stat '/mnt/u1/rw/receipt/u1.crontab': No such file or 
directory
4 -rw-r--r-- 1 u1 u1 54 Feb  5  2017 /mnt/1g/home/u1/data/u1.crontab
0 -rwxr-xr-x 1 u1 u1 54 Feb  5  2017 /mnt/u1/rw/receipt/u1.crontab
   File: /mnt/1g/home/u1/data/u1.crontab
   Size: 54            Blocks: 8          IO Block: 4096   regular file
Device: 831h/2097d    Inode: 58721601    Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/      u1)   Gid: ( 1000/      u1)
Access: 2020-04-29 11:31:03.908624390 -0400
Modify: 2017-02-05 22:05:36.481422339 -0500
Change: 2017-02-05 22:05:36.581420492 -0500
  Birth: -
   File: /mnt/u1/rw/receipt/u1.crontab
   Size: 54            Blocks: 0          IO Block: 16384  regular file
Device: 2ch/44d    Inode: 75374568    Links: 1
Access: (0755/-rwxr-xr-x)  Uid: ( 1000/      u1)   Gid: ( 1000/      u1)
Access: 2020-04-29 11:31:03.908624300 -0400
Modify: 2017-02-05 22:05:36.481422300 -0500
Change: 2020-04-30 10:04:07.613793700 -0400
  Birth: -
    4 -rw-r--r-- 1 u1 u1 54 Feb  5  2017 /mnt/1g/home/u1/data/u1.crontab
1024 -rwxr-xr-x 1 u1 u1 54 Apr 30 10:04 /mnt/u1/rw/receipt/u1.crontab
   File: /mnt/1g/home/u1/data/u1.crontab
   Size: 54            Blocks: 8          IO Block: 4096   regular file
Device: 831h/2097d    Inode: 58721601    Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/      u1)   Gid: ( 1000/      u1)
Access: 2020-04-29 11:31:03.908624390 -0400
Modify: 2017-02-05 22:05:36.481422339 -0500
Change: 2017-02-05 22:05:36.581420492 -0500
  Birth: -
   File: /mnt/u1/rw/receipt/u1.crontab
   Size: 54            Blocks: 2048       IO Block: 16384  regular file
Device: 2ch/44d    Inode: 75374568    Links: 1
Access: (0755/-rwxr-xr-x)  Uid: ( 1000/      u1)   Gid: ( 1000/      u1)
Access: 2020-04-29 11:31:03.908624300 -0400
Modify: 2020-04-30 10:04:07.611931500 -0400
Change: 2020-04-30 10:04:07.611931500 -0400
  Birth: -

On 4/30/20 6:39 AM, Thomas Schmitt wrote:
> Hi,
>
> Alberto Sentieri wrote:
>> I tried setfattr as you suggested with "user" and without "user". Both
>> failed with "Operation not supported" and none of them changed the
>> timestamp.
> This only leaves the idea to mimick the strace of cp -p in an own
> C program to see whether utimensat() has the desired effect and
> whether following calls spoil it.
> Maybe it is necessary to really create a new file and to write
> some realistic amount of data to it in order to set up the conditions
> for the problem.
>
> The code of cp.c and copy.c in
>    https://sources.debian.org/src/coreutils/8.30-3/src/
> looks not overly self-contained. So it might be difficult to get a
> debuggable version of cp, if not Debian package development magic can
> produce it.
>
>
> Have a nice day :)
>
> Thomas
>

[toc] | [prev] | [next] | [standalone]

#222149

Apparently there is something wrong with the debian stretch utimensat 
system call, or with its interaction with cifs. It works as expected 
when the destination file is on a ext4 file system, but it does not work 
when the destination file is on a SMB file system.

I wrote a simple C program, which I compiled with <gcc -g -o x2 x2.c>.

On my debian buster workstation, I run the program using a script, which is:

#!/bin/bash
NAME='/mnt/u1/rw/receipt/test2.txt'
rm -f "${NAME}"
ls -ls "${NAME}"
/mnt/1g/home/u1/data/cp-pi/x2 "${NAME}"
ls -ls "${NAME}"
sleep 1
ls -ls "${NAME}"

Note that the /mnt/u1/rw/receipt is a SMB folder. I got this result:

$ ./do2.sh
ls: cannot access '/mnt/u1/rw/receipt/test2.txt': No such file or directory
0 -rwxr-xr-x 1 u1 u1 10 Feb  5  2017 /mnt/u1/rw/receipt/test2.txt
1024 -rwxr-xr-x 1 u1 u1 10 Apr 30 12:20 /mnt/u1/rw/receipt/test2.txt

As you can see, the time stamp changes after one second.

This is the C code for x2.

#include <stdio.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/syscall.h>

static char *data = "test data\n";
static struct timespec timestamps [2] = {
     { 1588174263, 908624390 },
     { 1486350336, 481422339 }
};

int main (int argc, char **argv)
{
     int fd;

     if (argc > 1) {
         fd = openat (AT_FDCWD, argv [1], O_WRONLY|O_CREAT, 0666);
         if (fd >= 0) {
             write (fd, data, strlen (data));
             /* do not use glibc utimensat directly, it does not allow 
NULL as a second argument */
             syscall (SYS_utimensat, fd, NULL, timestamps, 0);
             close (fd);
             return 0;
         }
     }
     printf ("something went wrong\n");
     return 0;
}

About my buster workstation:
# cat /proc/version;dpkg -l |grep cifs
Linux version 4.19.0-8-amd64 (debian-kernel@lists.debian.org) (gcc 
version 8.3.0 (Debian 8.3.0-6)) #1 SMP Debian 4.19.98-1+deb10u1 (2020-04-27)
ii cifs-utils 2:6.8-2 amd64 Common Internet File System utilities

I run the same binary and script on my debian stretch workstation, and 
got this:

$ ./do2.sh
ls: cannot access '/mnt/u1/rw/receipt/test2.txt': No such file or directory
0 -rwxr-xr-x 1 u1 u1 10 Feb  5  2017 /mnt/u1/rw/receipt/test2.txt
4 -rwxr-xr-x 1 u1 u1 10 Feb  5  2017 /mnt/u1/rw/receipt/test2.txt

$ cat /proc/version
Linux version 4.9.0-12-amd64 (debian-kernel@lists.debian.org) (gcc 
version 6.3.0 20170516 (Debian 6.3.0-18+deb9u1) ) #1 SMP Debian 
4.9.210-1 (2020-01-20)
# dpkg -l |grep cifs
ii  cifs-utils 2:6.7-1 amd64 Common Internet File System utilities

I run the C program above, using the same binary, and got two different 
behaviors.

Can anyone help me with this?

On 4/29/20 5:31 PM, Thomas Schmitt wrote:
> Hi,
>
> assumed that the success of "touch" indicates that utimensat(2) works
> fine, i would pick the failed fsetxattr(2) as next suspect.
>
> Does this set the timestamps despite failing ?
>
> setfattr -n user.test_name -v test_value /mnt/u1/rw/receipt/u1.crontab
>
> (I expect an "Operation not supported" error as in strace.
> If it succeeds against our will, try -n "test_name", without prefix
> "user.".)
>
>
> Alberto Sentieri wrote:
>> So, the cp behavior on debian stretch and buster seems to be the same.
> But the implementation of the system calls is not.
>
>
>> touch does the trick of dup2 and close, before calling utimensat.
> Hm. To verify suchtheories you will have to create a C program which
> uses the traced system calls and by which you can test variations.
>
> Quite interesting would be to inquire the file timestamps immediately
> after utimensat() in order to learn whether it gets into effect at
> least for a short time.
>
>
> Have a nice day :)
>
> Thomas
>

[toc] | [prev] | [next] | [standalone]

#222150

Hi,

Alberto Sentieri wrote:
> ls -ls "${NAME}"
> Note that the /mnt/u1/rw/receipt is a SMB folder. I got this result:
> 1024 -rwxr-xr-x 1 u1 u1 10 Apr 30 12:20 /mnt/u1/rw/receipt/test2.txt
> [...]
> I run the same binary and script on my debian stretch workstation, and got
> 4 -rwxr-xr-x 1 u1 u1 10 Feb  5  2017 /mnt/u1/rw/receipt/test2.txt

It is quite remarkable that the SMB file is reported to use 1024 blocks
for storing 10 bytes.

-------------------------------------------------------------------------

I am still not convinced that SYS_utimensat is to blame.

What do you get if you sleep for 3 seconds before performing it ?

If still the file timestamp changes with that sleep:
What happens if you close the fd without calling utimensat and
re-open the file for applying utimensat only then ?
(What if you let sleep between between close and re-open ?)


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#222153

1 second, 3 seconds or 10 minutes have the same result.

About the 1024 1K-blocks, if one checks the file size on the samba 
server using the same <ls -ls> command, the return will be:
4 -rwxrw-r--  1 root cifsusers       10 Feb  5  2017 test2.txt

So, it is using only 4Kbytes on the server. I have no idea why smb 
interface always reports 1Mbyte. By the way, I have seen this behavior 
for many years. stat (on the workstation) also reports the same (2048 
512-byte blocks).

I will do the tests you suggested later. I am also thinking about trying 
a flush before utimensat. None of this, however, happens in cp.

It may not be utimensat. Between the kernel and the smb, there is a cifs 
package (the driver) which could be doing it wrong. Both the kernel and 
cifs package are different between the two workstations.

Thanks,

Alberto

On 4/30/20 1:32 PM, Thomas Schmitt wrote:
> Hi,
>
> Alberto Sentieri wrote:
>> ls -ls "${NAME}"
>> Note that the /mnt/u1/rw/receipt is a SMB folder. I got this result:
>> 1024 -rwxr-xr-x 1 u1 u1 10 Apr 30 12:20 /mnt/u1/rw/receipt/test2.txt
>> [...]
>> I run the same binary and script on my debian stretch workstation, and got
>> 4 -rwxr-xr-x 1 u1 u1 10 Feb  5  2017 /mnt/u1/rw/receipt/test2.txt
> It is quite remarkable that the SMB file is reported to use 1024 blocks
> for storing 10 bytes.
>
> -------------------------------------------------------------------------
>
> I am still not convinced that SYS_utimensat is to blame.
>
> What do you get if you sleep for 3 seconds before performing it ?
>
> If still the file timestamp changes with that sleep:
> What happens if you close the fd without calling utimensat and
> re-open the file for applying utimensat only then ?
> (What if you let sleep between between close and re-open ?)
>
>
> Have a nice day :)
>
> Thomas
>

[toc] | [prev] | [next] | [standalone]

#222155

I run tcpdump while running my simple program on both stretch and 
buster. On stretch, x2 uses SMB (version 1, I guess) protocol, while on 
buster it uses SMB2 (version 2 or 3).

So, the problem can be solved by adding vers=1.0 to mount.cifs options. 
Any version from 2.0 and above will incur in the same utimensat error. I 
set vers to 1.0 and it worked properly.

While using SMB2, I can also see FILE_INFO/SMB2_FILE_BASIC_INFO on 
buster as a result of utimensat, so something is being sent to the samba 
server.

The question to be answered now is if this is a problem on buster side, 
or or on samba stretch side. The later has all updates.



On 4/30/20 1:32 PM, Thomas Schmitt wrote:
> Hi,
>
> Alberto Sentieri wrote:
>> ls -ls "${NAME}"
>> Note that the /mnt/u1/rw/receipt is a SMB folder. I got this result:
>> 1024 -rwxr-xr-x 1 u1 u1 10 Apr 30 12:20 /mnt/u1/rw/receipt/test2.txt
>> [...]
>> I run the same binary and script on my debian stretch workstation, 
>> and got
>> 4 -rwxr-xr-x 1 u1 u1 10 Feb  5  2017 /mnt/u1/rw/receipt/test2.txt
> It is quite remarkable that the SMB file is reported to use 1024 blocks
> for storing 10 bytes.
>
> -------------------------------------------------------------------------
>
> I am still not convinced that SYS_utimensat is to blame.
>
> What do you get if you sleep for 3 seconds before performing it ?
>
> If still the file timestamp changes with that sleep:
> What happens if you close the fd without calling utimensat and
> re-open the file for applying utimensat only then ?
> (What if you let sleep between between close and re-open ?)
>
>
> Have a nice day :)
>
> Thomas
>

[toc] | [prev] | [next] | [standalone]

#222156

[Multipart message — attachments visible in raw view] — view raw

On 2020-04-30 at 15:27, Alberto Sentieri wrote:

> I run tcpdump while running my simple program on both stretch and 
> buster. On stretch, x2 uses SMB (version 1, I guess) protocol, while
> on buster it uses SMB2 (version 2 or 3).
> 
> So, the problem can be solved by adding vers=1.0 to mount.cifs
> options. Any version from 2.0 and above will incur in the same
> utimensat error. I set vers to 1.0 and it worked properly.

Beware: this is a security hole. My understanding (based on experiences
in my workplace) is that the SMBv1 protocol is known to be inherently,
unfixably insecure, to the point that current Windows versions will not
speak it unless explicitly told to in an out-of-the-way and inconvenient
configuration location.

I would not consider forcing the use of SMBv1 to be a satisfactory
resolution, unless you can be 100% sure that nothing malicious will ever
be in a position to mess around here.

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man.         -- George Bernard Shaw

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | linux.debian.user

csiph-web