Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #195339 > unrolled thread

Backup problem using "cp"

Back to article view | Back to linux.debian.user

Contents

  Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-06 16:30 +0200
    Re: Backup problem using "cp" "Thomas Schmitt" <scdbackup@gmx.net> - 2018-05-06 16:40 +0200
      Re: Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-06 16:50 +0200
        Re: Backup problem using "cp" "Thomas Schmitt" <scdbackup@gmx.net> - 2018-05-06 17:20 +0200
          Re: Backup problem using "cp" Cindy-Sue Causey <butterflybytes@gmail.com> - 2018-05-06 17:30 +0200
          Re: Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-07 13:40 +0200
            Re: Backup problem using "cp" "Thomas Schmitt" <scdbackup@gmx.net> - 2018-05-07 14:10 +0200
              Re: Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-07 15:10 +0200
                Re: Backup problem using "cp" "Thomas Schmitt" <scdbackup@gmx.net> - 2018-05-07 15:30 +0200
                  Re: Backup problem using "cp" Bob Weber <bobrweber@gmail.com> - 2018-05-07 16:00 +0200
                    Re: Backup problem using "cp" Dan Norton <dnorton@mindspring.com> - 2018-05-07 16:50 +0200
                  Re: Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-07 16:10 +0200
                  Re: Backup problem using "cp" David Wright <deblis@lionunicorn.co.uk> - 2018-05-08 03:10 +0200
                Re: Backup problem using "cp" Kushal Kumaran <kushal@locationd.net> - 2018-05-07 19:20 +0200
                  Re: Backup problem using "cp" Jude DaShiell <jdashiel@panix.com> - 2018-05-07 20:00 +0200
              Running of rrequested tests - [was Re: Backup problem using "cp"] Richard Owlett <rowlett@cloud85.net> - 2018-05-08 17:20 +0200
                Re: Running of rrequested tests - [was Backup problem using "cp"] "Thomas Schmitt" <scdbackup@gmx.net> - 2018-05-08 17:50 +0200
                  Re: Running of rrequested tests - [was Backup problem using "cp"] Richard Owlett <rowlett@cloud85.net> - 2018-05-08 19:40 +0200
                    Re: Running of rrequested tests - [was Backup problem using "cp"] "Thomas Schmitt" <scdbackup@gmx.net> - 2018-05-08 22:30 +0200
                      Re: Running of rrequested tests - [was Backup problem using "cp"] Richard Owlett <rowlett@cloud85.net> - 2018-05-09 14:00 +0200
                        Re: Running of rrequested tests - [was Backup problem using "cp"] "Thomas Schmitt" <scdbackup@gmx.net> - 2018-05-09 15:00 +0200
                Re: Running of rrequested tests - [was Re: Backup problem using "cp"] The Wanderer <wanderer@fastmail.fm> - 2018-05-08 17:50 +0200
                  Re: Running of rrequested tests - [was Re: Backup problem using "cp"] Richard Owlett <rowlett@cloud85.net> - 2018-05-08 19:50 +0200
                    Re: Running of rrequested tests - [was Re: Backup problem using "cp"] The Wanderer <wanderer@fastmail.fm> - 2018-05-09 04:50 +0200
                      Re: Running of rrequested tests - [was Re: Backup problem using "cp"] Richard Owlett <rowlett@cloud85.net> - 2018-05-09 14:30 +0200
                        Re: Running of rrequested tests - [was Re: Backup problem using "cp"] The Wanderer <wanderer@fastmail.fm> - 2018-05-09 14:50 +0200
                        Re: Running of rrequested tests - [was Re: Backup problem using "cp"] Richard Hector <richard@walnut.gen.nz> - 2018-05-09 15:00 +0200
                      Re: Running of rrequested tests - [was Re: Backup problem using "cp"] David Wright <deblis@lionunicorn.co.uk> - 2018-05-09 22:30 +0200
                        Re: Running of rrequested tests - [was Re: Backup problem using "cp"] Greg Wooledge <wooledg@eeg.ccf.org> - 2018-05-09 22:40 +0200
                          Re: Running of rrequested tests - [was Re: Backup problem using "cp"] David Wright <deblis@lionunicorn.co.uk> - 2018-05-10 16:00 +0200
                        Re: Running of rrequested tests - [was Re: Backup problem using "cp"] The Wanderer <wanderer@fastmail.fm> - 2018-05-10 05:00 +0200
                          Re: Running of rrequested tests - [was Re: Backup problem using "cp"] David Wright <deblis@lionunicorn.co.uk> - 2018-05-10 16:20 +0200
                  Re: Running of rrequested tests - [was Re: Backup problem using "cp"] David Wright <deblis@lionunicorn.co.uk> - 2018-05-08 21:50 +0200
            Re: Backup problem using "cp" Dan Purgert <dan@djph.net> - 2018-05-07 14:10 +0200
              Re: Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-07 15:00 +0200
                Re: Backup problem using "cp" Dan Purgert <dan@djph.net> - 2018-05-07 15:20 +0200
                Re: Backup problem using "cp" Andy Smith <andy@strugglers.net> - 2018-05-07 15:30 +0200
                  Re: Backup problem using "cp" "Thomas Schmitt" <scdbackup@gmx.net> - 2018-05-07 16:00 +0200
                    Re: Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-07 16:20 +0200
                      Re: Backup problem using "cp" David Wright <deblis@lionunicorn.co.uk> - 2018-05-08 03:20 +0200
                  Re: Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-07 16:10 +0200
            Re: Backup problem using "cp" David Griffith <dave@661.org> - 2018-05-07 15:00 +0200
              Re: Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-07 15:50 +0200
              Re: Backup problem using "cp" Richard Hector <richard@walnut.gen.nz> - 2018-05-07 16:00 +0200
                Re: Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-07 16:40 +0200
            Re: Backup problem using "cp" Charlie Gibbs <cgibbs@surfnaked.ca> - 2018-05-07 20:30 +0200
            Re: Backup problem using "cp" David Wright <deblis@lionunicorn.co.uk> - 2018-05-08 03:00 +0200
    Re: Backup problem using "cp" Dan Purgert <dan@djph.net> - 2018-05-06 16:50 +0200
    Re: Backup problem using "cp" Pascal Hambourg <pascal@plouf.fr.eu.org> - 2018-05-06 17:40 +0200
      Re: Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-06 21:00 +0200
        Re: Backup problem using "cp" David Wright <deblis@lionunicorn.co.uk> - 2018-05-07 00:00 +0200
        Re: Backup problem using "cp" Jonathan Dowland <jmtd@debian.org> - 2018-05-10 10:30 +0200
          Re: Running of rrequested tests - [was Backup problem using "cp"] "Thomas Schmitt" <scdbackup@gmx.net> - 2018-05-10 11:30 +0200
          Re: Backup problem using "cp" David Wright <deblis@lionunicorn.co.uk> - 2018-05-10 15:40 +0200
    SUCCESS!!!! - was {Re: Backup problem using "cp"} Richard Owlett <rowlett@cloud85.net> - 2018-05-11 15:10 +0200
      Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} <tomas@tuxteam.de> - 2018-05-11 15:20 +0200
        Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} Richard Owlett <rowlett@cloud85.net> - 2018-05-11 15:30 +0200
          Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} <tomas@tuxteam.de> - 2018-05-11 17:30 +0200
          Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} Curt <curty@free.fr> - 2018-05-11 17:40 +0200
            Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} Richard Owlett <rowlett@cloud85.net> - 2018-05-11 19:30 +0200
            Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} Gene Heskett <gheskett@shentel.net> - 2018-05-11 19:50 +0200
              Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} Richard Owlett <rowlett@cloud85.net> - 2018-05-11 22:30 +0200
      Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} Pascal Hambourg <pascal@plouf.fr.eu.org> - 2018-05-11 16:50 +0200
        Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} Richard Owlett <rowlett@cloud85.net> - 2018-05-11 20:00 +0200
          Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} Pascal Hambourg <pascal@plouf.fr.eu.org> - 2018-05-11 21:50 +0200
            Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} Richard Owlett <rowlett@cloud85.net> - 2018-05-11 22:40 +0200
      Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} David Wright <deblis@lionunicorn.co.uk> - 2018-05-11 17:30 +0200

Page 1 of 4  [1] 2 3 4  Next page →

#195339 — Backup problem using "cp"

I'm attempting to backup current partition  to a USB connected 1 TB drive.

I get:

> root@debian-jan13:/home/richard# cp -ax / "/media/richard/MISC backups/dev_sda14/"
> cp: cannot stat '/media/richard/MISC backups/dev_sda14/home/richard/.local/share/Trash/expunged/73080846/grub2 problem-2018-02-13/grub2 problem-2018-02-13/grub2 problem-2018-02-13/grub2 problem-2018-02-13/grub2 problem-2....018-02-13/grub2 problem-2018-02-13': File name too long

After multiple re-readings of man page for cp I suspect:

-t, --target-directory=DIRECTORY
     copy all SOURCE arguments into DIRECTORY

-T, --no-target-directory
     treat DEST as a normal file

but am actually quite lost.
HELP please.
TIA

[toc] | [next] | [standalone]

#195342

Hi,

Richard Owlett wrote:
> cp -ax / "/media/richard/MISC
> backups/dev_sda14/"
> cp: cannot stat '/media/richard/MISC
> ...
> File name too long

You tell cp to copy / including its sub tree /media/richard/... which is
the target of the copy process. So as soon as reading reaches that tree
it begins to copy itself into itself endlessly until it bonks against
a filesystem limit.

You need to exclude the target directory from being copied.
E.g. by not giving "/" as copy source but rather all its files and
directories except "/media".


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#195344

On 05/06/2018 09:33 AM, Thomas Schmitt wrote:
> Hi,
> 
> Richard Owlett wrote:
>> cp -ax / "/media/richard/MISC
>> backups/dev_sda14/"
>> cp: cannot stat '/media/richard/MISC
>> ...
>> File name too long
> 
> You tell cp to copy / including its sub tree /media/richard/... which is
> the target of the copy process. So as soon as reading reaches that tree
> it begins to copy itself into itself endlessly until it bonks against
> a filesystem limit.
> 
> You need to exclude the target directory from being copied.
> E.g. by not giving "/" as copy source but rather all its files and
> directories except "/media".
> 

Thought I was doing that by specifying -x.
Any way to accomplish that without explicitly listing all directories 
except /media ?

[toc] | [prev] | [next] | [standalone]

#195347

Hi,

Richard Owlett wrote:
> Thought I was doing that by specifying -x.

Either cp -x has a bug or the target directory is not in a different
filesystem than "/" and not a mount point of such a filesystem.

Check the device numbers of "/" and "/media/richard/MISC...".
E.g. like this

  $ stat / | fgrep Device
  Device: 803h/2051d      Inode: 2           Links: 25
  $ stat /bkp | fgrep Device
  Device: 814h/2068d      Inode: 2           Links: 7

Here "/bkp" has a different device number (2068) than "/" (2051).
So it (its inode, to be exacting) is in a different filesystem.

As contrast see a directory in the same filesystem as "/":

  $ stat /home | fgrep Device
  Device: 803h/2051d      Inode: 2228225     Links: 60


> Any way to accomplish that without explicitly listing all directories except
> /media ?

If it is indeed a bug with cp -x, then you could use some archiver like
"tar" which has options to exclude a file.

Totally untested fantasy:

  $ tar cf - --exclude=/media / | (cd /media/.../dev_sda14 ; tar xf - )

Get inspiration from googling "tar pipe for copying".


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#195349

On 5/6/18, Thomas Schmitt <scdbackup@gmx.net> wrote:
> Hi,
>
> Richard Owlett wrote:
>> Thought I was doing that by specifying -x.
>
> Either cp -x has a bug or the target directory is not in a different
> filesystem than "/" and not a mount point of such a filesystem.
>
> Check the device numbers of "/" and "/media/richard/MISC...".
> E.g. like this
>
>   $ stat / | fgrep Device
>   Device: 803h/2051d      Inode: 2           Links: 25
>   $ stat /bkp | fgrep Device
>   Device: 814h/2068d      Inode: 2           Links: 7
>
> Here "/bkp" has a different device number (2068) than "/" (2051).
> So it (its inode, to be exacting) is in a different filesystem.
>
> As contrast see a directory in the same filesystem as "/":
>
>   $ stat /home | fgrep Device
>   Device: 803h/2051d      Inode: 2228225     Links: 60
>
>
>> Any way to accomplish that without explicitly listing all directories
>> except
>> /media ?
>
> If it is indeed a bug with cp -x, then you could use some archiver like
> "tar" which has options to exclude a file.
>
> Totally untested fantasy:
>
>   $ tar cf - --exclude=/media / | (cd /media/.../dev_sda14 ; tar xf - )
>
> Get inspiration from googling "tar pipe for copying".


I use "--exclude" with rsync so I tried a quick search with that, too.
It's a handy keyword to throw into the search mix.

"tar" and "file" have been coming up for me for specific copy needs
lately, primarily related to plucking via time stamps within large,
single directories. :)

Cindy :)
-- 
Cindy-Sue Causey
Talking Rock, Pickens County, Georgia, USA

* runs with duct tape *

[toc] | [prev] | [next] | [standalone]

#195400

On 05/06/2018 10:11 AM, Thomas Schmitt wrote:
> Hi,
> 
> Richard Owlett wrote:
>> Thought I was doing that by specifying -x.
> 
> Either cp -x has a bug or the target directory is not in a different
> filesystem than "/" and not a mount point of such a filesystem.
> 
> Check the device numbers of "/" and "/media/richard/MISC...".
> E.g. like this
> 
>    $ stat / | fgrep Device
>    Device: 803h/2051d      Inode: 2           Links: 25
>    $ stat /bkp | fgrep Device
>    Device: 814h/2068d      Inode: 2           Links: 7
> 
> Here "/bkp" has a different device number (2068) than "/" (2051).
> So it (its inode, to be exacting) is in a different filesystem.
> 
> As contrast see a directory in the same filesystem as "/":
> 
>    $ stat /home | fgrep Device
>    Device: 803h/2051d      Inode: 2228225     Links: 60

I get:
richard@debian-jan13:~$ stat / | fgrep Device
Device: 80eh/2062d	Inode: 2           Links: 22
richard@debian-jan13:~$ stat /media | fgrep Device
Device: 80eh/2062d	Inode: 131073      Links: 5
richard@debian-jan13:~$

I gather that "cp" is then an inappropriate tool.

"tar" is inappropriate for my preferences - I was attempting to use "cp" 
as there would be multiple files &/or directories as input *and* output.

I suspect long term I want "rsync" [ *MUCH* reading to do! ]


>> Any way to accomplish that without explicitly listing all directories except
>> /media ?
> 
> If it is indeed a bug with cp -x, then you could use some archiver like
> "tar" which has options to exclude a file.
>  > Get inspiration from googling "tar pipe for copying".

Although I wish to avoid "tar", I did get inspiration for a brute force 
method - I'll try it first before commenting.

[toc] | [prev] | [next] | [standalone]

#195402

Hi,

Richard Owlett wrote:
> richard@debian-jan13:~$ stat / | fgrep Device
> Device: 80eh/2062d      Inode: 2           Links: 22
> richard@debian-jan13:~$ stat /media | fgrep Device
> Device: 80eh/2062d      Inode: 131073      Links: 5

"/media" was not the directory to examine. 
You need to examine the directory which you gave as second argument to cp.
In your initial post of this thread the cp command is quoted as

  cp -ax /  "/media/richard/MISC
  backups/dev_sda14/"

(Dunno whether the line break in that mail is significant.)


> I gather that "cp" is then an inappropriate tool.

That's not decided yet.

If your cp target directory had a different device number, then cp option -x
did not what it is supposed to do.

But if your target directory was not the mount point of the USB disk and
not inside the filesystem of the USB disk, then you made a mistake by
giving that directory as target.


> "tar" is inappropriate for my preferences - I was attempting to use "cp" as
> there would be multiple files &/or directories as input *and* output.

You could use multiple tar commands.
(I doubt that you can talk cp into addressing more than one target.)


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#195412

On 05/07/2018 07:01 AM, Thomas Schmitt wrote:
> Hi,
> 
> Richard Owlett wrote:
>> richard@debian-jan13:~$ stat / | fgrep Device
>> Device: 80eh/2062d      Inode: 2           Links: 22
>> richard@debian-jan13:~$ stat /media | fgrep Device
>> Device: 80eh/2062d      Inode: 131073      Links: 5
> 
> "/media" was not the directory to examine.
> You need to examine the directory which you gave as second argument to cp.
> In your initial post of this thread the cp command is quoted as
> 
>    cp -ax /  "/media/richard/MISC
>    backups/dev_sda14/"
> 
> (Dunno whether the line break in that mail is significant.)

No the _apparent_ line break was an artifact of email quoting process.
I can't rerun the diagnostic at the moment - hardware fully occupied 
with testing my proposed workaround, which works so far but progress 
resembles molasses running uphill on Pluto in winter ;/


> 
> 
>> I gather that "cp" is then an inappropriate tool.
> 
> That's not decided yet.
> 
> If your cp target directory had a different device number, then cp option -x
> did not what it is supposed to do.
> 
> But if your target directory was not the mount point of the USB disk and
> not inside the filesystem of the USB disk, then you made a mistake by
> giving that directory as target.

Understood.

> 
> 
>> "tar" is inappropriate for my preferences - I was attempting to use "cp" as
>> there would be multiple files &/or directories as input *and* output.
> 
> You could use multiple tar commands.
> (I doubt that you can talk cp into addressing more than one target.)

Communication confusion.
My goal was to copy root and its sub-directory to a directory on another 
physical device.

> 
> 
> Have a nice day :)
> 
> Thomas
> 
> 

[toc] | [prev] | [next] | [standalone]

#195416

Hi,

Richard Owlett wrote:
> My goal was to copy root and its sub-directory to a directory on another
> physical device.

Well understood.
In a slightly different scenario (backup on Blu-ray) i do this several
times per day.

But i would not dare to give the whole root tree as input to any copying
program or backup archiver. Not only because of the risk of stepping on my
own foot but also because there are several trees which do not deserve
backup or could even make trouble when being fully read.

In my root directory that would be: /dev /mnt /proc /run /sys
E.g. because of
  $ ls -l /proc/kcore
  -r-------- 1 root root 140737477877760 May  7 15:22 /proc/kcore

(Somebody else shall try whether it's really readable and what comes out.
 The announced size is nearly 128 TiB.)


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#195422

[Multipart message — attachments visible in raw view] — view raw

On 5/7/18 9:28 AM, Thomas Schmitt wrote:
> Hi,
>
> Richard Owlett wrote:
>> My goal was to copy root and its sub-directory to a directory on another
>> physical device.
> Well understood.
> In a slightly different scenario (backup on Blu-ray) i do this several
> times per day.
>
> But i would not dare to give the whole root tree as input to any copying
> program or backup archiver. Not only because of the risk of stepping on my
> own foot but also because there are several trees which do not deserve
> backup or could even make trouble when being fully read.
>
> In my root directory that would be: /dev /mnt /proc /run /sys
> E.g. because of
>    $ ls -l /proc/kcore
>    -r-------- 1 root root 140737477877760 May  7 15:22 /proc/kcore
>
> (Somebody else shall try whether it's really readable and what comes out.
>   The announced size is nearly 128 TiB.)
>
>
> Have a nice day :)
>
> Thomas
>
>
There is a program called rsnapshot that uses rsync for the actual work of 
copying but has a config file where you can supply exclude directories (like 
/media).  I just run "rsnapshot hourly" to copy my root file system before an 
apt upgrade command just in case a major problem occurs with the update.  The 
/proc /sys and /dev directories are not copied since they are "mounted" system 
directories.  rsnapshot uses hard links between backups so only the changed 
files are actually copied.  The number of versions to keep is configured in 
/etc/rsnapshot.conf.

In using your cp command, rsync or rsnapshot it is very important that the 
destination filesystem be able to handle hard links and all the file attributes 
of a linux file system.  So make sure that at least there is an ext3 or ext4 
type file system on the destination drive.  If you are not sure what file system 
is in use for the backup destination just run the mount command (as root) 
without any arguments and it will print out all the mounted file systems and 
types.  I get this line for my /home directory:

/dev/md2 on /home type ext4 (rw,relatime,data=ordered)

If I plug in a flash drive and have the system mount it I get (all one line):

/dev/sdg1 on /media/bob/3A78-573B type vfat 
(rw,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,showexec,utf8,flush,errors=remount-ro,uhelper=udisks2)

Notice the type vfat which is NOT what you want to see on the destination file 
system.

-- 


*...Bob*

[toc] | [prev] | [next] | [standalone]

#195430

On Mon, 7 May 2018 09:59:01 -0400
Bob Weber <bobrweber@gmail.com> wrote:

> On 5/7/18 9:28 AM, Thomas Schmitt wrote:
> > Hi,
> >
> > Richard Owlett wrote:  
> >> My goal was to copy root and its sub-directory to a directory on
> >> another physical device.  
> > Well understood.
> > In a slightly different scenario (backup on Blu-ray) i do this
> > several times per day.
> >
> > But i would not dare to give the whole root tree as input to any
> > copying program or backup archiver. Not only because of the risk of
> > stepping on my own foot but also because there are several trees
> > which do not deserve backup or could even make trouble when being
> > fully read.
> >
> > In my root directory that would be: /dev /mnt /proc /run /sys
> > E.g. because of
> >    $ ls -l /proc/kcore
> >    -r-------- 1 root root 140737477877760 May  7 15:22 /proc/kcore
> >
> > (Somebody else shall try whether it's really readable and what
> > comes out. The announced size is nearly 128 TiB.)
> >
> >
> > Have a nice day :)
> >
> > Thomas
> >
> >  
> There is a program called rsnapshot that uses rsync for the actual
> work of copying but has a config file where you can supply exclude
> directories (like /media).  I just run "rsnapshot hourly" to copy my
> root file system before an apt upgrade command just in case a major
> problem occurs with the update.  The /proc /sys and /dev directories
> are not copied since they are "mounted" system directories.
> rsnapshot uses hard links between backups so only the changed files
> are actually copied.  The number of versions to keep is configured
> in /etc/rsnapshot.conf.
> 
> In using your cp command, rsync or rsnapshot it is very important
> that the destination filesystem be able to handle hard links and all
> the file attributes of a linux file system.  So make sure that at
> least there is an ext3 or ext4 type file system on the destination
> drive.  If you are not sure what file system is in use for the backup
> destination just run the mount command (as root) without any
> arguments and it will print out all the mounted file systems and
> types.

Another way to show fs type, without showing so much system stuff:

$ lsblk -f

This also shows labels (if any) and mount points.

 - Dan

[toc] | [prev] | [next] | [standalone]

#195425

On 05/07/2018 08:28 AM, Thomas Schmitt wrote:
> Hi,
> 
> Richard Owlett wrote:
>> My goal was to copy root and its sub-directory to a directory on another
>> physical device.
> 
> Well understood.
> In a slightly different scenario (backup on Blu-ray) i do this several
> times per day.
> 
> But i would not dare to give the whole root tree as input to any copying
> program or backup archiver. Not only because of the risk of stepping on my
> own foot but also because there are several trees which do not deserve
> backup or could even make trouble when being fully read.
> 
> In my root directory that would be: /dev /mnt /proc /run /sys

For my purposes I would add /media .

[toc] | [prev] | [next] | [standalone]

#195474

On Mon 07 May 2018 at 15:28:14 (+0200), Thomas Schmitt wrote:
> Hi,
> 
> Richard Owlett wrote:
> > My goal was to copy root and its sub-directory to a directory on another
> > physical device.
> 
> Well understood.
> In a slightly different scenario (backup on Blu-ray) i do this several
> times per day.
> 
> But i would not dare to give the whole root tree as input to any copying
> program or backup archiver.

I've cloned live systems in the past by writing:

# cd /
# find -xdev -path './lost+found' -prune -o -print | cpio -vdamp /mnt

(omitting v to make it quieter).

> Not only because of the risk of stepping on my
> own foot but also because there are several trees which do not deserve
> backup or could even make trouble when being fully read.
> 
> In my root directory that would be: /dev /mnt /proc /run /sys
> E.g. because of
>   $ ls -l /proc/kcore
>   -r-------- 1 root root 140737477877760 May  7 15:22 /proc/kcore
> 
> (Somebody else shall try whether it's really readable and what comes out.
>  The announced size is nearly 128 TiB.)

The -xdev (or cp -x) deals with those.

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#195443

Richard Owlett <rowlett@cloud85.net> writes:

> On 05/07/2018 07:01 AM, Thomas Schmitt wrote:
>> Hi,
>>
>> Richard Owlett wrote:
>>> richard@debian-jan13:~$ stat / | fgrep Device
>>> Device: 80eh/2062d      Inode: 2           Links: 22
>>> richard@debian-jan13:~$ stat /media | fgrep Device
>>> Device: 80eh/2062d      Inode: 131073      Links: 5
>>
>> "/media" was not the directory to examine.
>> You need to examine the directory which you gave as second argument to cp.
>> In your initial post of this thread the cp command is quoted as
>>
>>    cp -ax /  "/media/richard/MISC
>>    backups/dev_sda14/"
>>
>> (Dunno whether the line break in that mail is significant.)
>
> No the _apparent_ line break was an artifact of email quoting process.
> I can't rerun the diagnostic at the moment - hardware fully occupied
> with testing my proposed workaround, which works so far but progress
> resembles molasses running uphill on Pluto in winter ;/
>
>
>>
>>
>>> I gather that "cp" is then an inappropriate tool.
>>
>> That's not decided yet.
>>
>> If your cp target directory had a different device number, then cp option -x
>> did not what it is supposed to do.
>>
>> But if your target directory was not the mount point of the USB disk and
>> not inside the filesystem of the USB disk, then you made a mistake by
>> giving that directory as target.
>
> Understood.
>
>>
>>
>>> "tar" is inappropriate for my preferences - I was attempting to use "cp" as
>>> there would be multiple files &/or directories as input *and* output.
>>
>> You could use multiple tar commands.
>> (I doubt that you can talk cp into addressing more than one target.)
>
> Communication confusion.
> My goal was to copy root and its sub-directory to a directory on
> another physical device.
>

Are you using an LVM volume as your root?  Then mounting a snapshot at
some location and copying from there is a good option.  This is what I
do for my own backups.

If you're not using LVM volumes, you could still use bind mounts for
this.  Run:

  # mkdir -p /mnt/source
  # mount --bind -o ro / /mnt/source

(The -o ro option is to make the /mnt/source mount read only.  I
understand you intend to use this as a backup source, so read-only
access should be sufficient.  If you change it to not be read-only, be
cautious when changing things in /mnt/source, because it is the same
filesystem as /)

Now /mnt/source is exactly the root filesystem.  Other filesystems
mounted (such as /dev or /proc) would only be present as empty
directories in /mnt/source.  After you're done copying, remove the bind
by running:

  # umount /mnt/source

This way, you do not have to figure out an exclude list or instruct
tools to stop at filesystem boundaries.

Make sure you do verify the files and directories you need are included.
If some of the required files are not part of the root filesystem, then
bind-mount them at the proper location under /mnt/source as well.  For
example, if your /home is a different filesystem, and you need it to be
part of your copying:

  # mkdir -p /mnt/source
  # mount --bind -o ro / /mnt/source
  # mount --bind -o ro /home /mnt/source/home

Then unmounting after you finish copying will be in the reverse order:

  # umount /mnt/source/home
  # umount /mnt/source

-- 
regards,
kushal

[toc] | [prev] | [next] | [standalone]

#195444

On Mon, 7 May 2018, Kushal Kumaran wrote:

> Date: Mon, 7 May 2018 13:15:48
> From: Kushal Kumaran <kushal@locationd.net>
> To: debian-user@lists.debian.org
> Subject: Re: Backup problem using "cp"
> Resent-Date: Mon,  7 May 2018 17:16:09 +0000 (UTC)
> Resent-From: debian-user@lists.debian.org
> 
> Richard Owlett <rowlett@cloud85.net> writes:
>
>> On 05/07/2018 07:01 AM, Thomas Schmitt wrote:
>>> Hi,
>>>
>>> Richard Owlett wrote:
>>>> richard@debian-jan13:~$ stat / | fgrep Device
>>>> Device: 80eh/2062d      Inode: 2           Links: 22
>>>> richard@debian-jan13:~$ stat /media | fgrep Device
>>>> Device: 80eh/2062d      Inode: 131073      Links: 5
>>>
>>> "/media" was not the directory to examine.
>>> You need to examine the directory which you gave as second argument to cp.
>>> In your initial post of this thread the cp command is quoted as
>>>
>>>    cp -ax /  "/media/richard/MISC
>>>    backups/dev_sda14/"
>>>
>>> (Dunno whether the line break in that mail is significant.)
>>
>> No the _apparent_ line break was an artifact of email quoting process.
>> I can't rerun the diagnostic at the moment - hardware fully occupied
>> with testing my proposed workaround, which works so far but progress
>> resembles molasses running uphill on Pluto in winter ;/
>>
>>
>>>
>>>
>>>> I gather that "cp" is then an inappropriate tool.
>>>
>>> That's not decided yet.
>>>
>>> If your cp target directory had a different device number, then cp option -x
>>> did not what it is supposed to do.
>>>
>>> But if your target directory was not the mount point of the USB disk and
>>> not inside the filesystem of the USB disk, then you made a mistake by
>>> giving that directory as target.
>>
>> Understood.
>>
>>>
>>>
>>>> "tar" is inappropriate for my preferences - I was attempting to use "cp" as
>>>> there would be multiple files &/or directories as input *and* output.
>>>
>>> You could use multiple tar commands.
>>> (I doubt that you can talk cp into addressing more than one target.)
>>
>> Communication confusion.
>> My goal was to copy root and its sub-directory to a directory on
>> another physical device.
>>
>
> Are you using an LVM volume as your root?  Then mounting a snapshot at
> some location and copying from there is a good option.  This is what I
> do for my own backups.
>
> If you're not using LVM volumes, you could still use bind mounts for
> this.  Run:
>
>  # mkdir -p /mnt/source
>  # mount --bind -o ro / /mnt/source
>
> (The -o ro option is to make the /mnt/source mount read only.  I
> understand you intend to use this as a backup source, so read-only
> access should be sufficient.  If you change it to not be read-only, be
> cautious when changing things in /mnt/source, because it is the same
> filesystem as /)
>
> Now /mnt/source is exactly the root filesystem.  Other filesystems
> mounted (such as /dev or /proc) would only be present as empty
> directories in /mnt/source.  After you're done copying, remove the bind
> by running:
>
>  # umount /mnt/source
>
> This way, you do not have to figure out an exclude list or instruct
> tools to stop at filesystem boundaries.
>
> Make sure you do verify the files and directories you need are included.
> If some of the required files are not part of the root filesystem, then
> bind-mount them at the proper location under /mnt/source as well.  For
> example, if your /home is a different filesystem, and you need it to be
> part of your copying:
>
>  # mkdir -p /mnt/source
>  # mount --bind -o ro / /mnt/source
>  # mount --bind -o ro /home /mnt/source/home
>
> Then unmounting after you finish copying will be in the reverse order:
>
>  # umount /mnt/source/home
>  # umount /mnt/source
>
Tar has a -T file and an -X file set of options too.  Of course files 
with includes and excludes will have to exist before the copying for that 
to work.

>

-- 

[toc] | [prev] | [next] | [standalone]

#195497 — Running of rrequested tests - [was Re: Backup problem using "cp"]

On 05/07/2018 07:01 AM, Thomas Schmitt wrote:
> Hi,
> 
> Richard Owlett wrote:
>> richard@debian-jan13:~$ stat / | fgrep Device
>> Device: 80eh/2062d      Inode: 2           Links: 22
>> richard@debian-jan13:~$ stat /media | fgrep Device
>> Device: 80eh/2062d      Inode: 131073      Links: 5
> 
> "/media" was not the directory to examine.
> You need to examine the directory which you gave as second argument to cp.
> In your initial post of this thread the cp command is quoted as
> 
>    cp -ax /  "/media/richard/MISC
>    backups/dev_sda14/"
> 
> (Dunno whether the line break in that mail is significant.)
> [SNIP]

The line break was an email artifact and have changed partition label to 
prevent future confusion.

This is the output of script command:

> 

> Script started, file is typescript

> #

> #

> rm -rv /media/root/MISC-backups/

> #

> #

> ls /media/richard/MISC-backups/dev_sda14/

> #

> #

> stat / | fgrep Device

> #

> #

> stat /media | fgrep Device

> #

> #

> stat /media/richard/MISC-backups/dev_sda14/ | fgrep Device

> #

> #

> cp -ax /  /media/richard/MISC-backups/dev_sda14/

> #

> #

> ls /media/richard/MISC-backups/dev_sda14/

> #

> #

> ls /media/richard/MISC-backups/dev_sda14/*

> #

> #

> ls /media/richard/MISC-backups/dev_sda14/home/richard

> #

> #

> ls -aRil /media/richard/MISC-backups/dev_sda14/

> #

> #

> exit

> root@debian-jan13:/home/richard# #

> root@debian-jan13:/home/richard# #

> root@debian-jan13:/home/richard# rm -rv /media/root/MISC-backups/

> rm: cannot remove '/media/root/MISC-backups/': No such file or directory

> root@debian-jan13:/home/richard# #

> root@debian-jan13:/home/richard# #

> root@debian-jan13:/home/richard# ls /media/richard/MISC-backups/dev_sda14/

> home  lost+found

> root@debian-jan13:/home/richard# #

> root@debian-jan13:/home/richard# #

> root@debian-jan13:/home/richard# stat / | fgrep Device

> Device: 80eh/2062d	Inode: 2           Links: 22

> root@debian-jan13:/home/richard# #

> root@debian-jan13:/home/richard# #

> root@debian-jan13:/home/richard# stat /media | fgrep Device

> Device: 80eh/2062d	Inode: 131073      Links: 5

> root@debian-jan13:/home/richard# #

> root@debian-jan13:/home/richard# #

> root@debian-jan13:/home/richard# stat /media/richard/MISC-backups/dev_sda14/ | fgrep Device

> Device: 819h/2073d	Inode: 15728641    Links: 4

> root@debian-jan13:/home/richard# #

> root@debian-jan13:/home/richard# #

> root@debian-jan13:/home/richard# cp -ax /  /media/richard/MISC-backups/dev_sda14/

> 

> cp: cannot stat '/media/richard/MISC-backups/dev_sda14/home/richard/.local/share/Trash/expunged/1449727740/grub2 problem-2018-02-13/grub2 problem-2018-02-13/...
> ... 
> ...
> ... grub2 problem-2018-02-13': File name too long

> ^C

> root@debian-jan13:/home/richard# 

> root@debian-jan13:/home/richard# #

> root@debian-jan13:/home/richard# #

> root@debian-jan13:/home/richard# ls /media/richard/MISC-backups/dev_sda14/

> home  lost+found

> root@debian-jan13:/home/richard# #

> root@debian-jan13:/home/richard# #

> root@debian-jan13:/home/richard# ls /media/richard/MISC-backups/dev_sda14/*

> /media/richard/MISC-backups/dev_sda14/home:

> richard

> 

> /media/richard/MISC-backups/dev_sda14/lost+found:

> root@debian-jan13:/home/richard# #

> root@debian-jan13:/home/richard# #

> root@debian-jan13:/home/richard# ls /media/richard/MISC-backups/dev_sda14/home/richard

> Desktop  Documents  Downloads  Music  Pictures	Public	Templates  Videos

> root@debian-jan13:/home/richard# #

> root@debian-jan13:/home/richard# #

I also have the output of
   ls -aRil /media/richard/MISC-backups/dev_sda14/
available. IIRC someone had requested I run a similar command.
TIA

[toc] | [prev] | [next] | [standalone]

#195498 — Re: Running of rrequested tests - [was Backup problem using "cp"]

Hi,

Richard Owlett wrote:
> # stat / | fgrep Device
> Device: 80eh/2062d    Inode: 2           Links: 22
> ...
> # stat /media/richard/MISC-backups/dev_sda14/ | fgrep Device
> Device: 819h/2073d    Inode: 15728641    Links: 4

That's different filesystems.
So cp -x should not try to copy again what it just had copied.


> cp: cannot stat
> '/media/richard/MISC-backups/dev_sda14/home/richard/.local/share/Trash/exp
> unged/1449727740/grub2 problem-2018-02-13/grub2 problem-2018-02-13/...
> ... ...
> ... grub2 problem-2018-02-13': File name too long

It seems that my theory is not valid.

Something could be wrong with the input tree instead. Especially around file
"grub2 problem-2018-02-13".

What do you get from

  ls -lR /home/richard/.local/share/Trash/expunged/1449727740/"grub2 problem-2018-02-13" 2>&1 | less

This could be many text lines. So i propose to pipe the output into "less"
and to then explore it without flooding the terminal.

If nothing spectacular shows up: What does this yield:

  ls -lR /home/richard/.local/share/Trash/expunged/1449727740 2>&1 | less


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#195502 — Re: Running of rrequested tests - [was Backup problem using "cp"]

On 05/08/2018 10:38 AM, Thomas Schmitt wrote:
> [snip]
> 
> Something could be wrong with the input tree instead. Especially around file
> "grub2 problem-2018-02-13".
> 
> What do you get from
> 
>    ls -lR /home/richard/.local/share/Trash/expunged/1449727740/"grub2 problem-2018-02-13" 2>&1 | less
> 
> This could be many text lines. So i propose to pipe the output into "less"
> and to then explore it without flooding the terminal.
> 
> If nothing spectacular shows up: What does this yield:
>  > Script started on Tue 08 May 2018 12:03:45 PM CDT
> root@debian-jan13:/home/richard# ############# What do you get from
> root@debian-jan13:/home/richard# ##  ls -lR /home/richard/.local/share/Trash/expunged/1449727740/"grub2 
>  problem-2018-02-13" 2>&1
> root@debian-jan13:/home/richard# 
> root@debian-jan13:/home/richard# ls -lR /home/richard/.local/share/Trash/expunged/1449727740/"grub2 prob
> blem-2018-02-13" 2>&1
> /home/richard/.local/share/Trash/expunged/1449727740/grub2 problem-2018-02-13: 
> 
> #### Insert visual break#### 
> 
> 
> total 4
> drwx------ 3 richard richard 4096 Mar  6 09:41 grub2 problem-2018-02-13
> 
> /home/richard/.local/share/Trash/expunged/1449727740/grub2 problem-2018-02-13/grub2 problem-2018-02-13: 
> 
> #### Insert visual break#### 
> 
> 
> total 4
> drwx------ 3 richard richard 4096 Mar  6 09:41 grub2 problem-2018-02-13
> 
> /home/richard/.local/share/Trash/expunged/1449727740/grub2 problem-2018-02-13/grub2 problem-2018-02-13/grub2 problem-2018-02-13: 
> 
> #### Insert visual break#### 
> 
> 
> total 4
> drwx------ 3 richard richard 4096 Mar  6 09:41 grub2 problem-2018-02-13
> 
> /home/richard/.local/share/Trash/expunged/1449727740/grub2 problem-2018-02-13/grub2 problem-2018-02-13/grub2 problem-2018-02-13/grub2 problem-2018-02-13: 
> 
> #### Insert visual break#### 
> 
> 
> total 4
> drwx------ 3 richard richard 4096 Mar  6 09:41 grub2 problem-2018-02-13
> 
> /home/richard/.local/share/Trash/expunged/1449727740/grub2 problem-2018-02-13/grub2 problem-2018-02-13/grub2 problem-2018-02-13/grub2 problem-2018-02-13/grub2 problem-2018-02-13: 
> 
> #### Insert visual break#### 
> 
> root@debian-jan13:/home/richard# ############# If nothing spectacular shows up: What does this yield:
> root@debian-jan13:/home/richard# ##ls -lR /home/richard/.local/share/Trash/expunged/1449727740 2>&1
> root@debian-jan13:/home/richard# 
> root@debian-jan13:/home/richard# ls -lR /home/richard/.local/share/Trash/expunged/1449727740 2>&1
> /home/richard/.local/share/Trash/expunged/1449727740:
> total 4
> drwx------ 3 richard richard 4096 Mar  6 09:41 grub2 problem-2018-02-13
> 
> /home/richard/.local/share/Trash/expunged/1449727740/grub2 problem-2018-02-13: 
> 
> #### Insert visual break#### 
> 
> 
> total 4
> drwx------ 3 richard richard 4096 Mar  6 09:41 grub2 problem-2018-02-13
> 
> /home/richard/.local/share/Trash/expunged/1449727740/grub2 problem-2018-02-13/grub2 problem-2018-02-13: 
> 
> #### Insert visual break#### 
> 
> 
> total 4
> drwx------ 3 richard richard 4096 Mar  6 09:41 grub2 problem-2018-02-13
> 
> /home/richard/.local/share/Trash/expunged/1449727740/grub2 problem-2018-02-13/grub2 problem-2018-02-13/grub2 problem-2018-02-13: 
> 
> #### Insert visual break#### 
> 
> 
> root@debian-jan13:/home/richard# 
> root@debian-jan13:/home/richard# ############# exit script command
> root@debian-jan13:/home/richard# exit
> exit
> 
> Script done on Tue 08 May 2018 12:08:04 PM CDT


>    ls -lR /home/richard/.local/share/Trash/expunged/1449727740 2>&1 | less
>


I couldn't interpret what I was seeing so below are excerpts of what was 
captured by script command.

[toc] | [prev] | [next] | [standalone]

#195512 — Re: Running of rrequested tests - [was Backup problem using "cp"]

Hi,

Richard Owlett wrote:
> I couldn't interpret what I was seeing so below are excerpts of what was
> captured by script command.

It is hard to understand what "script" wants to tell us.
"less" would have been more useful for our purpose of displaying output
and navigating in it.


Whatever, we see a strange directory branch:
> /home/richard/.local/share/Trash/expunged/1449727740/grub2
> problem-2018-02-13/grub2 problem-2018-02-13/grub2 problem-2018-02-13/grub2
> problem-2018-02-13/grub2 problem-2018-02-13: 

Whether it goes deeper is not clear. Your original message had:

> > cp: cannot stat '/media/richard/MISC
> > backups/dev_sda14/home/richard/.local/share/Trash/expunged/73080846/grub2
> > problem-2018-02-13/grub2 problem-2018-02-13/grub2 problem-2018-02-13/grub2
> > problem-2018-02-13/grub2 problem-2....018-02-13/grub2 problem-2018-02-13':
> > File name too long

That's at least 6 of the "grub2 problem-2018-02-13", not five as in your
"script" text. 

We need to apply clear classic commands:

  cd /home/richard/.local/share/Trash/expunged/1449727740

  find . -exec ls -ld '{}' ';' >/tmp/weird_tree.txt 2>&1

This will create a file named /tmp/weird_tree.txt with the output of
"ls -ld" of each file underneath "1449727740".

Please upload this file somewhere, so that its lines do not get hacked
into pieces or abbreviated. Or send it as attachment to my mail address.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#195526 — Re: Running of rrequested tests - [was Backup problem using "cp"]

On 05/08/2018 03:22 PM, Thomas Schmitt wrote:
> Hi,
> 
> Richard Owlett wrote:
>> I couldn't interpret what I was seeing so below are excerpts of what was
>> captured by script command.
> 
> It is hard to understand what "script" wants to tell us.
> "less" would have been more useful for our purpose of displaying output
> and navigating in it.
> 

I couldn't figure out how to post a useful segment. I suspect some of my 
confusion revolves around not understanding "2>&1".


> [snip]
>
> 
> We need to apply clear classic commands:
> 
>    cd /home/richard/.local/share/Trash/expunged/1449727740
> 
>    find . -exec ls -ld '{}' ';' >/tmp/weird_tree.txt 2>&1
> 
> This will create a file named /tmp/weird_tree.txt with the output of
> "ls -ld" of each file underneath "1449727740".
> 
> Please upload this file somewhere, so that its lines do not get hacked
> into pieces or abbreviated. Or send it as attachment to my mail address.

I have run that if you need it.
I think I have a more compact view of that information.
I ran:
   tree -adRA -L 8 /home/richard/.local/share/Trash/expunged/1449727740/
The output was:

> /home/richard/.local/share/Trash/expunged/1449727740/
> └── grub2 problem-2018-02-13
>     └── grub2 problem-2018-02-13
>         └── grub2 problem-2018-02-13
>             └── grub2 problem-2018-02-13
>                 └── grub2 problem-2018-02-13
>                     └── grub2 problem-2018-02-13
>                         └── grub2 problem-2018-02-13
>                             └── grub2 problem-2018-02-13

Goes on for 161 directories

[toc] | [prev] | [next] | [standalone]

Page 1 of 4  [1] 2 3 4  Next page →

Back to top | Article view | linux.debian.user

csiph-web