Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #195339 > unrolled thread

Backup problem using "cp"

Back to article view | Back to linux.debian.user

Contents

  Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-06 16:30 +0200
    Re: Backup problem using "cp" "Thomas Schmitt" <scdbackup@gmx.net> - 2018-05-06 16:40 +0200
      Re: Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-06 16:50 +0200
        Re: Backup problem using "cp" "Thomas Schmitt" <scdbackup@gmx.net> - 2018-05-06 17:20 +0200
          Re: Backup problem using "cp" Cindy-Sue Causey <butterflybytes@gmail.com> - 2018-05-06 17:30 +0200
          Re: Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-07 13:40 +0200
            Re: Backup problem using "cp" "Thomas Schmitt" <scdbackup@gmx.net> - 2018-05-07 14:10 +0200
              Re: Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-07 15:10 +0200
                Re: Backup problem using "cp" "Thomas Schmitt" <scdbackup@gmx.net> - 2018-05-07 15:30 +0200
                  Re: Backup problem using "cp" Bob Weber <bobrweber@gmail.com> - 2018-05-07 16:00 +0200
                    Re: Backup problem using "cp" Dan Norton <dnorton@mindspring.com> - 2018-05-07 16:50 +0200
                  Re: Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-07 16:10 +0200
                  Re: Backup problem using "cp" David Wright <deblis@lionunicorn.co.uk> - 2018-05-08 03:10 +0200
                Re: Backup problem using "cp" Kushal Kumaran <kushal@locationd.net> - 2018-05-07 19:20 +0200
                  Re: Backup problem using "cp" Jude DaShiell <jdashiel@panix.com> - 2018-05-07 20:00 +0200
              Running of rrequested tests - [was Re: Backup problem using "cp"] Richard Owlett <rowlett@cloud85.net> - 2018-05-08 17:20 +0200
                Re: Running of rrequested tests - [was Backup problem using "cp"] "Thomas Schmitt" <scdbackup@gmx.net> - 2018-05-08 17:50 +0200
                  Re: Running of rrequested tests - [was Backup problem using "cp"] Richard Owlett <rowlett@cloud85.net> - 2018-05-08 19:40 +0200
                    Re: Running of rrequested tests - [was Backup problem using "cp"] "Thomas Schmitt" <scdbackup@gmx.net> - 2018-05-08 22:30 +0200
                      Re: Running of rrequested tests - [was Backup problem using "cp"] Richard Owlett <rowlett@cloud85.net> - 2018-05-09 14:00 +0200
                        Re: Running of rrequested tests - [was Backup problem using "cp"] "Thomas Schmitt" <scdbackup@gmx.net> - 2018-05-09 15:00 +0200
                Re: Running of rrequested tests - [was Re: Backup problem using "cp"] The Wanderer <wanderer@fastmail.fm> - 2018-05-08 17:50 +0200
                  Re: Running of rrequested tests - [was Re: Backup problem using "cp"] Richard Owlett <rowlett@cloud85.net> - 2018-05-08 19:50 +0200
                    Re: Running of rrequested tests - [was Re: Backup problem using "cp"] The Wanderer <wanderer@fastmail.fm> - 2018-05-09 04:50 +0200
                      Re: Running of rrequested tests - [was Re: Backup problem using "cp"] Richard Owlett <rowlett@cloud85.net> - 2018-05-09 14:30 +0200
                        Re: Running of rrequested tests - [was Re: Backup problem using "cp"] The Wanderer <wanderer@fastmail.fm> - 2018-05-09 14:50 +0200
                        Re: Running of rrequested tests - [was Re: Backup problem using "cp"] Richard Hector <richard@walnut.gen.nz> - 2018-05-09 15:00 +0200
                      Re: Running of rrequested tests - [was Re: Backup problem using "cp"] David Wright <deblis@lionunicorn.co.uk> - 2018-05-09 22:30 +0200
                        Re: Running of rrequested tests - [was Re: Backup problem using "cp"] Greg Wooledge <wooledg@eeg.ccf.org> - 2018-05-09 22:40 +0200
                          Re: Running of rrequested tests - [was Re: Backup problem using "cp"] David Wright <deblis@lionunicorn.co.uk> - 2018-05-10 16:00 +0200
                        Re: Running of rrequested tests - [was Re: Backup problem using "cp"] The Wanderer <wanderer@fastmail.fm> - 2018-05-10 05:00 +0200
                          Re: Running of rrequested tests - [was Re: Backup problem using "cp"] David Wright <deblis@lionunicorn.co.uk> - 2018-05-10 16:20 +0200
                  Re: Running of rrequested tests - [was Re: Backup problem using "cp"] David Wright <deblis@lionunicorn.co.uk> - 2018-05-08 21:50 +0200
            Re: Backup problem using "cp" Dan Purgert <dan@djph.net> - 2018-05-07 14:10 +0200
              Re: Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-07 15:00 +0200
                Re: Backup problem using "cp" Dan Purgert <dan@djph.net> - 2018-05-07 15:20 +0200
                Re: Backup problem using "cp" Andy Smith <andy@strugglers.net> - 2018-05-07 15:30 +0200
                  Re: Backup problem using "cp" "Thomas Schmitt" <scdbackup@gmx.net> - 2018-05-07 16:00 +0200
                    Re: Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-07 16:20 +0200
                      Re: Backup problem using "cp" David Wright <deblis@lionunicorn.co.uk> - 2018-05-08 03:20 +0200
                  Re: Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-07 16:10 +0200
            Re: Backup problem using "cp" David Griffith <dave@661.org> - 2018-05-07 15:00 +0200
              Re: Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-07 15:50 +0200
              Re: Backup problem using "cp" Richard Hector <richard@walnut.gen.nz> - 2018-05-07 16:00 +0200
                Re: Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-07 16:40 +0200
            Re: Backup problem using "cp" Charlie Gibbs <cgibbs@surfnaked.ca> - 2018-05-07 20:30 +0200
            Re: Backup problem using "cp" David Wright <deblis@lionunicorn.co.uk> - 2018-05-08 03:00 +0200
    Re: Backup problem using "cp" Dan Purgert <dan@djph.net> - 2018-05-06 16:50 +0200
    Re: Backup problem using "cp" Pascal Hambourg <pascal@plouf.fr.eu.org> - 2018-05-06 17:40 +0200
      Re: Backup problem using "cp" Richard Owlett <rowlett@cloud85.net> - 2018-05-06 21:00 +0200
        Re: Backup problem using "cp" David Wright <deblis@lionunicorn.co.uk> - 2018-05-07 00:00 +0200
        Re: Backup problem using "cp" Jonathan Dowland <jmtd@debian.org> - 2018-05-10 10:30 +0200
          Re: Running of rrequested tests - [was Backup problem using "cp"] "Thomas Schmitt" <scdbackup@gmx.net> - 2018-05-10 11:30 +0200
          Re: Backup problem using "cp" David Wright <deblis@lionunicorn.co.uk> - 2018-05-10 15:40 +0200
    SUCCESS!!!! - was {Re: Backup problem using "cp"} Richard Owlett <rowlett@cloud85.net> - 2018-05-11 15:10 +0200
      Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} <tomas@tuxteam.de> - 2018-05-11 15:20 +0200
        Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} Richard Owlett <rowlett@cloud85.net> - 2018-05-11 15:30 +0200
          Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} <tomas@tuxteam.de> - 2018-05-11 17:30 +0200
          Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} Curt <curty@free.fr> - 2018-05-11 17:40 +0200
            Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} Richard Owlett <rowlett@cloud85.net> - 2018-05-11 19:30 +0200
            Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} Gene Heskett <gheskett@shentel.net> - 2018-05-11 19:50 +0200
              Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} Richard Owlett <rowlett@cloud85.net> - 2018-05-11 22:30 +0200
      Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} Pascal Hambourg <pascal@plouf.fr.eu.org> - 2018-05-11 16:50 +0200
        Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} Richard Owlett <rowlett@cloud85.net> - 2018-05-11 20:00 +0200
          Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} Pascal Hambourg <pascal@plouf.fr.eu.org> - 2018-05-11 21:50 +0200
            Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} Richard Owlett <rowlett@cloud85.net> - 2018-05-11 22:40 +0200
      Re: SUCCESS!!!! - was {Re: Backup problem using "cp"} David Wright <deblis@lionunicorn.co.uk> - 2018-05-11 17:30 +0200

Page 2 of 4 — ← Prev page 1 [2] 3 4  Next page →

#195531 — Re: Running of rrequested tests - [was Backup problem using "cp"]

Hi,

Richard Owlett wrote:
> > /home/richard/.local/share/Trash/expunged/1449727740/
> └── grub2 problem-2018-02-13
> ...
> Goes on for 161 directories

The name is 24 bytes long.
Plus one slash yields 25.
161 times 25 = 4025 bytes for the problem directories.
Plus 53 for "/home/...40/" = 4078.
So far your system accepted the path.

But plus 37 for "/media/richard/...sda14" yields 4115 and caused protest.

Google finds me
  https://github.com/torvalds/linux/blob/master/include/uapi/linux/limits.h
saying
  #define PATH_MAX 4096	/* # chars in a path name including nul */
My local
  /usr/include/linux/limits.h
says the same.

So Linux publishes a limit of at most 4095 bytes for paths.
Many filesystem will actually have this limit or lower ones. It is not
clear whether you bonked against a fixed limit of Linux or a specific
limit of the particular filesystem on the USB disk.

The origin of this insane directory chain is obscure, of course. Maybe
the name parts "grub2" and "2018-02-13" help Richard to remember what
might have happened back then.

For the purpose of disk sanity (and maybe successful "cp -ax") i'd then
consider to remove these directories.


Congrats:
David Wright made the better initial guess, compared to mine.

-------------------------------------------------------------------------

> I suspect some of my confusion revolves around not understanding "2>&1".

That's about
- "standard output" (result channel of classical Unix programs
- "standard error" (non-result message channel)
- "redirection" (plugging together such channels)
- "file descriptors" (a programmer's thing, actually) which get set at
  program start to:
    0 = standard input (the channel for input text, normally your keyboard)
    1 = standard output (normally your terminal)
    2 = standard error (normally your terminal, too)

All together "2>&1" funnels the non-result messages of a program into the
result channel. This is done typically before a pipe that consumes standard
output

  program 2>&1 | other_program

or _after_ a redirection of standard output to a file

  program >log_file 2>&1 

In both cases it shall join both output channels into a single channel.

(If you write "program 2>&1 >log_file" the non-result messages will
 appear on the former standard output and the results will go into the
 file "log_file". Sequence matters.)


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#195499 — Re: Running of rrequested tests - [was Re: Backup problem using "cp"]

[Multipart message — attachments visible in raw view] — view raw

On 2018-05-08 at 11:10, Richard Owlett wrote:

> On 05/07/2018 07:01 AM, Thomas Schmitt wrote:
>
>> Hi,
>> 
>> Richard Owlett wrote:
>>> richard@debian-jan13:~$ stat / | fgrep Device
>>> Device: 80eh/2062d      Inode: 2           Links: 22
>>> richard@debian-jan13:~$ stat /media | fgrep Device
>>> Device: 80eh/2062d      Inode: 131073      Links: 5
>> 
>> "/media" was not the directory to examine.
>> You need to examine the directory which you gave as second argument to cp.
>> In your initial post of this thread the cp command is quoted as
>> 
>>    cp -ax /  "/media/richard/MISC
>>    backups/dev_sda14/"
>> 
>> (Dunno whether the line break in that mail is significant.)
>> [SNIP]
> 
> The line break was an email artifact and have changed partition label to 
> prevent future confusion.
> 
> This is the output of script command:

<snip>

Have you tried

stat /home/richard/.local/share/Trash/expunged/1449727740/grub2
problem-2018-02-13/

and/or a 'ls' of the same directory?

Since we've apparently confirmed that /media/richard/MISC-backups/ is on
a separate filesystem from /, it really looks to me as if this too-deep
directory chain may exist within the source tree, in some form.

The only comment I've found from you on this point seems to be a
statement that yes, such a chain existed, but after you deleted it, it
came back the next time you tried the copy.

If that's correct, then there must be something intervening to cause the
chain to be re-created; the cp command should not be capable of
modifying its source argument, unless its destination is under its
source (which would obviously cause problems).

If it's not correct, then I'd be interested to know what is in fact
there, and what might have caused it to exist (or not).

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man.         -- George Bernard Shaw

[toc] | [prev] | [next] | [standalone]

#195503 — Re: Running of rrequested tests - [was Re: Backup problem using "cp"]

On 05/08/2018 10:38 AM, The Wanderer wrote:
> On 2018-05-08 at 11:10, Richard Owlett wrote:
> 
>> On 05/07/2018 07:01 AM, Thomas Schmitt wrote:
>>
>>> Hi,
>>>
>>> Richard Owlett wrote:
>>>> richard@debian-jan13:~$ stat / | fgrep Device
>>>> Device: 80eh/2062d      Inode: 2           Links: 22
>>>> richard@debian-jan13:~$ stat /media | fgrep Device
>>>> Device: 80eh/2062d      Inode: 131073      Links: 5
>>>
>>> "/media" was not the directory to examine.
>>> You need to examine the directory which you gave as second argument to cp.
>>> In your initial post of this thread the cp command is quoted as
>>>
>>>     cp -ax /  "/media/richard/MISC
>>>     backups/dev_sda14/"
>>>
>>> (Dunno whether the line break in that mail is significant.)
>>> [SNIP]
>>
>> The line break was an email artifact and have changed partition label to
>> prevent future confusion.
>>
>> This is the output of script command:
> 
> <snip>
> 
> Have you tried
> 
> stat /home/richard/.local/share/Trash/expunged/1449727740/grub2
> problem-2018-02-13/
> 
> and/or a 'ls' of the same directory?


After adding required quotation marks:
> richard@debian-jan13:~$ stat "/home/richard/.local/share/Trash/expunged/1449727740/grub2 problem-2018-02-13/"
>   File: /home/richard/.local/share/Trash/expunged/1449727740/grub2 problem-2018-02-13/
>   Size: 4096      	Blocks: 8          IO Block: 4096   directory
> Device: 80eh/2062d	Inode: 141462      Links: 3
> Access: (0700/drwx------)  Uid: ( 1000/ richard)   Gid: ( 1000/ richard)
> Access: 2018-05-08 08:10:45.509914304 -0500
> Modify: 2018-03-06 09:41:14.972933512 -0600
> Change: 2018-05-07 04:50:02.664296985 -0500
>  Birth: -
> richard@debian-jan13:~$ ls "/home/richard/.local/share/Trash/expunged/1449727740/grub2 problem-2018-02-13/"
> grub2 problem-2018-02-13
> richard@debian-jan13:~$



> 
> Since we've apparently confirmed that /media/richard/MISC-backups/ is on
> a separate filesystem from /, it really looks to me as if this too-deep
> directory chain may exist within the source tree, in some form.
> 
> The only comment I've found from you on this point seems to be a
> statement that yes, such a chain existed, but after you deleted it, it
> came back the next time you tried the copy.

That is correct.
Also I did the same before running today's test.

> 
> If that's correct, then there must be something intervening to cause the
> chain to be re-created; the cp command should not be capable of
> modifying its source argument, unless its destination is under its
> source (which would obviously cause problems).
> 
> If it's not correct, then I'd be interested to know what is in fact
> there, and what might have caused it to exist (or not).
> 

Me to ;/

[toc] | [prev] | [next] | [standalone]

#195518 — Re: Running of rrequested tests - [was Re: Backup problem using "cp"]

[Multipart message — attachments visible in raw view] — view raw

On 2018-05-08 at 13:48, Richard Owlett wrote:

> On 05/08/2018 10:38 AM, The Wanderer wrote:

>> Have you tried
>> 
>> stat /home/richard/.local/share/Trash/expunged/1449727740/grub2
>> problem-2018-02-13/
>> 
>> and/or a 'ls' of the same directory?
> 
> After adding required quotation marks:

Yes, sorry about that - I noticed it after hitting Send.

>> richard@debian-jan13:~$ stat "/home/richard/.local/share/Trash/expunged/1449727740/grub2 problem-2018-02-13/"
>>   File: /home/richard/.local/share/Trash/expunged/1449727740/grub2 problem-2018-02-13/
>>   Size: 4096      	Blocks: 8          IO Block: 4096   directory
>> Device: 80eh/2062d	Inode: 141462      Links: 3

Just for the sake of exhaustiveness, can you check the next level down,
and confirm that it actually does have a different inode?

>> Access: (0700/drwx------)  Uid: ( 1000/ richard)   Gid: ( 1000/ richard)
>> Access: 2018-05-08 08:10:45.509914304 -0500
>> Modify: 2018-03-06 09:41:14.972933512 -0600
>> Change: 2018-05-07 04:50:02.664296985 -0500
>>  Birth: -

As David Wright points out, this indicates that this was last modified
in early March, which should mean that it can't have been deleted in the
meantime.

>> Since we've apparently confirmed that /media/richard/MISC-backups/
>> is on a separate filesystem from /, it really looks to me as if
>> this too-deep directory chain may exist within the source tree, in
>> some form.
>> 
>> The only comment I've found from you on this point seems to be a
>> statement that yes, such a chain existed, but after you deleted it,
>> it came back the next time you tried the copy.
> 
> That is correct. Also I did the same before running today's test.

What method are you using to delete it?

If you haven't already, I'd recommend trying 'rm -r', *very* carefully,
from a command prompt. (Unless you have an extremely unusual setup, that
should avoid any possibility of an intermediary "Trash" to need emptying.)

Do you have mlocate installed on your system? If so, the output of

locate 'grub2 problem-2018-02-13'

might be informative; if it finds a copy of the directory somewhere
else, that might tell us where it's being restored from, if it is indeed
being restored.

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man.         -- George Bernard Shaw

[toc] | [prev] | [next] | [standalone]

#195529 — Re: Running of rrequested tests - [was Re: Backup problem using "cp"]

On 05/08/2018 09:39 PM, The Wanderer wrote:
> On 2018-05-08 at 13:48, Richard Owlett wrote:
> [massive SNIP]
> What method are you using to delete it?

Moving directory to trash and then emptying the trash with MATE's 
graphical tools.

> 
> If you haven't already, I'd recommend trying 'rm -r', *very* carefully,
> from a command prompt. (Unless you have an extremely unusual setup, that
> should avoid any possibility of an intermediary "Trash" to need emptying.)
> 
> Do you have mlocate installed on your system? If so, the output of
> 
> locate 'grub2 problem-2018-02-13'

It starts off with:
> /home/richard/.local/share/Trash/expunged/1449727740/grub2 problem-2018-02-13

> /home/richard/.local/share/Trash/expunged/1449727740/grub2 problem-2018-02-13/grub2 problem-2018-02-13

> /home/richard/.local/share/Trash/expunged/1449727740/grub2 problem-2018-02-13/grub2 problem-2018-02-13/grub2 problem-2018-02-13

> /home/richard/.local/share/Trash/expunged/1449727740/grub2 problem-2018-02-13/grub2 problem-2018-02-13/grub2 problem-2018-02-13/grub2 problem-2018-02-13
and goes on for >100 kB.

Running
   /home/richard# tree -adRA -L 8 
/home/richard/.local/share/Trash/expunged/1449727740/
gives:>
> /home/richard/.local/share/Trash/expunged/1449727740/
> └── grub2 problem-2018-02-13
>     └── grub2 problem-2018-02-13
>         └── grub2 problem-2018-02-13
>             └── grub2 problem-2018-02-13
>                 └── grub2 problem-2018-02-13
>                     └── grub2 problem-2018-02-13
>                         └── grub2 problem-2018-02-13
>                             └── grub2 problem-2018-02-13

Goes on for 161 directories

> 
> might be informative; if it finds a copy of the directory somewhere
> else, that might tell us where it's being restored from, if it is indeed
> being restored.
> 

I'll try this afternoon or tomorrow. I'll read documentation for "rm" 
first and use it to purge what I know I'm not interested in. Then re-run 
"locate". I'll wait until I know that I don't have to preserve the 
machine' current state.

Thanks

[toc] | [prev] | [next] | [standalone]

#195530 — Re: Running of rrequested tests - [was Re: Backup problem using "cp"]

[Multipart message — attachments visible in raw view] — view raw

On 2018-05-09 at 08:28, Richard Owlett wrote:

> On 05/08/2018 09:39 PM, The Wanderer wrote:

>> If you haven't already, I'd recommend trying 'rm -r', *very*
>> carefully, from a command prompt. (Unless you have an extremely
>> unusual setup, that should avoid any possibility of an intermediary
>> "Trash" to need emptying.)
>> 
>> Do you have mlocate installed on your system? If so, the output of
>> 
>> locate 'grub2 problem-2018-02-13'

<snip>

>> might be informative; if it finds a copy of the directory
>> somewhere else, that might tell us where it's being restored from,
>> if it is indeed being restored.
> 
> I'll try this afternoon or tomorrow. I'll read documentation for "rm"
> first and use it to purge what I know I'm not interested in. Then
> re-run "locate". I'll wait until I know that I don't have to preserve
> the machine' current state.

Note that 'locate' doesn't update instantly; it draws from an index,
which is normally updated by a cron job, on a nightly basis or so.

If you want it to update its index right away without waiting, you need
to run 'updatedb'. IIRC that might need to be done as root (or with
similar levels of access to /var/lib/mlocate/mlocate.db), though I
haven't checked lately.

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man.         -- George Bernard Shaw

[toc] | [prev] | [next] | [standalone]

#195532 — Re: Running of rrequested tests - [was Re: Backup problem using "cp"]

[Multipart message — attachments visible in raw view] — view raw

On 10/05/18 00:28, Richard Owlett wrote:

>> /home/richard/.local/share/Trash/expunged/1449727740/
>> └── grub2 problem-2018-02-13
>>     └── grub2 problem-2018-02-13
>>         └── grub2 problem-2018-02-13
>>             └── grub2 problem-2018-02-13
>>                 └── grub2 problem-2018-02-13
>>                     └── grub2 problem-2018-02-13
>>                         └── grub2 problem-2018-02-13
>>                             └── grub2 problem-2018-02-13
> 
> Goes on for 161 directories

161 'grub2 problem-2018-02-13' directories? That makes sense; if you
added one more you'd go over the 4016-character (byte?) path length limit.

I'm intrigued as to how cp can create directories in the source tree though.

Are there symlinks involved?

Does

find /home/richard/.local/share/Trash/expunged/1449727740 -type l

show up anything useful?

If the new tree ends up linking back into the old one, for instance, it
could get interesting. (my quick tests suggest not that interesting,
actually, but I might not be testing the right scenario)

Richard

[toc] | [prev] | [next] | [standalone]

#195540 — Re: Running of rrequested tests - [was Re: Backup problem using "cp"]

On Tue 08 May 2018 at 22:39:55 (-0400), The Wanderer wrote:
> On 2018-05-08 at 13:48, Richard Owlett wrote:
> 
> > On 05/08/2018 10:38 AM, The Wanderer wrote:
> 
> >> Have you tried
> >> 
> >> stat /home/richard/.local/share/Trash/expunged/1449727740/grub2
> >> problem-2018-02-13/
> >> 
> >> and/or a 'ls' of the same directory?
> > 
> > After adding required quotation marks:
> 
> Yes, sorry about that - I noticed it after hitting Send.
> 
> >> richard@debian-jan13:~$ stat "/home/richard/.local/share/Trash/expunged/1449727740/grub2 problem-2018-02-13/"
> >>   File: /home/richard/.local/share/Trash/expunged/1449727740/grub2 problem-2018-02-13/
> >>   Size: 4096      	Blocks: 8          IO Block: 4096   directory
> >> Device: 80eh/2062d	Inode: 141462      Links: 3
> 
> Just for the sake of exhaustiveness, can you check the next level down,
> and confirm that it actually does have a different inode?
> 
> >> Access: (0700/drwx------)  Uid: ( 1000/ richard)   Gid: ( 1000/ richard)
> >> Access: 2018-05-08 08:10:45.509914304 -0500
> >> Modify: 2018-03-06 09:41:14.972933512 -0600
> >> Change: 2018-05-07 04:50:02.664296985 -0500
> >>  Birth: -
> 
> As David Wright points out, this indicates that this was last modified
> in early March, which should mean that it can't have been deleted in the
> meantime.
> 
> >> Since we've apparently confirmed that /media/richard/MISC-backups/
> >> is on a separate filesystem from /, it really looks to me as if
> >> this too-deep directory chain may exist within the source tree, in
> >> some form.
> >> 
> >> The only comment I've found from you on this point seems to be a
> >> statement that yes, such a chain existed, but after you deleted it,
> >> it came back the next time you tried the copy.
> > 
> > That is correct. Also I did the same before running today's test.
> 
> What method are you using to delete it?
> 
> If you haven't already, I'd recommend trying 'rm -r', *very* carefully,
> from a command prompt. (Unless you have an extremely unusual setup, that
> should avoid any possibility of an intermediary "Trash" to need emptying.)

As you're deleting a chain of directories, the appropriate command is rmdir.

This bash function might prove useful:

remove-empties () 
{ 
    [ -z "$1" ] && printf '%s\n' "Usage:	$FUNCNAME directories ...
	removes any empty directories under the directories given after prompting." 1>&2 && return 1;
    local IFS="
";
    find $* -depth -xdev -type d -empty -ok rmdir {} \;
}

though you could, for now, just cut and paste this line:

find /home/richard/.local/share/Trash/expunged/1449727740/ -depth -xdev -type d -empty -ok rmdir {} \;

Having to type y 161 times might serve as punishment :)

Alternatively, there's this function:

remove-empties-unprompted () 
{ 
    [ -z "$1" ] && printf '%s\n' "Usage:	$FUNCNAME directories ...
	removes any empty directories under the directories given WITHOUT prompting." 1>&2 && return 1;
    local IFS="
";
    find $* -depth -xdev -type d -empty -print -exec rmdir {} \;
}

which does the job more quickly. Remove the "-print" if you want it
done silently too.

(To Greg: is there a better IFS to use?)

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#195541 — Re: Running of rrequested tests - [was Re: Backup problem using "cp"]

On Wed, May 09, 2018 at 03:20:06PM -0500, David Wright wrote:
> remove-empties () 
> { 
>     [ -z "$1" ] && printf '%s\n' "Usage:	$FUNCNAME directories ...
> 	removes any empty directories under the directories given after prompting." 1>&2 && return 1;
>     local IFS="
> ";
>     find $* -depth -xdev -type d -empty -ok rmdir {} \;
> }

$* should be "$@" with the quotes.

find "$@" -depth -xdev -type d -empty -ok rmdir {} \;

> (To Greg: is there a better IFS to use?)

IFS is irrelevant when you quote properly.  You can simply remove that
assignment command.

[toc] | [prev] | [next] | [standalone]

#195575 — Re: Running of rrequested tests - [was Re: Backup problem using "cp"]

On Wed 09 May 2018 at 16:32:03 (-0400), Greg Wooledge wrote:
> On Wed, May 09, 2018 at 03:20:06PM -0500, David Wright wrote:
> > remove-empties () 
> > { 
> >     [ -z "$1" ] && printf '%s\n' "Usage:	$FUNCNAME directories ...
> > 	removes any empty directories under the directories given after prompting." 1>&2 && return 1;
> >     local IFS="
> > ";
> >     find $* -depth -xdev -type d -empty -ok rmdir {} \;
> > }
> 
> $* should be "$@" with the quotes.
> 
> find "$@" -depth -xdev -type d -empty -ok rmdir {} \;

Reviewing my .bashrc, I can see that I had converted 74 occurrences
of $* to "$@" long ago, but 5 had escaped me; I don't know why.

> > (To Greg: is there a better IFS to use?)
> 
> IFS is irrelevant when you quote properly.  You can simply remove that
> assignment command.

OTOH I hadn't bothered to remove their accompanying IFS assignment in
about 50 cases, so I've had a useful tidy up. Probably originated in
some script I copied in the late 20th century. Thanks.

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#195553 — Re: Running of rrequested tests - [was Re: Backup problem using "cp"]

[Multipart message — attachments visible in raw view] — view raw

On 2018-05-09 at 16:20, David Wright wrote:

> On Tue 08 May 2018 at 22:39:55 (-0400), The Wanderer wrote:

>> What method are you using to delete it?
>> 
>> If you haven't already, I'd recommend trying 'rm -r', *very*
>> carefully, from a command prompt. (Unless you have an extremely
>> unusual setup, that should avoid any possibility of an intermediary
>> "Trash" to need emptying.)
> 
> As you're deleting a chain of directories, the appropriate command is
> rmdir.

When I initially wrote that, I didn't know that the chain of directories
didn't contain any non-directory items. (In fact, I'm still not 100%
certain I do know that, although I think I do.)

I also wasn't aware that rmdir could operate recursively; looking now, I
don't see any indication in its man page that it can. Without that, it
can only be used to delete one directory at a time, not "a chain of
directories". Even your own solution handles the "chain" part by using
find - which admittedly is the canonical default-correct way of doing
recursive operation in an *nix environment, but also adds a layer of
complexity not present in a simple 'rm -r'.

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man.         -- George Bernard Shaw

[toc] | [prev] | [next] | [standalone]

#195576 — Re: Running of rrequested tests - [was Re: Backup problem using "cp"]

On Wed 09 May 2018 at 22:56:48 (-0400), The Wanderer wrote:
> On 2018-05-09 at 16:20, David Wright wrote:
> 
> > On Tue 08 May 2018 at 22:39:55 (-0400), The Wanderer wrote:
> 
> >> What method are you using to delete it?
> >> 
> >> If you haven't already, I'd recommend trying 'rm -r', *very*
> >> carefully, from a command prompt. (Unless you have an extremely
> >> unusual setup, that should avoid any possibility of an intermediary
> >> "Trash" to need emptying.)
> > 
> > As you're deleting a chain of directories, the appropriate command is
> > rmdir.
> 
> When I initially wrote that, I didn't know that the chain of directories
> didn't contain any non-directory items. (In fact, I'm still not 100%
> certain I do know that, although I think I do.)

Fair enough. But the reason I wanted to use rmdir is precisely because
it will only remove empty directories. Any "real" information that
might happen to be present is protected from loss.

> I also wasn't aware that rmdir could operate recursively; looking now, I
> don't see any indication in its man page that it can. Without that, it
> can only be used to delete one directory at a time, not "a chain of
> directories". Even your own solution handles the "chain" part by using
> find - which admittedly is the canonical default-correct way of doing
> recursive operation in an *nix environment, but also adds a layer of
> complexity not present in a simple 'rm -r'.

Yes, by definition, the directories only become empty (and hence
subject to rmdir) when starting at the bottom. find -depth
solves that.

The problem with using rm -r is that it wipes out any files too.
Also you can't sensibly combine it with -i (beyond the trivial).
So naturally the complexity increases with the care taken.

In summary, those two functions will remove only directories that are
empty, and only prompt for such directories regardless of how deep
they lie.

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#195511 — Re: Running of rrequested tests - [was Re: Backup problem using "cp"]

On Tue 08 May 2018 at 11:38:52 (-0400), The Wanderer wrote:
> On 2018-05-08 at 11:10, Richard Owlett wrote:
> 
> > On 05/07/2018 07:01 AM, Thomas Schmitt wrote:
> >
> >> Hi,
> >> 
> >> Richard Owlett wrote:
> >>> richard@debian-jan13:~$ stat / | fgrep Device
> >>> Device: 80eh/2062d      Inode: 2           Links: 22
> >>> richard@debian-jan13:~$ stat /media | fgrep Device
> >>> Device: 80eh/2062d      Inode: 131073      Links: 5
> >> 
> >> "/media" was not the directory to examine.
> >> You need to examine the directory which you gave as second argument to cp.
> >> In your initial post of this thread the cp command is quoted as
> >> 
> >>    cp -ax /  "/media/richard/MISC
> >>    backups/dev_sda14/"
> >> 
> >> (Dunno whether the line break in that mail is significant.)
> >> [SNIP]
> > 
> > The line break was an email artifact and have changed partition label to 
> > prevent future confusion.
> > 
> > This is the output of script command:
> 
> <snip>
> 
> Have you tried
> 
> stat /home/richard/.local/share/Trash/expunged/1449727740/grub2
> problem-2018-02-13/
> 
> and/or a 'ls' of the same directory?
> 
> Since we've apparently confirmed that /media/richard/MISC-backups/ is on
> a separate filesystem from /, it really looks to me as if this too-deep
> directory chain may exist within the source tree, in some form.

It's unwise to distrust cp -x without a *lot* more evidence.

> The only comment I've found from you on this point seems to be a
> statement that yes, such a chain existed, but after you deleted it, it
> came back the next time you tried the copy.

Reiterating my question: what's the evidence that anything has
been deleted? "Emptied the trash" was the phrase used.

> If that's correct, then there must be something intervening to cause the
> chain to be re-created; the cp command should not be capable of
> modifying its source argument, unless its destination is under its
> source (which would obviously cause problems).
> 
> If it's not correct, then I'd be interested to know what is in fact
> there, and what might have caused it to exist (or not).

Recreating these directories is problematical. Whatever created them
happened on the morning of the first Tuesday in March. They could
only come back from the dead by being copied from yet another copy.
(Look at the file modification dates.)

So I think they've existed all the time. It might be interesting to
know what else has timestamps in that period. The find command can
do that rather easily with its -newer and -not switches.

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#195403

Richard Owlett wrote:
> [...]
> I suspect long term I want "rsync" [ *MUCH* reading to do! ]

Perhaps; although depending on the source / target pathing, you may run
into the same length issues.


-- 
|_|O|_| Registered Linux user #585947
|_|_|O| Github: https://github.com/dpurgert
|O|O|O| PGP: 05CA 9A50 3F2E 1335 4DC5  4AEE 8E11 DDF3 1279 A281

[toc] | [prev] | [next] | [standalone]

#195410

On 05/07/2018 06:54 AM, Dan Purgert wrote:
> Richard Owlett wrote:
>> [...]
>> I suspect long term I want "rsync" [ *MUCH* reading to do! ]
> 
> Perhaps; although depending on the source / target pathing, you may run
> into the same length issues.
> 

1st - I suspect that --exclude will eliminate multiple pit falls.
2nd - It appears that cp creates a tangle of sub-directories >100
       levels deep - each sub-directory has same string as its name ;{

I'll likely abandon further immediate investigation of cp. I've other 
projects to complete <grin>

[toc] | [prev] | [next] | [standalone]

#195414

Richard Owlett wrote:
> On 05/07/2018 06:54 AM, Dan Purgert wrote:
>> Richard Owlett wrote:
>>> [...]
>>> I suspect long term I want "rsync" [ *MUCH* reading to do! ]
>> 
>> Perhaps; although depending on the source / target pathing, you may run
>> into the same length issues.
>> 
>
> 1st - I suspect that --exclude will eliminate multiple pit falls.

Yep, it should.
> 2nd - It appears that cp creates a tangle of sub-directories >100
>        levels deep - each sub-directory has same string as its name ;{

If you did something that resolved to stick 'cp' into a recursive loop,
yup that'd definitely happen. Same with rsync.


-- 
|_|O|_| Registered Linux user #585947
|_|_|O| Github: https://github.com/dpurgert
|O|O|O| PGP: 05CA 9A50 3F2E 1335 4DC5  4AEE 8E11 DDF3 1279 A281

[toc] | [prev] | [next] | [standalone]

#195415

Hello,

On Mon, May 07, 2018 at 07:51:23AM -0500, Richard Owlett wrote:
> I'll likely abandon further immediate investigation of cp. I've other
> projects to complete <grin>

It would still be good to establish why "cp -x" was seemingly able
to cross filesystem boundaries as that would be a bug.

Cheers,
Andy

[toc] | [prev] | [next] | [standalone]

#195420

Hi,

Andy Smith wrote:
> It would still be good to establish why "cp -x" was seemingly able
> to cross filesystem boundaries as that would be a bug.

Yep. Leaving behind too many maybe-bugs can make the ground swampy.

I forgot to mention that the theory of David Wright is not outruled yet:

David Wright wrote:
> The loop is generating a source filename
> /home/richard/.local/share/Trash/expunged/73080846/grub2
> problem-2018-02-13/grub2 problem-2....018-02-13/grub2 problem-2018-02-13
> which is likely within length limits, and resides on the correct
> filesytem.

Loop and size overflow could indeed have been separated.
- First some step-on-own-foot loop would have created the deep directory
  tree until it failed due to path size. May have happened months ago.
  Now barely legal paths are lurking.
- This would then cause another failure when a non-looping copy attempt
  lengthens the path by prefix "/media/richard/MISCbackups/dev_sda14".

Richard would have to check whether there is such a deep tree on the disk
that shall be source of the copy.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#195428

On 05/07/2018 08:49 AM, Thomas Schmitt wrote:
> Hi,
> 
> Andy Smith wrote:
>> It would still be good to establish why "cp -x" was seemingly able
>> to cross filesystem boundaries as that would be a bug.
> 
> Yep. Leaving behind too many maybe-bugs can make the ground swampy.
> 
> I forgot to mention that the theory of David Wright is not outruled yet:
> 
> David Wright wrote:
>> The loop is generating a source filename
>> /home/richard/.local/share/Trash/expunged/73080846/grub2
>> problem-2018-02-13/grub2 problem-2....018-02-13/grub2 problem-2018-02-13
>> which is likely within length limits, and resides on the correct
>> filesytem.
> 
> Loop and size overflow could indeed have been separated.
> - First some step-on-own-foot loop would have created the deep directory
>    tree until it failed due to path size. May have happened months ago.
>    Now barely legal paths are lurking.
> - This would then cause another failure when a non-looping copy attempt
>    lengthens the path by prefix "/media/richard/MISCbackups/dev_sda14".
> 
> Richard would have to check whether there is such a deep tree on the disk
> that shall be source of the copy.

It was. I deleted. I emptied trash. It reappeared on my next run of cp.
My machine is still tied up and haven't rerun 'Check the device numbers 
of "/" and "/media/richard/MISC...". '

[toc] | [prev] | [next] | [standalone]

#195475

On Mon 07 May 2018 at 09:11:08 (-0500), Richard Owlett wrote:
> On 05/07/2018 08:49 AM, Thomas Schmitt wrote:
> >Hi,
> >
> >Andy Smith wrote:
> >>It would still be good to establish why "cp -x" was seemingly able
> >>to cross filesystem boundaries as that would be a bug.
> >
> >Yep. Leaving behind too many maybe-bugs can make the ground swampy.
> >
> >I forgot to mention that the theory of David Wright is not outruled yet:
> >
> >David Wright wrote:
> >>The loop is generating a source filename
> >>/home/richard/.local/share/Trash/expunged/73080846/grub2
> >>problem-2018-02-13/grub2 problem-2....018-02-13/grub2 problem-2018-02-13
> >>which is likely within length limits, and resides on the correct
> >>filesytem.
> >
> >Loop and size overflow could indeed have been separated.
> >- First some step-on-own-foot loop would have created the deep directory
> >   tree until it failed due to path size. May have happened months ago.
> >   Now barely legal paths are lurking.
> >- This would then cause another failure when a non-looping copy attempt
> >   lengthens the path by prefix "/media/richard/MISCbackups/dev_sda14".
> >
> >Richard would have to check whether there is such a deep tree on the disk
> >that shall be source of the copy.
> 
> It was. I deleted. I emptied trash.

I read these words, but have no idea what events actually take place.

> It reappeared on my next run of cp.
> My machine is still tied up and haven't rerun 'Check the device
> numbers of "/" and "/media/richard/MISC...". '

What would interest me is a listing of these files
(including their inodes (-i) too):

/home/richard/.local/share/Trash/expunged/73080846/
/home/richard/.local/share/Trash/expunged/73080846/grub2 problem-2018-02-13/
/home/richard/.local/share/Trash/expunged/73080846/grub2 problem-2018-02-13/grub2 problem-2018-02-13/

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

Page 2 of 4 — ← Prev page 1 [2] 3 4  Next page →

Back to top | Article view | linux.debian.user

csiph-web