Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #165317 > unrolled thread

Need help salvaging what I can from major disaster

Back to article view | Back to linux.debian.user

Contents

  Need help salvaging what I can from major disaster Kynn Jones <kynnjo@gmail.com> - 2016-03-13 01:20 +0100
    Re: Need help salvaging what I can from major disaster Kynn Jones <kynnjo@gmail.com> - 2016-03-13 02:10 +0100
      Re: Need help salvaging what I can from major disaster Kynn Jones <kynnjo@gmail.com> - 2016-03-13 05:00 +0100
        Re: Need help salvaging what I can from major disaster rlharris@oplink.net - 2016-03-13 06:00 +0100
      Re: Need help salvaging what I can from major disaster Pascal Hambourg <pascal@plouf.fr.eu.org> - 2016-03-13 10:50 +0100
        Re: Need help salvaging what I can from major disaster Kynn Jones <kynnjo@gmail.com> - 2016-03-13 21:40 +0100
          Re: Need help salvaging what I can from major disaster Andrew McGlashan <andrew.mcglashan@affinityvision.com.au> - 2016-03-14 20:50 +0100
            Re: Need help salvaging what I can from major disaster Andrew McGlashan <andrew.mcglashan@affinityvision.com.au> - 2016-03-14 21:00 +0100
            Re: Need help salvaging what I can from major disaster Andrew McGlashan <andrew.mcglashan@affinityvision.com.au> - 2016-03-14 21:00 +0100

#165317 — Need help salvaging what I can from major disaster

The plan was:

  1. back up my hard drive to a new external drive;
  2. reinstall Debian;
  3. restore my data from the external drive;

I did 1 and 2, but when I went to restore data from the backup, there
was nothing there (except for a lost+found).

I don't understand how this could have happened.  I've done this sort
of thing many times, and this has never happened before.  (Needless to
say, I'm ready to shoot myself.)

Now my most pressing need is to salvage whatever I can from the hard disk.

Any advice/recommendations would be much appreciated.

kj

[toc] | [next] | [standalone]

#165319

I realize that my last e-mail included too little information.  Sorry
about that.  Here are some more relevant details.

It turns out that I backed up the hard disk (HD) to the external drive
(XD) twice.  The first backup went well, as far as I can tell.  I
spot-checked individual files, etc.  Everything looked normal.

This first backup would have sufficed, but after I had done it I came
across the recommendation that a disk of that size (2TB) should be
formatted for ext4, whereas I had originally formatted it for ext2.  I
reformatted the disk for ext4 (fdisk + mkfs.ext4), and then repeated
the backup.  This backup procedure was in the form of a shell script,
that I had tested repeatedly.  I just re-ran it.  My memory is fuzzy
on the subsequent details.  Since everything had gone so well up to
that point, I probably did not check everything as obsessively as I
had up to that point, but something must have gone very wrong.

At any rate, at the moment, my only hopes are that

  1. something may be recoverable from the first backup in XD;
  2. something may be recoverable from HD.

The second of these is (I think) less likely, because I changed the
partition scheme for HD, going from a single partition, to separate
partitions for /, /var, /tmp, and swap.  In contrast, XD has always
consisted of a single partition.

Now I'm looking for some data recovery tool (preferably from Debian)
that either is part of some bootable media, or that I can use from a
USB stick after first booting from a Debian Live CD.

Any recommendations for these would be appreciated.

Many thanks in advance!

kj

[toc] | [prev] | [next] | [standalone]

#165321

OK, post-mortem:

  1. after reviewing everything, the best hypothesis I have is that
the root of the problem was a bug in my backup script: it should have
checked that the backup disk was properly mounted before proceeding
with the back-up; something may have gone wrong with the mounting of
XD after it was reformatted, but this failure went undetected; rsync
just created the target directory under /media in HD.

  2. it looks like photorec can find a fair bit of recoverable data
from the first XD backup, but it's going to be a long slog to sort out
the recovered files (400GB's worth); for most of this I have an older
backup from 5 months ago; so the problem will be to identify the stuff
added since then.

I guess this is what learning Debian in the "school of hard knocks" is
all about...  This one knocked me out cold.

kj


On 3/12/16, Kynn Jones <kynnjo@gmail.com> wrote:
> I realize that my last e-mail included too little information.  Sorry
> about that.  Here are some more relevant details.
>
> It turns out that I backed up the hard disk (HD) to the external drive
> (XD) twice.  The first backup went well, as far as I can tell.  I
> spot-checked individual files, etc.  Everything looked normal.
>
> This first backup would have sufficed, but after I had done it I came
> across the recommendation that a disk of that size (2TB) should be
> formatted for ext4, whereas I had originally formatted it for ext2.  I
> reformatted the disk for ext4 (fdisk + mkfs.ext4), and then repeated
> the backup.  This backup procedure was in the form of a shell script,
> that I had tested repeatedly.  I just re-ran it.  My memory is fuzzy
> on the subsequent details.  Since everything had gone so well up to
> that point, I probably did not check everything as obsessively as I
> had up to that point, but something must have gone very wrong.
>
> At any rate, at the moment, my only hopes are that
>
>   1. something may be recoverable from the first backup in XD;
>   2. something may be recoverable from HD.
>
> The second of these is (I think) less likely, because I changed the
> partition scheme for HD, going from a single partition, to separate
> partitions for /, /var, /tmp, and swap.  In contrast, XD has always
> consisted of a single partition.
>
> Now I'm looking for some data recovery tool (preferably from Debian)
> that either is part of some bootable media, or that I can use from a
> USB stick after first booting from a Debian Live CD.
>
> Any recommendations for these would be appreciated.
>
> Many thanks in advance!
>
> kj
>

[toc] | [prev] | [next] | [standalone]

#165323

On Sat, March 12, 2016 9:59 pm, Kynn Jones wrote:
> OK, post-mortem:
...
> 1. after reviewing everything, the best hypothesis I have is that
> the root of the problem was a bug in my backup script: it should have...

I use gnome-disk-utility to mount the external usb3 drive and then I open
a terminal running Midnight Commander to check that I am using the correct
rsync command, which command I run manually.  Not quite as automated, but
I think it is safer.

I do use a script for jigdo download of weekly ISO images, but those mount
points are on the hard drive and do not change.

You may find fslint useful for detecting duplicates between your old
backup and files in the recovery directory.

Russ

[toc] | [prev] | [next] | [standalone]

#165326

Kynn Jones a écrit :
> 
> It turns out that I backed up the hard disk (HD) to the external drive
> (XD) twice.  The first backup went well, as far as I can tell.  I
> spot-checked individual files, etc.  Everything looked normal.
> 
> This first backup would have sufficed, but after I had done it I came
> across the recommendation that a disk of that size (2TB) should be
> formatted for ext4, whereas I had originally formatted it for ext2.  I

Ext2 can handle 2 TB just fine.

> reformatted the disk for ext4 (fdisk + mkfs.ext4), and then repeated
> the backup.  This backup procedure was in the form of a shell script,
> that I had tested repeatedly.  I just re-ran it.  My memory is fuzzy
> on the subsequent details.  Since everything had gone so well up to
> that point, I probably did not check everything as obsessively as I
> had up to that point, but something must have gone very wrong.

Before reading your next post, I also thought that the backup disk was
not properly mounted after being reformatted. There should be some form
of size limit or quota on /media to warn against this kind of common
mistake.

> At any rate, at the moment, my only hopes are that
> 
>   1. something may be recoverable from the first backup in XD;
>   2. something may be recoverable from HD.
> 
> The second of these is (I think) less likely, because I changed the
> partition scheme for HD, going from a single partition, to separate
> partitions for /, /var, /tmp, and swap.  In contrast, XD has always
> consisted of a single partition.

Yes, but not for this reason. Creating partitions does very little
modification to a disk.
IMO there are two better reasons :
- There were much fewer overwrites to the XD (only filesystem metadata)
than the HD (complete system installation).
- Files on the XD were written in sequence and not modified, meaning
that they are less fragmented and easier to recover.

> Now I'm looking for some data recovery tool (preferably from Debian)
> that either is part of some bootable media, or that I can use from a
> USB stick after first booting from a Debian Live CD.

I would have suggested photorec but you already found out.

[toc] | [prev] | [next] | [standalone]

#165341

@Russ, @Pascal: thanks for your comments and suggestions.  They are
very helpful.

kj

[toc] | [prev] | [next] | [standalone]

#165378

[Multipart message — attachments visible in raw view] — view raw

On 14/03/2016 7:38 AM, Kynn Jones wrote:
> @Russ, @Pascal: thanks for your comments and suggestions.  They are
> very helpful.

A couple of things.

ext2 is virtually the same as ext3 and ext4, but the latter two have
journal logs.

Earlier today I was working on a script and I accidentally deleteed it,
brought it back with this tool [1].

I wonder if you could re-align your original partition and then try to
do recovery of that?  There won't be any journal to help though, so the
tool I used isn't going to help; but there might be other options to
reconstruct the file system from super blocks that haven't been trashed.

Cheers
AndrewM

[1]  http://extundelete.sourceforge.net/

# aptitude show extundelete
Monday 14 March 22:32:14 AEDT 2016 -- show extundelete
Package: extundelete
State: installed
Automatically installed: no
Version: 0.2.0-2.1
Priority: optional
Section: utils
Maintainer: Debian Forensics <forensics-devel@lists.alioth.debian.org>
Architecture: amd64
Uncompressed Size: 157 k
Depends: e2fslibs (>= 1.42), libc6 (>= 2.2.5), libcomerr2 (>= 1.01),
libgcc1 (>= 1:4.1.1), libstdc++6 (>= 4.6)
Description: utility to recover deleted files from ext3/ext4 partition
 extundelete uses the information stored in the partition's journal to
attempt to recover a file that has been deleted. There is no guarantee
that any particular file will be able to be undeleted.
Homepage: http://extundelete.sourceforge.net/


# umount /home/andrewm/other
# extundelete --restore-file install-wrk/veracrypt/1.17/get-files.sh
/dev/mapper/n4800eco--a----vg1-home--andrewm--other
# mount /home/andrewm/other

# find /root/RECOVERED_FILES/
/root/RECOVERED_FILES/
/root/RECOVERED_FILES/install-wrk
/root/RECOVERED_FILES/install-wrk/veracrypt
/root/RECOVERED_FILES/install-wrk/veracrypt/1.17
/root/RECOVERED_FILES/install-wrk/veracrypt/1.17/get-files.sh

# cp -p RECOVERED_FILES/install-wrk/veracrypt/1.17/get-files.sh
/home/andrewm/other/install-wrk/veracrypt/1.17/

[toc] | [prev] | [next] | [standalone]

#165380

[Multipart message — attachments visible in raw view] — view raw

This might help, using one of the super blocks that hasn't been overwritten:

http://www.cyberciti.biz/faq/linux-find-alternative-superblocks/

Cheers
AndrewM

[toc] | [prev] | [next] | [standalone]

#165381

[Multipart message — attachments visible in raw view] — view raw

On 15/03/2016 6:46 AM, Andrew McGlashan wrote:
> I wonder if you could re-align your original partition and then try to
> do recovery of that?  There won't be any journal to help though, so the
> tool I used isn't going to help; but there might be other options to
> reconstruct the file system from super blocks that haven't been trashed.

Oh and I would not reformat it once the partition is back to original;
perhaps if you can do a dry run and then you can see where the
superblocks are probably written as they were before.


Part of the man page for mke2fs:

  -n     Causes mke2fs to not actually create a filesystem


Cheers
AndrewM

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web