Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #274049 > unrolled thread

recover files

Back to article view | Back to linux.debian.user

Contents

  recover files louletian@sina.com - 2024-10-25 20:40 +0200
    Re: recover files Hans <hans.ullrich@loop.de> - 2024-10-25 21:10 +0200
      Re: recover files Max Nikulin <manikulin@gmail.com> - 2024-10-26 05:00 +0200
        Re: recover files <tomas@tuxteam.de> - 2024-10-26 08:30 +0200
        Re: recover files Hans <hans.ullrich@loop.de> - 2024-10-26 10:00 +0200
          Re: recover files Max Nikulin <manikulin@gmail.com> - 2024-10-26 13:40 +0200
            Re: recover files Hans <hans.ullrich@loop.de> - 2024-10-26 15:30 +0200
              Re: recover files Max Nikulin <manikulin@gmail.com> - 2024-10-26 17:10 +0200
                Re: recover files Hans <hans.ullrich@loop.de> - 2024-10-26 18:40 +0200
                  Re: recover files <tomas@tuxteam.de> - 2024-10-26 18:50 +0200

#274049 — recover files

[Multipart message — attachments visible in raw view] — view raw

Hi folksIs there possible to recover deleted files in ext4 filesystem?
--------------------------------

[toc] | [next] | [standalone]

#274052

Am Freitag, 25. Oktober 2024, 20:32:29 CEST schrieb louletian@sina.com:
> Hi folksIs there possible to recover deleted files in ext4 filesystem?
> --------------------------------

Try extundelete.

Hint: Make an image from the whole partition using dd before do any recover 
tries. Then use the image to recover files. Make a copy of the image, so you 
always have a untouched original available.

Using an image, you can try nice tools like foremost, scalpel or autopsy to 
recover files.

Also you can use CAINE, KALI or DEFT (these are forensis suites) for data 
recovery.

If you are carefully, you might get many files back.

Good luck!

Hans

[toc] | [prev] | [next] | [standalone]

#274061

On 26/10/2024 02:03, Hans wrote:
> Am Freitag, 25. Oktober 2024, 20:32:29 CEST schrieb louletian@sina.com:
>> Hi folksIs there possible to recover deleted files in ext4 filesystem?
> 
> Try extundelete.
[...]
> Using an image, you can try nice tools like foremost, scalpel or autopsy to
> recover files.
> 
> Also you can use CAINE, KALI or DEFT (these are forensis suites) for data
> recovery.

Have you tried these tools in action? I believe that removing files from 
ext4 wipes list of used blocks and explicitly zeroes size in inode 
records, so a chance of recovery is quite low. Some info for recently 
accessed files may be restored from filesystem journal. In the case of 
contiguous block spans signature-based search (e.g. photorec from the 
testdisk package) may find some files, but they will be buried in the 
heap of false positives without any hints related to file names and 
directory structures.

I am quite skeptical concerning fraction of successfully recovered files.

[toc] | [prev] | [next] | [standalone]

#274064

[Multipart message — attachments visible in raw view] — view raw

On Sat, Oct 26, 2024 at 09:57:11AM +0700, Max Nikulin wrote:
> On 26/10/2024 02:03, Hans wrote:
> > Am Freitag, 25. Oktober 2024, 20:32:29 CEST schrieb louletian@sina.com:
> > > Hi folksIs there possible to recover deleted files in ext4 filesystem?
> > 
> > Try extundelete.
> [...]
> > Using an image, you can try nice tools like foremost, scalpel or autopsy to
> > recover files.
> > 
> > Also you can use CAINE, KALI or DEFT (these are forensis suites) for data
> > recovery.
> 
> Have you tried these tools in action? I believe that removing files from
> ext4 wipes list of used blocks and explicitly zeroes size in inode records,
> so a chance of recovery is quite low. Some info for recently accessed files
> may be restored from filesystem journal. In the case of contiguous block
> spans signature-based search (e.g. photorec from the testdisk package) may
> find some files, but they will be buried in the heap of false positives
> without any hints related to file names and directory structures.

I actually tried photorec a while ago. It uses heuristics to guess which
free blocks to stitch together and it's not *that* bad. But: you have to
be quick (the more you use your file system the worse your recovery chances),
you need *a lot* of extra storage (there's more than one way to stitch
a junkyard together) and you'll have to wade through quite a bit of stuff.

And oh, it takes quite a bit of time to chew through a file system.

Cheers
-- 
t

[toc] | [prev] | [next] | [standalone]

#274065

Yes, whilst extundelete is not so easy to use, I was very successfull with 
photorec and autopsy.

Last time I had to revover 2 TB music files for a friend, and photorec gave me 
all files back. However, i had to rename the filenames to the title of the 
music, but here puddletag could help. As ypou mentioned, blocks were deleteed, 
too, so filenames could not be recoverewd any more.

For special files you can also use scalpel. It looks for header, then footer 
and everything between is the file. However, scalpel is not easy to configure.

Also foremost is another tool of my favourites, as it is easy to use.

And last but not least not to forget about testdisk (for lost partitions) ans 
autopsy (latest version!) are doing great jobs.

Ah, and the suite I am working with is CAINE and also KALI, because these are 
livefile systems and this is necessary, as you want to rescue files from an 
unmounted device.

Best

Hans 



> Have you tried these tools in action? I believe that removing files from
> ext4 wipes list of used blocks and explicitly zeroes size in inode
> records, so a chance of recovery is quite low. Some info for recently
> accessed files may be restored from filesystem journal. In the case of
> contiguous block spans signature-based search (e.g. photorec from the
> testdisk package) may find some files, but they will be buried in the
> heap of false positives without any hints related to file names and
> directory structures.
> 
> I am quite skeptical concerning fraction of successfully recovered files.

[toc] | [prev] | [next] | [standalone]

#274072

On 26/10/2024 14:53, Hans wrote:
> Yes, whilst extundelete is not so easy to use,

Thank you for the detailed answer.

I have tried ext4magic. My impression is that it might have an issue 
with reading journal and that it is unnecessary strict walking through 
inodes (zeroing invalidates checksums if I remember it correctly). It 
may restore some files, however I can not figure out what approach 
extundelete or other tools may use to noticeably improve success rate 
since important data is overwritten.

> I was very successfull with photorec and autopsy.

Does autopsy/sleuthkit use some heuristic that allows to restore 
significantly more data than extundelete and photorec in the case of 
unintentional removing of directories?

> Last time I had to revover 2 TB music files for a friend, and photorec gave me
> all files back.

Of course, a few MB size files with reach metadata (audio, image, zip) 
is an optimal case for photorec and foremost. For 1 hour long .mp3 files 
fragmentation causes recovery of only some parts of files (at least in 
the case of FAT32).

> Also foremost is another tool of my favourites, as it is easy to use.

I am curious what are cases when it may perform noticeably better than 
photorec.

[toc] | [prev] | [next] | [standalone]

#274076

> Thank you for the detailed answer.

youre welcome.
> 
> I have tried ext4magic. My impression is that it might have an issue
> with reading journal and that it is unnecessary strict walking through
> inodes (zeroing invalidates checksums if I remember it correctly). It
> may restore some files, however I can not figure out what approach
> extundelete or other tools may use to noticeably improve success rate
> since important data is overwritten.
> 
As far as I know, it does not use journal. It is looking at the data it reads 
(form the imagefile) and then finds headers and footers (similar to scalpel) 
and as what it reada, it creates the surname (like jpg , doc whatever). As it 
reads also the metadata in the header, it knows, what kind of file it reaches.

> > I was very successfull with photorec and autopsy.
> 
> Does autopsy/sleuthkit use some heuristic that allows to restore
> significantly more data than extundelete and photorec in the case of
> unintentional removing of directories?

as fatr as I know, it doews, but I am no coder, so I can not prove it.
> 
> > Last time I had to revover 2 TB music files for a friend, and photorec
> > gave me all files back.
> 
> Of course, a few MB size files with reach metadata (audio, image, zip)
> is an optimal case for photorec and foremost. For 1 hour long .mp3 files
> fragmentation causes recovery of only some parts of files (at least in
> the case of FAT32).

Ah no, these were not only a few audio files. These were about 90.000 audio 
files (which I all could recover) whilst about 2000 I could not rename again 
(mp3 was too old version). The whole process lasted per run about 36 hours! 

First I had to create an image of the drive (this was about 2TB), which lasted 
about 28 hours (as it was an USB-drive I had to use the USB-port), then 
several scnas with different tools. Each scan about 36 hours, then after I got 
all files, renaming (using puddletag) and sorting (using find). 

All about several days and nights (it was for a good friend of mine!) and I 
was happy of the result. An he, too!

What I wanted to say: I am always using several and different tools to get 
best results over all. It is time consuming, yes, but that is what I in my 
first post meant: If you are doing carefully, you will get most data 
recovered.

> 
> > Also foremost is another tool of my favourites, as it is easy to use.
> 
> I am curious what are cases when it may perform noticeably better than
> photorec.

Oh, you will have noticed, that I not mentioned some of the commercial tools 
like FTK or ENCASE. I am not using these, I do not like those (for some 
personal reasons) and the free tools are fully satisfying my needs.

Hans

Best

Hans

[toc] | [prev] | [next] | [standalone]

#274079

On 26/10/2024 20:26, Hans wrote:
> On 26/10/2024 18:37, Max Nikulin wrote:
>> however I can not figure out what approach
>> extundelete or other tools may use to noticeably improve success rate
>> since important data is overwritten.
>>
> As far as I know, it does not use journal. It is looking at the data it reads
> (form the imagefile) and then finds headers and footers (similar to scalpel)

apt show extundelete
[...]
Description: utility to recover deleted files from ext3/ext4 partition
  extundelete uses the information stored in the partition's journal to 
attempt
  to recover a file that has been deleted.

So it sounds like FS journal is the primary source for recovery and thus 
my expectation that it can not restore files written long time ago and 
that are not accessed recently. (Updating access time causes creating a 
journal record for inode with a copy of block list.)

>>> Last time I had to revover 2 TB music files for a friend, and photorec
>>> gave me all files back.
>>
>> Of course, a few MB size files with reach metadata (audio, image, zip)
>> is an optimal case for photorec and foremost. For 1 hour long .mp3 files
>> fragmentation causes recovery of only some parts of files (at least in
>> the case of FAT32).
> 
> Ah no, these were not only a few audio files. These were about 90.000 audio
> files

Sorry, I was unclear. I mean fair probability to recover files in the 
range of 1-5 MB each, but large files (50-200 MB or more) may be 
troublesome. The tool limitation is contiguous span of blocks. A disk 
dedicated to music collection is a much easier case than e.g. mix of 
files having wide range of types and sizes in home directories.

>>> Also foremost is another tool of my favourites, as it is easy to use.
>>
>> I am curious what are cases when it may perform noticeably better than
>> photorec.
> 
> Oh, you will have noticed, that I not mentioned some of the commercial tools
> like FTK or ENCASE. I am not using these, I do not like those (for some
> personal reasons) and the free tools are fully satisfying my needs.

My question was specific to foremost vs. photorec. Your words make me 
thinking that I have missed some feature of foremost.

[toc] | [prev] | [next] | [standalone]

#274081

Even very big files should not be the problem, because, when the header is 
found all date until the footer are the file.

When you are quick and do not overwrite your device (thus create an iumage as 
soon as possible!), also big data can be saved.

And here comes scalpel in handy: scalpel has the ability, that you can define, 
how a header is looking and how the footer is looking. This makes it super 
felixible. But, and this is the problem, you must exactly know, what you 
define. Its best for coders, bad for no-coders like me. :)

Data rescue is not an easy task! And I am still learning.

Hans
 
> Sorry, I was unclear. I mean fair probability to recover files in the
> range of 1-5 MB each, but large files (50-200 MB or more) may be
> troublesome. The tool limitation is contiguous span of blocks. A disk
> dedicated to music collection is a much easier case than e.g. mix of
> files having wide range of types and sizes in home directories.
> 

[toc] | [prev] | [next] | [standalone]

#274082

[Multipart message — attachments visible in raw view] — view raw

On Sat, Oct 26, 2024 at 06:30:07PM +0200, Hans wrote:
> Even very big files should not be the problem, because, when the header is 
> found all date until the footer are the file.

Well, but that file content is cut up in little 4K snippets strewn around
your disk's free space.

The "backbone" holding that together is also strewn around like that.
Some parts have been overwritten.

It's... messy.

Cheers
-- 
t

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web