Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #211872 > unrolled thread

Debian Buster: Is it safe to use on autodefrag on a Btrfs filesystem that is used for (Restic) backup only with no Btrfs snapshots or subvolumes?

Back to article view | Back to linux.debian.user

Contents

  Debian Buster: Is it safe to use on autodefrag on a Btrfs filesystem  that is used for (Restic) backup only with no Btrfs snapshots or subvolumes? Judah Richardson <judahrichardson@gmail.com> - 2019-07-29 17:30 +0200
    Re: Debian Buster: Is it safe to use on autodefrag on a Btrfs filesystem that is used for (Restic) backup only with no Btrfs snapshots or subvolumes? Stefan Monnier <monnier@iro.umontreal.ca> - 2019-07-29 20:10 +0200
      Re: Debian Buster: Is it safe to use on autodefrag on a Btrfs  filesystem that is used for (Restic) backup only with no Btrfs snapshots or  subvolumes? David Wright <deblis@lionunicorn.co.uk> - 2019-07-29 23:50 +0200
        Re: Debian Buster: Is it safe to use on autodefrag on a Btrfs filesystem that is used for (Restic) backup only with no Btrfs snapshots or subvolumes? Stefan Monnier <monnier@iro.umontreal.ca> - 2019-07-30 00:00 +0200
      Re: Debian Buster: Is it safe to use on autodefrag on a Btrfs filesystem that is used for (Restic) backup only with no Btrfs snapshots or subvolumes? Mart van de Wege <mvdwege@gmail.com> - 2019-07-31 13:00 +0200
        Re: Debian Buster: Is it safe to use on autodefrag on a Btrfs  filesystem that is used for (Restic) backup only with no Btrfs snapshots or subvolumes? Judah Richardson <judahrichardson@gmail.com> - 2019-07-31 16:10 +0200

#211872 — Debian Buster: Is it safe to use on autodefrag on a Btrfs filesystem that is used for (Restic) backup only with no Btrfs snapshots or subvolumes?

[Multipart message — attachments visible in raw view] — view raw

Hi All,

First time on the mailing list.

*System:*

OS: Debian Buster with KDE
CPU: Intel Core i3-2100
RAM: 8 GB
OS SSD: 1 TB Crucial MX500 SSD, where /home folder is located. Formatted as
ext4
Other HDD: 2 x 2 TB Toshiba L200 HDD, both used completely for Btrfs RAID1
(files and metadata) with a single subvolume, no Btrfs snapshots

*Goal:*

To store rotating snapshots of some folders in /home/MyUsername/ (located
on SSD) on the Btrfs filesystem located on the 2 HDDs. I've chosen Restic
<https://restic.net/> for this since rsnapshot's developer switched to Borg
and Borg doesn't do snapshots.

*Problem:*

I want to minimize fragmentation on the Btrfs file system, but Debian's
Wiki recommends against usin <https://wiki.debian.org/Btrfs#Recommendations>g
-autodefrag. The reason for this seems to be found in the Warnings section
<https://wiki.debian.org/Btrfs#Warnings>:

>
>    - Mounting with -o autodefrag will often duplicate reflinked or
>    snapshotted files when a balance is run. (TODO: is this still current for
>    linux-4.19.x?)
>    - Any "btrfs filesystem defrag" operation can potentially duplicate
>    reflinked or snapshotted blocks. Files with shared extents lose their
>    shared reflinks, which are then duplicated with n-copies. The effect of the
>    lack of "snapshot aware defrag" is that volumes that make heavy use of
>    reflinks or snapshots will unexpectedly run out of free space. Avoid this
>    by minimizing the use of snapshots, and instead use deduplicating backup
>    software to store backups efficiently (eg: borgbackup).
>
> *Question:*

Given that Restic:

   - doesn't use reflinks
   - uses its own snapshot system (not Btrfs')
   - performs deduplication

Is it safe to use autodefrag for my use case?

[toc] | [next] | [standalone]

#211882 — Re: Debian Buster: Is it safe to use on autodefrag on a Btrfs filesystem that is used for (Restic) backup only with no Btrfs snapshots or subvolumes?

> Is it safe to use autodefrag for my use case?

It sounds like it might be "safe" (the text doesn't actually say it's
unsafe, but just that it has downsides).

I do wonder why you'd want to do that, tho.  Fragmentation is typically
something that clueless Windows users worry about (a left over from the
days of FAT filesystems; and even tho it became largely useless under
NTFS it persisted because it's easier to tell people how to defragment
than to explain to them why it's not needed any more).

Do you have a concrete reason to think that defragmentation will bring
you any tangible benefit?


        Stefan

[toc] | [prev] | [next] | [standalone]

#211896 — Re: Debian Buster: Is it safe to use on autodefrag on a Btrfs filesystem that is used for (Restic) backup only with no Btrfs snapshots or subvolumes?

On Mon 29 Jul 2019 at 14:07:25 (-0400), Stefan Monnier wrote:
> > Is it safe to use autodefrag for my use case?
> 
> It sounds like it might be "safe" (the text doesn't actually say it's
> unsafe, but just that it has downsides).
> 
> I do wonder why you'd want to do that, tho.  Fragmentation is typically
> something that clueless Windows users worry about (a left over from the
> days of FAT filesystems; and even tho it became largely useless under
> NTFS it persisted because it's easier to tell people how to defragment
> than to explain to them why it's not needed any more).
> 
> Do you have a concrete reason to think that defragmentation will bring
> you any tangible benefit?

Btrfs is not my thing, but I looked at the reference that was posted,
and the Gotchas referred to within, and they mention log files having
tens of thousands of extents. Doesn't sound very good.

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#211898 — Re: Debian Buster: Is it safe to use on autodefrag on a Btrfs filesystem that is used for (Restic) backup only with no Btrfs snapshots or subvolumes?

> Btrfs is not my thing, but I looked at the reference that was posted,
> and the Gotchas referred to within, and they mention log files having
> tens of thousands of extents. Doesn't sound very good.

Maybe it doesn't sound very good, but except for very specific
circumstances, it makes no visible difference.


        Stefan

[toc] | [prev] | [next] | [standalone]

#211959 — Re: Debian Buster: Is it safe to use on autodefrag on a Btrfs filesystem that is used for (Restic) backup only with no Btrfs snapshots or subvolumes?

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Is it safe to use autodefrag for my use case?
>
> It sounds like it might be "safe" (the text doesn't actually say it's
> unsafe, but just that it has downsides).
>
> I do wonder why you'd want to do that, tho.  Fragmentation is typically
> something that clueless Windows users worry about

No. Fragmentation is an issue with all copy-on-write filesystems
(including ZFS, which avoids periodic defrag by keeping an enormous
amount of information in memory and doing defrag on the fly on that).

Mart

-- 
"We will need a longer wall when the revolution comes."
--- AJS, quoting an uncertain source.

[toc] | [prev] | [next] | [standalone]

#211977 — Re: Debian Buster: Is it safe to use on autodefrag on a Btrfs filesystem that is used for (Restic) backup only with no Btrfs snapshots or subvolumes?

[Multipart message — attachments visible in raw view] — view raw

This. From the Btfrs Gotchas page:

Files with a lot of random writes can become heavily fragmented (10000+
> extents) causing thrashing on HDDs and excessive multi-second spikes of CPU
> load on systems with an SSD or large amount a RAM.
>
>    - On servers and workstations this affects databases and virtual
>    machine images.
>       - The nodatacow mount option
>       <https://btrfs.wiki.kernel.org/index.php/Mount_options> may be of
>       use here, with associated gotchas.
>    - On desktops this primarily affects application databases (including
>    Firefox and Chromium profiles, GNOME Zeitgeist, Ubuntu Desktop Couch,
>    Banshee, and Evolution's datastore.)
>       - Workarounds include manually defragmenting your home directory
>       using btrfs fi defragment. Auto-defragment (mount option autodefrag) should
>       solve this problem in 3.0.
>    - Symptoms include btrfs-transacti and btrfs-endio-wri taking up a lot
>    of CPU time (in spikes, possibly triggered by syncs). You can use filefrag
>    to locate heavily fragmented files (may not work correctly with
>    compression).
>
>
On Wed, Jul 31, 2019 at 5:50 AM Mart van de Wege <mvdwege@gmail.com> wrote:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
> >> Is it safe to use autodefrag for my use case?
> >
> > It sounds like it might be "safe" (the text doesn't actually say it's
> > unsafe, but just that it has downsides).
> >
> > I do wonder why you'd want to do that, tho.  Fragmentation is typically
> > something that clueless Windows users worry about
>
> No. Fragmentation is an issue with all copy-on-write filesystems
> (including ZFS, which avoids periodic defrag by keeping an enormous
> amount of information in memory and doing defrag on the fly on that).
>
> Mart
>
> --
> "We will need a longer wall when the revolution comes."
> --- AJS, quoting an uncertain source.
>
>

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web