Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.bugs.dist > #1120875 > unrolled thread

Bug#1021750: general: the nodelalloc mount option should be used by default for ext4 in /etc/fstab

Back to article view | Back to linux.debian.bugs.dist

Contents

  Bug#1021750: general: the nodelalloc mount option should be used by default for ext4 in /etc/fstab Vincent Lefevre <vincent@vinc17.net> - 2022-10-14 03:10 +0200
    Bug#1021750: general: the nodelalloc mount option should be used by default for ext4 in /etc/fstab nick black <dankamongmen@gmail.com> - 2022-10-14 08:10 +0200
    Bug#1021750: general: the nodelalloc mount option should be used by default for ext4 in /etc/fstab Marco d'Itri <md@Linux.IT> - 2022-10-14 12:10 +0200
      Bug#1021750: general: the nodelalloc mount option should be used by default for ext4 in /etc/fstab Vincent Lefevre <vincent@vinc17.net> - 2022-10-14 12:40 +0200
        Bug#1021750: general: the nodelalloc mount option should be used by default for ext4 in /etc/fstab Marco d'Itri <md@linux.it> - 2022-10-14 12:50 +0200
          Bug#1021750: general: the nodelalloc mount option should be used by default for ext4 in /etc/fstab Vincent Lefevre <vincent@vinc17.net> - 2022-10-14 14:00 +0200
            Bug#1021750: general: the nodelalloc mount option should be used by default for ext4 in /etc/fstab Marco d'Itri <md@linux.it> - 2022-10-14 15:40 +0200
              Bug#1021750: general: the nodelalloc mount option should be used by default for ext4 in /etc/fstab "Theodore Ts'o" <tytso@mit.edu> - 2022-10-14 20:10 +0200

#1120875 — Bug#1021750: general: the nodelalloc mount option should be used by default for ext4 in /etc/fstab

Package: general
Severity: normal

The /etc/fstab file is created using by default ext4 with just
the errors=remount-ro option. However, the Debian FAQ recommends
the nodelalloc mount option to avoid performance degradation and
preserve data safety:

https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Why_is_dpkg_so_slow_when_using_new_filesystems_such_as_btrfs_or_ext4.3F

  For ext4, use instead the "nodelalloc" mount option, which should
  fix both the performance degradation and the data safety issues,
  and not for just dpkg, but for any application in the system.

So this option should be used by default for btrfs and ext4.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

[toc] | [next] | [standalone]

#1120883

i'm pretty sure that the corruption issues leading to the
nodelalloc option were considered largely remedied by the
"auto_da_alloc" capability introduced (and enabled by default)
in 2.6.30? how would nodelalloc equal the performance of
delalloc? nodelalloc was all about reliability for programs that
weren't conforming to certain POSIX semantics, as i recall.

[toc] | [prev] | [next] | [standalone]

#1120899

[Multipart message — attachments visible in raw view] — view raw

On Oct 14, Vincent Lefevre <vincent@vinc17.net> wrote:

> The /etc/fstab file is created using by default ext4 with just
> the errors=remount-ro option. However, the Debian FAQ recommends
> the nodelalloc mount option to avoid performance degradation and
> preserve data safety:
This is not "the Debian FAQ" but "the DPKG FAQ", which has been known to 
recommend awful things in the past.

There is a reason if delayed allocation is enabled by default and this 
should not be changed without a detailed analysis.

-- 
ciao,
Marco

[toc] | [prev] | [next] | [standalone]

#1120900

On 2022-10-14 11:59:09 +0200, Marco d'Itri wrote:
> On Oct 14, Vincent Lefevre <vincent@vinc17.net> wrote:
> 
> > The /etc/fstab file is created using by default ext4 with just
> > the errors=remount-ro option. However, the Debian FAQ recommends
> > the nodelalloc mount option to avoid performance degradation and
> > preserve data safety:
> This is not "the Debian FAQ" but "the DPKG FAQ", which has been known to 
> recommend awful things in the past.

But it is still considered in the present times by the dpkg developers.
Bug 923423 was closed several hours ago based on this dpkg FAQ:

------------------------------------------------------------------------
All these fsync()s you see in rapid succession are used as a
synchronization points, way after the data has been requested to be
synced to disk asynchronously via sync_file_range().

What this is trying to achieve is durability, so that dpkg can know
the data is on the disk, so that it can mark the package as installed.

This is explained on the dpkg FAQ:

  <https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Why_is_dpkg_so_slow_when_using_new_filesystems_such_as_btrfs_or_ext4.3F>
------------------------------------------------------------------------

(the issue here is that there is a fsync for *every* file, so that for
a package with many small files, such as texlive-fonts-extra, which
currently has more than 92000 files, this can be awfully slow).

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

[toc] | [prev] | [next] | [standalone]

#1120901

[Multipart message — attachments visible in raw view] — view raw

On Oct 14, Vincent Lefevre <vincent@vinc17.net> wrote:

> > This is not "the Debian FAQ" but "the DPKG FAQ", which has been known to 
> > recommend awful things in the past.
> But it is still considered in the present times by the dpkg developers.
> Bug 923423 was closed several hours ago based on this dpkg FAQ:
This is obviously convenient on Guillem's part, but we have to optimize 
systems by default for the general case and not just for dpkg -i.

> (the issue here is that there is a fsync for *every* file, so that for
And at this point I am wondering if this is a good design, or if it 
would be better to use a different strategy.

-- 
ciao,
Marco

[toc] | [prev] | [next] | [standalone]

#1120915

On 2022-10-14 12:44:25 +0200, Marco d'Itri wrote:
> On Oct 14, Vincent Lefevre <vincent@vinc17.net> wrote:
> > > This is not "the Debian FAQ" but "the DPKG FAQ", which has been known to 
> > > recommend awful things in the past.
> > But it is still considered in the present times by the dpkg developers.
> > Bug 923423 was closed several hours ago based on this dpkg FAQ:
> This is obviously convenient on Guillem's part, but we have to optimize 
> systems by default for the general case and not just for dpkg -i.

This dpkg FAQ says that this is not beneficial for just dpkg,
but also "for any application in the system".

> > (the issue here is that there is a fsync for *every* file, so that for
> And at this point I am wondering if this is a good design, or if it 
> would be better to use a different strategy.

IMHO, this should time based or perhaps the strategy could depend
on the package being installed (critical packages like libc6 could
use fsync for every file, but all these intermediate fsync's are
useless for packages that can simply be reinstalled in case of
crash or hardware related failure).

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

[toc] | [prev] | [next] | [standalone]

#1120925

[Multipart message — attachments visible in raw view] — view raw

On Oct 14, Vincent Lefevre <vincent@vinc17.net> wrote:

> > This is obviously convenient on Guillem's part, but we have to optimize 
> > systems by default for the general case and not just for dpkg -i.
> This dpkg FAQ says that this is not beneficial for just dpkg,
> but also "for any application in the system".
[citation needed]

I hope that you understand why at this point I cannot trust as is the 
opinions of the dpkg maintainer.

-- 
ciao,
Marco

[toc] | [prev] | [next] | [standalone]

#1120952

On Fri, Oct 14, 2022 at 03:37:29PM +0200, Marco d'Itri wrote:
> On Oct 14, Vincent Lefevre <vincent@vinc17.net> wrote:
> 
> > > This is obviously convenient on Guillem's part, but we have to optimize 
> > > systems by default for the general case and not just for dpkg -i.
> > This dpkg FAQ says that this is not beneficial for just dpkg,
> > but also "for any application in the system".
> [citation needed]
> 
> I hope that you understand why at this point I cannot trust as is the 
> opinions of the dpkg maintainer.

The dpkg FAQ is just wrong.  It relates to controversy which is over a
dozen years old.  For more information see, see Josef Sipek's blog
post from 2009, "O_PONIES & Other Assorted Wishes"

	https://blahg.josefsipek.net/?p=364

The O_PONIES mention references an 2009 April Fool's patch:

	https://lore.kernel.org/linux-fsdevel/20090401041843.GN19690@josefsipek.net/

Because buggy applications and clueless application programmers vastly
outnumbers file system maintainers, at the 2009 LSF/MM workshop, a
number of the major file system developers agreed on the following
workaround.  If the application opens a pre-existing file with
O_TRUNC, or renames a newly created file on top of pre-existing file,
we will force the delayed allocation to be automatically resolved when
the file is closed (in the first case) or the rename (in the second
case).  It does *not* force a file system commit.  So if you crash
within 5 seconds of the close(2) or rename(2), you will still suffer
data loss.

HOWEVER, this was always the case for buggy applicatons that refused
to call fsync(2) who were relying on the old ext3 file system
semantics.  It did not guarantee that things would work; it would just
*mostly* work, since *usually* you didn't crash within 5 seconds of
rewriting a file.

So no, we're not going to be making the default change to /etc/fstab.

NACK.

						- Ted

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.bugs.dist

csiph-web