Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #180881 > unrolled thread

NVMe SSD and discard mount option

Back to article view | Back to linux.debian.user

Contents

  NVMe SSD and discard mount option David Guyot <david.guyot@europecamions-interactive.com> - 2017-05-09 15:40 +0200
    Re: NVMe SSD and discard mount option Dan Ritter <dsr@randomstring.org> - 2017-05-09 17:10 +0200
      Re: NVMe SSD and discard mount option David Guyot <david.guyot@europecamions-interactive.com> - 2017-05-09 17:20 +0200
        Re: NVMe SSD and discard mount option Dan Ritter <dsr@randomstring.org> - 2017-05-09 18:00 +0200
    Re: NVMe SSD and discard mount option Dan Ritter <dsr@randomstring.org> - 2017-05-09 17:10 +0200
    Re: NVMe SSD and discard mount option Henrique de Moraes Holschuh <hmh@debian.org> - 2017-05-09 18:40 +0200

#180881 — NVMe SSD and discard mount option

[Multipart message — attachments visible in raw view] — view raw

Hello, there.

I recently loaned a server with NVMe SSD and saw, during my research on
the relevance of the discard mount option for them, that its use is
discouraged for NVMe SSDs. Why? Does NVMe SSDs not need trimming? Is it
integrated in the NVMe driver for Linux?

Awaiting your answers,

Regards.
-- 
David Guyot
Administrateur système, réseau et télécom / Sysadmin
Europe Camions Interactive / Stockway
Moulin Collot
F-88500 Ambacourt
03 29 30 47 85

[toc] | [next] | [standalone]

#180883

On Tue, May 09, 2017 at 11:02:46AM -0400, Dan Ritter wrote:
> On Tue, May 09, 2017 at 03:35:04PM +0200, David Guyot wrote:
> > Hello, there.
> > 
> > I recently loaned a server with NVMe SSD and saw, during my research on
> > the relevance of the discard mount option for them, that its use is
> > discouraged for NVMe SSDs. Why? Does NVMe SSDs not need trimming? Is it
> > integrated in the NVMe driver for Linux?
> 
> Intel, among other NVMe SSD manufacturers, recommends using
> fstrim periodically rather than enabling continuous trimming
> with the discard option.
> 
> They don't specifically state why, but I will hazard a guess
> that performance is impacted during a trim, and doing so for
> a short period once a day or once a week is more useful than
> decreasing performance all the time.

Evidence from Ted Ts'o:

https://forums.freebsd.org/threads/56951/#post-328912

which boils down to:

A) yes, there's a performance impact
B) the systems which listen to TRIM commands do better on large
   allocations than on small, so batching them up with fstrim
   called by cron is preferred.

-dsr-

[toc] | [prev] | [next] | [standalone]

#180885

[Multipart message — attachments visible in raw view] — view raw

I also thought about that at first, but asked to my server provider, and
they quoted Intel docs: "Be sure to turn off the discard option when
making your Linux filesystem. You want to allow the SSD manage blocks
and its activity between the NVM (non-volatile memory) and host with
more advanced and consistent approaches in the SSD Controller.Core
Filesystems:•ext4– the default extended option is not to discard blocks
at filesystem make time, retain this, and do not add the “discard”
extended option as some information will tell you to do"

If I understand correctly, the SSD already manages discarding the old
blocks, probably using some NVMe magic. Is my interpretation OK?

Le mardi 09 mai 2017 à 11:07 -0400, Dan Ritter a écrit :
> On Tue, May 09, 2017 at 11:02:46AM -0400, Dan Ritter wrote:
> > On Tue, May 09, 2017 at 03:35:04PM +0200, David Guyot wrote:
> > > Hello, there.
> > > 
> > > I recently loaned a server with NVMe SSD and saw, during my research on
> > > the relevance of the discard mount option for them, that its use is
> > > discouraged for NVMe SSDs. Why? Does NVMe SSDs not need trimming? Is it
> > > integrated in the NVMe driver for Linux?
> > 
> > Intel, among other NVMe SSD manufacturers, recommends using
> > fstrim periodically rather than enabling continuous trimming
> > with the discard option.
> > 
> > They don't specifically state why, but I will hazard a guess
> > that performance is impacted during a trim, and doing so for
> > a short period once a day or once a week is more useful than
> > decreasing performance all the time.
> 
> Evidence from Ted Ts'o:
> 
> https://forums.freebsd.org/threads/56951/#post-328912
> 
> which boils down to:
> 
> A) yes, there's a performance impact
> B) the systems which listen to TRIM commands do better on large
>    allocations than on small, so batching them up with fstrim
>    called by cron is preferred.
> 
> -dsr-

-- 
David Guyot
Administrateur système, réseau et télécom / Sysadmin
Europe Camions Interactive / Stockway
Moulin Collot
F-88500 Ambacourt
03 29 30 47 85

[toc] | [prev] | [next] | [standalone]

#180889

On Tue, May 09, 2017 at 05:17:42PM +0200, David Guyot wrote:
> I also thought about that at first, but asked to my server provider, and
> they quoted Intel docs: "Be sure to turn off the discard option when
> making your Linux filesystem. You want to allow the SSD manage blocks
> and its activity between the NVM (non-volatile memory) and host with
> more advanced and consistent approaches in the SSD Controller.Core
> Filesystems:•ext4– the default extended option is not to discard blocks
> at filesystem make time, retain this, and do not add the “discard”
> extended option as some information will tell you to do"
> 
> If I understand correctly, the SSD already manages discarding the old
> blocks, probably using some NVMe magic. Is my interpretation OK?

That's not what I said, no.

NVMe is an interface, like SATA. There's nothing magic about it.

All the data quoted so far suggests that you should not add the
discard option.

Trim operations should be done via cron calling the fstrim
command.

-dsr-

[toc] | [prev] | [next] | [standalone]

#180884

On Tue, May 09, 2017 at 03:35:04PM +0200, David Guyot wrote:
> Hello, there.
> 
> I recently loaned a server with NVMe SSD and saw, during my research on
> the relevance of the discard mount option for them, that its use is
> discouraged for NVMe SSDs. Why? Does NVMe SSDs not need trimming? Is it
> integrated in the NVMe driver for Linux?

Intel, among other NVMe SSD manufacturers, recommends using
fstrim periodically rather than enabling continuous trimming
with the discard option.

They don't specifically state why, but I will hazard a guess
that performance is impacted during a trim, and doing so for
a short period once a day or once a week is more useful than
decreasing performance all the time.

-dsr-

[toc] | [prev] | [next] | [standalone]

#180891

On Tue, 09 May 2017, David Guyot wrote:
> I recently loaned a server with NVMe SSD and saw, during my research on
> the relevance of the discard mount option for them, that its use is
> discouraged for NVMe SSDs. Why? Does NVMe SSDs not need trimming? Is it
> integrated in the NVMe driver for Linux?

Linux does better with filesystem-level TRIM every so often (how often
depends really on your write load and level of overprovisioning) as a
rule, anyway.

That said: NVMe usually dislikes frequent use of TRIM because it
typically will play badly with the many-queue scheduler inside the
device: the device will typically have to issue a device-wide write
barrier internally, which has to drain (and freeze) every [write?] queue
up to the barrier point before the barrier can be cleared.  The blocks
are then marked as free for future garbage collection, and all queues
unfrozen, thus resuming operation.  This hurts multi-stream streaming
performance quite a lot...

Even if it had to freeze just one queue, it would still hurt when
compared to an fstrim every hour/day/week/month.

Non-NVMe devices have far less command-path paralellism, so the
device-wide write barrier should typically hurt less (in relative terms)
than it would on a NVMe device.

Besides, I/O latency becomes *utterly unpredictable* when online discard
is active, which can cause all sort of stuttering on the default I/O
scheduler.  Latency will get unpredictable during fstrim as well, but
you can schedule the fstrim to a time of your choice, instead of every
time the filesystem frees a data or metadata block...

As for flash wear, on a modern SSD (NVMe or otherwise), to keep it at
the bare minimum it should be enough to overprovision it properly and
issue an fstrim (on average) after writing about 50%[1] of the size of
the overprovisioned area.  That might even give you less flash wear than
online-discard over time...


[1] 50% is just a hunch.  You could test that, but please keep in mind
that it will be device-firmware dependent.  I bet there are a few
devices that are going to priorize copying data around to free
almost-empty erase blocks for erasure, no matter how many fully-erased
blocks are already available...

-- 
  Henrique Holschuh

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web