Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #239106

Re: nvme SSD and poor performance

Show all headers | View raw

On 8/17/21 2:54 AM, Pierre Willaime wrote:
 > I have a nvme SSD (CAZ-82512-Q11 NVMe LITEON 512GB) on debian stable
 > (bulleye now).
 >
 > For a long time, I suffer poor I/O performances which slow down a lot of
 > tasks (apt upgrade when unpacking for example).


On 8/20/21 1:50 AM, Pierre Willaime wrote:
> Thanks all.
> 
> I activated `# systemctl enable fstrim.timer` (thanks Linux-Fan).
> 
> But I do not think my issue is trim related after all. 


I agree.


> I have always a 
> lot of I/O activities from jdb2 even just after booting and even when 
> the computer is doing nothing for hours.
> 
> Here is an extended  log of iotop where you can see jdb2 anormal 
> activities: https://pastebin.com/eyGcGdUz


Was your command something like this?

# iotop -o -k -t -q -q -q > iotop.out


> I cannot (yet) find what process is generating this activities. I tried 
> to kill a lot of jobs seing in atop output with no results.


Analyzing the first ten minutes worth of data with an improvised Perl 
script:

index of field 'read_bw'
     1256840.19 firefox-esr
       77926.07 apt
         316.08 perl
          22.74 dpkg
          15.27 [kworker/6:0-events]
index of field 'write_bw'
      220512.79 thunderbird
       29613.87 firefox-esr
       23221.20 dpkg
       15211.66 [jbd2/nvme0n1p2-]
        5529.57 [dpkg]
        4148.09 systemd-journald
        1699.13 perl
         533.28 mandb
         507.15 apt
         145.61 rsyslogd
         131.59 atop
         115.77 syncthing
          46.35 xfce4-terminal
          38.60 smartd
          15.48 Xorg
          15.44 NetworkManager
           7.64 bash
index of field 'swap_percent'
index of field 'io_wait_percent'
       12427.58 [jbd2/nvme0n1p2-]
        1334.15 firefox-esr
         568.91 dpkg
         385.31 thunderbird
         293.57 mandb
          99.99 syncthing
          64.82 [kworker/13:3-events_freezable_power_]
          63.86 smartd
          55.12 [kworker/u32:3+flush-259:0]
          38.64 [kworker/u32:2-flush-259:0]
          25.27 [kworker/u32:3-events_unbound]
          23.13 [kworker/4:0-events_freezable_power_]
          22.68 [kworker/u32:2-events_unbound]
          21.55 apt
          12.51 [kworker/u32:1-ext4-rsv-conversion]
           9.87 [kworker/u32:2-ext4-rsv-conversion]
           9.63 [kworker/9:1-events]
           8.90 [kworker/u32:1-flush-259:0]
           8.58 perl
           8.11 [kworker/9:1-events_freezable_power_]
           5.85 [dpkg]
           4.33 [kworker/u32:3-ext4-rsv-conversion]
           4.26 NetworkManager
           3.57 [kworker/13:3-mm_percpu_wq]
           2.85 [kworker/9:1-mm_percpu_wq]
           2.71 [kworker/4:0-mm_percpu_wq]
           0.40 [kworker/13:3-events] [kworker/4:0-events]
           0.38 [kworker/6:1-events]
           0.36 [kworker/9:1-rcu_gp]
           0.30 [kworker/u32:3-flush-259:0]
           0.26 [kworker/6:0-events_freezable_power_]
           0.16 systemd-journald
           0.03 [kworker/6:0-events]


I appears:

- firefox-esr used the most read bandwidth -- 1256840.19 K/s total

- thunderbird used the most write bandwdith -- 220512.79 K/s total

- No processes were swapping.

- jbd2/nvme0n1p2- waited the longest for I/O -- 12427.58 % total


Both apt(8) and dpkg(1) were also running and using a small amount of 
I/O.  While I may leave Firefox and Thunderbird running when installing 
a package or two, I shut them down for updates and upgrades.  Was the 
iotop data collected during a long-running upgrade?


AIUI the jbd2/nvme0n1p2 I/O corresponds to the bottom half of the kernel 
(e.g. device driver interface, DDI) in response to I/O via the top half 
of the kernel (e.g. application programming interface, API).  The way to 
reduce jdb2 I/O is to reduce application I/O.


 > On 8/17/21 7:07 AM, Dan Ritter wrote:
 >> I don't think you have a significant performance problem, but
 >> you are definitely feeling some pain -- so can you tell us more
 >> about what feels slow? Does it happen during the ordinary course
 >> of the day?

> Program are slow to start. Sometimes there is a delay when I type 
> (letters are displayed few second after typing). Apt unpack take forewer 
> (5 hours to unpack packages when upgrading to bulleye).
> 
> The computer is a recent Dell precision desktop with i9-9900 as CPU, an 
> NVIDIA GP107GL [Quadro P400] (and the GPU integrated to the CPU). The 
> nvme SSD is supposed to be a decent one. This desktop is yet a lot 
> slower than my (more basic) laptop.
> 
> Complete system info: https://pastebin.com/zaGVEpae


That's a good workstation.  :-)


Firefox and Thunderbird are habitual trouble-makers on my Debian 
desktops.  I run Xfce with the CPU Graph panel applet.  If I leave 
Firefox or Thunderbird open long enough, eventually I will see a core 
pegged at 100% and the CPU fan will spin up.  Both applications keep 
working in this state; but a swirling toilet bowl mouse pointer in 
Thunderbird is a danger sign -- I have lost e-mail messages when moving 
a message produced that symptom.  The only cure is to close the 
offending program(s) and implement additional measures to duplicate 
incoming and outgoing e-mail.  My conclusion is that both Firefox and 
Thunderbird have multi-threaded programming bugs.  The next time either 
malfunctions, I may capture some iotop(8) data and analyze it.


I have also noticed that Linux is not very good at sharing I/O between 
processes and/or drives.  If one process is hammering I/O, any other 
processes that attempts I/O will stutter.  I have seen this both when 
two processes access the same drive and when two processes access 
different drives.


The last consideration is that Debian 11.0 has just been released.  I 
have encountered many problems with Debian *.0 releases over the years, 
and typically wait several months and a couple of minor version upgrades 
before doing a fresh install and migrating services and data.  (The 
Debian 10 nouveau driver did not like the NVIDIA Optimus graphics in my 
Dell Latitude E6520 laptop until recently -- Debian 10.9?)


I suggest that you do a fresh install of Debian 10.10 and evaluate that. 
  If you still see problems, try Debian 9.13.


David

Back to linux.debian.user | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

nvme SSD and poor performance Pierre Willaime <pierre.willaime@univ-lorraine.fr> - 2021-08-17 12:10 +0200
  Re: nvme SSD and poor performance Dan Ritter <dsr@randomstring.org> - 2021-08-17 14:10 +0200
    Re: nvme SSD and poor performance Pierre Willaime <pierre.willaime@univ-lorraine.fr> - 2021-08-17 15:20 +0200
      Re: nvme SSD and poor performance Dan Ritter <dsr@randomstring.org> - 2021-08-17 16:10 +0200
      Re: nvme SSD and poor performance David Christensen <dpchrist@holgerdanske.com> - 2021-08-18 00:30 +0200
        Re: nvme SSD and poor performance Pierre Willaime <pierre.willaime@univ-lorraine.fr> - 2021-08-20 11:00 +0200
          Re: nvme SSD and poor performance Christian Britz <cbritz@t-online.de> - 2021-08-20 17:50 +0200
            Re: nvme SSD and poor performance Tixy <tixy@yxit.co.uk> - 2021-08-20 19:40 +0200
          Re: nvme SSD and poor performance Linux-Fan <Ma_Sys.ma@web.de> - 2021-08-20 19:20 +0200
          Re: nvme SSD and poor performance "Alexander V. Makartsev" <avbetev@gmail.com> - 2021-08-20 19:50 +0200
            Re: nvme SSD and poor performance Pierre Willaime <pierre.willaime@univ-lorraine.fr> - 2021-08-23 10:50 +0200
              Re: nvme SSD and poor performance "Alexander V. Makartsev" <avbetev@gmail.com> - 2021-08-23 14:10 +0200
                Re: nvme SSD and poor performance piorunz <piorunz@gmx.com> - 2021-08-23 14:20 +0200
          Re: nvme SSD and poor performance David Christensen <dpchrist@holgerdanske.com> - 2021-08-21 04:10 +0200
  Re: nvme SSD and poor performance Linux-Fan <Ma_Sys.ma@web.de> - 2021-08-17 15:40 +0200
    Re: nvme SSD and poor performance Marco Möller <talby@debianlists.mobilxpress.net> - 2021-08-17 16:10 +0200
      Re: nvme SSD and poor performance piorunz <piorunz@gmx.com> - 2021-08-20 18:30 +0200
        Re: nvme SSD and poor performance Marco Möller <talby@debianlists.mobilxpress.net> - 2021-08-20 19:20 +0200
          Re: nvme SSD and poor performance piorunz <piorunz@gmx.com> - 2021-08-20 20:10 +0200
    Re: nvme SSD and poor performance Christian Britz <cbritz@t-online.de> - 2021-08-17 17:30 +0200
      Re: nvme SSD and poor performance Linux-Fan <Ma_Sys.ma@web.de> - 2021-08-17 17:40 +0200
  Re: nvme SSD and poor performance Stefan Monnier <monnier@iro.umontreal.ca> - 2021-08-17 17:00 +0200
  Re: nvme SSD and poor performance Jochen Spieker <ml@well-adjusted.de> - 2021-08-18 22:00 +0200
    Re: nvme SSD and poor performance Anssi Saari <as@sci.fi> - 2021-08-19 08:10 +0200
      Re: nvme SSD and poor performance Sven Hartge <sven@svenhartge.de> - 2021-08-21 15:00 +0200

csiph-web