Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #219995 > unrolled thread

Re: System unusably slow after Debian upgrade.

Back to article view | Back to linux.debian.user

Contents

  Re: System unusably slow after Debian upgrade. "G.W. Haywood" <debian-bugs@jubileegroup.co.uk> - 2020-03-09 12:50 +0100
    Re: System unusably slow after Debian upgrade. "Thomas Schmitt" <scdbackup@gmx.net> - 2020-03-09 13:20 +0100
    Re: System unusably slow after Debian upgrade. "G.W. Haywood" <debian-bugs@jubileegroup.co.uk> - 2020-03-12 13:10 +0100

#219995 — Re: System unusably slow after Debian upgrade.

[Multipart message — attachments visible in raw view] — view raw

Hi Marco,

Thanks very much for the trouble you have taken in your reply, I do
appreciate it very much.

On Sun, 8 Mar 2020, Marco Möller wrote:

> Upgrading your OS could result in the indexed search database to
> become rebuild, and this can take a very(!) long time and meanwhile
> painfully rendering your system almost unresponsive ...

Yes, true enough, but I have to say in this case -

(1) this would be obvious as a process using resources (e.g. in top,
or atop, which has already been covered in this thread, and my Nagios
monitoring would immediately and very clearly have shown me what was
happening, and thus I would not have been posting to this list;) and

> I assume that you have maybe GNOME or KDE Plasma ...

(2) as already mentioned the desktop is XFCE, which I have not seen do
such things (although I'll happily admit that I may just not have been
looking hard enough:).

> I am still curious to read about the outcome of your investigations.

Given what I found last night, so, I imagine, will be everyone else!

As mentioned in the thread I suspected the kernel was the problem, and
I am now sure of that.  Below are the timings of the simple rsync copy
operation that I've been using as a quick test.  The first two I have
already posted in earlier messages; the last two are new - I only
finished building the last custom kernel last night, and I tried the
Stretch stock kernel just for fun.  This is a tab-separated list, I'm
sorry if you don't do fixed pitch. :(

Kernel	Debian	Time	Notes

4.19	Buster	8m42s	Stock kernel, Buster
3.16	Jessie	1m17s	Stock kernel, Jessie
4.9	Stretch	2m40s	Stock kernel, Stretch
4.19-F	Buster	0m52s	Custom kernel, Buster

This is an 'rsync -avP' copy of a roughly 3 gigabyte file to /dev/shm/
and it generally gives an indication of what's going to happen within
a couple of seconds of hitting 'return'.  The transfer rate is about
40 MBytes/s with the Jessie 3.16 kernel, and about 6 MBytes/s with the
stock Buster kernel.  With my custom kernel it's about 60 MBytes/s on
exactly the same hardware (and, otherwise, exactly the same OS) as in
the measurement above (stock 4.19 kernel) which gives 6 MBytes/s, i.e.
it's literally an order of magnitude faster.  Food for thought.

The kernel shipped with Debian Jessie was OK, but even though Buster
has left it available for booting, and it does in fact boot, X won't
run with it so it's effectively useless.  The stock kernel in Stretch
is bad enough, but the kernel shipped with Debian Buster is absolutely
hopeless.  The performance of the _same_ version 4.19 kernel, but with
a mountain of junk removed using 'make menuconfig', is something like
50% better in this crude comparison than even the stock Jessie kernel.
X seems to be fine with the custom 4.19 kernel but it hasn't yet stood
the test of users hammering away at it doing actual work.  Most likely
after a few more exercises on the bench I'll install it at the farm
tomorrow and hope that it survives.

Unfortunately my tweaks to the stock kernel configuration resembled
wading through the jungle with a machete, and a diff between the stock
Buster kernel .config and my own .config amounts to 7,173 lines.  Most
of these I am sure are irrelevant to this issue, because they're about
building modules which will never be used in any case on the systems
I'm working with.  I would like to narrow it down to which differences
matter and which do not, but time is pressing and I do need to get the
system I'm working with back into service for its users.  I am very
happy to post the custom kernel .config as it stands, if anyone would
like to see it.  My first suspicion is that SPECTRE etc. mitigations
might be causing more serious problems on this architecture than they
do on some others, but that's really a wild guess.

> Wishing you all the best!

And the same you (and everyone else) too!

-- 

73,
Ged.

[toc] | [next] | [standalone]

#219997

Hi,

G.W. Haywood wrote:
> Unfortunately my tweaks to the stock kernel configuration resembled
> wading through the jungle with a machete, and a diff between the stock
> Buster kernel .config and my own .config amounts to 7,173 lines.  Most
> of these I am sure are irrelevant to this issue, because they're about
> building modules which will never be used in any case on the systems
> I'm working with.  I would like to narrow it down to which differences
> matter and which do not, but time is pressing and I do need to get the
> system I'm working with back into service for its users.  I am very
> happy to post the custom kernel .config as it stands, if anyone would
> like to see it.

I guess some readers would righteously complain if you send ~ 10000 lines
here.

But if you have some webspace where to expose the fast running configuration
and the diff to the stock configuration, then people could start making
theories and future victims of the problem could continue your effort
by developing a smaller diff.

(I would have expected a bitrot-like source change in the kernel, not
 a configurable issue. Maybe i will participate in riddling ...)


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#220095

Hi there,

On Mon, 09 Mar 2020, Thomas Schmitt wrote:
> G.W. Haywood wrote:
> > ... I am very happy to post the custom kernel .config as it stands,
> > if anyone would like to see it.
> 
> I guess some readers would righteously complain if you send ~ 10000 lines
> here.

I did't mean that I was offering to post the diff to the mailing list. :)

> But if you have some webspace where to expose the fast running configuration
> and the diff to the stock configuration, then people could start making
> theories and future victims of the problem could continue your effort
> by developing a smaller diff.

I've opened bug #953545 in the Debian bug tacking system.  My custom
kernel configuration is attached to that report.  I suspect that the
diff will not be useful to the bug-hunters, so that is not attached
but it will be easy to reproduce from the stock kernel configs.

-- 

73,
Ged.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web