Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #245229 > unrolled thread

Memory leak

Back to article view | Back to linux.debian.user

Contents

  Memory leak Celejar <celejar@gmail.com> - 2022-02-11 17:10 +0100
    Re: Memory leak Charles Curley <charlescurley@charlescurley.com> - 2022-02-11 18:00 +0100
      Re: Memory leak Celejar <celejar@gmail.com> - 2022-02-11 18:10 +0100
        Re: Memory leak Charles Curley <charlescurley@charlescurley.com> - 2022-02-11 19:50 +0100
          Re: Memory leak Celejar <celejar@gmail.com> - 2022-02-11 21:10 +0100
            Re: Memory leak Charles Curley <charlescurley@charlescurley.com> - 2022-02-11 23:40 +0100
          Re: Memory leak piorunz <piorunz@gmx.com> - 2022-02-11 21:20 +0100
        Re: Memory leak Celejar <celejar@gmail.com> - 2022-02-11 20:50 +0100
    Re: Memory leak Stefan Monnier <monnier@iro.umontreal.ca> - 2022-02-11 19:50 +0100
      Re: Memory leak Linux-Fan <Ma_Sys.ma@web.de> - 2022-02-11 20:20 +0100
        Re: Memory leak David Wright <deblis@lionunicorn.co.uk> - 2022-02-13 18:10 +0100
      Re: Memory leak Celejar <celejar@gmail.com> - 2022-02-11 20:50 +0100
        Re: Memory leak Stefan Monnier <monnier@iro.umontreal.ca> - 2022-02-12 18:50 +0100
          Re: Memory leak Celejar <celejar@gmail.com> - 2022-02-14 22:50 +0100
    Re: Memory leak "Roy J. Tellason, Sr." <roy@rtellason.com> - 2022-02-12 00:20 +0100
      Re: Memory leak piorunz <piorunz@gmx.com> - 2022-02-12 09:10 +0100
        Re: Memory leak Curt <curty@free.fr> - 2022-02-12 13:50 +0100
          Re: Memory leak piorunz <piorunz@gmx.com> - 2022-02-14 02:30 +0100
      Re: Memory leak Curt <curty@free.fr> - 2022-02-12 11:20 +0100
        Re: Memory leak rhkramer@gmail.com - 2022-02-12 15:30 +0100
          Re: Memory leak Felmon Davis <moelmoel2714@gmail.com> - 2022-02-12 23:50 +0100
          Re: Memory leak <tomas@tuxteam.de> - 2022-02-13 07:30 +0100
            Re: Memory leak rhkramer@gmail.com - 2022-02-13 16:10 +0100
              Re: Memory leak <tomas@tuxteam.de> - 2022-02-13 16:20 +0100
              Re: Memory leak "Andrew M.A. Cater" <amacater@einval.com> - 2022-02-13 17:10 +0100
          Re: Memory leak "Roy J. Tellason, Sr." <roy@rtellason.com> - 2022-02-13 21:00 +0100
            Re: Memory leak rhkramer@gmail.com - 2022-02-14 02:20 +0100
        Re: Memory leak Felmon Davis <moelmoel2714@gmail.com> - 2022-02-12 23:50 +0100
          Re: Memory leak <tomas@tuxteam.de> - 2022-02-13 07:40 +0100
            Re: Memory leak Tixy <tixy@yxit.co.uk> - 2022-02-13 09:10 +0100
              Re: Memory leak Andrei POPESCU <andreimpopescu@gmail.com> - 2022-02-13 17:10 +0100

Page 1 of 2  [1] 2  Next page →

#245229 — Memory leak

Hello,

I seem to have a serious memory leak on my system (Lenovo W550s) - the
memory usage seems to slowly but more or less steadily keep increasing.

This is a more or less normal (I think) desktop installation of Sid,
running Xfce4. Typical applications used are Firefox (currently with
just one extension: uBlock Origin), LibreOffice Writer, Sylpheed, Xfce4
Terminal, and Liferea, all from the official repos.

I used to have 8 GB on the system, and it would start to thrash at
about 7+ GB usage. I recently ugrade to 16 GB; memory usage is
currently over 8 GB, and it seems to be slowly but steadily increasing.

I have never been able to wrap my head around linux memory usage
reporting. htop currently reports about 8.13 GB used (up from about
8.03 when I began this email). I sorted the processes by PERCENT_MEM,
and here are the leaders (VIRT / RES / SHR):

A half dozen or so Xorg processes: 4340M / 2947M / 90948

Numerous 'firefox' processes: 5102M / 1068M / 369M.

Some soffice.bin processes: 1096M / 301M / 128M.

A bunch more '/usr/lib/firefox/firefox' processes: 27.2G / 213M / 88620.

Some webkit processes: 100G / 212M / 143M.

Some liferea processes: 84.3G / 198M / 112M

A bunch *more* '/usr/lib/firefox/firefox' processes: 2835M / 186M / 127M

And a bunch *more* '/usr/lib/firefox/firefox' processes: 2456M / 180M /
150M

And then several more bunches of '/usr/lib/firefox/firefox' processes,
with values of the same orders of magnitude to the preceeding ones.

My understanding is that it's the RES and SHR values that are
important, not the VIRT ones. I have the impression that the Xorg
values are high. I saw recommendations to use xrestop to check things
like pixmap usage. I don't really know much about this, but currently
the total pixmap usage is about 360M, mostly used by a Writer process,
and about 40M for Xfwm4 and 30M by Firefox.

Any ideas?

Celejar

[toc] | [next] | [standalone]

#245232

On Fri, 11 Feb 2022 11:06:01 -0500
Celejar <celejar@gmail.com> wrote:

> I seem to have a serious memory leak on my system (Lenovo W550s) - the
> memory usage seems to slowly but more or less steadily keep
> increasing.
> 
> This is a more or less normal (I think) desktop installation of Sid,
> running Xfce4. Typical applications used are Firefox (currently with
> just one extension: uBlock Origin), LibreOffice Writer, Sylpheed,
> Xfce4 Terminal, and Liferea, all from the official repos.

Firefox left running for days on end is a possible culprit.

-- 
Does anybody read signatures any more?

https://charlescurley.com
https://charlescurley.com/blog/

[toc] | [prev] | [next] | [standalone]

#245233

On Fri, 11 Feb 2022 09:53:17 -0700
Charles Curley <charlescurley@charlescurley.com> wrote:

> On Fri, 11 Feb 2022 11:06:01 -0500
> Celejar <celejar@gmail.com> wrote:
> 
> > I seem to have a serious memory leak on my system (Lenovo W550s) - the
> > memory usage seems to slowly but more or less steadily keep
> > increasing.
> > 
> > This is a more or less normal (I think) desktop installation of Sid,
> > running Xfce4. Typical applications used are Firefox (currently with
> > just one extension: uBlock Origin), LibreOffice Writer, Sylpheed,
> > Xfce4 Terminal, and Liferea, all from the official repos.
> 
> Firefox left running for days on end is a possible culprit.

So I've heard. So is this something I just have to live with? Does
everyone have this problem? I actually did used to kill firefox when I
was experiencing memory pressure - it certainly relieved the immediate
problem, but I think I found that not all the memory used was returned,
and when I restarted firefox, the problem of running out of memory
often returned before long (i.e., in much less time than after a fresh
install).

Celejar

[toc] | [prev] | [next] | [standalone]

#245237

On Fri, 11 Feb 2022 12:01:59 -0500
Celejar <celejar@gmail.com> wrote:

> So I've heard. So is this something I just have to live with? Does
> everyone have this problem?

It is widely rumored, backed by experiments I've done here. I've not
seen anything official from the Mozilla folks, but then I don't pay
close attention to them.

My solution is simple: I switched to Vivaldi over a year ago, and
haven't looked back. https://vivaldi.com. They have packages for
Debian, and run a roughly two week release cycle. It's based on
Chromium, but with better privacy settings for the defaults.

-- 
Does anybody read signatures any more?

https://charlescurley.com
https://charlescurley.com/blog/

[toc] | [prev] | [next] | [standalone]

#245248

On Fri, 11 Feb 2022 11:40:15 -0700
Charles Curley <charlescurley@charlescurley.com> wrote:

> On Fri, 11 Feb 2022 12:01:59 -0500
> Celejar <celejar@gmail.com> wrote:
> 
> > So I've heard. So is this something I just have to live with? Does
> > everyone have this problem?
> 
> It is widely rumored, backed by experiments I've done here. I've not
> seen anything official from the Mozilla folks, but then I don't pay
> close attention to them.
> 
> My solution is simple: I switched to Vivaldi over a year ago, and
> haven't looked back. https://vivaldi.com. They have packages for
> Debian, and run a roughly two week release cycle. It's based on
> Chromium, but with better privacy settings for the defaults.

There are several reasons I'm not ready to do that:

1) Vivaldi is not open source.

2) It's based on Chrome, which empowers Google and its ability to
control web standards.

3) I am worried about a web monoculture.

Celejar

[toc] | [prev] | [next] | [standalone]

#245260

On Fri, 11 Feb 2022 15:00:39 -0500
Celejar <celejar@gmail.com> wrote:

> There are several reasons I'm not ready to do that:

Fine, get another browser.

-- 
Does anybody read signatures any more?

https://charlescurley.com
https://charlescurley.com/blog/

[toc] | [prev] | [next] | [standalone]

#245249

On 11/02/2022 18:40, Charles Curley wrote:

> My solution is simple: I switched to Vivaldi over a year ago, and
> haven't looked back. https://vivaldi.com. They have packages for
> Debian, and run a roughly two week release cycle. It's based on
> Chromium, but with better privacy settings for the defaults.

Unfortunately, Vivaldi is not acceptable for inclusion in Debian
project, because it's not GPL licensed. It's not fully open source, it
comes with EULA and privacy policy which you must accept in order to use
it. Binaries are not reproducible due to lack of 100% source code.

Therefore I must stay with Firefox for now, it's 100% FOSS, it's
maintained by Debian as well. I won't entrust commercial company,
Vivaldi Technologies AS from Oslo, with my browsing data.

--
With kindest regards, Piotr.

⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org/
⠈⠳⣄⠀⠀⠀⠀

[toc] | [prev] | [next] | [standalone]

#245244

On Fri, 11 Feb 2022 18:06:52 +0000
piorunz <piorunz@gmx.com> wrote:

> On 11/02/2022 17:01, Celejar wrote:
> > So I've heard. So is this something I just have to live with? Does
> > everyone have this problem? I actually did used to kill firefox when I
> > was experiencing memory pressure - it certainly relieved the immediate
> > problem, but I think I found that not all the memory used was returned,
> > and when I restarted firefox, the problem of running out of memory
> > often returned before long (i.e., in much less time than after a fresh
> > install).
> 
> Yes that's typical for today's browsers. I never close browsers/reboot
> my home-work computer, right now I have 2 days uptime and 17 GB memory
> allocated between my normal day to day tools.
> 
> 
> Go to Firefox Settings / Performance
> Untick Recommended settings and change Content process limit to 1. Like
> in the screenshot.
> That will decrease memory usage.

Thanks - I'll look into this.

Celejar

[toc] | [prev] | [next] | [standalone]

#245238

> I used to have 8 GB on the system, and it would start to thrash at
> about 7+ GB usage. I recently ugrade to 16 GB; memory usage is
> currently over 8 GB, and it seems to be slowly but steadily increasing.

Presumably you bought 16GB to make use of it, right?
So it's only natural for your OS to try and put that memory to use.
Any "free memory" is memory that could potentially be used for something
more useful (IOW "free" = "wasted" in some sense).

It's normal for memory use to increase over time, as your OS finds more
things to put into it.

Of course, that doesn't mean that the concept of excessive memory use
doesn't exist.  Just that it's hard to characterize.

Do you actually experience a concrete problem (like your system getting
sluggish and making too many disk accesses to swap things in&out of the
RAM)?


        Stefan

[toc] | [prev] | [next] | [standalone]

#245240

[Multipart message — attachments visible in raw view] — view raw

Stefan Monnier writes:

> > I used to have 8 GB on the system, and it would start to thrash at
> > about 7+ GB usage. I recently ugrade to 16 GB; memory usage is
> > currently over 8 GB, and it seems to be slowly but steadily increasing.
>
> Presumably you bought 16GB to make use of it, right?
> So it's only natural for your OS to try and put that memory to use.
> Any "free memory" is memory that could potentially be used for something
> more useful (IOW "free" = "wasted" in some sense).
>
> It's normal for memory use to increase over time, as your OS finds more
> things to put into it.

That was my first intuition, too. There is even a classic website about this  
very topic: https://www.linuxatemyram.com/

HOWEVER, given that the OP mentions looking at the RSS sizes I think the  
classic "all memory used" issue is already ruled-out. The issue seems to be  
modern webbrowsers which could be considered OSes on their own already hence  
they also claim more resources whenever it is useful for them.

Firefox takes just above 1600 MiB here with only six tabs open for four  
hours. Yet I am pretty sure it would take less were this a "lower-end"  
system e.g. fewer CPU cores would cause fewer processes to be spawned and  
hence the memory efficiency might be better in such cases.

HTH and YMMV
Linux-Fan

öö

[...]

[toc] | [prev] | [next] | [standalone]

#245337

On Fri 11 Feb 2022 at 20:04:35 (+0100), Linux-Fan wrote:
> Stefan Monnier writes:
> 
> > > I used to have 8 GB on the system, and it would start to thrash at
> > > about 7+ GB usage. I recently ugrade to 16 GB; memory usage is
> > > currently over 8 GB, and it seems to be slowly but steadily increasing.
> > 
> > Presumably you bought 16GB to make use of it, right?
> > So it's only natural for your OS to try and put that memory to use.
> > Any "free memory" is memory that could potentially be used for something
> > more useful (IOW "free" = "wasted" in some sense).
> > 
> > It's normal for memory use to increase over time, as your OS finds more
> > things to put into it.
> 
> That was my first intuition, too. There is even a classic website
> about this very topic: https://www.linuxatemyram.com/
> 
> HOWEVER, given that the OP mentions looking at the RSS sizes I think
> the classic "all memory used" issue is already ruled-out. The issue
> seems to be modern webbrowsers which could be considered OSes on their
> own already hence they also claim more resources whenever it is useful
> for them.
> 
> Firefox takes just above 1600 MiB here with only six tabs open for
> four hours. Yet I am pretty sure it would take less were this a
> "lower-end" system e.g. fewer CPU cores would cause fewer processes to
> be spawned and hence the memory efficiency might be better in such
> cases.

Very slow, but I can run FF (≤3 tabs) and mpv in bullseye on a 2004-era
laptop: Pentium M i386 1.5GHz 512MB with 1GB swap (encrypted, as is /home).
I don't run it for hours on end¹. Compared with buster, it's slower to
start FF, but no slowdown with loading pages. (Of course, buster was
running FF 6X.X rather than the new 91.6.)

¹ It's a non-portable laptop. Remove the power cable and it's dead.

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#245243

On Fri, 11 Feb 2022 13:43:55 -0500
Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> > I used to have 8 GB on the system, and it would start to thrash at
> > about 7+ GB usage. I recently ugrade to 16 GB; memory usage is
> > currently over 8 GB, and it seems to be slowly but steadily increasing.
> 
> Presumably you bought 16GB to make use of it, right?

Yes, and so I can do things like development, with IDEs and emulators
that consume additional GBs of RAM, while leaving my usual applications
running.

> So it's only natural for your OS to try and put that memory to use.
> Any "free memory" is memory that could potentially be used for something
> more useful (IOW "free" = "wasted" in some sense).
> 
> It's normal for memory use to increase over time, as your OS finds more
> things to put into it.
> 
> Of course, that doesn't mean that the concept of excessive memory use
> doesn't exist.  Just that it's hard to characterize.

I understand all this, at least in general terms.

> Do you actually experience a concrete problem (like your system getting
> sluggish and making too many disk accesses to swap things in&out of the
> RAM)?

As I mentioned (briefly) in my original post, yes, I experience concrete
problems: the system either grinds to a halt or becomes unresponsive,
or hits swap and becomes intolerably slow.

Since upgrading to 16 GB a couple of days ago, I haven't had problems
yet, but I used to see the problems I mentioned when I had "only" 8
GB ;/

Celejar

[toc] | [prev] | [next] | [standalone]

#245287

> As I mentioned (briefly) in my original post, yes, I experience concrete
> problems: the system either grinds to a halt or becomes unresponsive,
> or hits swap and becomes intolerably slow.

Sorry I missed that part.
I think that's what you should focus on: try and run some background
collection of timestamped system state (CPU use and memory use) and then
try and investigate to see what it is that was eating all the resources
during those times where the system grinds to a halt.


        Stefan

[toc] | [prev] | [next] | [standalone]

#245388

On Sat, 12 Feb 2022 12:49:15 -0500
Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> > As I mentioned (briefly) in my original post, yes, I experience concrete
> > problems: the system either grinds to a halt or becomes unresponsive,
> > or hits swap and becomes intolerably slow.
> 
> Sorry I missed that part.
> I think that's what you should focus on: try and run some background
> collection of timestamped system state (CPU use and memory use) and then
> try and investigate to see what it is that was eating all the resources
> during those times where the system grinds to a halt.

Thank you. I actually just did a complete rebuild of my system from
scratch: the old SSD was almost full and I installed a new one, so I
decided to rebuild from scratch to get rid of accumulated cruft. So far
things have been much better, but I'll see if problems return.

Celejar

[toc] | [prev] | [next] | [standalone]

#245262

On Friday 11 February 2022 11:06:01 am Celejar wrote:
> I seem to have a serious memory leak on my system (Lenovo W550s) - the
> memory usage seems to slowly but more or less steadily keep increasing.
> 
> This is a more or less normal (I think) desktop installation of Sid,
> running Xfce4. Typical applications used are Firefox (currently with
> just one extension: uBlock Origin), LibreOffice Writer, Sylpheed, Xfce4
> Terminal, and Liferea, all from the official repos.
 
I have noticed similar behavior here,  and the culprit seems to be firefox.  If I ignore it things get sluggish, and then the machine starts to thrash,  until the only recourse is the power switch.  If I notice it in time,  close firefox and restart it,  the memory used after I do that is significantly less than it was before.

Somewhere in their help or documentation they even say that you shouldn't leave it running for extended periods of time.

I wish they'd fix it!


-- 
Member of the toughest, meanest, deadliest, most unrelenting -- and
ablest -- form of life in this section of space,  a critter that can
be killed but can't be tamed.  --Robert A. Heinlein, "The Puppet Masters"
-
Information is more dangerous than cannon to a society ruled by lies. --James 
M Dakin

[toc] | [prev] | [next] | [standalone]

#245270

On 11/02/2022 22:16, Roy J. Tellason, Sr. wrote:
> Somewhere in their help or documentation they even say that you shouldn't leave it running for extended periods of time.

Never heard such a thing. Do you have source?

-- 
With kindest regards, Piotr.

⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org/
⠈⠳⣄⠀⠀⠀⠀

[toc] | [prev] | [next] | [standalone]

#245277

On 2022-02-12, piorunz <piorunz@gmx.com> wrote:
> On 11/02/2022 22:16, Roy J. Tellason, Sr. wrote:
>> Somewhere in their help or documentation they even say that you shouldn't leave it running for extended periods of time.
>
> Never heard such a thing. Do you have source?
>

See my other post in this thread. 

[toc] | [prev] | [next] | [standalone]

#245354

On 12/02/2022 12:43, Curt wrote:
> On 2022-02-12, piorunz <piorunz@gmx.com> wrote:
>> On 11/02/2022 22:16, Roy J. Tellason, Sr. wrote:
>>> Somewhere in their help or documentation they even say that you shouldn't leave it running for extended periods of time.
>>
>> Never heard such a thing. Do you have source?
>>

> See my other post in this thread.

Yes, see that, thanks.

--
With kindest regards, Piotr.

⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org/
⠈⠳⣄⠀⠀⠀⠀

[toc] | [prev] | [next] | [standalone]

#245273

On 2022-02-11, Roy J. Tellason, Sr. <roy@rtellason.com> wrote:
> On Friday 11 February 2022 11:06:01 am Celejar wrote:
>> I seem to have a serious memory leak on my system (Lenovo W550s) - the
>> memory usage seems to slowly but more or less steadily keep increasing.
>> 
>> This is a more or less normal (I think) desktop installation of Sid,
>> running Xfce4. Typical applications used are Firefox (currently with
>> just one extension: uBlock Origin), LibreOffice Writer, Sylpheed, Xfce4
>> Terminal, and Liferea, all from the official repos.
>  
> I have noticed similar behavior here,  and the culprit seems to be firefox.  If I ignore it things get sluggish, and then the machine starts to thrash,  until the only recourse is the power switch.  If I notice it in time,  close firefox and restart it,  the memory used after I do that is significantly less than it was before.
>
> Somewhere in their help or documentation they even say that you shouldn't leave it running for extended periods of time.
>
> I wish they'd fix it!
>
>

I don't think they're headed in the direction of a fix.

https://support.mozilla.org/en-US/kb/firefox-uses-too-much-memory-or-cpu-resources

 Firefox may use more system resources if it's left open for long periods
 of time. A workaround for this is to periodically restart Firefox. You
 can configure Firefox to save your tabs and windows so that when you
 start it again, you can start where you left off. 

[toc] | [prev] | [next] | [standalone]

#245284

On Saturday, February 12, 2022 05:11:31 AM Curt wrote:
>  Firefox may use more system resources if it's left open for long periods
>  of time. A workaround for this is to periodically restart Firefox. You
>  can configure Firefox to save your tabs and windows so that when you
>  start it again, you can start where you left off.

The version of Firefox used in Jessie (and presumably later versions) creates 
(typically mutlitple) files named "Web Content".  I don't know how Firefox 
decides what to put in each of those (e.g., content from how many tabs), but 
...

I keep top running in a VT and check it every once in a while, and when too 
much memory is used, I kill one or more of those files, usually the largest 
first.

My tabs remain as tabs (with the associated URL), but the content is gone, but 
I can get it back by reloading the tab.

When too much memory is used on my Jessie system, the system hangs (on 
occasion i've even left it running overnight to see if it was triyng to do 
something to free up memory itself (like the, iirc, OOM killer), but so far it 
never has.

On that Jessie system (with 12 GB of RAM and 2 GB of swap), I try to kill one 
or more Web Content files when RAM usage reaches the high 9GB  of RAM (I don't 
pay much attention to swap usage, but I guess at least some is used at that 
point), because when RAM usage reaches 10 GB I run into the hang problem.

Maybe things would work better with more swap, but I haven/t (and probably 
won't try that) -- in the reasonably near future (maybe after tax season), I 
plan to set up a new system with Debian 1 (whatever that is "code named").

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | linux.debian.user

csiph-web