Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #227460 > unrolled thread

"ps -o %mem" and free memory in Linux

Back to article view | Back to linux.debian.user

Contents

  "ps -o %mem" and free memory in Linux Victor Sudakov <vas@sibptus.ru> - 2020-09-29 05:30 +0200
    Re: "ps -o %mem" and free memory in Linux Tixy <tixy@yxit.co.uk> - 2020-09-29 09:20 +0200
      Re: "ps -o %mem" and free memory in Linux Victor Sudakov <vas@sibptus.ru> - 2020-09-29 17:20 +0200
        Re: "ps -o %mem" and free memory in Linux Klaus Singvogel <deb-user-ml@singvogel.net> - 2020-09-29 17:40 +0200
          Re: "ps -o %mem" and free memory in Linux Victor Sudakov <vas@sibptus.ru> - 2020-09-29 18:00 +0200
            Re: "ps -o %mem" and free memory in Linux Fabrice BAUZAC-STEHLY <noon@mykolab.com> - 2020-10-02 21:50 +0200
    Re: "ps -o %mem" and free memory in Linux <tomas@tuxteam.de> - 2020-09-29 11:20 +0200
      Re: "ps -o %mem" and free memory in Linux Victor Sudakov <vas@sibptus.ru> - 2020-09-29 17:20 +0200
        Re: "ps -o %mem" and free memory in Linux <tomas@tuxteam.de> - 2020-09-29 17:30 +0200
          Re: "ps -o %mem" and free memory in Linux Victor Sudakov <vas@sibptus.ru> - 2020-09-29 18:00 +0200

#227460 — "ps -o %mem" and free memory in Linux

[Multipart message — attachments visible in raw view] — view raw

Dear Colleagues,

Could you please clarify for me how the following is possible. `ps` shows
that the php-fpm workers have occupied 62% of physical memory, while
`free` shows that only 1.3Gi (which is 17% of total RAM) is used:

$ ps axww -o cmd,%mem |awk '/php-fpm/{sum+=$NF}END{print sum}'
62.1
$ free -h
              total        used        free      shared  buff/cache available
Mem:          7.5Gi       1.3Gi       4.4Gi       113Mi       1.8Gi
5.8Gi
Swap:            0B          0B          0B
$ 


-- 
Victor Sudakov,  VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/

[toc] | [next] | [standalone]

#227461

On Tue, 2020-09-29 at 10:24 +0700, Victor Sudakov wrote:
> Dear Colleagues,
> 
> Could you please clarify for me how the following is possible. `ps` shows
> that the php-fpm workers have occupied 62% of physical memory, while
> `free` shows that only 1.3Gi (which is 17% of total RAM) is used:
> 
> $ ps axww -o cmd,%mem |awk '/php-fpm/{sum+=$NF}END{print sum}'
> 62.1
> $ free -h
>               total        used        free      shared  buff/cache available
> Mem:          7.5Gi       1.3Gi       4.4Gi       113Mi       1.8Gi
> 5.8Gi
> Swap:            0B          0B          0B
> $ 

Perhaps because the php-fpm workers were forked from the same parent
and so a lot of theie 'physical' RAM is actually the same RAM as each
other, because it's not been modified?

-- 
Tixy

[toc] | [prev] | [next] | [standalone]

#227484

[Multipart message — attachments visible in raw view] — view raw

Tixy wrote:
> > 
> > Could you please clarify for me how the following is possible. `ps` shows
> > that the php-fpm workers have occupied 62% of physical memory, while
> > `free` shows that only 1.3Gi (which is 17% of total RAM) is used:
> > 
> > $ ps axww -o cmd,%mem |awk '/php-fpm/{sum+=$NF}END{print sum}'
> > 62.1
> > $ free -h
> >               total        used        free      shared  buff/cache available
> > Mem:          7.5Gi       1.3Gi       4.4Gi       113Mi       1.8Gi
> > 5.8Gi
> > Swap:            0B          0B          0B
> > $ 
> 
> Perhaps because the php-fpm workers were forked from the same parent
> and so a lot of theie 'physical' RAM is actually the same RAM as each
> other, because it's not been modified?

I see your point, but ps(1) talks about real physical RAM:

%mem        %MEM      ratio of the process's resident set size  to the physical memory on the machine, expressed as a percentage.  (alias pmem).

If those php-fpm workers share a lot of virtual (?) memory between one
another, shouldn't `ps` show it as such?

-- 
Victor Sudakov,  VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/

[toc] | [prev] | [next] | [standalone]

#227489

Victor Sudakov wrote:
> > Perhaps because the php-fpm workers were forked from the same parent
> > and so a lot of theie 'physical' RAM is actually the same RAM as each
> > other, because it's not been modified?
> 
> I see your point, but ps(1) talks about real physical RAM:
> 
> %mem        %MEM      ratio of the process's resident set size  to the physical memory on the machine, expressed as a percentage.  (alias pmem).
> 
> If those php-fpm workers share a lot of virtual (?) memory between one
> another, shouldn't `ps` show it as such?

You sum up this:

< php-fpm individual 1><----   php-fpm   ----   shared  ---->
< php-fpm individual 2><----   php-fpm   ----   shared  ---->
...
< php-fpm individual n><----   php-fpm   ----   shared  ---->

You summed up with awk: indivual[1..n] + n * shared

But the real memory sum is:  indivual[1..n] + 1 * shared

Do you see the difference?
 
Regards,
	Klaus.
-- 
Klaus Singvogel
GnuPG-Key-ID: 1024R/5068792D  1994-06-27

[toc] | [prev] | [next] | [standalone]

#227493

[Multipart message — attachments visible in raw view] — view raw

Klaus Singvogel wrote:
> Victor Sudakov wrote:
> > > Perhaps because the php-fpm workers were forked from the same parent
> > > and so a lot of theie 'physical' RAM is actually the same RAM as each
> > > other, because it's not been modified?
> > 
> > I see your point, but ps(1) talks about real physical RAM:
> > 
> > %mem        %MEM      ratio of the process's resident set size  to the physical memory on the machine, expressed as a percentage.  (alias pmem).
> > 
> > If those php-fpm workers share a lot of virtual (?) memory between one
> > another, shouldn't `ps` show it as such?
> 
> You sum up this:
> 
> < php-fpm individual 1><----   php-fpm   ----   shared  ---->
> < php-fpm individual 2><----   php-fpm   ----   shared  ---->
> ...
> < php-fpm individual n><----   php-fpm   ----   shared  ---->
> 
> You summed up with awk: indivual[1..n] + n * shared

I summed up with awk the values of %mem, which are supposed to be "ratio
of the process's resident set size  to the physical memory", correct?

In my understanding, the value of %mem should indicate how much physical
memory is spent on the "individual" part of the process, otherwise the
parameter is either useless or misdocumented.

> 
> But the real memory sum is:  indivual[1..n] + 1 * shared
> 
> Do you see the difference?

I see your point but still don't understand how that comes from the
definition of "%mem" in the man page.

-- 
Victor Sudakov,  VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/

[toc] | [prev] | [next] | [standalone]

#227550

Victor Sudakov writes:

> I summed up with awk the values of %mem, which are supposed to be "ratio
> of the process's resident set size  to the physical memory", correct?
>
> In my understanding, the value of %mem should indicate how much physical
> memory is spent on the "individual" part of the process, otherwise the
> parameter is either useless or misdocumented.

No, in resident memory you can find memory private to a process as well
as memory shared between processes.

And in memory private to a process, you can find resident memory and
non-resident memory.

According to Linux's Documentation/filesystems/proc.txt, we can find
this information in /proc/PID/status:

      VmData   size of private data segments
e.g.  VmData:  123004 kB

This looks like the amount of private memory, but I'm not sure.

--
Fabrice BAUZAC-STEHLY
PGP 015AE9B25DCB0511D200A75DE5674DEA514C891D

[toc] | [prev] | [next] | [standalone]

#227463

[Multipart message — attachments visible in raw view] — view raw

On Tue, Sep 29, 2020 at 10:24:35AM +0700, Victor Sudakov wrote:
> Dear Colleagues,
> 
> Could you please clarify for me how the following is possible. `ps` shows
> that the php-fpm workers have occupied 62% of physical memory, while
> `free` shows that only 1.3Gi (which is 17% of total RAM) is used:
> 
> $ ps axww -o cmd,%mem |awk '/php-fpm/{sum+=$NF}END{print sum}'

Ah, but you're adding the resident set sizes of many processes here,
right?

Remember that they do share quite a bit of that set: libraries, binaries,
and so on: you are counting that shared stuff more than once.

Cheers
 - t

[toc] | [prev] | [next] | [standalone]

#227485

[Multipart message — attachments visible in raw view] — view raw

tomas@tuxteam.de wrote:
> On Tue, Sep 29, 2020 at 10:24:35AM +0700, Victor Sudakov wrote:
> > Dear Colleagues,
> > 
> > Could you please clarify for me how the following is possible. `ps` shows
> > that the php-fpm workers have occupied 62% of physical memory, while
> > `free` shows that only 1.3Gi (which is 17% of total RAM) is used:
> > 
> > $ ps axww -o cmd,%mem |awk '/php-fpm/{sum+=$NF}END{print sum}'
> 
> Ah, but you're adding the resident set sizes of many processes here,
> right?

Indeed, as written in the man page:

%mem        %MEM      ratio of the process's resident set size  to the physical
memory on the machine, expressed as a percentage.  (alias pmem).

And it is even not mine, but Zabbix' idea, but that's another story.

> 
> Remember that they do share quite a bit of that set: libraries, binaries,
> and so on: you are counting that shared stuff more than once.

Those are kind of virtual things, as far as I understand. If not %mem, then
what `ps` parameter can show me how many php-fpm workers I can safely start
before RAM is exhausted?



-- 
Victor Sudakov,  VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/

[toc] | [prev] | [next] | [standalone]

#227487

[Multipart message — attachments visible in raw view] — view raw

On Tue, Sep 29, 2020 at 10:19:17PM +0700, Victor Sudakov wrote:

[...]

> Those are kind of virtual things, as far as I understand. If not %mem, then
> what `ps` parameter can show me how many php-fpm workers I can safely start
> before RAM is exhausted?

This is a seemingly easy question with a surprisingly difficult answer.

I don't have a good reference ready for you, but I poked a bit the
Intertubes and this [1] at least might give you a rough impression
of what kinds of things are involved.

Cheers

[1] https://stackoverflow.com/questions/131303/how-can-i-measure-the-actual-memory-usage-of-an-application-or-process

 - t

[toc] | [prev] | [next] | [standalone]

#227494

[Multipart message — attachments visible in raw view] — view raw

tomas@tuxteam.de wrote:
> 
> > Those are kind of virtual things, as far as I understand. If not %mem, then
> > what `ps` parameter can show me how many php-fpm workers I can safely start
> > before RAM is exhausted?
> 
> This is a seemingly easy question with a surprisingly difficult answer.
> 
> I don't have a good reference ready for you, but I poked a bit the
> Intertubes and this [1] at least might give you a rough impression
> of what kinds of things are involved.
> 
> Cheers
> 
> [1] https://stackoverflow.com/questions/131303/how-can-i-measure-the-actual-memory-usage-of-an-application-or-process
> 

Thank you, a very helpful, and about /proc/*/spams too.



-- 
Victor Sudakov,  VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web