Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1611919 > unrolled thread

Re: [BUG nohz]: wrong user and system time accounting

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [BUG nohz]: wrong user and system time accounting Frederic Weisbecker <fweisbec@gmail.com> - 2017-03-29 15:10 +0200
    Re: [BUG nohz]: wrong user and system time accounting Rik van Riel <riel@redhat.com> - 2017-03-29 15:20 +0200
      Re: [BUG nohz]: wrong user and system time accounting Luiz Capitulino <lcapitulino@redhat.com> - 2017-03-29 15:30 +0200
        Re: [BUG nohz]: wrong user and system time accounting Frederic Weisbecker <fweisbec@gmail.com> - 2017-03-29 23:20 +0200
          Re: [BUG nohz]: wrong user and system time accounting Luiz Capitulino <lcapitulino@redhat.com> - 2017-03-30 03:50 +0200

#1611919 — Re: [BUG nohz]: wrong user and system time accounting

On Thu, Mar 23, 2017 at 04:55:12PM -0400, Luiz Capitulino wrote:
> 
> When there are two or more tasks executing in user-space and
> taking 100% of a nohz_full CPU, top reports 70% system time
> and 30% user time utilization. Sometimes I'm even able to get
> 100% system time and 0% user time.
> 
> This was reproduced with latest Linus tree (093b995), but I
> don't believe it's a regression (at least not a recent one)
> as I can reproduce it with older kernels. Also, I have
> CONFIG_IRQ_TIME_ACCOUNTING=y and haven't tried to reproduce
> without it yet.
> 
> Below you'll find the steps to reproduce and some initial
> analysis.
> 
> Steps to reproduce
> ------------------
> 
> 1. Set up a CPU for nohz_full with isolcpus= nohz_full=
> 
> 2. Pin two tasks that hog the CPU 100% of the time to that CPU

I failed to reproduce with your config. I'm still getting 99% userspace
cputime. So I'm wondering if the hogging style plays a role.

I run pure user loops:

    int main(int argc, char **argv)
    {
        for (;;);
        return 0
    }

Does your user program perform syscalls or IOs of some sort?

[toc] | [next] | [standalone]

#1611922

On Wed, 2017-03-29 at 15:04 +0200, Frederic Weisbecker wrote:
> On Thu, Mar 23, 2017 at 04:55:12PM -0400, Luiz Capitulino wrote:
> > 
> > When there are two or more tasks executing in user-space and
> > taking 100% of a nohz_full CPU, top reports 70% system time
> > and 30% user time utilization. Sometimes I'm even able to get
> > 100% system time and 0% user time.
> > 
> > This was reproduced with latest Linus tree (093b995), but I
> > don't believe it's a regression (at least not a recent one)
> > as I can reproduce it with older kernels. Also, I have
> > CONFIG_IRQ_TIME_ACCOUNTING=y and haven't tried to reproduce
> > without it yet.
> > 
> > Below you'll find the steps to reproduce and some initial
> > analysis.
> > 
> > Steps to reproduce
> > ------------------
> > 
> > 1. Set up a CPU for nohz_full with isolcpus= nohz_full=
> > 
> > 2. Pin two tasks that hog the CPU 100% of the time to that CPU
> 
> I failed to reproduce with your config. I'm still getting 99%
> userspace
> cputime. So I'm wondering if the hogging style plays a role.
> 
> I run pure user loops:
> 
>     int main(int argc, char **argv)
>     {
>         for (;;);
>         return 0
>     }
> 
> Does your user program perform syscalls or IOs of some sort?

Luiz's program makes a syscall every millisecond,
if started with the arguments he gave as his
reproducer.

[toc] | [prev] | [next] | [standalone]

#1611931

[Multipart message — attachments visible in raw view] — view raw

On Wed, 29 Mar 2017 09:14:32 -0400
Rik van Riel <riel@redhat.com> wrote:

> > I failed to reproduce with your config. I'm still getting 99%
> > userspace
> > cputime. So I'm wondering if the hogging style plays a role.
> > 
> > I run pure user loops:
> > 
> >     int main(int argc, char **argv)
> >     {
> >         for (;;);
> >         return 0
> >     }
> > 
> > Does your user program perform syscalls or IOs of some sort?  
> 
> Luiz's program makes a syscall every millisecond,
> if started with the arguments he gave as his
> reproducer.

There are various reproducers actually. I started off with the simple
loop above, then wrote the attach program and then wrote the one
you're mentioning:

 http://people.redhat.com/~lcapitul/real-time/acct-bug.c

All of them reproduce the issue 100% of the time for me.

[toc] | [prev] | [next] | [standalone]

#1612367

On Wed, Mar 29, 2017 at 09:23:57AM -0400, Luiz Capitulino wrote:
> 
> There are various reproducers actually. I started off with the simple
> loop above, then wrote the attach program and then wrote the one
> you're mentioning:
> 
>  http://people.redhat.com/~lcapitul/real-time/acct-bug.c
> 
> All of them reproduce the issue 100% of the time for me.

> #define _GNU_SOURCE
> #include <stdio.h>
> #include <unistd.h>
> #include <stdlib.h>
> #include <sched.h>
> #include <sys/types.h>
> 
> static int move_to_cpu(int cpu)
> {
>         cpu_set_t set;
> 
>         CPU_ZERO(&set);
>         CPU_SET(cpu, &set);
>         return sched_setaffinity(0, sizeof(set), &set);
> }
> 
> static void loop(void)
> {
>         for (;;) ;
> }
> 
> static int fork_hog(int cpu)
> {
>         int pid;
> 
>         pid = (int) fork();
>         if (pid == 0) {
>                 move_to_cpu(cpu);
>                 loop();
>                 exit(0);
>         }
> 
>         return pid;
> }
> 
> int main(int argc, char *argv[])
> {
>         int i, pid, cpu, nr_procs;
> 
> 		if (argc != 3) {
> 			printf("usage: hog < nr-procs > < CPU >\n");
> 			exit(1);
> 		}
> 
> 		cpu = atoi(argv[2]);
> 		nr_procs = atoi(argv[1]);
> 
>         for (i = 0; i < nr_procs; i++) {
>                 pid = fork_hog(cpu);
>                 fprintf(stderr, "created hog%d pid=%d\n", i, pid);
>         }
> 
>         fprintf(stderr, "pausing...\n");
>         pause();
> 
>         return 0;
> }

I just tried both of these and none seem to show incorrect cputime :-/
I'm wondering if that bug depends on some hardware.

[toc] | [prev] | [next] | [standalone]

#1612487

On Wed, 29 Mar 2017 23:12:00 +0200
Frederic Weisbecker <fweisbec@gmail.com> wrote:

> On Wed, Mar 29, 2017 at 09:23:57AM -0400, Luiz Capitulino wrote:
> > 
> > There are various reproducers actually. I started off with the simple
> > loop above, then wrote the attach program and then wrote the one
> > you're mentioning:
> > 
> >  http://people.redhat.com/~lcapitul/real-time/acct-bug.c
> > 
> > All of them reproduce the issue 100% of the time for me.  
> 
> > #define _GNU_SOURCE
> > #include <stdio.h>
> > #include <unistd.h>
> > #include <stdlib.h>
> > #include <sched.h>
> > #include <sys/types.h>
> > 
> > static int move_to_cpu(int cpu)
> > {
> >         cpu_set_t set;
> > 
> >         CPU_ZERO(&set);
> >         CPU_SET(cpu, &set);
> >         return sched_setaffinity(0, sizeof(set), &set);
> > }
> > 
> > static void loop(void)
> > {
> >         for (;;) ;
> > }
> > 
> > static int fork_hog(int cpu)
> > {
> >         int pid;
> > 
> >         pid = (int) fork();
> >         if (pid == 0) {
> >                 move_to_cpu(cpu);
> >                 loop();
> >                 exit(0);
> >         }
> > 
> >         return pid;
> > }
> > 
> > int main(int argc, char *argv[])
> > {
> >         int i, pid, cpu, nr_procs;
> > 
> > 		if (argc != 3) {
> > 			printf("usage: hog < nr-procs > < CPU >\n");
> > 			exit(1);
> > 		}
> > 
> > 		cpu = atoi(argv[2]);
> > 		nr_procs = atoi(argv[1]);
> > 
> >         for (i = 0; i < nr_procs; i++) {
> >                 pid = fork_hog(cpu);
> >                 fprintf(stderr, "created hog%d pid=%d\n", i, pid);
> >         }
> > 
> >         fprintf(stderr, "pausing...\n");
> >         pause();
> > 
> >         return 0;
> > }  
> 
> I just tried both of these and none seem to show incorrect cputime :-/
> I'm wondering if that bug depends on some hardware.

Are you running on x86? My CPU is:

Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz

I wonder if this issue depends on the timer used by the hrtimer
subsystem.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web