Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1443968 > unrolled thread

Re: [PATCH -v4 2/2] printk: Add kernel parameter to control writes to /dev/kmsg

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH -v4 2/2] printk: Add kernel parameter to control writes  to /dev/kmsg Dave Young <dyoung@redhat.com> - 2016-07-15 08:30 +0200
    Re: [PATCH -v4 2/2] printk: Add kernel parameter to control writes  to /dev/kmsg Borislav Petkov <bp@alien8.de> - 2016-07-15 14:50 +0200
      Re: [PATCH -v4 2/2] printk: Add kernel parameter to control writes  to /dev/kmsg Dave Young <dyoung@redhat.com> - 2016-07-16 12:50 +0200
        Re: [PATCH -v4 2/2] printk: Add kernel parameter to control writes  to /dev/kmsg Borislav Petkov <bp@alien8.de> - 2016-07-17 07:50 +0200
          Re: [PATCH -v4 2/2] printk: Add kernel parameter to control writes  to /dev/kmsg Dave Young <dyoung@redhat.com> - 2016-07-18 04:20 +0200
            Re: [PATCH -v4 2/2] printk: Add kernel parameter to control writes  to /dev/kmsg Borislav Petkov <bp@alien8.de> - 2016-07-18 06:50 +0200
              Re: [PATCH -v4 2/2] printk: Add kernel parameter to control writes  to /dev/kmsg Dave Young <dyoung@redhat.com> - 2016-07-18 07:30 +0200
                Re: [PATCH -v4 2/2] printk: Add kernel parameter to control writes  to /dev/kmsg Borislav Petkov <bp@alien8.de> - 2016-07-18 09:30 +0200
                  Re: [PATCH -v4 2/2] printk: Add kernel parameter to control writes  to /dev/kmsg Dave Young <dyoung@redhat.com> - 2016-07-18 09:40 +0200
                    Re: [PATCH -v4 2/2] printk: Add kernel parameter to control writes  to /dev/kmsg Borislav Petkov <bp@alien8.de> - 2016-07-18 10:10 +0200
                      Re: [PATCH -v4 2/2] printk: Add kernel parameter to control writes  to /dev/kmsg Dave Young <dyoung@redhat.com> - 2016-07-18 10:20 +0200
                        Re: [PATCH -v4 2/2] printk: Add kernel parameter to control writes  to /dev/kmsg Borislav Petkov <bp@alien8.de> - 2016-07-18 11:10 +0200
                          Re: [PATCH -v4 2/2] printk: Add kernel parameter to control writes  to /dev/kmsg Dave Young <dyoung@redhat.com> - 2016-07-19 02:40 +0200
                            Re: [PATCH -v4 2/2] printk: Add kernel parameter to control writes  to /dev/kmsg Borislav Petkov <bp@alien8.de> - 2016-07-19 08:50 +0200
                              Re: [PATCH -v4 2/2] printk: Add kernel parameter to control writes  to /dev/kmsg Dave Young <dyoung@redhat.com> - 2016-07-19 09:10 +0200
                        Re: [PATCH -v4 2/2] printk: Add kernel parameter to control writes  to /dev/kmsg Steven Rostedt <rostedt@goodmis.org> - 2016-07-25 17:20 +0200

#1443968 — Re: [PATCH -v4 2/2] printk: Add kernel parameter to control writes to /dev/kmsg

On 07/08/16 at 11:18am, Borislav Petkov wrote:
> From: Borislav Petkov <bp@suse.de>
> 
> Add a "printk.devkmsg" kernel command line parameter which controls how
> userspace writes into /dev/kmsg. It has three options:
> 
> * ratelimit - ratelimit logging from userspace.
> * on  - unlimited logging from userspace
> * off - logging from userspace gets ignored
> 
> The default setting is to ratelimit the messages written to it.

Sorry for jumping in late. But I just see this today and I really like the
idea to add a switch to turn off the kmsg writing from userspace because
I suffer from it also.

I may missed the background, what is the reason for "ratelimit"?  It sounds
a little odd. I think use default "on" should be safe to keep same behavior
as before.

Thanks
Dave

[toc] | [next] | [standalone]

#1444271

On Fri, Jul 15, 2016 at 02:21:09PM +0800, Dave Young wrote:
> Sorry for jumping in late. But I just see this today and I really like the
> idea to add a switch to turn off the kmsg writing from userspace because
> I suffer from it also.

Thanks, I'll send v5 soon.

> I may missed the background, what is the reason for "ratelimit"?  It sounds
> a little odd. I think use default "on" should be safe to keep same behavior
> as before.

Yeah, so we want to ratelimit by default so that excessive logging
doesn't disturb system operation. People who want to see all that
blubber can do so with "on" so they should be taken care of too.

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--

[toc] | [prev] | [next] | [standalone]

#1444807

On 07/15/16 at 02:45pm, Borislav Petkov wrote:
> On Fri, Jul 15, 2016 at 02:21:09PM +0800, Dave Young wrote:
> > Sorry for jumping in late. But I just see this today and I really like the
> > idea to add a switch to turn off the kmsg writing from userspace because
> > I suffer from it also.
> 
> Thanks, I'll send v5 soon.
> 
> > I may missed the background, what is the reason for "ratelimit"?  It sounds
> > a little odd. I think use default "on" should be safe to keep same behavior
> > as before.
> 
> Yeah, so we want to ratelimit by default so that excessive logging
> doesn't disturb system operation. People who want to see all that
> blubber can do so with "on" so they should be taken care of too.

Ratelimit the writing to kmsg sounds not a good way, the ratelimit use cases
are for "callbacks" being mentioned in other thread. Basiclly specific
printk from same line of a function will be suppressed, that means the
supressed messages are somehow same error/message/warnings. But different
writings to /dev/msg could be from different userspace software so that the
messages being supressed could be different and the *ratelimit* is not
fine-grained as *ratelimt* supposed to be.

Mis-ratelimit cause critical userspace messages being lost, that is worse
than use off as default. Suppose we turn off devkmsg by default distributions
can still turn on it with sysctl and for us who do not want the flooding we can
use printk.devkmsg=off in kernel cmdline to override it.

Of course if we turn off it by default we can print a warning to alert user.

BTW, for userspace messages maybe they should not go to same log buffer, maybe
a separate log buffer for /dev/msg will be better.

Thanks
Dave

[toc] | [prev] | [next] | [standalone]

#1445027

On Sat, Jul 16, 2016 at 06:44:25PM +0800, Dave Young wrote:

...

> Mis-ratelimit cause critical userspace messages being lost, that is worse

The current setting is quite generous so that all critical messages
should land in dmesg. Besides, we don't ratelimit during boot. The idea
is that userspace should switch to a different logging facility once the
system is up... which userspace does reportedly.

IOW, /dev/kmsg should handle a relatively big logging amount without
ratelimiting.

> than use off as default. Suppose we turn off devkmsg by default distributions
> can still turn on it with sysctl and for us who do not want the flooding we can
> use printk.devkmsg=off in kernel cmdline to override it.

That part I cannot parse.

> Of course if we turn off it by default we can print a warning to alert user.
> 
> BTW, for userspace messages maybe they should not go to same log buffer, maybe
> a separate log buffer for /dev/msg will be better.

See above.

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--

[toc] | [prev] | [next] | [standalone]

#1445157

On 07/17/16 at 07:40am, Borislav Petkov wrote:
> On Sat, Jul 16, 2016 at 06:44:25PM +0800, Dave Young wrote:
> 
> ...
> 
> > Mis-ratelimit cause critical userspace messages being lost, that is worse
> 
> The current setting is quite generous so that all critical messages
> should land in dmesg. Besides, we don't ratelimit during boot. The idea
> is that userspace should switch to a different logging facility once the
> system is up... which userspace does reportedly.

I would say avoiding ratelimit during boot make no much sense. Userspace can not
write to /dev/kmsg when system_state == SYSTEM_BOOTING because init process
has not run yet.

> 
> IOW, /dev/kmsg should handle a relatively big logging amount without
> ratelimiting.
> 
> > than use off as default. Suppose we turn off devkmsg by default distributions
> > can still turn on it with sysctl and for us who do not want the flooding we can
> > use printk.devkmsg=off in kernel cmdline to override it.
> 
> That part I cannot parse.

I do not understand, care to elaborate a bit?

Let me explain my comments, I means to set printk.devkmsg=off by default,
userspace can set it to on by sysctl. User can provide kernel cmdline
printk.devkmsg=off if he/she want.

Or set printk.devkmsg=on by default to avoid break userspace, it is also fine.

> 
> > Of course if we turn off it by default we can print a warning to alert user.
> > 
> > BTW, for userspace messages maybe they should not go to same log buffer, maybe
> > a separate log buffer for /dev/msg will be better.
> 
> See above.
> 

Thanks
Dave

[toc] | [prev] | [next] | [standalone]

#1445201

On Mon, Jul 18, 2016 at 10:18:09AM +0800, Dave Young wrote:
> I would say avoiding ratelimit during boot make no much sense. Userspace can not
> write to /dev/kmsg when system_state == SYSTEM_BOOTING because init process
> has not run yet.

You're right - kernel_init() sets SYSTEM_RUNNING before running the init
process. I probably should kill all that logic in the second patch.

> I means to set printk.devkmsg=off by default, userspace can set it to
> on by sysctl.

That can't happen: DEVKMSG_LOG_MASK_LOCK.

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--

[toc] | [prev] | [next] | [standalone]

#1445216

On 07/18/16 at 06:44am, Borislav Petkov wrote:
> On Mon, Jul 18, 2016 at 10:18:09AM +0800, Dave Young wrote:
> > I would say avoiding ratelimit during boot make no much sense. Userspace can not
> > write to /dev/kmsg when system_state == SYSTEM_BOOTING because init process
> > has not run yet.
> 
> You're right - kernel_init() sets SYSTEM_RUNNING before running the init
> process. I probably should kill all that logic in the second patch.
> 
> > I means to set printk.devkmsg=off by default, userspace can set it to
> > on by sysctl.
> 
> That can't happen: DEVKMSG_LOG_MASK_LOCK.

Sorry, seems I do not get your point, suppose using the bis defined in your
patch, shouldn't below work?

#define DEVKMSG_LOG_MASK_DEFAULT	2

Thanks
Dave

[toc] | [prev] | [next] | [standalone]

#1445272

On Mon, Jul 18, 2016 at 01:20:32PM +0800, Dave Young wrote:
> Sorry, seems I do not get your point, suppose using the bis defined in your
> patch, shouldn't below work?

When you supply "printk.devkmsg=" on the kernel command line, then
devkmsg logging setting is final for the current boot because we lock it
with DEVKMSG_LOG_MASK_LOCK, see control_devkmsg() here:

https://lkml.kernel.org/r/20160716061745.15795-3-bp@alien8.de

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--

[toc] | [prev] | [next] | [standalone]

#1445278

On 07/18/16 at 09:21am, Borislav Petkov wrote:
> On Mon, Jul 18, 2016 at 01:20:32PM +0800, Dave Young wrote:
> > Sorry, seems I do not get your point, suppose using the bis defined in your
> > patch, shouldn't below work?
> 
> When you supply "printk.devkmsg=" on the kernel command line, then
> devkmsg logging setting is final for the current boot because we lock it
> with DEVKMSG_LOG_MASK_LOCK, see control_devkmsg() here:

Boris, I saw that point and I agreed about that design, what I do not like is
the ratelimit part. What in my mind is like below (suppose default=off)

printk.devkmsg= on kernel cmdline
   -> locked, one can not use sysctl to change it
   -> follow the kernel cmdline setting
no printk.devkmsg= on kernel cmdline
   -> unlocked, default is off, writing to /dev/kmsg is forbidded
      -> usespace set sysctl printk.devkmsg=on
         -> writing to /dev/kmsg is allowed.

Thanks
Dave

[toc] | [prev] | [next] | [standalone]

#1445287

On Mon, Jul 18, 2016 at 03:38:45PM +0800, Dave Young wrote:
> no printk.devkmsg= on kernel cmdline
>    -> unlocked, default is off, writing to /dev/kmsg is forbidded
>       -> usespace set sysctl printk.devkmsg=on
>          -> writing to /dev/kmsg is allowed.

And that helps how exactly?

I don't understand why you're harping on the ratelimiting. It is a sane
default for the *default* case. Userspace should switch to its own
logging scheme when it has started anyway.

If you still want to see *all* messages, you boot with "=on" or set it
through sysctl.

So what is the problem?!

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--

[toc] | [prev] | [next] | [standalone]

#1445292

On 07/18/16 at 10:08am, Borislav Petkov wrote:
> On Mon, Jul 18, 2016 at 03:38:45PM +0800, Dave Young wrote:
> > no printk.devkmsg= on kernel cmdline
> >    -> unlocked, default is off, writing to /dev/kmsg is forbidded
> >       -> usespace set sysctl printk.devkmsg=on
> >          -> writing to /dev/kmsg is allowed.
> 
> And that helps how exactly?
> 
> I don't understand why you're harping on the ratelimiting. It is a sane
> default for the *default* case. Userspace should switch to its own
> logging scheme when it has started anyway.

Because IMHO it is wrong, they can not be ratelimited because the writing could
be from different userspace programs. Simply ratelimiting different sources of
writing is pointless to me. One can only see messages they would like to see
by luck, it is worse than off.

> 
> If you still want to see *all* messages, you boot with "=on" or set it
> through sysctl.
> 
> So what is the problem?!

As above mentioned..

Thanks
Dave

[toc] | [prev] | [next] | [standalone]

#1445317

On Mon, Jul 18, 2016 at 04:17:12PM +0800, Dave Young wrote:
> Because IMHO it is wrong, they can not be ratelimited because the writing could
> be from different userspace programs.

It is ratelimited by interface openers.

> Simply ratelimiting different sources of writing is pointless to me.

So what are you arguing for? What is the *actual* *real-life* *use* case
you think will be handicapped?

> One can only see messages they would like to see by luck, it is worse
> than off.

THAT'S WHY YOU BOOT WITH "printk.devkmsg=on" TO SEE THEM ALL!

The /dev/kmsg thing was added for the more or less, wrong, historic
reasons and userspace started abusing it and interfering with kernel
operation. That's why we're adding this tristate option.

In the default case we're ratelimiting writes to it because they should
not interfere with kernel operation.

  [ Frankly, those writes are pretty much useless to the normal user so
    we can just as well ignore them but WTH. ]

So give me a concrete problem you see with the ralimiting and not some
notion of a feeling you might have of it being pointless, ok?

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--

[toc] | [prev] | [next] | [standalone]

#1445957

On 07/18/16 at 11:06am, Borislav Petkov wrote:
> On Mon, Jul 18, 2016 at 04:17:12PM +0800, Dave Young wrote:
> > Because IMHO it is wrong, they can not be ratelimited because the writing could
> > be from different userspace programs.
> 
> It is ratelimited by interface openers.

You are thinking from kernel point of view, but it will be different from
a user point of view.

> 
> > Simply ratelimiting different sources of writing is pointless to me.
> 
> So what are you arguing for? What is the *actual* *real-life* *use* case
> you think will be handicapped?
> 
> > One can only see messages they would like to see by luck, it is worse
> > than off.
> 
> THAT'S WHY YOU BOOT WITH "printk.devkmsg=on" TO SEE THEM ALL!
> 
> The /dev/kmsg thing was added for the more or less, wrong, historic
> reasons and userspace started abusing it and interfering with kernel
> operation. That's why we're adding this tristate option.
> 
> In the default case we're ratelimiting writes to it because they should
> not interfere with kernel operation.
> 
>   [ Frankly, those writes are pretty much useless to the normal user so
>     we can just as well ignore them but WTH. ]
> 
> So give me a concrete problem you see with the ralimiting and not some
> notion of a feeling you might have of it being pointless, ok?

Ok, for example below (A and B can be any program, systemd or other logging
utility, dracut scripts...)

Program A:
for (i = 0; i < 100; i++)
	write an error message A1

for (i =0; i < 100; i++)
	write an error message A2

Program B:
for (i = 0; i < 100; i++)
	write an error message B1

for (i =0; i < 100; i++)
	write an error message B2

In above case, ratelimit specific message like A1 is reasonable, but simply
ratelimit the writing to /dev/kmsg will cause the final messages appear are
uncertain. This is not ratelimit should do.

We provide a default printk.devkmsg, it should either works or not, the
middle point is just wrong because it helps nothing, so what is the benefit
of the ratelimit compares with "off"? If there's no benefits why we will
make the code complicate?
 
Thanks
Dave

[toc] | [prev] | [next] | [standalone]

#1446067

On Tue, Jul 19, 2016 at 08:35:07AM +0800, Dave Young wrote:
> Ok, for example below (A and B can be any program, systemd or other logging
> utility, dracut scripts...)

Are you even reading what I'm writing to you?

I asked for real-life use case and I told you the ratelimiting is done
by openers of /dev/kmsg.

> Program A:
> for (i = 0; i < 100; i++)
> 	write an error message A1

Because this is a real use case, right? Write A1 100 times in a loop?!
You gotta be f*cking kidding me!

Now I'm going to ignore you know until you start reading what I'm
writing to you.

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--

[toc] | [prev] | [next] | [standalone]

#1446110

On 07/19/16 at 08:49am, Borislav Petkov wrote:
> On Tue, Jul 19, 2016 at 08:35:07AM +0800, Dave Young wrote:
> > Ok, for example below (A and B can be any program, systemd or other logging
> > utility, dracut scripts...)
> 
> Are you even reading what I'm writing to you?
> 
> I asked for real-life use case and I told you the ratelimiting is done
> by openers of /dev/kmsg.
> 
> > Program A:
> > for (i = 0; i < 100; i++)
> > 	write an error message A1
> 
> Because this is a real use case, right? Write A1 100 times in a loop?!
> You gotta be f*cking kidding me!

It is just emulating a use case, in a real life use case of course it shouldn't
be a for loop. Suppose checking a device existance every some period, it
is possible for same error messages being printed again and again, may for
same device, may for different device, but it is from same source code
line. I suppose you can understand, but you did not.

Please stop being angry, I was trying to understand you. I hope you can
also think from different point of view.

> 
> Now I'm going to ignore you know until you start reading what I'm
> writing to you.

Same to me, I will stop reply because I have tried my best but I'm
beginning have same feeling as you that is you just do not want to read
what I wrote, sigh.

Thanks
Dave

[toc] | [prev] | [next] | [standalone]

#1449566

On Mon, 18 Jul 2016 16:17:12 +0800
Dave Young <dyoung@redhat.com> wrote:

> On 07/18/16 at 10:08am, Borislav Petkov wrote:
> > On Mon, Jul 18, 2016 at 03:38:45PM +0800, Dave Young wrote:  
> > > no printk.devkmsg= on kernel cmdline  
> > >    -> unlocked, default is off, writing to /dev/kmsg is forbidded
> > >       -> usespace set sysctl printk.devkmsg=on
> > >          -> writing to /dev/kmsg is allowed.  
> > 
> > And that helps how exactly?
> > 
> > I don't understand why you're harping on the ratelimiting. It is a sane
> > default for the *default* case. Userspace should switch to its own
> > logging scheme when it has started anyway.  
> 
> Because IMHO it is wrong, they can not be ratelimited because the writing could
> be from different userspace programs. Simply ratelimiting different sources of
> writing is pointless to me. One can only see messages they would like to see
> by luck, it is worse than off.
> 

[ sorry for coming in late, I've been on vacation ]

Note, this is a kernel resource that, IMHO should never had been
accessed by userspace in the first place. Perhaps we should create a
separate buffer for userspace instead. But until then, we need this
usage.

If ratelimit is triggered, then something is blasting the limited
resource of the kernel, and information will be lost. Most of the time,
it's the first lines that are needed to determine what happened, so if
something starts writing a lot into the buffer, you want to save what
was first written. That's the most critical part. Rate limiting will
help save the part you want to see most.

And it could have been the kernel that caused the issue. Rate limiting
userspace to preserve the kernel output is a good thing.

-- Steve

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web