Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1351295 > unrolled thread
| Started by | Wang Nan <wangnan0@huawei.com> |
|---|---|
| First post | 2016-03-07 05:00 +0100 |
| Last post | 2016-03-08 22:10 +0100 |
| Articles | 16 on this page of 36 — 6 participants |
Back to article view | Back to linux.kernel
[RESEND PATCH 0/5] perf core: Support overwrite ring buffer Wang Nan <wangnan0@huawei.com> - 2016-03-07 05:00 +0100
[RESEND PATCH 5/5] perf core: Reduce perf event output overhead by new overflow handler Wang Nan <wangnan0@huawei.com> - 2016-03-07 05:00 +0100
[RESEND PATCH 4/5] perf core: Add backward attribute to perf event Wang Nan <wangnan0@huawei.com> - 2016-03-07 05:00 +0100
[RESEND PATCH 3/5] perf core: Prepare writing into ring buffer from end Wang Nan <wangnan0@huawei.com> - 2016-03-07 05:00 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Peter Zijlstra <peterz@infradead.org> - 2016-03-08 14:50 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Ingo Molnar <mingo@kernel.org> - 2016-03-08 14:50 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Peter Zijlstra <peterz@infradead.org> - 2016-03-08 15:00 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Ingo Molnar <mingo@kernel.org> - 2016-03-08 16:30 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Dmitry Vyukov <dvyukov@google.com> - 2016-03-08 16:40 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Ingo Molnar <mingo@kernel.org> - 2016-03-08 17:00 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Dmitry Vyukov <dvyukov@google.com> - 2016-03-08 17:20 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Ingo Molnar <mingo@kernel.org> - 2016-03-08 17:30 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Dmitry Vyukov <dvyukov@google.com> - 2016-03-08 17:40 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Peter Zijlstra <peterz@infradead.org> - 2016-03-08 17:40 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Ingo Molnar <mingo@kernel.org> - 2016-03-08 17:50 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Ingo Molnar <mingo@kernel.org> - 2016-03-08 17:50 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Dmitry Vyukov <dvyukov@google.com> - 2016-03-08 18:10 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Ingo Molnar <mingo@kernel.org> - 2016-03-08 18:30 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Dmitry Vyukov <dvyukov@google.com> - 2016-03-08 18:30 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Ingo Molnar <mingo@kernel.org> - 2016-03-08 18:40 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Dmitry Vyukov <dvyukov@google.com> - 2016-03-08 18:50 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Ingo Molnar <mingo@kernel.org> - 2016-03-08 18:50 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Peter Zijlstra <peterz@infradead.org> - 2016-03-08 19:00 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Ingo Molnar <mingo@kernel.org> - 2016-03-08 19:00 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Ingo Molnar <mingo@kernel.org> - 2016-03-08 19:10 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Ingo Molnar <mingo@kernel.org> - 2016-03-08 19:30 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Ingo Molnar <mingo@kernel.org> - 2016-03-08 19:40 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Dmitry Vyukov <dvyukov@google.com> - 2016-03-08 19:00 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Peter Zijlstra <peterz@infradead.org> - 2016-03-08 19:00 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Peter Zijlstra <peterz@infradead.org> - 2016-03-08 17:40 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Borislav Petkov <bp@alien8.de> - 2016-03-09 12:00 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Dmitry Vyukov <dvyukov@google.com> - 2016-03-09 12:30 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Jiri Olsa <jolsa@redhat.com> - 2016-03-08 21:00 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Peter Zijlstra <peterz@infradead.org> - 2016-03-08 21:10 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Jiri Olsa <jolsa@redhat.com> - 2016-03-08 21:50 +0100
Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Peter Zijlstra <peterz@infradead.org> - 2016-03-08 22:10 +0100
Page 2 of 2 — ← Prev page 1 [2]
| From | Dmitry Vyukov <dvyukov@google.com> |
|---|---|
| Date | 2016-03-08 18:50 +0100 |
| Message-ID | <ratVF-47w-21@gated-at.bofh.it> |
| In reply to | #1353258 |
On Tue, Mar 8, 2016 at 6:37 PM, Ingo Molnar <mingo@kernel.org> wrote: > > * Dmitry Vyukov <dvyukov@google.com> wrote: > >> > fomalhaut:~/go/src/github.com/google/syzkaller> ps aux | grep -i syz >> > mingo 1374 0.0 0.0 118476 2376 pts/2 S+ 18:23 0:00 grep --color=auto -i syz >> > >> > and with no kernel messages in dmesg - and with a fully functional system. >> > >> > I'm running the 16-task load on a 120 CPU system - should I increase it to 120? >> > Does the code expect to saturate the system? >> >> No, it does not expect to saturate the system. Set "procs" to 480, or >> something like that. > > Does not seem to help much: > > fomalhaut:~> vmstat 10 > procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- > r b swpd free buff cache si so bi bo in cs us sy id wa st > > 1 0 0 257465904 219940 4736092 0 0 0 102 16022 4396 0 1 99 0 0 > 2 0 0 257452144 220496 4755052 0 0 2 3649 14286 4627 0 1 99 0 0 > 2 0 0 257473408 221188 4770824 0 0 15 1898 17175 4474 0 1 99 0 0 > > Only around 1% system utilization. Should I go for 1,000 or more? :) > > Peter, do you experience with running syz-kaller on larger CPU count Intel > systems? Try to set "dropprivs": false in config. I've noticed that creation/destruction of namespaces is very slow and globally serialized. So sometimes it takes tens of seconds for each worker processes to startup. For perf-related syscalls it should be "safe" to just run as root. And perf subsystem operation is also unaffected by namespaces as far as I know, so it should not affect behavior as well.
[toc] | [prev] | [next] | [standalone]
| From | Ingo Molnar <mingo@kernel.org> |
|---|---|
| Date | 2016-03-08 18:50 +0100 |
| Message-ID | <ratVI-47w-79@gated-at.bofh.it> |
| In reply to | #1353262 |
* Dmitry Vyukov <dvyukov@google.com> wrote: > On Tue, Mar 8, 2016 at 6:37 PM, Ingo Molnar <mingo@kernel.org> wrote: > > > > * Dmitry Vyukov <dvyukov@google.com> wrote: > > > >> > fomalhaut:~/go/src/github.com/google/syzkaller> ps aux | grep -i syz > >> > mingo 1374 0.0 0.0 118476 2376 pts/2 S+ 18:23 0:00 grep --color=auto -i syz > >> > > >> > and with no kernel messages in dmesg - and with a fully functional system. > >> > > >> > I'm running the 16-task load on a 120 CPU system - should I increase it to 120? > >> > Does the code expect to saturate the system? > >> > >> No, it does not expect to saturate the system. Set "procs" to 480, or > >> something like that. > > > > Does not seem to help much: > > > > fomalhaut:~> vmstat 10 > > procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- > > r b swpd free buff cache si so bi bo in cs us sy id wa st > > > > 1 0 0 257465904 219940 4736092 0 0 0 102 16022 4396 0 1 99 0 0 > > 2 0 0 257452144 220496 4755052 0 0 2 3649 14286 4627 0 1 99 0 0 > > 2 0 0 257473408 221188 4770824 0 0 15 1898 17175 4474 0 1 99 0 0 > > > > Only around 1% system utilization. Should I go for 1,000 or more? :) > > > > Peter, do you experience with running syz-kaller on larger CPU count Intel > > systems? > > > Try to set "dropprivs": false in config. Things got a lot more lively after that! But most of the overhead seems to come from systemd trying to dump core or something like that: 85872 mingo 20 0 34712 3016 2656 S 4.6 0.0 0:00.14 systemd-coredum 85440 mingo 20 0 34712 3028 2664 S 4.2 0.0 0:00.13 systemd-coredum 85751 mingo 20 0 34712 3076 2716 S 4.2 0.0 0:00.13 systemd-coredum 85840 mingo 20 0 34712 2988 2624 S 4.2 0.0 0:00.13 systemd-coredum 85861 mingo 20 0 34712 3080 2720 S 4.2 0.0 0:00.13 systemd-coredum 85954 mingo 20 0 34712 3028 2664 S 4.2 0.0 0:00.13 systemd-coredum and I have: fomalhaut:~/go/src/github.com/google/syzkaller> ulimit -c 0 weird ... Has any of you seen such behavior? Thanks, Ingo
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2016-03-08 19:00 +0100 |
| Message-ID | <rau5k-4bI-7@gated-at.bofh.it> |
| In reply to | #1353266 |
On Tue, Mar 08, 2016 at 06:48:56PM +0100, Ingo Molnar wrote: > weird ... Has any of you seen such behavior? No systemd on any of my machines ;-)
[toc] | [prev] | [next] | [standalone]
| From | Ingo Molnar <mingo@kernel.org> |
|---|---|
| Date | 2016-03-08 19:00 +0100 |
| Message-ID | <rau5k-4bI-13@gated-at.bofh.it> |
| In reply to | #1353266 |
* Ingo Molnar <mingo@kernel.org> wrote: > Things got a lot more lively after that! > > But most of the overhead seems to come from systemd trying to dump core or > something like that: > > 85872 mingo 20 0 34712 3016 2656 S 4.6 0.0 0:00.14 systemd-coredum > 85440 mingo 20 0 34712 3028 2664 S 4.2 0.0 0:00.13 systemd-coredum > 85751 mingo 20 0 34712 3076 2716 S 4.2 0.0 0:00.13 systemd-coredum > 85840 mingo 20 0 34712 2988 2624 S 4.2 0.0 0:00.13 systemd-coredum > 85861 mingo 20 0 34712 3080 2720 S 4.2 0.0 0:00.13 systemd-coredum > 85954 mingo 20 0 34712 3028 2664 S 4.2 0.0 0:00.13 systemd-coredum > > and I have: > > fomalhaut:~/go/src/github.com/google/syzkaller> ulimit -c > 0 > > weird ... Has any of you seen such behavior? So the workaround for that is to disable systemd trying to log every core dump to the system journal (!), via: echo > /proc/sys/kernel/core_pattern Thanks, Ingo
[toc] | [prev] | [next] | [standalone]
| From | Ingo Molnar <mingo@kernel.org> |
|---|---|
| Date | 2016-03-08 19:10 +0100 |
| Message-ID | <raueZ-4va-3@gated-at.bofh.it> |
| In reply to | #1353269 |
* Ingo Molnar <mingo@kernel.org> wrote: > > * Ingo Molnar <mingo@kernel.org> wrote: > > > Things got a lot more lively after that! > > > > But most of the overhead seems to come from systemd trying to dump core or > > something like that: > > > > 85872 mingo 20 0 34712 3016 2656 S 4.6 0.0 0:00.14 systemd-coredum > > 85440 mingo 20 0 34712 3028 2664 S 4.2 0.0 0:00.13 systemd-coredum > > 85751 mingo 20 0 34712 3076 2716 S 4.2 0.0 0:00.13 systemd-coredum > > 85840 mingo 20 0 34712 2988 2624 S 4.2 0.0 0:00.13 systemd-coredum > > 85861 mingo 20 0 34712 3080 2720 S 4.2 0.0 0:00.13 systemd-coredum > > 85954 mingo 20 0 34712 3028 2664 S 4.2 0.0 0:00.13 systemd-coredum > > > > and I have: > > > > fomalhaut:~/go/src/github.com/google/syzkaller> ulimit -c > > 0 > > > > weird ... Has any of you seen such behavior? > > So the workaround for that is to disable systemd trying to log every core dump to > the system journal (!), via: > > echo > /proc/sys/kernel/core_pattern So with that fixed, I finally started fuzzing for real. With nproc set to 120 it seems to be chugging along at about 25% system utilization: Tasks: 1271 total, 1 running, 1270 sleeping, 0 stopped, 0 zombie %Cpu(s): 4.5 us, 35.3 sy, 0.0 ni, 55.2 id, 0.5 wa, 4.5 hi, 0.0 si, 0.0 st KiB Mem : 26401230+total, 25401017+free, 1624640 used, 8377496 buff/cache KiB Swap: 0 total, 0 free, 0 used. 26143329+avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 87593 mingo 20 0 7850436 162284 11672 S 2941 0.1 33:01.83 syz-fuzzer 923 root 20 0 84772 44344 43840 S 22.8 0.0 1:23.86 systemd-journal 1369 root 16 -4 114636 3256 2832 S 15.7 0.0 0:29.03 auditd 1379 root 12 -8 80236 1764 1432 S 8.3 0.0 0:15.79 audispd 878 root 20 0 0 0 0 S 6.9 0.0 0:16.55 jbd2/sda1-8 1381 root 16 -4 52216 3232 2892 S 3.8 0.0 0:07.08 sedispatch Even that one is not ideal - obviously there's way too much systemd-journal overhead, but I'm unable to turn the darn thing off ... with nproc=480 it does not seem to be working very well - it quickly generates: 2016/03/08 18:59:38 local-0: lost connection: exit status 2 2016/03/08 18:59:38 local-0: saving crash 'lost connection' to crash-local-0-1457459978295423413 and then after some time starts to ramp up again. It's mostly idling around. Thanks, Ingo
[toc] | [prev] | [next] | [standalone]
| From | Ingo Molnar <mingo@kernel.org> |
|---|---|
| Date | 2016-03-08 19:30 +0100 |
| Message-ID | <rauyn-4FV-29@gated-at.bofh.it> |
| In reply to | #1353276 |
* Ingo Molnar <mingo@kernel.org> wrote:
> With nproc set to 120 it seems to be chugging along at about 25% system
> utilization:
>
> Tasks: 1271 total, 1 running, 1270 sleeping, 0 stopped, 0 zombie
> %Cpu(s): 4.5 us, 35.3 sy, 0.0 ni, 55.2 id, 0.5 wa, 4.5 hi, 0.0 si, 0.0 st
> KiB Mem : 26401230+total, 25401017+free, 1624640 used, 8377496 buff/cache
> KiB Swap: 0 total, 0 free, 0 used. 26143329+avail Mem
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 87593 mingo 20 0 7850436 162284 11672 S 2941 0.1 33:01.83 syz-fuzzer
> 923 root 20 0 84772 44344 43840 S 22.8 0.0 1:23.86 systemd-journal
> 1369 root 16 -4 114636 3256 2832 S 15.7 0.0 0:29.03 auditd
> 1379 root 12 -8 80236 1764 1432 S 8.3 0.0 0:15.79 audispd
> 878 root 20 0 0 0 0 S 6.9 0.0 0:16.55 jbd2/sda1-8
> 1381 root 16 -4 52216 3232 2892 S 3.8 0.0 0:07.08 sedispatch
>
> Even that one is not ideal - obviously there's way too much systemd-journal
> overhead, but I'm unable to turn the darn thing off ...
The journal.conf man page suggests that putting 'Storage=none' into
/etc/systemd/journal.conf disables journalling - but that's not true.
It's apparently impossible to disable systemd logging via any normal means ...
So the workaround for all that is a brutal:
mv /var/log/journal /var/log/journal.dontuse
after that there does not seem to be systemd logging anymore.
... but there is tons of auditd logging now! ;-)
Fortunately that's easily stopped via:
service stop auditd
with that there's no log IO anymore during fuzzing (yay!).
Except that systemd journald rears its ugly head back:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
111464 mingo 20 0 7249024 143348 11476 S 2741 0.1 7:41.31 syz-fuzzer
122632 root 20 0 29200 2768 2524 S 9.5 0.0 0:28.74 systemd-journal
111463 root 20 0 165596 5772 3840 R 4.8 0.0 0:00.80 top
112078 mingo 20 0 20928 704 644 S 2.9 0.0 0:00.17 syz-executor
112516 mingo 20 0 20928 708 644 S 2.9 0.0 0:00.16 syz-executor
878 root 20 0 0 0 0 S 1.9 0.0 0:53.83 jbd2/sda1-8
111711 mingo 20 0 20928 704 644 S 1.9 0.0 0:00.16 syz-executor
111901 mingo 20 0 20928 704 644 S 1.9 0.0 0:00.15 syz-executor
8 root 20 0 0 0 0 S 1.0 0.0 1:00.55 rcu_sched
so it's eating about 10% of system overhead despite doing nothing (!).
The workaround for that systemd bug is a brutal:
[root@fomalhaut ~]# mv /usr/lib/systemd/systemd-journald /usr/lib/systemd/systemd-journald.dontuse
[root@fomalhaut ~]#
Thanks,
Ingo
[toc] | [prev] | [next] | [standalone]
| From | Ingo Molnar <mingo@kernel.org> |
|---|---|
| Date | 2016-03-08 19:40 +0100 |
| Message-ID | <rauI2-4Kx-21@gated-at.bofh.it> |
| In reply to | #1353297 |
* Ingo Molnar <mingo@kernel.org> wrote: > so it's eating about 10% of system overhead despite doing nothing (!). > > The workaround for that systemd bug is a brutal: > > [root@fomalhaut ~]# mv /usr/lib/systemd/systemd-journald /usr/lib/systemd/systemd-journald.dontuse > [root@fomalhaut ~]# Except that if I do that, systemd stops working altogether - for example: [root@fomalhaut ~]# systemctl disable audit Failed to execute operation: Connection timed out only reinstating the binary and re-starting journald fixes that. systemd is the most passive-agressive utility I've ever seen. So it's not possible to disable journald. Does anyone know any solution for that, which does not involve reinstalling a whole distro? Thanks, Ingo
[toc] | [prev] | [next] | [standalone]
| From | Dmitry Vyukov <dvyukov@google.com> |
|---|---|
| Date | 2016-03-08 19:00 +0100 |
| Message-ID | <rau5k-4bI-29@gated-at.bofh.it> |
| In reply to | #1353266 |
On Tue, Mar 8, 2016 at 6:48 PM, Ingo Molnar <mingo@kernel.org> wrote: > > * Dmitry Vyukov <dvyukov@google.com> wrote: > >> On Tue, Mar 8, 2016 at 6:37 PM, Ingo Molnar <mingo@kernel.org> wrote: >> > >> > * Dmitry Vyukov <dvyukov@google.com> wrote: >> > >> >> > fomalhaut:~/go/src/github.com/google/syzkaller> ps aux | grep -i syz >> >> > mingo 1374 0.0 0.0 118476 2376 pts/2 S+ 18:23 0:00 grep --color=auto -i syz >> >> > >> >> > and with no kernel messages in dmesg - and with a fully functional system. >> >> > >> >> > I'm running the 16-task load on a 120 CPU system - should I increase it to 120? >> >> > Does the code expect to saturate the system? >> >> >> >> No, it does not expect to saturate the system. Set "procs" to 480, or >> >> something like that. >> > >> > Does not seem to help much: >> > >> > fomalhaut:~> vmstat 10 >> > procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- >> > r b swpd free buff cache si so bi bo in cs us sy id wa st >> > >> > 1 0 0 257465904 219940 4736092 0 0 0 102 16022 4396 0 1 99 0 0 >> > 2 0 0 257452144 220496 4755052 0 0 2 3649 14286 4627 0 1 99 0 0 >> > 2 0 0 257473408 221188 4770824 0 0 15 1898 17175 4474 0 1 99 0 0 >> > >> > Only around 1% system utilization. Should I go for 1,000 or more? :) >> > >> > Peter, do you experience with running syz-kaller on larger CPU count Intel >> > systems? >> >> >> Try to set "dropprivs": false in config. > > Things got a lot more lively after that! > > But most of the overhead seems to come from systemd trying to dump core or > something like that: > > 85872 mingo 20 0 34712 3016 2656 S 4.6 0.0 0:00.14 systemd-coredum > 85440 mingo 20 0 34712 3028 2664 S 4.2 0.0 0:00.13 systemd-coredum > 85751 mingo 20 0 34712 3076 2716 S 4.2 0.0 0:00.13 systemd-coredum > 85840 mingo 20 0 34712 2988 2624 S 4.2 0.0 0:00.13 systemd-coredum > 85861 mingo 20 0 34712 3080 2720 S 4.2 0.0 0:00.13 systemd-coredum > 85954 mingo 20 0 34712 3028 2664 S 4.2 0.0 0:00.13 systemd-coredum > > and I have: > > fomalhaut:~/go/src/github.com/google/syzkaller> ulimit -c > 0 > > weird ... Has any of you seen such behavior? I have not seen it. Probably I need to directly disable core dumps within the syz-executor process.
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2016-03-08 19:00 +0100 |
| Message-ID | <rau5l-4bI-31@gated-at.bofh.it> |
| In reply to | #1353258 |
On Tue, Mar 08, 2016 at 06:37:09PM +0100, Ingo Molnar wrote: > Peter, do you experience with running syz-kaller on larger CPU count Intel > systems? I run it on a 20 core (40 cpu) system mostly, as that tends to not take ages to come back up again once I wrecked it.
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2016-03-08 17:40 +0100 |
| Message-ID | <rasPT-3kP-3@gated-at.bofh.it> |
| In reply to | #1353192 |
On Tue, Mar 08, 2016 at 05:27:03PM +0100, Ingo Molnar wrote:
> > > triton:~/go/src/github.com/google/syzkaller> cat perf.cfg
> > > {
> > > "http": "localhost:50000",
> > > "workdir": "/home/mingo/go/src/github.com/google/syzkaller/workdir",
> > > "syzkaller": "/home/mingo/go/src/github.com/google/syzkaller",
> > > "vmlinux": "-",
> > > "type": "local",
> > > "count": 1,
> > > "procs": 16,
> > > "nocover": true,
this^^^
> I now get periodic output of:
>
> fomalhaut:~/go/src/github.com/google/syzkaller> bin/syz-manager -config src/github.com/google/syzkaller/perf.cfg
> 2016/03/08 17:24:07 failed to read config file: open src/github.com/google/syzkaller/perf.cfg: no such file or directory
> fomalhaut:~/go/src/github.com/google/syzkaller> bin/syz-manager -config perf.cfg
> 2016/03/08 17:24:19 loading corpus...
> 2016/03/08 17:24:19 loaded 0 programs
> 2016/03/08 17:24:19 serving http on http://localhost:50000
> 2016/03/08 17:24:19 serving rpc on tcp://127.0.0.1:37006
> 2016/03/08 17:24:34 local-0: saving crash 'BUG: /sys/kernel/debug/kcov is missing (permission denied). Enable CONFIG_KCOV and mount debugfs.' to crash-local-0-1457454274467286949
> 2016/03/08 17:24:34 local-0: lost connection: exit status 1
> is CONFIG_KCOV=y a must-have feature? There's no KCOV support upstream that I can
> see.
Should have disabled that.
[toc] | [prev] | [next] | [standalone]
| From | Borislav Petkov <bp@alien8.de> |
|---|---|
| Date | 2016-03-09 12:00 +0100 |
| Message-ID | <raK0q-6IA-19@gated-at.bofh.it> |
| In reply to | #1353172 |
On Tue, Mar 08, 2016 at 04:54:23PM +0100, Ingo Molnar wrote:
> triton:~/go/src/github.com/google/syzkaller> cat perf.cfg
> {
> "http": "localhost:50000",
> "workdir": "/home/mingo/go/src/github.com/google/syzkaller/workdir",
> "syzkaller": "/home/mingo/go/src/github.com/google/syzkaller",
> "vmlinux": "-",
> "type": "local",
> "count": 1,
> "procs": 16,
> "nocover": true,
> "nodropprivs": true,
> "enable_syscalls": [
> "getpid",
> "perf_event_open",
Btw, is there a way to specify range of arguments to feed into
perf_event_open? Like, limit @attr_uptr to single or multiple event IDs
or so, for example?
--
Regards/Gruss,
Boris.
ECO tip #101: Trim your mails when you reply.
[toc] | [prev] | [next] | [standalone]
| From | Dmitry Vyukov <dvyukov@google.com> |
|---|---|
| Date | 2016-03-09 12:30 +0100 |
| Message-ID | <raKtt-7bN-39@gated-at.bofh.it> |
| In reply to | #1353985 |
On Wed, Mar 9, 2016 at 11:53 AM, Borislav Petkov <bp@alien8.de> wrote:
> On Tue, Mar 08, 2016 at 04:54:23PM +0100, Ingo Molnar wrote:
>> triton:~/go/src/github.com/google/syzkaller> cat perf.cfg
>> {
>> "http": "localhost:50000",
>> "workdir": "/home/mingo/go/src/github.com/google/syzkaller/workdir",
>> "syzkaller": "/home/mingo/go/src/github.com/google/syzkaller",
>> "vmlinux": "-",
>> "type": "local",
>> "count": 1,
>> "procs": 16,
>> "nocover": true,
>> "nodropprivs": true,
>> "enable_syscalls": [
>> "getpid",
>> "perf_event_open",
>
> Btw, is there a way to specify range of arguments to feed into
> perf_event_open? Like, limit @attr_uptr to single or multiple event IDs
> or so, for example?
No, there is no _that_ level of granularity in the config.
If you really-really want that, then you can alter description of the
perf_event_open syscall in sys/perf.txt file:
https://github.com/google/syzkaller/blob/master/sys/perf.txt
to be more restrictive. For example, you can limit set of values
passed in a particular argument, or even set some args/fields to const
values.
Then you will need to do:
$ make generate LINUX=/path/to/fresh/linux/checkout
$ make
But I would suggest to not do that. That perf config already limits
set of syscalls to a very small set. The fuzzer should be able to
examine all interesting combinations of arguments for these syscalls
in a reasonable time (provided that you use CONFIG_KCOV). And in the
end you don't know where the bugs. They are usually where you don't
expect them. So I would suggest the opposite: describe and more
perf-related syscalls, describe arguments in greater detail, enable
other syscalls that can have effect on perf subsystem. And then just
run it for longer using more machines.
[toc] | [prev] | [next] | [standalone]
| From | Jiri Olsa <jolsa@redhat.com> |
|---|---|
| Date | 2016-03-08 21:00 +0100 |
| Message-ID | <ravXs-5uX-15@gated-at.bofh.it> |
| In reply to | #1353069 |
On Tue, Mar 08, 2016 at 02:57:59PM +0100, Peter Zijlstra wrote:
> On Tue, Mar 08, 2016 at 02:49:01PM +0100, Ingo Molnar wrote:
> >
> > * Peter Zijlstra <peterz@infradead.org> wrote:
> >
> > > On Mon, Mar 07, 2016 at 03:50:14AM +0000, Wang Nan wrote:
> > > > This patch set has been posted multiple times (with and without
> > > > corresponding 'perf tool' patches), and doesn't receive further
> > > > comment. I think it should be okay to merge them into mainline.
> > > > There are many perf's improvement depend on it. However, Peter
> > > > is not responsive after I fixed some problems he pointed out.
> > > >
> > > > Introduces 'write_backward' into perf_event_attr, allows kernel
> > > > writing the ring buffer from the end of it. This feature allows
> > > > extracting data from overwritable ring buffer.
> > > >
> > > > Wang Nan (5):
> > > > perf core: Introduce new ioctl options to pause and resume ring buffer
> > > > perf core: Set event's default overflow_handler
> > > > perf core: Prepare writing into ring buffer from end
> > > > perf core: Add backward attribute to perf event
> > > > perf core: Reduce perf event output overhead by new overflow handler
> > >
> > > perf kernel features are currently on hold until I can manage to run a
> > > fuzzer for more than a few minutes without my machine having a seizure.
> >
> > Btw., could you describe exactly what commands you are running, with what
> > configuration options (if that matters), so that people who'd like our feature
> > freeze to be lifted can help out?
>
> Mostly syz-kaller, but also Vince's perf-fuzzer and your perf-stress
> script, which I'm not sure is publicly available.
>
> perf_fuzzer lives at:
>
> https://github.com/deater/perf_event_tests.git
>
> Here's a thread on syz-kaller:
>
> lkml.kernel.org/r/CACT4Y+Ym0TZLkmRrM0ZGgLpu8kqS-YjoWTMrvaLz=tx2tnyO3w@mail.gmail.com
>
> If things have shifted again I'm sure Dmitry is willing to help.
>
> I run the thing natively on actual real hardware, which ensure I get to
> test the PMU drivers too.
>
> # cat go-fuzz.sh
> #!/bin/bash
>
> echo 1 > /proc/sys/kernel/traceoff_on_warning
> echo 1 > /debug/tracing/options/stacktrace
> echo 1 > /debug/tracing/events/sched/enable
are you running this under root?
jirka
> cd gopath/src/github.com/google/syzkaller/
> ./bin/syz-manager -config perf.cfg
>
> # cat gopath/src/github.com/google/syzkaller/perf.cfg
>
> {
> "http": "localhost:50000",
> "workdir": "/root/gopath/src/github.com/google/syzkaller/workdir",
> "syzkaller": "/root/gopath/src/github.com/google/syzkaller",
> "vmlinux": "-",
> "type": "local",
> "count": 1,
> "procs": 160,
> "cover": false,
> "dropprivs": false,
> "enable_syscalls": [
> "getpid",
> "gettid",
> "perf_event_open",
> "ioctl$PERF*",
> "prctl$void",
> "bpf$*",
> "sched_yield"
> ]
> }
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2016-03-08 21:10 +0100 |
| Message-ID | <raw78-5OE-17@gated-at.bofh.it> |
| In reply to | #1353358 |
On Tue, Mar 08, 2016 at 08:56:42PM +0100, Jiri Olsa wrote: > > # cat go-fuzz.sh > > #!/bin/bash > > > > echo 1 > /proc/sys/kernel/traceoff_on_warning > > echo 1 > /debug/tracing/options/stacktrace > > echo 1 > /debug/tracing/events/sched/enable > > are you running this under root? Of course ;-) I'm not even sure these machines have a user account. Also, what's the point of fuzzing if half your 'fun' options return -EPERM.
[toc] | [prev] | [next] | [standalone]
| From | Jiri Olsa <jolsa@redhat.com> |
|---|---|
| Date | 2016-03-08 21:50 +0100 |
| Message-ID | <rawJP-659-3@gated-at.bofh.it> |
| In reply to | #1353372 |
On Tue, Mar 08, 2016 at 09:07:46PM +0100, Peter Zijlstra wrote: > On Tue, Mar 08, 2016 at 08:56:42PM +0100, Jiri Olsa wrote: > > > # cat go-fuzz.sh > > > #!/bin/bash > > > > > > echo 1 > /proc/sys/kernel/traceoff_on_warning > > > echo 1 > /debug/tracing/options/stacktrace > > > echo 1 > /debug/tracing/events/sched/enable > > > > are you running this under root? > > Of course ;-) I'm not even sure these machines have a user account. > > Also, what's the point of fuzzing if half your 'fun' options return > -EPERM. IIRC the perf fuzzer (one from Vince) could screw your machine having it run under root jirka
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2016-03-08 22:10 +0100 |
| Message-ID | <rax3b-6qY-5@gated-at.bofh.it> |
| In reply to | #1353386 |
On Tue, Mar 08, 2016 at 09:44:58PM +0100, Jiri Olsa wrote: > On Tue, Mar 08, 2016 at 09:07:46PM +0100, Peter Zijlstra wrote: > > On Tue, Mar 08, 2016 at 08:56:42PM +0100, Jiri Olsa wrote: > > > > # cat go-fuzz.sh > > > > #!/bin/bash > > > > > > > > echo 1 > /proc/sys/kernel/traceoff_on_warning > > > > echo 1 > /debug/tracing/options/stacktrace > > > > echo 1 > /debug/tracing/events/sched/enable > > > > > > are you running this under root? > > > > Of course ;-) I'm not even sure these machines have a user account. > > > > Also, what's the point of fuzzing if half your 'fun' options return > > -EPERM. > > IIRC the perf fuzzer (one from Vince) could screw your machine having > it run under root Never noticed :-) Guess how I usually run it..
[toc] | [prev] | [standalone]
Page 2 of 2 — ← Prev page 1 [2]
Back to top | Article view | linux.kernel
csiph-web