Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1522953 > unrolled thread

perf: fuzzer KASAN unwind_get_return_address

Back to article view | Back to linux.kernel

Contents

  perf: fuzzer KASAN unwind_get_return_address Vince Weaver <vincent.weaver@maine.edu> - 2016-11-15 18:50 +0100
    Re: perf: fuzzer KASAN unwind_get_return_address Peter Zijlstra <peterz@infradead.org> - 2016-11-15 20:00 +0100
      Re: perf: fuzzer KASAN unwind_get_return_address Vince Weaver <vincent.weaver@maine.edu> - 2016-11-15 20:10 +0100
        Re: perf: fuzzer KASAN unwind_get_return_address Josh Poimboeuf <jpoimboe@redhat.com> - 2016-11-15 22:00 +0100
          Re: perf: fuzzer KASAN unwind_get_return_address Peter Zijlstra <peterz@infradead.org> - 2016-11-16 14:10 +0100
            Re: perf: fuzzer KASAN unwind_get_return_address Dmitry Vyukov <dvyukov@google.com> - 2016-11-16 14:20 +0100
            Re: perf: fuzzer KASAN unwind_get_return_address Josh Poimboeuf <jpoimboe@redhat.com> - 2016-11-16 15:40 +0100
              Re: perf: fuzzer KASAN unwind_get_return_address Josh Poimboeuf <jpoimboe@redhat.com> - 2016-11-16 16:00 +0100
              Re: perf: fuzzer KASAN unwind_get_return_address Peter Zijlstra <peterz@infradead.org> - 2016-11-16 16:00 +0100
                Re: perf: fuzzer KASAN unwind_get_return_address Josh Poimboeuf <jpoimboe@redhat.com> - 2016-11-17 05:50 +0100
                  Re: perf: fuzzer KASAN unwind_get_return_address Peter Zijlstra <peterz@infradead.org> - 2016-11-17 10:20 +0100
                    Re: perf: fuzzer KASAN unwind_get_return_address Josh Poimboeuf <jpoimboe@redhat.com> - 2016-11-17 18:10 +0100
                      Re: perf: fuzzer KASAN unwind_get_return_address Vince Weaver <vincent.weaver@maine.edu> - 2016-11-17 18:20 +0100
                      Re: perf: fuzzer KASAN unwind_get_return_address Dmitry Vyukov <dvyukov@google.com> - 2016-11-17 18:50 +0100
                    Re: perf: fuzzer KASAN unwind_get_return_address Josh Poimboeuf <jpoimboe@redhat.com> - 2016-11-17 18:40 +0100
                      Re: perf: fuzzer KASAN unwind_get_return_address Vince Weaver <vincent.weaver@maine.edu> - 2016-11-17 19:20 +0100
                  Re: perf: fuzzer KASAN unwind_get_return_address Peter Zijlstra <peterz@infradead.org> - 2016-11-17 18:20 +0100
                    Re: perf: fuzzer KASAN unwind_get_return_address Peter Zijlstra <peterz@infradead.org> - 2016-11-17 18:20 +0100
                      [tip:perf/urgent] perf/x86/intel: Cure bogus unwind from PEBS  entries tip-bot for Peter Zijlstra <tipbot@zytor.com> - 2016-11-22 13:40 +0100
                  Re: perf: fuzzer KASAN unwind_get_return_address Josh Poimboeuf <jpoimboe@redhat.com> - 2016-11-17 18:50 +0100
              Re: perf: fuzzer KASAN unwind_get_return_address Peter Zijlstra <peterz@infradead.org> - 2016-11-16 16:00 +0100
                Re: perf: fuzzer KASAN unwind_get_return_address Vince Weaver <vincent.weaver@maine.edu> - 2016-11-16 16:10 +0100
            [PATCH 1/2] unwind: prevent KASAN false positive warnings in guess unwinder Josh Poimboeuf <jpoimboe@redhat.com> - 2016-11-17 18:20 +0100
              Re: [PATCH 1/2] unwind: prevent KASAN false positive warnings in  guess unwinder Josh Poimboeuf <jpoimboe@redhat.com> - 2016-11-17 21:30 +0100
                Re: [PATCH 1/2] unwind: prevent KASAN false positive warnings in  guess unwinder Ingo Molnar <mingo@kernel.org> - 2016-11-18 09:40 +0100
              [tip:x86/urgent] x86/unwind: Prevent KASAN false positive warnings  in guess unwinder tip-bot for Josh Poimboeuf <tipbot@zytor.com> - 2016-11-18 10:10 +0100
            [PATCH 2/2] dumpstack: prevent KASAN false positive warnings Josh Poimboeuf <jpoimboe@redhat.com> - 2016-11-17 18:20 +0100
              [tip:x86/urgent] x86/dumpstack: Prevent KASAN false positive  warnings tip-bot for Josh Poimboeuf <tipbot@zytor.com> - 2016-11-18 10:10 +0100
      Re: perf: fuzzer KASAN unwind_get_return_address Dmitry Vyukov <dvyukov@google.com> - 2016-11-15 20:10 +0100
        Re: perf: fuzzer KASAN unwind_get_return_address Josh Poimboeuf <jpoimboe@redhat.com> - 2016-11-15 22:00 +0100

Page 2 of 2 — ← Prev page 1 [2]

#1523555

On Wed, Nov 16, 2016 at 08:37:46AM -0600, Josh Poimboeuf wrote:
> On Wed, Nov 16, 2016 at 02:03:37PM +0100, Peter Zijlstra wrote:
> > On Tue, Nov 15, 2016 at 02:57:48PM -0600, Josh Poimboeuf wrote:
> > > Would you mind posting a disassembly of unwind_get_return_address()?
> > > Any idea how recreatable it is?  (In particular I'd be interested in
> > > seeing this dump with the latest unwinder improvements in the -tip tree,
> > > which dump the pt_regs associated with an interrupt.)
> > 
> > Fairly reproducable it seems, doesn't seem to include pt_regs dumps
> > though :/
> > 
> > tip/master as of this morning.
> 
> Thanks.  This is actually a different issue than the one reported by
> Vince.  In this case FRAME_POINTER is disabled, so it uses the "guess"
> unwinder which scans every address on the stack, looking for text
> addresses.  So the kasan errors are expected.
> 
> (The missing pt_regs are also expected: the guess unwinder doesn't show
> them.)
> 
> I'll work up a patch to fix this.  I still have no idea what's causing
> Vince's bug in the frame pointer unwinder.

Hurm,.. by the number of '?' entries in Vince's backtrace I was assuming
it was without frame pointers.

Let me enable those and run again, it didn't insta-trigger like it does
without.

[toc] | [prev] | [next] | [standalone]

#1523564

On Wed, 16 Nov 2016, Peter Zijlstra wrote:

> On Wed, Nov 16, 2016 at 08:37:46AM -0600, Josh Poimboeuf wrote:
> > On Wed, Nov 16, 2016 at 02:03:37PM +0100, Peter Zijlstra wrote:
> > > On Tue, Nov 15, 2016 at 02:57:48PM -0600, Josh Poimboeuf wrote:
> > > > Would you mind posting a disassembly of unwind_get_return_address()?
> > > > Any idea how recreatable it is?  (In particular I'd be interested in
> > > > seeing this dump with the latest unwinder improvements in the -tip tree,
> > > > which dump the pt_regs associated with an interrupt.)
> > > 
> > > Fairly reproducable it seems, doesn't seem to include pt_regs dumps
> > > though :/
> > > 
> > > tip/master as of this morning.
> > 
> > Thanks.  This is actually a different issue than the one reported by
> > Vince.  In this case FRAME_POINTER is disabled, so it uses the "guess"
> > unwinder which scans every address on the stack, looking for text
> > addresses.  So the kasan errors are expected.
> > 
> > (The missing pt_regs are also expected: the guess unwinder doesn't show
> > them.)
> > 
> > I'll work up a patch to fix this.  I still have no idea what's causing
> > Vince's bug in the frame pointer unwinder.
> 
> Hurm,.. by the number of '?' entries in Vince's backtrace I was assuming
> it was without frame pointers.
> 
> Let me enable those and run again, it didn't insta-trigger like it does
> without.

Yes, that machine does have frame pointers enabled.

Sorry for the delay responding, the machine crashed right after I had left 
last night and so I wasn't able to take a look at what was going on until 
right now.

It triggers fairly quickly on the Haswell machine but not my other 
machines (although they possibly don't have quite as many debug options 
turned on).

Vince

[toc] | [prev] | [next] | [standalone]

#1524514 — [PATCH 1/2] unwind: prevent KASAN false positive warnings in guess unwinder

The guess unwinder scans the entire stack, which can cause KASAN
"stack-out-of-bounds" false positive warnings.  Tell KASAN to ignore it.

Reported-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
---
 arch/x86/kernel/unwind_guess.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kernel/unwind_guess.c b/arch/x86/kernel/unwind_guess.c
index 2d721e5..b80e8bf 100644
--- a/arch/x86/kernel/unwind_guess.c
+++ b/arch/x86/kernel/unwind_guess.c
@@ -7,11 +7,13 @@
 
 unsigned long unwind_get_return_address(struct unwind_state *state)
 {
+	unsigned long addr = READ_ONCE_NOCHECK(*state->sp);
+
 	if (unwind_done(state))
 		return 0;
 
 	return ftrace_graph_ret_addr(state->task, &state->graph_idx,
-				     *state->sp, state->sp);
+				     addr, state->sp);
 }
 EXPORT_SYMBOL_GPL(unwind_get_return_address);
 
@@ -23,8 +25,10 @@ bool unwind_next_frame(struct unwind_state *state)
 		return false;
 
 	do {
+		unsigned long addr = READ_ONCE_NOCHECK(*state->sp);
+
 		for (state->sp++; state->sp < info->end; state->sp++)
-			if (__kernel_text_address(*state->sp))
+			if (__kernel_text_address(addr))
 				return true;
 
 		state->sp = info->next_sp;
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]

#1524796 — Re: [PATCH 1/2] unwind: prevent KASAN false positive warnings in guess unwinder

On Thu, Nov 17, 2016 at 09:57:23AM -0600, Josh Poimboeuf wrote:
> The guess unwinder scans the entire stack, which can cause KASAN
> "stack-out-of-bounds" false positive warnings.  Tell KASAN to ignore it.
> 
> Reported-by: Peter Zijlstra <peterz@infradead.org>
> Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>

Whoops, forgot to prefix the patch subjects with "x86/".

-- 
Josh

[toc] | [prev] | [next] | [standalone]

#1525086 — Re: [PATCH 1/2] unwind: prevent KASAN false positive warnings in guess unwinder

* Josh Poimboeuf <jpoimboe@redhat.com> wrote:

> On Thu, Nov 17, 2016 at 09:57:23AM -0600, Josh Poimboeuf wrote:
> > The guess unwinder scans the entire stack, which can cause KASAN
> > "stack-out-of-bounds" false positive warnings.  Tell KASAN to ignore it.
> > 
> > Reported-by: Peter Zijlstra <peterz@infradead.org>
> > Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
> 
> Whoops, forgot to prefix the patch subjects with "x86/".

No problem, fixed it up.

Thanks,

	Ingo

[toc] | [prev] | [next] | [standalone]

#1525100 — [tip:x86/urgent] x86/unwind: Prevent KASAN false positive warnings in guess unwinder

Commit-ID:  c2d75e03d6307bda0e14b616818a6f7b09fd623a
Gitweb:     http://git.kernel.org/tip/c2d75e03d6307bda0e14b616818a6f7b09fd623a
Author:     Josh Poimboeuf <jpoimboe@redhat.com>
AuthorDate: Thu, 17 Nov 2016 09:57:23 -0600
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 18 Nov 2016 09:38:00 +0100

x86/unwind: Prevent KASAN false positive warnings in guess unwinder

The guess unwinder scans the entire stack, which can cause KASAN
"stack-out-of-bounds" false positive warnings.  Tell KASAN to ignore it.

Reported-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Stephane Eranian <eranian@gmail.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vince Weaver <vincent.weaver@maine.edu>
Cc: davej@codemonkey.org.uk
Cc: dvyukov@google.com
Link: http://lkml.kernel.org/r/61939c0b2b2d63ce97ba59cba3b00fd47c2962cf.1479398226.git.jpoimboe@redhat.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/kernel/unwind_guess.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kernel/unwind_guess.c b/arch/x86/kernel/unwind_guess.c
index 2d721e5..b80e8bf 100644
--- a/arch/x86/kernel/unwind_guess.c
+++ b/arch/x86/kernel/unwind_guess.c
@@ -7,11 +7,13 @@
 
 unsigned long unwind_get_return_address(struct unwind_state *state)
 {
+	unsigned long addr = READ_ONCE_NOCHECK(*state->sp);
+
 	if (unwind_done(state))
 		return 0;
 
 	return ftrace_graph_ret_addr(state->task, &state->graph_idx,
-				     *state->sp, state->sp);
+				     addr, state->sp);
 }
 EXPORT_SYMBOL_GPL(unwind_get_return_address);
 
@@ -23,8 +25,10 @@ bool unwind_next_frame(struct unwind_state *state)
 		return false;
 
 	do {
+		unsigned long addr = READ_ONCE_NOCHECK(*state->sp);
+
 		for (state->sp++; state->sp < info->end; state->sp++)
-			if (__kernel_text_address(*state->sp))
+			if (__kernel_text_address(addr))
 				return true;
 
 		state->sp = info->next_sp;

[toc] | [prev] | [next] | [standalone]

#1524536 — [PATCH 2/2] dumpstack: prevent KASAN false positive warnings

The oops stack dump code scans the entire stack, which can cause KASAN
"stack-out-of-bounds" false positive warnings.  Tell KASAN to ignore it.

Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
---
 arch/x86/kernel/dumpstack.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/dumpstack.c b/arch/x86/kernel/dumpstack.c
index 9b7cf5c..85f854b 100644
--- a/arch/x86/kernel/dumpstack.c
+++ b/arch/x86/kernel/dumpstack.c
@@ -112,7 +112,7 @@ void show_trace_log_lvl(struct task_struct *task, struct pt_regs *regs,
 		for (; stack < stack_info.end; stack++) {
 			unsigned long real_addr;
 			int reliable = 0;
-			unsigned long addr = *stack;
+			unsigned long addr = READ_ONCE_NOCHECK(*stack);
 			unsigned long *ret_addr_p =
 				unwind_get_return_address_ptr(&state);
 
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]

#1525103 — [tip:x86/urgent] x86/dumpstack: Prevent KASAN false positive warnings

Commit-ID:  91e08ab0c8515450258d7ad9033bfe69bebad25a
Gitweb:     http://git.kernel.org/tip/91e08ab0c8515450258d7ad9033bfe69bebad25a
Author:     Josh Poimboeuf <jpoimboe@redhat.com>
AuthorDate: Thu, 17 Nov 2016 09:57:24 -0600
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 18 Nov 2016 09:38:00 +0100

x86/dumpstack: Prevent KASAN false positive warnings

The oops stack dump code scans the entire stack, which can cause KASAN
"stack-out-of-bounds" false positive warnings.  Tell KASAN to ignore it.

Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@gmail.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vince Weaver <vincent.weaver@maine.edu>
Cc: davej@codemonkey.org.uk
Cc: dvyukov@google.com
Link: http://lkml.kernel.org/r/5f6e80c4b0c7f7f0b6211900847a247cdaad753c.1479398226.git.jpoimboe@redhat.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/kernel/dumpstack.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/dumpstack.c b/arch/x86/kernel/dumpstack.c
index 9b7cf5c..85f854b 100644
--- a/arch/x86/kernel/dumpstack.c
+++ b/arch/x86/kernel/dumpstack.c
@@ -112,7 +112,7 @@ void show_trace_log_lvl(struct task_struct *task, struct pt_regs *regs,
 		for (; stack < stack_info.end; stack++) {
 			unsigned long real_addr;
 			int reliable = 0;
-			unsigned long addr = *stack;
+			unsigned long addr = READ_ONCE_NOCHECK(*stack);
 			unsigned long *ret_addr_p =
 				unwind_get_return_address_ptr(&state);
 

[toc] | [prev] | [next] | [standalone]

#1523021

On Tue, Nov 15, 2016 at 10:57 AM, Peter Zijlstra <peterz@infradead.org> wrote:
> On Tue, Nov 15, 2016 at 12:43:56PM -0500, Vince Weaver wrote:
>>
>> Running on my haswell machine with the imc/uncore patch applied, the
>> perf_fuzzer next tripped over this issue.
>>
>> [  202.034495] BAD LUCK: lost 371 message(s) from NMI context!
>> [  202.034496] ==================================================================
>> [  202.048327] BUG: KASAN: stack-out-of-bounds in unwind_get_return_address+0x35/0x80 at addr ffff8800cff0bd90
>> [  202.058826] Read of size 8 by task perf_fuzzer/16254
>> [  202.064186] page:ffffea00033fc2c0 count:1 mapcount:0 mapping:          (null) index:0x0^Ac
>> [  202.073068] flags: 0x1ffff8000000400(reserved)
>> [  202.077885] page dumped because: kasan: bad access detected
>> [  202.083880] CPU: 4 PID: 16254 Comm: perf_fuzzer Not tainted 4.9.0-rc5+ #5
>> [  202.091204] Hardware name: LENOVO 10AM000AUS/SHARKBAY, BIOS FBKT72AUS 01/26/2014
>> [  202.099181]  ffff8800cff0b1d8^Ac ffffffff816bb796^Ac ffff8800cff0b270^Ac ffff8800cff0bd90^Ac
>> [  202.107896]  ffff8800cff0b260^Ac ffffffff812fbe95^Ac 00007ffc9d1ab480^Ac 0000000000000000^Ac
>> [  202.116638]  ffffffff8125117d^Ac 0000000000000092^Ac 0000000000000000^Ac ffff8800cff0b7c0^Ac
>> [  202.125339] Call Trace:
>> [  202.127994]  <NMI>  [<ffffffff816bb796>] dump_stack+0x63/0x8d
>> [  202.134184]  [<ffffffff812fbe95>] kasan_report_error+0x495/0x4c0
>> [  202.140680]  [<ffffffff8125117d>] ? perf_output_begin+0x28d/0x4c0
>> [  202.147228]  [<ffffffff812fc319>] kasan_report+0x39/0x40
>> [  202.152987]  [<ffffffff81095ce5>] ? unwind_get_return_address+0x35/0x80
>> [  202.160094]  [<ffffffff812fa8fe>] __asan_load8+0x5e/0x70
>> [  202.165859]  [<ffffffff81095ce5>] unwind_get_return_address+0x35/0x80
>
> Josh, any ideas?

I think this is a false positive due to imprecise unwind that hits a
stack redzone.
We probably need to use READ_ONCE_NOCHECK as in get_wchan.

[toc] | [prev] | [next] | [standalone]

#1523063

On Tue, Nov 15, 2016 at 11:04:45AM -0800, Dmitry Vyukov wrote:
> On Tue, Nov 15, 2016 at 10:57 AM, Peter Zijlstra <peterz@infradead.org> wrote:
> > On Tue, Nov 15, 2016 at 12:43:56PM -0500, Vince Weaver wrote:
> >>
> >> Running on my haswell machine with the imc/uncore patch applied, the
> >> perf_fuzzer next tripped over this issue.
> >>
> >> [  202.034495] BAD LUCK: lost 371 message(s) from NMI context!
> >> [  202.034496] ==================================================================
> >> [  202.048327] BUG: KASAN: stack-out-of-bounds in unwind_get_return_address+0x35/0x80 at addr ffff8800cff0bd90
> >> [  202.058826] Read of size 8 by task perf_fuzzer/16254
> >> [  202.064186] page:ffffea00033fc2c0 count:1 mapcount:0 mapping:          (null) index:0x0^Ac
> >> [  202.073068] flags: 0x1ffff8000000400(reserved)
> >> [  202.077885] page dumped because: kasan: bad access detected
> >> [  202.083880] CPU: 4 PID: 16254 Comm: perf_fuzzer Not tainted 4.9.0-rc5+ #5
> >> [  202.091204] Hardware name: LENOVO 10AM000AUS/SHARKBAY, BIOS FBKT72AUS 01/26/2014
> >> [  202.099181]  ffff8800cff0b1d8^Ac ffffffff816bb796^Ac ffff8800cff0b270^Ac ffff8800cff0bd90^Ac
> >> [  202.107896]  ffff8800cff0b260^Ac ffffffff812fbe95^Ac 00007ffc9d1ab480^Ac 0000000000000000^Ac
> >> [  202.116638]  ffffffff8125117d^Ac 0000000000000092^Ac 0000000000000000^Ac ffff8800cff0b7c0^Ac
> >> [  202.125339] Call Trace:
> >> [  202.127994]  <NMI>  [<ffffffff816bb796>] dump_stack+0x63/0x8d
> >> [  202.134184]  [<ffffffff812fbe95>] kasan_report_error+0x495/0x4c0
> >> [  202.140680]  [<ffffffff8125117d>] ? perf_output_begin+0x28d/0x4c0
> >> [  202.147228]  [<ffffffff812fc319>] kasan_report+0x39/0x40
> >> [  202.152987]  [<ffffffff81095ce5>] ? unwind_get_return_address+0x35/0x80
> >> [  202.160094]  [<ffffffff812fa8fe>] __asan_load8+0x5e/0x70
> >> [  202.165859]  [<ffffffff81095ce5>] unwind_get_return_address+0x35/0x80
> >
> > Josh, any ideas?
> 
> I think this is a false positive due to imprecise unwind that hits a
> stack redzone.
> We probably need to use READ_ONCE_NOCHECK as in get_wchan.

I'm not so sure about that.  The unwind should be precise here: it
should only be looking at the frame pointers and return addresses on the
current task's stack.  I can't see any reason why it would be reading
into the KASAN stack redzone.

-- 
Josh

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | linux.kernel

csiph-web