Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1660686 > unrolled thread

Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to reduce memory consumption

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Joonsoo Kim <js1304@gmail.com> - 2017-06-08 04:50 +0200
    Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Joonsoo Kim <js1304@gmail.com> - 2017-06-14 02:20 +0200

#1660686 — Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to reduce memory consumption

On Thu, Jun 01, 2017 at 08:06:02PM +0200, Dmitry Vyukov wrote:
> On Tue, May 30, 2017 at 4:16 PM, Andrey Ryabinin
> <aryabinin@virtuozzo.com> wrote:
> > On 05/29/2017 06:29 PM, Dmitry Vyukov wrote:
> >> Joonsoo,
> >>
> >> I guess mine (and Andrey's) main concern is the amount of additional
> >> complexity (I am still struggling to understand how it all works) and
> >> more arch-dependent code in exchange for moderate memory win.
> >>
> >> Joonsoo, Andrey,
> >>
> >> I have an alternative proposal. It should be conceptually simpler and
> >> also less arch-dependent. But I don't know if I miss something
> >> important that will render it non working.
> >> Namely, we add a pointer to shadow to the page struct. Then, create a
> >> slab allocator for 512B shadow blocks. Then, attach/detach these
> >> shadow blocks to page structs as necessary. It should lead to even
> >> smaller memory consumption because we won't need a whole shadow page
> >> when only 1 out of 8 corresponding kernel pages are used (we will need
> >> just a single 512B block). I guess with some fragmentation we need
> >> lots of excessive shadow with the current proposed patch.
> >> This does not depend on TLB in any way and does not require hooking
> >> into buddy allocator.
> >> The main downside is that we will need to be careful to not assume
> >> that shadow is continuous. In particular this means that this mode
> >> will work only with outline instrumentation and will need some ifdefs.
> >> Also it will be slower due to the additional indirection when
> >> accessing shadow, but that's meant as "small but slow" mode as far as
> >> I understand.
> >
> > It seems that you are forgetting about stack instrumentation.
> > You'll have to disable it completely, at least with current implementation of it in gcc.
> >
> >> But the main win as I see it is that that's basically complete support
> >> for 32-bit arches. People do ask about arm32 support:
> >> https://groups.google.com/d/msg/kasan-dev/Sk6BsSPMRRc/Gqh4oD_wAAAJ
> >> https://groups.google.com/d/msg/kasan-dev/B22vOFp-QWg/EVJPbrsgAgAJ
> >> and probably mips32 is relevant as well.
> >
> > I don't see how above is relevant for 32-bit arches. Current design
> > is perfectly fine for 32-bit arches. I did some POC arm32 port couple years
> > ago - https://github.com/aryabinin/linux/commits/kasan/arm_v0_1
> > It has some ugly hacks and non-critical bugs. AFAIR it also super-slow because I (mistakenly)
> > made shadow memory uncached. But otherwise it works.
> >
> >> Such mode does not require a huge continuous address space range, has
> >> minimal memory consumption and requires minimal arch-dependent code.
> >> Works only with outline instrumentation, but I think that's a
> >> reasonable compromise.
> >>
> >> What do you think?
> >
> > I don't understand why we trying to invent some hacky/complex schemes when we already have
> > a simple one - scaling shadow to 1/32. It's easy to implement and should be more performant comparing
> > to suggested schemes.
> 
> 
> If 32-bits work with the current approach, then I would also prefer to
> keep things simpler.
> FWIW clang supports settings shadow scale via a command line flag
> (-asan-mapping-scale).

Hello,

To confirm the final consensus, I did a quick comparison of scaling
approach and mine. Note that scaling approach can be co-exist with
mine. And, there is an assumption that we can disable quarantine and
other optional feature of KASAN.

Scaling vs Mine

Memory usage: 1/32 of total memory. vs can be far less than 1/32.
Slab object layout: should be changed. vs none.
Usability: hard. vs simple. (Updating compiler is not required)
Implementation complexity: simple. vs complex.
Porting to other ARCH: simple. vs hard (But, not mandatory)

So, do both you disagree to merge my per-page shadow? If so, I will
not submit v2. Please let me know your decision.

Thanks.

[toc] | [next] | [standalone]

#1665314

On Tue, Jun 13, 2017 at 07:49:47PM +0300, Andrey Ryabinin wrote:
> On 06/08/2017 05:40 AM, Joonsoo Kim wrote:
> >>>
> >>> I don't understand why we trying to invent some hacky/complex schemes when we already have
> >>> a simple one - scaling shadow to 1/32. It's easy to implement and should be more performant comparing
> >>> to suggested schemes.
> >>
> >>
> >> If 32-bits work with the current approach, then I would also prefer to
> >> keep things simpler.
> >> FWIW clang supports settings shadow scale via a command line flag
> >> (-asan-mapping-scale).
> > 
> > Hello,
> > 
> > To confirm the final consensus, I did a quick comparison of scaling
> > approach and mine. Note that scaling approach can be co-exist with
> > mine. And, there is an assumption that we can disable quarantine and
> > other optional feature of KASAN.
> > 
> > Scaling vs Mine
> > 
> > Memory usage: 1/32 of total memory. vs can be far less than 1/32.
> > Slab object layout: should be changed. vs none.
> > Usability: hard. vs simple. (Updating compiler is not required)
> > Implementation complexity: simple. vs complex.
> > Porting to other ARCH: simple. vs hard (But, not mandatory)
> 
> 
> My main concern is a huge amount of complex and fragile code that comes with this patchset.
> Basically you are building a completely new algorithm on the fundamentals that were designed
> for the current algorithm. Hence you have to do these hacks with black shadow, tlb flushing, etc.
> 
> Yes, it does consume less memory, but I'm not convinced that such aggressive memory saving
> are mandatory. I guess that for the most of the users (if not all) that currently unsatisfied with 1/8 shadow
> 1/32 will be good enough.
> FWIW I did run sanitized kernel (1/8 shadow) on the smart TVs with 1Gb of ram.
> 
> > So, do both you disagree to merge my per-page shadow? If so, I will
> > not submit v2. Please let me know your decision.
> > 
> 
> Sorry, but it's a nack from me.

I understand your decision. Thanks for looking this.

Thanks.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web