Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to reduce memory consumption

From	Joonsoo Kim <js1304@gmail.com>
Newsgroups	linux.kernel
Subject	Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to reduce memory consumption
Date	2017-05-25 02:50 +0200
Message-ID	<tKP8t-11Y-3@gated-at.bofh.it> (permalink)
References	(1 earlier) <tIFmW-3k7-17@gated-at.bofh.it> <tIFmW-3k7-15@gated-at.bofh.it> <tJOHw-1Mg-15@gated-at.bofh.it> <tKxEB-6Td-3@gated-at.bofh.it> <tKHui-4G7-7@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

On Wed, May 24, 2017 at 06:31:04PM +0200, Dmitry Vyukov wrote:
> On Wed, May 24, 2017 at 8:04 AM, Joonsoo Kim <js1304@gmail.com> wrote:
> >> >> > From: Joonsoo Kim <iamjoonsoo.kim@lge.com>
> >> >> >
> >> >> > Hello, all.
> >> >> >
> >> >> > This is an attempt to recude memory consumption of KASAN. Please see
> >> >> > following description to get the more information.
> >> >> >
> >> >> > 1. What is per-page shadow memory
> >> >> >
> >> >> > This patch introduces infrastructure to support per-page shadow memory.
> >> >> > Per-page shadow memory is the same with original shadow memory except
> >> >> > the granualarity. It's one byte shows the shadow value for the page.
> >> >> > The purpose of introducing this new shadow memory is to save memory
> >> >> > consumption.
> >> >> >
> >> >> > 2. Problem of current approach
> >> >> >
> >> >> > Until now, KASAN needs shadow memory for all the range of the memory
> >> >> > so the amount of statically allocated memory is so large. It causes
> >> >> > the problem that KASAN cannot run on the system with hard memory
> >> >> > constraint. Even if KASAN can run, large memory consumption due to
> >> >> > KASAN changes behaviour of the workload so we cannot validate
> >> >> > the moment that we want to check.
> >> >> >
> >> >> > 3. How does this patch fix the problem
> >> >> >
> >> >> > This patch tries to fix the problem by reducing memory consumption for
> >> >> > the shadow memory. There are two observations.
> >> >> >
> >> >>
> >> >>
> >> >> I think that the best way to deal with your problem is to increase shadow scale size.
> >> >>
> >> >> You'll need to add tunable to gcc to control shadow size. I expect that gcc has some
> >> >> places where 8-shadow scale size is hardcoded, but it should be fixable.
> >> >>
> >> >> The kernel also have some small amount of code written with KASAN_SHADOW_SCALE_SIZE == 8 in mind,
> >> >> which should be easy to fix.
> >> >>
> >> >> Note that bigger shadow scale size requires bigger alignment of allocated memory and variables.
> >> >> However, according to comments in gcc/asan.c gcc already aligns stack and global variables and at
> >> >> 32-bytes boundary.
> >> >> So we could bump shadow scale up to 32 without increasing current stack consumption.
> >> >>
> >> >> On a small machine (1Gb) 1/32 of shadow is just 32Mb which is comparable to yours 30Mb, but I expect it to be
> >> >> much faster. More importantly, this will require only small amount of simple changes in code, which will be
> >> >> a *lot* more easier to maintain.
> >>
> >>
> >> Interesting option. We never considered increasing scale in user space
> >> due to performance implications. But the algorithm always supported up
> >> to 128x scale. Definitely worth considering as an option.
> >
> > Could you explain me how does increasing scale reduce performance? I
> > tried to guess the reason but failed.
> 
> 
> The main reason is inline instrumentation. Inline instrumentation for
> a check of 8-byte access (which are very common in 64-bit code) is
> just a check of the shadow byte for 0. For smaller accesses we have
> more complex instrumentation that first checks shadow for 0 and then
> does precise check based on size/offset of the access + shadow value.
> That's slower and also increases register pressure and code size
> (which can further reduce performance due to icache overflow). If we
> increase scale to 16/32, all accesses will need that slow path.
> Another thing is stack instrumentation: larger scale will require
> larger redzones to ensure proper alignment. That will increase stack
> frames and also more instructions to poison/unpoison stack shadow on
> function entry/exit.

Now, I see. Thanks for explanation.

Thanks.

Thread

[PATCH v1 00/11] mm/kasan: support per-page shadow memory to reduce memory consumption js1304@gmail.com - 2017-05-16 03:20 +0200
  [PATCH v1 09/11] x86/kasan: support on-demand shadow mapping js1304@gmail.com - 2017-05-16 03:20 +0200
  [PATCH v1 11/11] mm/kasan: change the order of shadow memory check js1304@gmail.com - 2017-05-16 03:20 +0200
  [PATCH v1 10/11] mm/kasan: support dynamic shadow memory free js1304@gmail.com - 2017-05-16 03:20 +0200
  [PATCH v1 05/11] mm/kasan: introduce per-page shadow memory infrastructure js1304@gmail.com - 2017-05-16 03:20 +0200
  [PATCH v1 08/11] mm/kasan: support on-demand shadow allocation/mapping js1304@gmail.com - 2017-05-16 03:20 +0200
  [PATCH v1 02/11] mm/kasan: don't fetch the next shadow value speculartively js1304@gmail.com - 2017-05-16 03:20 +0200
  [PATCH v1 07/11] x86/kasan: use per-page shadow memory js1304@gmail.com - 2017-05-16 03:20 +0200
  [PATCH(RE-RESEND) v1 01/11] mm/kasan: rename _is_zero to _is_nonzero Joonsoo Kim <js1304@gmail.com> - 2017-05-16 03:30 +0200
  Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Dmitry Vyukov <dvyukov@google.com> - 2017-05-16 06:40 +0200
    Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Dmitry Vyukov <dvyukov@google.com> - 2017-05-16 06:50 +0200
    Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Joonsoo Kim <js1304@gmail.com> - 2017-05-16 08:30 +0200
      Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Dmitry Vyukov <dvyukov@google.com> - 2017-05-16 22:50 +0200
        Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Joonsoo Kim <js1304@gmail.com> - 2017-05-17 09:30 +0200
        Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Joonsoo Kim <js1304@gmail.com> - 2017-05-17 09:30 +0200
        Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Dmitry Vyukov <dvyukov@google.com> - 2017-05-24 09:00 +0200
          Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Joonsoo Kim <js1304@gmail.com> - 2017-05-24 09:50 +0200
            Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Dmitry Vyukov <dvyukov@google.com> - 2017-05-24 19:30 +0200
              Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Joonsoo Kim <js1304@gmail.com> - 2017-05-25 02:50 +0200
                Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Dmitry Vyukov <dvyukov@google.com> - 2017-05-29 17:10 +0200
                Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Dmitry Vyukov <dvyukov@google.com> - 2017-05-29 17:20 +0200
                Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Dmitry Vyukov <dvyukov@google.com> - 2017-05-29 17:40 +0200
                Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Vladimir Murzin <vladimir.murzin@arm.com> - 2017-05-30 10:00 +0200
                Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Dmitry Vyukov <dvyukov@google.com> - 2017-05-30 10:20 +0200
                Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Vladimir Murzin <vladimir.murzin@arm.com> - 2017-05-30 10:40 +0200
                Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Vladimir Murzin <vladimir.murzin@arm.com> - 2017-05-30 10:50 +0200
                Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Dmitry Vyukov <dvyukov@google.com> - 2017-05-30 10:50 +0200
                Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Vladimir Murzin <vladimir.murzin@arm.com> - 2017-05-30 11:10 +0200
                Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Dmitry Vyukov <dvyukov@google.com> - 2017-05-30 11:30 +0200
                Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Vladimir Murzin <vladimir.murzin@arm.com> - 2017-05-30 11:40 +0200
                Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Dmitry Vyukov <dvyukov@google.com> - 2017-05-30 11:50 +0200
                Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Vladimir Murzin <vladimir.murzin@arm.com> - 2017-05-30 12:00 +0200
                Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Joonsoo Kim <js1304@gmail.com> - 2017-05-31 08:00 +0200
                Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Dmitry Vyukov <dvyukov@google.com> - 2017-06-01 20:10 +0200
  Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Joonsoo Kim <js1304@gmail.com> - 2017-05-19 04:00 +0200
    Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Dmitry Vyukov <dvyukov@google.com> - 2017-05-22 08:10 +0200
      Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Joonsoo Kim <js1304@gmail.com> - 2017-05-24 08:10 +0200
        Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Dmitry Vyukov <dvyukov@google.com> - 2017-05-24 18:40 +0200
          Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Joonsoo Kim <js1304@gmail.com> - 2017-05-25 02:50 +0200
    Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to  reduce memory consumption Joonsoo Kim <js1304@gmail.com> - 2017-05-24 08:20 +0200

csiph-web