Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1671846
| Path | csiph.com!goblin1!goblin2!goblin.stu.neva.ru!news.unit0.net!news.panservice.it!bofh.it!news.nic.it!robomod |
|---|---|
| From | Kees Cook <keescook@chromium.org> |
| Newsgroups | linux.kernel |
| Subject | Re: [PATCH] x86/uaccess: use unrolled string copy for short strings |
| Date | Wed, 21 Jun 2017 19:40:03 +0200 |
| Message-ID | <tURLJ-Mh-37@gated-at.bofh.it> (permalink) |
| References | <tULPY-5nY-1@gated-at.bofh.it> |
| X-Original-To | Paolo Abeni <pabeni@redhat.com> |
| Dkim-Signature | v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=9paaZ3HV9gR4CRYgkM0a3FOEkWLlsljZOJkTsFzYHXg=; b=BnJyvF5XGDVx0bvKtMc9B2H9JISnr+Y6l89mpyllf29y2zaYAqXHumULznCaSo3Kw9 FqeUqLrnHS0vxv3mKgwQQDBqHP4UK8VYYBxXZXhc+oYLM12UlilUCxjgXdZwYPz2yWI3 KEjdHz8vdMU7biSJlGy5fKgBrxwTMvrz5Qsd1A6/+W6PKTVk+6GU1zwFMHZFgaGWv96l UPqKtKLf9FmmKCm3IngJrYVRSwr1QoXvCG25YIHcGy/QC86fVClcZ0QZkVD6gvC43Zg3 +1v6xYzW2T+1b8TO24xMSDTV8iUbZliDUpa42nzmgF+Vh7PcR2Bv5EVqIpHQu1e2k6Ae 1O8Q== |
| Dkim-Signature | v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=9paaZ3HV9gR4CRYgkM0a3FOEkWLlsljZOJkTsFzYHXg=; b=NUVbdHDgHYBJXtVpYQs19K/RtY+HdUR8iRR68NKQ3cKyvqZ/JA3um0JmFvRT2cb1pW UVN5r/TRrKySziHhR3rAdJ8XYrhFY343gLZrKE9EMJaGklNe74Vg5hVR0K0H7GtvjNBM KnDtBcVir8fvwpCVKGJ8IyZ41XYcl9tQ8lP18= |
| X-Google-Dkim-Signature | v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=9paaZ3HV9gR4CRYgkM0a3FOEkWLlsljZOJkTsFzYHXg=; b=O4cUERkzWmdgsxT2OC/3R5pLoiUYzweOZZvhudlqcyC9dRoOhFYMWuZHhDHIU1CuEG wr/+cN/JwL2Mb2vP8xTv8vHmGFcXvlfXlVXqJHuQPEVQVE4vsZvEPA0E/DZTnZepXalP kZzVTWPkwM22vCvPi70HoxnUOHWS/Y8F1gjf8aijNbFCBnWVFOf43K4X/k7qN1vq6Cml 1nwWmNZDIZ706FAPQxl7VwZUneRrD1GpApL0CiGDp7bEgPResMAG5sp+DilLaBLky/8u w57ab3CwegTpBucLg+18Deex7vpt4uP7nrFfyflT6sR4Hmad72OXfcqe0dFH/rG56Hbj 2hrQ== |
| X-Gm-Message-State | AKS2vOz3vVOe2i+++okMhY0msCFFqy//eExIyBQDhBH3q0T3dDv/cYIp SPq3NUalddH1VhHNAKQ7zBIbEzN3eWn8 |
| X-Received | by 10.36.70.66 with SMTP id j63mr10239576itb.6.1498066727442; Wed, 21 Jun 2017 10:38:47 -0700 (PDT) |
| MIME-Version | 1.0 |
| X-Google-Sender-Auth | BiyMJ1_eJPvSpT9y3ate9BkcJsI |
| Content-Type | text/plain; charset="UTF-8" |
| Sender | robomod@news.nic.it |
| List-ID | <linux-kernel.vger.kernel.org> |
| X-Mailing-List | linux-kernel@vger.kernel.org |
| Approved | robomod@news.nic.it |
| Lines | 123 |
| Organization | linux.* mail to news gateway |
| X-Original-Cc | "x86@kernel.org" <x86@kernel.org>, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, "H. Peter Anvin" <hpa@zytor.com>, Al Viro <viro@zeniv.linux.org.uk>, Hannes Frederic Sowa <hannes@stressinduktion.org>, LKML <linux-kernel@vger.kernel.org> |
| X-Original-Date | Wed, 21 Jun 2017 10:38:46 -0700 |
| X-Original-Message-ID | <CAGXu5j+1yjqgbfYT-JVQC=MstJxUNzqhr3E4WGKS+82rSHaaVA@mail.gmail.com> |
| X-Original-References | <63d913f28bc64bd4ea66a39a532f0b59ee015382.1498039056.git.pabeni@redhat.com> |
| X-Original-Sender | linux-kernel-owner@vger.kernel.org |
| Xref | csiph.com linux.kernel:1671846 |
Show key headers only | View raw
On Wed, Jun 21, 2017 at 4:09 AM, Paolo Abeni <pabeni@redhat.com> wrote:
> The 'rep' prefix suffers for a relevant "setup cost"; as a result
> string copies with unrolled loops are faster than even
> optimized string copy using 'rep' variant, for short string.
>
> This change updates __copy_user_generic() to use the unrolled
> version for small string length. The threshold length for short
> string - 64 - has been selected with empirical measures as the
> larger value that still ensure a measurable gain.
>
> A micro-benchmark of __copy_from_user() with different lengths shows
> the following:
>
> string len vanilla patched delta
> bytes ticks ticks tick(%)
>
> 0 58 26 32(55%)
> 1 49 29 20(40%)
> 2 49 31 18(36%)
> 3 49 32 17(34%)
> 4 50 34 16(32%)
> 5 49 35 14(28%)
> 6 49 36 13(26%)
> 7 49 38 11(22%)
> 8 50 31 19(38%)
> 9 51 33 18(35%)
> 10 52 36 16(30%)
> 11 52 37 15(28%)
> 12 52 38 14(26%)
> 13 52 40 12(23%)
> 14 52 41 11(21%)
> 15 52 42 10(19%)
> 16 51 34 17(33%)
> 17 51 35 16(31%)
> 18 52 37 15(28%)
> 19 51 38 13(25%)
> 20 52 39 13(25%)
> 21 52 40 12(23%)
> 22 51 42 9(17%)
> 23 51 46 5(9%)
> 24 52 35 17(32%)
> 25 52 37 15(28%)
> 26 52 38 14(26%)
> 27 52 39 13(25%)
> 28 52 40 12(23%)
> 29 53 42 11(20%)
> 30 52 43 9(17%)
> 31 52 44 8(15%)
> 32 51 36 15(29%)
> 33 51 38 13(25%)
> 34 51 39 12(23%)
> 35 51 41 10(19%)
> 36 52 41 11(21%)
> 37 52 43 9(17%)
> 38 51 44 7(13%)
> 39 52 46 6(11%)
> 40 51 37 14(27%)
> 41 50 38 12(24%)
> 42 50 39 11(22%)
> 43 50 40 10(20%)
> 44 50 42 8(16%)
> 45 50 43 7(14%)
> 46 50 43 7(14%)
> 47 50 45 5(10%)
> 48 50 37 13(26%)
> 49 49 38 11(22%)
> 50 50 40 10(20%)
> 51 50 42 8(16%)
> 52 50 42 8(16%)
> 53 49 46 3(6%)
> 54 50 46 4(8%)
> 55 49 48 1(2%)
> 56 50 39 11(22%)
> 57 50 40 10(20%)
> 58 49 42 7(14%)
> 59 50 42 8(16%)
> 60 50 46 4(8%)
> 61 50 47 3(6%)
> 62 50 48 2(4%)
> 63 50 48 2(4%)
> 64 51 38 13(25%)
>
> Above 64 bytes the gain fades away.
>
> Very similar values are collectd for __copy_to_user().
> UDP receive performances under flood with small packets using recvfrom()
> increase by ~5%.
>
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Since there are no regressions here, this seems sensible to me. :)
Reviewed-by: Kees Cook <keescook@chromium.org>
-Kees
> ---
> arch/x86/include/asm/uaccess_64.h | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/arch/x86/include/asm/uaccess_64.h b/arch/x86/include/asm/uaccess_64.h
> index c5504b9..16a8871 100644
> --- a/arch/x86/include/asm/uaccess_64.h
> +++ b/arch/x86/include/asm/uaccess_64.h
> @@ -28,6 +28,9 @@ copy_user_generic(void *to, const void *from, unsigned len)
> {
> unsigned ret;
>
> + if (len <= 64)
> + return copy_user_generic_unrolled(to, from, len);
> +
> /*
> * If CPU has ERMS feature, use copy_user_enhanced_fast_string.
> * Otherwise, if CPU has rep_good feature, use copy_user_generic_string.
> --
> 2.9.4
>
--
Kees Cook
Pixel Security
Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
[PATCH] x86/uaccess: use unrolled string copy for short strings Paolo Abeni <pabeni@redhat.com> - 2017-06-21 13:20 +0200
Re: [PATCH] x86/uaccess: use unrolled string copy for short strings Kees Cook <keescook@chromium.org> - 2017-06-21 19:40 +0200
Re: [PATCH] x86/uaccess: use unrolled string copy for short strings Alan Cox <gnomes@lxorguk.ukuu.org.uk> - 2017-06-22 17:00 +0200
Re: [PATCH] x86/uaccess: use unrolled string copy for short strings Ingo Molnar <mingo@kernel.org> - 2017-06-22 10:50 +0200
Re: [PATCH] x86/uaccess: use unrolled string copy for short strings Paolo Abeni <pabeni@redhat.com> - 2017-06-22 19:10 +0200
Re: [PATCH] x86/uaccess: use unrolled string copy for short strings Linus Torvalds <torvalds@linux-foundation.org> - 2017-06-22 19:40 +0200
Re: [PATCH] x86/uaccess: use unrolled string copy for short strings Paolo Abeni <pabeni@redhat.com> - 2017-06-22 20:00 +0200
[PATCH] x86/uaccess: optimize copy_user_enhanced_fast_string for short string Paolo Abeni <pabeni@redhat.com> - 2017-06-29 16:00 +0200
Re: [PATCH] x86/uaccess: optimize copy_user_enhanced_fast_string for short string Linus Torvalds <torvalds@linux-foundation.org> - 2017-06-29 23:50 +0200
[tip:x86/asm] x86/uaccess: Optimize copy_user_enhanced_fast_string() for short strings tip-bot for Paolo Abeni <tipbot@zytor.com> - 2017-06-30 15:20 +0200
csiph-web