Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #267340

Re: Fast Random Data Generation (Was: Re: Unidentified subject!)

From Jeffrey Walton <noloader@gmail.com>
Newsgroups linux.debian.user
Subject Re: Fast Random Data Generation (Was: Re: Unidentified subject!)
Date 2024-02-12 22:30 +0100
Message-ID <I6M2d-9Jqf-9@gated-at.bofh.it> (permalink)
References (1 earlier) <I6dea-9oi8-3@gated-at.bofh.it> <I6eWC-9plk-7@gated-at.bofh.it> <I6ffX-9pry-7@gated-at.bofh.it> <I6xPA-9AWS-3@gated-at.bofh.it> <I6Hvz-9GDo-3@gated-at.bofh.it>
Organization linux.* mail to news gateway

Show all headers | View raw

On Mon, Feb 12, 2024 at 3:02 PM Linux-Fan <Ma_Sys.ma@web.de> wrote:
>
> David Christensen writes:
>
> > On 2/11/24 02:26, Linux-Fan wrote:
> >> I wrote a program to automatically generate random bytes in multiple threads:
> >> https://masysma.net/32/big4.xhtml
> >>
> >> Before knowing about `fio` this way my way to benchmark SSDs :)
> >>
> >> Example:
> >>
> >> | $ big4 -b /dev/null 100 GiB
> >> | Ma_Sys.ma Big 4.0.2, Copyright (c) 2014, 2019, 2020 Ma_Sys.ma.
> >> | For further info send an e-mail to Ma_Sys.ma@web.de.
>
> [...]
>
> >> | 99.97% +8426 MiB 7813 MiB/s 102368/102400 MiB
> >> | Wrote 102400 MiB in 13 s @ 7812.023 MiB/s
> >
> >
> > What algorithm did you implement?
>
> I copied the algorithm from here:
> https://www.javamex.com/tutorials/random_numbers/numerical_recipes.shtml
>
> I found it during the development of another application where I needed a
> lot of random data for simulation purposes :)

A PRNG for a simulation has different requirements than a PRNG for
cryptographic purposes. A simulation usually needs numbers fast from a
uniform distribution. Simulations can use predictable numbers. Often a
Linear Congurential Generator (LCG) will do just fine even though they
were broken about 35 years ago. See Also see Joan Boyer's Inferring
Sequences Produced by Pseudo-Random Number Generators,
<https://asterix.cs.gsu.edu/crypto/p129-boyar.pdf>.

A cryptographic application will have more stringent requirements. A
cryptographic generator may (will?) take longer to generate a number,
the numbers need to come from a uniform distribution, and the numbers
need to be prediction resistant. You can read about the cryptographic
qualities of random numbers in NIST SP800-90 and friends.

> My implementation code is here:
> https://github.com/m7a/bo-big/blob/master/latest/Big4.java
>
> If I were to do it again today, I'd probably switch to any of these PRNGS:
>
>  * https://burtleburtle.net/bob/rand/smallprng.html
>  * https://www.pcg-random.org/
>
> >> Secure Random can be obtained from OpenSSL:
> >>
> >> | $ time for i in `seq 1 100`; do openssl rand -out /dev/null $((1024 * 1024
> >> * 1024)); done
> >> |
> >> | real    0m49.288s
> >> | user    0m44.710s
> >> | sys    0m4.579s
> >>
> >> Effectively 2078 MiB/s (quite OK for single-threaded operation). It is not
> >> designed to generate large amounts of random data as the size is limited by
> >> integer range...

Jeff

Back to linux.debian.user | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Unidentified subject! gene heskett <gheskett@shentel.net> - 2024-02-07 21:40 +0100
  Re: Unidentified subject! Stefan Monnier <monnier@iro.umontreal.ca> - 2024-02-08 05:30 +0100
    Re: Unidentified subject! gene heskett <gheskett@shentel.net> - 2024-02-10 11:00 +0100
      Re: Unidentified subject! "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-10 11:40 +0100
        shred bug? [was: Unidentified subject!] <tomas@tuxteam.de> - 2024-02-10 13:50 +0100
          Re: shred bug? [was: Unidentified subject!] tomas@tuxteam.de - 2024-02-10 14:00 +0100
            Re: shred bug? "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-10 15:00 +0100
              Re: shred bug? <tomas@tuxteam.de> - 2024-02-10 15:40 +0100
          Re: shred bug? Gremlin <scott-andrews@columbus.rr.com> - 2024-02-10 14:40 +0100
          Re: shred bug? "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-10 14:40 +0100
          Re: shred bug? [was: Unidentified subject!] David Christensen <dpchrist@holgerdanske.com> - 2024-02-11 01:10 +0100
            Re: shred bug? [was: Unidentified subject!] Greg Wooledge <greg@wooledge.org> - 2024-02-11 01:20 +0100
              Re: shred bug? [was: Unidentified subject!] David Christensen <dpchrist@holgerdanske.com> - 2024-02-11 01:30 +0100
                Re: shred bug? [was: Unidentified subject!] debian-user@howorth.org.uk - 2024-02-11 14:00 +0100
                Re: shred bug? "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-11 14:30 +0100
              Re: shred bug? [was: Unidentified subject!] <tomas@tuxteam.de> - 2024-02-11 08:10 +0100
                Re: shred bug? [was: Unidentified subject!] Greg Wooledge <greg@wooledge.org> - 2024-02-11 15:40 +0100
                Re: shred bug? [was: Unidentified subject!] <tomas@tuxteam.de> - 2024-02-11 15:50 +0100
                Re: shred bug? [was: Unidentified subject!] Greg Wooledge <greg@wooledge.org> - 2024-02-11 16:00 +0100
                Re: shred bug? [was: Unidentified subject!] <tomas@tuxteam.de> - 2024-02-11 16:00 +0100
                Re: shred bug? [was: Unidentified subject!] Greg Wooledge <greg@wooledge.org> - 2024-02-12 18:00 +0100
                Re: shred bug? [was: Unidentified subject!] Curt <curty@free.fr> - 2024-02-12 18:00 +0100
                Re: shred bug? [was: Unidentified subject!] David Christensen <dpchrist@holgerdanske.com> - 2024-02-12 22:00 +0100
                Re: shred bug? [was: Unidentified subject!] "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-12 22:10 +0100
                Re: shred bug? [was: Unidentified subject!] <tomas@tuxteam.de> - 2024-02-13 06:50 +0100
                Re: shred bug? [was: Unidentified subject!] David Wright <deblis@lionunicorn.co.uk> - 2024-02-11 16:30 +0100
                Re: shred bug? [was: Unidentified subject!] Greg Wooledge <greg@wooledge.org> - 2024-02-13 13:20 +0100
                Re: shred bug? [was: Unidentified subject!] Greg Wooledge <greg@wooledge.org> - 2024-02-13 13:40 +0100
                Re: shred bug? [was: Unidentified subject!] <tomas@tuxteam.de> - 2024-02-13 13:50 +0100
                Re: shred bug? [was: Unidentified subject!] "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-13 14:00 +0100
                Re: shred bug? [was: Unidentified subject!] David Wright <deblis@lionunicorn.co.uk> - 2024-02-13 16:40 +0100
                Re: shred bug? [was: Unidentified subject!] Greg Wooledge <greg@wooledge.org> - 2024-02-13 17:30 +0100
                Re: shred bug? [was: Unidentified subject!] debian-user@howorth.org.uk - 2024-02-13 18:50 +0100
                Re: shred bug? [was: Unidentified subject!] David Christensen <dpchrist@holgerdanske.com> - 2024-02-13 22:10 +0100
                Re: shred bug? [was: Unidentified subject!] <tomas@tuxteam.de> - 2024-02-14 06:50 +0100
                Re: shred bug? [was: Unidentified subject!] "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-13 19:00 +0100
                Re: shred bug? [was: Unidentified subject!] Greg Wooledge <greg@wooledge.org> - 2024-02-13 20:00 +0100
                Re: shred bug? [was: Unidentified subject!] "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-13 20:40 +0100
                Re: shred bug? [was: Unidentified subject!] gene heskett <gheskett@shentel.net> - 2024-02-13 20:40 +0100
                Re: shred bug? [was: Unidentified subject!] "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-13 20:50 +0100
                Re: shred bug? [was: Unidentified subject!] gene heskett <gheskett@shentel.net> - 2024-02-13 21:30 +0100
                Re: shred bug? [was: Unidentified subject!] David Christensen <dpchrist@holgerdanske.com> - 2024-02-13 22:10 +0100
                Re: shred bug? [was: Unidentified subject!] gene heskett <gheskett@shentel.net> - 2024-02-13 22:50 +0100
                Re: shred bug? [was: Unidentified subject!] David Wright <deblis@lionunicorn.co.uk> - 2024-02-15 06:40 +0100
                Re: shred bug? [was: Unidentified subject!] David Christensen <dpchrist@holgerdanske.com> - 2024-02-11 23:50 +0100
                Re: shred bug? [was: Unidentified subject!] Max Nikulin <manikulin@gmail.com> - 2024-02-13 03:40 +0100
                Re: shred bug? [was: Unidentified subject!] "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-11 16:30 +0100
                Re: shred bug? [was: Unidentified subject!] Michael Stone <mstone@debian.org> - 2024-02-16 16:00 +0100
        Re: Unidentified subject! gene heskett <gheskett@shentel.net> - 2024-02-10 18:50 +0100
          Re: Unidentified subject! "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-10 19:40 +0100
            Re: Unidentified subject! gene heskett <gheskett@shentel.net> - 2024-02-10 20:30 +0100
            Re: Unidentified subject! David Christensen <dpchrist@holgerdanske.com> - 2024-02-11 00:50 +0100
              Re: Unidentified subject! "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-11 09:10 +0100
                Re: Unidentified subject! David Christensen <dpchrist@holgerdanske.com> - 2024-02-11 21:50 +0100
        Re: Unidentified subject! David Christensen <dpchrist@holgerdanske.com> - 2024-02-11 00:40 +0100
          Re: Unidentified subject! "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-11 09:20 +0100
            Re: Unidentified subject! David Christensen <dpchrist@holgerdanske.com> - 2024-02-11 11:10 +0100
              Fast Random Data Generation (Was: Re: Unidentified subject!) Linux-Fan <Ma_Sys.ma@web.de> - 2024-02-11 11:30 +0100
                Re: Fast Random Data Generation "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-11 12:30 +0100
                Re: Fast Random Data Generation (Was: Re: Unidentified subject!) Gremlin <scott-andrews@columbus.rr.com> - 2024-02-11 13:20 +0100
                Re: Fast Random Data Generation (Was: Re: Unidentified subject!) David Christensen <dpchrist@holgerdanske.com> - 2024-02-12 07:20 +0100
                Re: Fast Random Data Generation (Was: Re: Unidentified subject!) Linux-Fan <Ma_Sys.ma@web.de> - 2024-02-12 17:40 +0100
                Re: Fast Random Data Generation (Was: Re: Unidentified subject!) David Christensen <dpchrist@holgerdanske.com> - 2024-02-12 22:30 +0100
                Re: Fast Random Data Generation (Was: Re: Unidentified subject!) Linux-Fan <Ma_Sys.ma@web.de> - 2024-02-13 18:40 +0100
                Re: Fast Random Data Generation (Was: Re: Unidentified subject!) Jeffrey Walton <noloader@gmail.com> - 2024-02-12 22:30 +0100
              Re: Unidentified subject! "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-11 12:20 +0100
                Re: Unidentified subject! Jeffrey Walton <noloader@gmail.com> - 2024-02-11 16:20 +0100
                Re: Unidentified subject! David Christensen <dpchrist@holgerdanske.com> - 2024-02-11 23:00 +0100
      Re: Unidentified subject! Stefan Monnier <monnier@iro.umontreal.ca> - 2024-02-10 15:40 +0100
  Things I don't touch with a 3.048m barge pole: USB storage (Was Re:  Unidentified subject!) Andy Smith <andy@strugglers.net> - 2024-02-08 16:40 +0100
    Re: Things I don't touch with a 3.048m barge pole: USB storage (Was  Re: Unidentified subject!) Gremlin <scott-andrews@columbus.rr.com> - 2024-02-08 17:20 +0100
      Re: Things I don't touch with a 3.048m barge pole: USB storage (Was  Re: Unidentified subject!) Andy Smith <andy@strugglers.net> - 2024-02-08 17:30 +0100
      Re: Things I don't touch with a 3.048m barge pole: USB storage  (WasRe: Unidentified subject!) gene heskett <gheskett@shentel.net> - 2024-02-09 09:50 +0100
    Re: Things I don't touch with a 3.048m barge pole: USB storage (Was  Re: Unidentified subject!) Ralph Aichinger <ra@h5.or.at> - 2024-02-08 18:00 +0100
      Re: Things I don't touch with a 3.048m barge pole: USB storage (Was  Re: Unidentified subject!) Jeffrey Walton <noloader@gmail.com> - 2024-02-08 20:30 +0100
        Re: Things I don't touch with a 3.048m barge pole: USB storage (Was  Re: Unidentified subject!) Andy Smith <andy@strugglers.net> - 2024-02-08 21:50 +0100
        Re: Things I don't touch with a 3.048m barge pole: USB storage (Was  Re: Unidentified subject!) Andy Smith <andy@strugglers.net> - 2024-02-08 21:50 +0100
          Re: Things I don't touch with a 3.048m barge pole: USB storage (Was  Re: Unidentified subject!) Gremlin <scott-andrews@columbus.rr.com> - 2024-02-08 22:00 +0100
            Re: Things I don't touch with a 3.048m barge pole: USB storage (Was  Re: Unidentified subject!) Andy Smith <andy@strugglers.net> - 2024-02-08 22:20 +0100
              Re: Things I don't touch with a 3.048m barge pole: USB storage (Was  Re: Unidentified subject!) Andy Smith <andy@strugglers.net> - 2024-02-08 22:30 +0100
                Re: Things I don't touch with a 3.048m barge pole: USB storage (Was  Re: Unidentified subject!) Gremlin <scott-andrews@columbus.rr.com> - 2024-02-08 23:10 +0100
              Re: Things I don't touch with a 3.048m barge pole: USB storage (Was  Re: Unidentified subject!) Gremlin <scott-andrews@columbus.rr.com> - 2024-02-08 22:30 +0100
          Re: Things I don't touch with a 3.048m barge pole: USB storage  (WasRe: Unidentified subject!) gene heskett <gheskett@shentel.net> - 2024-02-09 14:00 +0100
            Re: Things I don't touch with a 3.048m barge pole "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-09 14:50 +0100
            Re: Things I don't touch with a 3.048m barge pole: USB storage  (WasRe: Unidentified subject!) David Christensen <dpchrist@holgerdanske.com> - 2024-02-10 07:00 +0100
              Re: Things I don't touch with a 3.048m barge pole: USB storage(WasRe:  Unidentified subject!) gene heskett <gheskett@shentel.net> - 2024-02-10 18:40 +0100
      Re: Things I don't touch with a 3.048m barge pole: USB storage (Was  Re: Unidentified subject!) Andy Smith <andy@strugglers.net> - 2024-02-08 21:40 +0100
        Re: Things I don't touch with a 3.048m barge pole: USB storage (Was  Re: Unidentified subject!) Arno Lehmann <al@its-lehmann.de> - 2024-02-09 09:50 +0100
    Re: Things I don't touch with a 3.048m barge pole: USB storage (Was  Re: Unidentified subject!) Max Nikulin <manikulin@gmail.com> - 2024-02-08 18:30 +0100
      Re: Things I don't touch with a 3.048m barge pole: USB storage (Was  Re: Unidentified subject!) Andy Smith <andy@strugglers.net> - 2024-02-08 21:40 +0100
        Re: Things I don't touch with a 3.048m barge pole: USB storage (Was  Re: Unidentified subject!) Gremlin <scott-andrews@columbus.rr.com> - 2024-02-08 22:10 +0100
          Re: Things I don't touch with a 3.048m barge pole: USB storage (Was  Re: Unidentified subject!) Andy Smith <andy@strugglers.net> - 2024-02-08 22:20 +0100
  Re: Unidentified subject! Richmond <dnomhcir@gmx.com> - 2024-02-08 23:50 +0100
    Re: Unidentified subject! Stefan Monnier <monnier@iro.umontreal.ca> - 2024-02-09 00:10 +0100
      Re: Unidentified subject! Charles Curley <charlescurley@charlescurley.com> - 2024-02-09 00:50 +0100
        Re: Unidentified subject! Richmond <dnomhcir@gmx.com> - 2024-02-09 05:50 +0100
          Re: Unidentified flying subject! Charles Curley <charlescurley@charlescurley.com> - 2024-02-09 07:50 +0100
            Re: Unidentified flying subject! Richmond <dnomhcir@gmx.com> - 2024-02-09 14:20 +0100

csiph-web