Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder1.news.weretis.net!feeder.erje.net!eu.feeder.erje.net!xlned.com!feeder1.xlned.com!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
In-Reply-To: <CAHVvXxQmBunWVRL-k8eNv-dM3OzZ2dcg84EbkcdLdfyLgNenxw@mail.gmail.com>
References: <24dc619b-7abd-4be3-aa92-f858eb4ab85f@n4g2000yqj.googlegroups.com> <51666aae$0$29977$c3e8da3$5496439d@news.astraweb.com> <mailman.453.1365673692.3114.python-list@python.org> <51669576$0$29977$c3e8da3$5496439d@news.astraweb.com> <CAHVvXxQmBunWVRL-k8eNv-dM3OzZ2dcg84EbkcdLdfyLgNenxw@mail.gmail.com>
Date: Thu, 11 Apr 2013 23:56:27 +1000
Subject: Re: performance of script to write very long lines of random chars
From: Chris Angelico <rosuav@gmail.com>
To: python-list@python.org
Content-Type: text/plain; charset=ISO-8859-1
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.474.1365688589.3114.python-list@python.org>
Lines: 50
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:43370

On Thu, Apr 11, 2013 at 10:05 PM, Oscar Benjamin
<oscar.j.benjamin@gmail.com> wrote:
> On 11 April 2013 11:50, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
>> Some (most?) modern operating systems provide a cryptographically strong
>> source of non-deterministic randomness. The non-deterministic part comes
>> from external "stuff", which is called "entropy". Typical sources of
>> entropy include network events, user key-presses, moving the mouse, and
>> (presumably in machines with special hardware), even thermal noise in
>> electrical components.
>
>> Entropy is used and discarded, so urandom needs the OS to continually
>> replenish the amount of entropy. Under normal circumstances, this it
>> does, but if you grab lots of urandom output on a system which is
>> otherwise quiet and not doing anything, it could run out.
>
> Okay, so I understand what entropy is in the thermodynamic sense and
> also in the mathematical (Shannon) sense but I'm still confused about
> what it means that the OS is somehow storing entropy. Do you mean that
> it is always maintaining a buffer of what it considers to be random
> bytes that it slowly builds up from noise that is made accessible to
> the OS from the hardware?

Correct. And Steven's right about most of what he says (modulo the
urandom vs random distinction, as Robert Kern pointed out - urandom
won't block, but it's not guaranteed to be cryptographically secure);
I'll just add that one of the best sources of entropy is a solid
cylinder, rotated at high velocity in a sealed container filled with a
fluid, and entropy is found in the eddies. Many computers have a
device of this nature - the solid cylinder is thin and flat and
referred to as a "disk", the fluid it's in is air, and the sealed
container is your hard disk drive.

The details will vary, but broadly speaking, the /dev/random driver
(or its equivalent) maintains an ever-increasing buffer of entropic
bits, accumulated as they arrive from the various sources, and often
saved to disk on shutdown to permit faster boot (which helps to avoid
the problem Steven described of 70-minute boot times - on an all-SSD
computer with no human being attached, entropy really can be very hard
to obtain); whenever a program asks for bytes from it, it delivers
them and removes that much "recorded entropy" from its buffer. For
many purposes, it's sufficient to take 4 or 8 bytes of /dev/random
entropy and use that to seed a PRNG, but if you're using /dev/urandom
and it's not a critical server, I wouldn't worry too much about
drawing too much off it. (On a web server that's constantly serving
HTTPS requests, for instance, I'd be cautious about reading too much
from /dev/urandom as it might cause the web server to block waiting
for /dev/random. Might kill your TPS for a while.)

ChrisA