Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.debian.user > #267280
| From | "Thomas Schmitt" <scdbackup@gmx.net> |
|---|---|
| Newsgroups | linux.debian.user |
| Subject | Re: Unidentified subject! |
| Date | 2024-02-11 12:20 +0100 |
| Message-ID | <I6g2l-9pWM-1@gated-at.bofh.it> (permalink) |
| References | <I6eWC-9plk-7@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
Hi,
David Christensen wrote:
> Concurrency:
> threads throughput
> 8 205+198+180+195+205+184+184+189=1,540 MB/s
There remains the question how to join these streams without losing speed
in order to produce a single checksum. (Or one would have to divide the
target into 8 areas which get checked separately.)
Does this 8 thread generator cause any problems with the usability of
the rest of the system ? Sluggish program behavior or so ?
The main reason to have my own low-quality implementation of a random
stream was that /dev/urandom was too slow for 12x speed (= 1.8 MB/s) CD-RW
media and that higher random quality still put too much load on a
single-core 600 MHz Pentium system. That was nearly 25 years ago.
I wrote:
> > Last time i tested /dev/urandom it was much slower on comparable machines
> > and also became slower as the amount grew.
> Did you figure out why the Linux random number subsystem slowed, and at what
> amount?
No. I cannot even remember when and why i had reason to compare it with
my own stream generator. Maybe 5 or 10 years ago. The throughput was
more than 100 times better.
I have to correct my previous measurement on the 4 GHz Xeon, which was
made with a debuggable version of the program that produced the stream.
The production binary which is compiled with -O2 can write 2500 MB/s into
a pipe with a pacifier program which counts the data:
$ time $(scdbackup -where bin)/cd_backup_planer -write_random - 100g 2s62gss463ar46492bni | $(scdbackup -where bin)/raedchen -step 100m -no_output -print_count
100.0g bytes
real 0m39.884s
user 0m30.629s
sys 0m41.013s
(One would have to install scdbackup to reproduce this and to see raedchen
count the bytes while spinning the classic SunOS boot wheel: |/-\|/-\|/-\
http://scdbackup.webframe.org/main_eng.html
http://scdbackup.webframe.org/examples.html
Oh nostalgy ...
)
Totally useless but yielding nearly 4000 MB/s:
$ time $(scdbackup -where bin)/cd_backup_planer -write_random - 100g 2s62gss463ar46492bni >/dev/null
real 0m27.064s
user 0m23.433s
sys 0m3.646s
The main bottleneck in my proposal would be the checksummer:
$ time $(scdbackup -where bin)/cd_backup_planer -write_random - 100g 2s62gss463ar46492bni | md5sum
5a6ba41c2c18423fa33355005445c183 -
real 2m8.160s
user 2m25.599s
sys 0m22.663s
That's quite exactly 800 MiB/s ~= 6.7 Gbps.
Still good enough for vanilla USB-3 with a fast SSD, i'd say.
> TIMTOWTDI. :-)
Looks like another example of a weak random stream. :))
Have a nice day :)
Thomas
Back to linux.debian.user | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Unidentified subject! gene heskett <gheskett@shentel.net> - 2024-02-07 21:40 +0100
Re: Unidentified subject! Stefan Monnier <monnier@iro.umontreal.ca> - 2024-02-08 05:30 +0100
Re: Unidentified subject! gene heskett <gheskett@shentel.net> - 2024-02-10 11:00 +0100
Re: Unidentified subject! "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-10 11:40 +0100
shred bug? [was: Unidentified subject!] <tomas@tuxteam.de> - 2024-02-10 13:50 +0100
Re: shred bug? [was: Unidentified subject!] tomas@tuxteam.de - 2024-02-10 14:00 +0100
Re: shred bug? "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-10 15:00 +0100
Re: shred bug? <tomas@tuxteam.de> - 2024-02-10 15:40 +0100
Re: shred bug? Gremlin <scott-andrews@columbus.rr.com> - 2024-02-10 14:40 +0100
Re: shred bug? "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-10 14:40 +0100
Re: shred bug? [was: Unidentified subject!] David Christensen <dpchrist@holgerdanske.com> - 2024-02-11 01:10 +0100
Re: shred bug? [was: Unidentified subject!] Greg Wooledge <greg@wooledge.org> - 2024-02-11 01:20 +0100
Re: shred bug? [was: Unidentified subject!] David Christensen <dpchrist@holgerdanske.com> - 2024-02-11 01:30 +0100
Re: shred bug? [was: Unidentified subject!] debian-user@howorth.org.uk - 2024-02-11 14:00 +0100
Re: shred bug? "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-11 14:30 +0100
Re: shred bug? [was: Unidentified subject!] <tomas@tuxteam.de> - 2024-02-11 08:10 +0100
Re: shred bug? [was: Unidentified subject!] Greg Wooledge <greg@wooledge.org> - 2024-02-11 15:40 +0100
Re: shred bug? [was: Unidentified subject!] <tomas@tuxteam.de> - 2024-02-11 15:50 +0100
Re: shred bug? [was: Unidentified subject!] Greg Wooledge <greg@wooledge.org> - 2024-02-11 16:00 +0100
Re: shred bug? [was: Unidentified subject!] <tomas@tuxteam.de> - 2024-02-11 16:00 +0100
Re: shred bug? [was: Unidentified subject!] Greg Wooledge <greg@wooledge.org> - 2024-02-12 18:00 +0100
Re: shred bug? [was: Unidentified subject!] Curt <curty@free.fr> - 2024-02-12 18:00 +0100
Re: shred bug? [was: Unidentified subject!] David Christensen <dpchrist@holgerdanske.com> - 2024-02-12 22:00 +0100
Re: shred bug? [was: Unidentified subject!] "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-12 22:10 +0100
Re: shred bug? [was: Unidentified subject!] <tomas@tuxteam.de> - 2024-02-13 06:50 +0100
Re: shred bug? [was: Unidentified subject!] David Wright <deblis@lionunicorn.co.uk> - 2024-02-11 16:30 +0100
Re: shred bug? [was: Unidentified subject!] Greg Wooledge <greg@wooledge.org> - 2024-02-13 13:20 +0100
Re: shred bug? [was: Unidentified subject!] Greg Wooledge <greg@wooledge.org> - 2024-02-13 13:40 +0100
Re: shred bug? [was: Unidentified subject!] <tomas@tuxteam.de> - 2024-02-13 13:50 +0100
Re: shred bug? [was: Unidentified subject!] "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-13 14:00 +0100
Re: shred bug? [was: Unidentified subject!] David Wright <deblis@lionunicorn.co.uk> - 2024-02-13 16:40 +0100
Re: shred bug? [was: Unidentified subject!] Greg Wooledge <greg@wooledge.org> - 2024-02-13 17:30 +0100
Re: shred bug? [was: Unidentified subject!] debian-user@howorth.org.uk - 2024-02-13 18:50 +0100
Re: shred bug? [was: Unidentified subject!] David Christensen <dpchrist@holgerdanske.com> - 2024-02-13 22:10 +0100
Re: shred bug? [was: Unidentified subject!] <tomas@tuxteam.de> - 2024-02-14 06:50 +0100
Re: shred bug? [was: Unidentified subject!] "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-13 19:00 +0100
Re: shred bug? [was: Unidentified subject!] Greg Wooledge <greg@wooledge.org> - 2024-02-13 20:00 +0100
Re: shred bug? [was: Unidentified subject!] "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-13 20:40 +0100
Re: shred bug? [was: Unidentified subject!] gene heskett <gheskett@shentel.net> - 2024-02-13 20:40 +0100
Re: shred bug? [was: Unidentified subject!] "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-13 20:50 +0100
Re: shred bug? [was: Unidentified subject!] gene heskett <gheskett@shentel.net> - 2024-02-13 21:30 +0100
Re: shred bug? [was: Unidentified subject!] David Christensen <dpchrist@holgerdanske.com> - 2024-02-13 22:10 +0100
Re: shred bug? [was: Unidentified subject!] gene heskett <gheskett@shentel.net> - 2024-02-13 22:50 +0100
Re: shred bug? [was: Unidentified subject!] David Wright <deblis@lionunicorn.co.uk> - 2024-02-15 06:40 +0100
Re: shred bug? [was: Unidentified subject!] David Christensen <dpchrist@holgerdanske.com> - 2024-02-11 23:50 +0100
Re: shred bug? [was: Unidentified subject!] Max Nikulin <manikulin@gmail.com> - 2024-02-13 03:40 +0100
Re: shred bug? [was: Unidentified subject!] "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-11 16:30 +0100
Re: shred bug? [was: Unidentified subject!] Michael Stone <mstone@debian.org> - 2024-02-16 16:00 +0100
Re: Unidentified subject! gene heskett <gheskett@shentel.net> - 2024-02-10 18:50 +0100
Re: Unidentified subject! "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-10 19:40 +0100
Re: Unidentified subject! gene heskett <gheskett@shentel.net> - 2024-02-10 20:30 +0100
Re: Unidentified subject! David Christensen <dpchrist@holgerdanske.com> - 2024-02-11 00:50 +0100
Re: Unidentified subject! "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-11 09:10 +0100
Re: Unidentified subject! David Christensen <dpchrist@holgerdanske.com> - 2024-02-11 21:50 +0100
Re: Unidentified subject! David Christensen <dpchrist@holgerdanske.com> - 2024-02-11 00:40 +0100
Re: Unidentified subject! "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-11 09:20 +0100
Re: Unidentified subject! David Christensen <dpchrist@holgerdanske.com> - 2024-02-11 11:10 +0100
Fast Random Data Generation (Was: Re: Unidentified subject!) Linux-Fan <Ma_Sys.ma@web.de> - 2024-02-11 11:30 +0100
Re: Fast Random Data Generation "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-11 12:30 +0100
Re: Fast Random Data Generation (Was: Re: Unidentified subject!) Gremlin <scott-andrews@columbus.rr.com> - 2024-02-11 13:20 +0100
Re: Fast Random Data Generation (Was: Re: Unidentified subject!) David Christensen <dpchrist@holgerdanske.com> - 2024-02-12 07:20 +0100
Re: Fast Random Data Generation (Was: Re: Unidentified subject!) Linux-Fan <Ma_Sys.ma@web.de> - 2024-02-12 17:40 +0100
Re: Fast Random Data Generation (Was: Re: Unidentified subject!) David Christensen <dpchrist@holgerdanske.com> - 2024-02-12 22:30 +0100
Re: Fast Random Data Generation (Was: Re: Unidentified subject!) Linux-Fan <Ma_Sys.ma@web.de> - 2024-02-13 18:40 +0100
Re: Fast Random Data Generation (Was: Re: Unidentified subject!) Jeffrey Walton <noloader@gmail.com> - 2024-02-12 22:30 +0100
Re: Unidentified subject! "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-11 12:20 +0100
Re: Unidentified subject! Jeffrey Walton <noloader@gmail.com> - 2024-02-11 16:20 +0100
Re: Unidentified subject! David Christensen <dpchrist@holgerdanske.com> - 2024-02-11 23:00 +0100
Re: Unidentified subject! Stefan Monnier <monnier@iro.umontreal.ca> - 2024-02-10 15:40 +0100
Things I don't touch with a 3.048m barge pole: USB storage (Was Re: Unidentified subject!) Andy Smith <andy@strugglers.net> - 2024-02-08 16:40 +0100
Re: Things I don't touch with a 3.048m barge pole: USB storage (Was Re: Unidentified subject!) Gremlin <scott-andrews@columbus.rr.com> - 2024-02-08 17:20 +0100
Re: Things I don't touch with a 3.048m barge pole: USB storage (Was Re: Unidentified subject!) Andy Smith <andy@strugglers.net> - 2024-02-08 17:30 +0100
Re: Things I don't touch with a 3.048m barge pole: USB storage (WasRe: Unidentified subject!) gene heskett <gheskett@shentel.net> - 2024-02-09 09:50 +0100
Re: Things I don't touch with a 3.048m barge pole: USB storage (Was Re: Unidentified subject!) Ralph Aichinger <ra@h5.or.at> - 2024-02-08 18:00 +0100
Re: Things I don't touch with a 3.048m barge pole: USB storage (Was Re: Unidentified subject!) Jeffrey Walton <noloader@gmail.com> - 2024-02-08 20:30 +0100
Re: Things I don't touch with a 3.048m barge pole: USB storage (Was Re: Unidentified subject!) Andy Smith <andy@strugglers.net> - 2024-02-08 21:50 +0100
Re: Things I don't touch with a 3.048m barge pole: USB storage (Was Re: Unidentified subject!) Andy Smith <andy@strugglers.net> - 2024-02-08 21:50 +0100
Re: Things I don't touch with a 3.048m barge pole: USB storage (Was Re: Unidentified subject!) Gremlin <scott-andrews@columbus.rr.com> - 2024-02-08 22:00 +0100
Re: Things I don't touch with a 3.048m barge pole: USB storage (Was Re: Unidentified subject!) Andy Smith <andy@strugglers.net> - 2024-02-08 22:20 +0100
Re: Things I don't touch with a 3.048m barge pole: USB storage (Was Re: Unidentified subject!) Andy Smith <andy@strugglers.net> - 2024-02-08 22:30 +0100
Re: Things I don't touch with a 3.048m barge pole: USB storage (Was Re: Unidentified subject!) Gremlin <scott-andrews@columbus.rr.com> - 2024-02-08 23:10 +0100
Re: Things I don't touch with a 3.048m barge pole: USB storage (Was Re: Unidentified subject!) Gremlin <scott-andrews@columbus.rr.com> - 2024-02-08 22:30 +0100
Re: Things I don't touch with a 3.048m barge pole: USB storage (WasRe: Unidentified subject!) gene heskett <gheskett@shentel.net> - 2024-02-09 14:00 +0100
Re: Things I don't touch with a 3.048m barge pole "Thomas Schmitt" <scdbackup@gmx.net> - 2024-02-09 14:50 +0100
Re: Things I don't touch with a 3.048m barge pole: USB storage (WasRe: Unidentified subject!) David Christensen <dpchrist@holgerdanske.com> - 2024-02-10 07:00 +0100
Re: Things I don't touch with a 3.048m barge pole: USB storage(WasRe: Unidentified subject!) gene heskett <gheskett@shentel.net> - 2024-02-10 18:40 +0100
Re: Things I don't touch with a 3.048m barge pole: USB storage (Was Re: Unidentified subject!) Andy Smith <andy@strugglers.net> - 2024-02-08 21:40 +0100
Re: Things I don't touch with a 3.048m barge pole: USB storage (Was Re: Unidentified subject!) Arno Lehmann <al@its-lehmann.de> - 2024-02-09 09:50 +0100
Re: Things I don't touch with a 3.048m barge pole: USB storage (Was Re: Unidentified subject!) Max Nikulin <manikulin@gmail.com> - 2024-02-08 18:30 +0100
Re: Things I don't touch with a 3.048m barge pole: USB storage (Was Re: Unidentified subject!) Andy Smith <andy@strugglers.net> - 2024-02-08 21:40 +0100
Re: Things I don't touch with a 3.048m barge pole: USB storage (Was Re: Unidentified subject!) Gremlin <scott-andrews@columbus.rr.com> - 2024-02-08 22:10 +0100
Re: Things I don't touch with a 3.048m barge pole: USB storage (Was Re: Unidentified subject!) Andy Smith <andy@strugglers.net> - 2024-02-08 22:20 +0100
Re: Unidentified subject! Richmond <dnomhcir@gmx.com> - 2024-02-08 23:50 +0100
Re: Unidentified subject! Stefan Monnier <monnier@iro.umontreal.ca> - 2024-02-09 00:10 +0100
Re: Unidentified subject! Charles Curley <charlescurley@charlescurley.com> - 2024-02-09 00:50 +0100
Re: Unidentified subject! Richmond <dnomhcir@gmx.com> - 2024-02-09 05:50 +0100
Re: Unidentified flying subject! Charles Curley <charlescurley@charlescurley.com> - 2024-02-09 07:50 +0100
Re: Unidentified flying subject! Richmond <dnomhcir@gmx.com> - 2024-02-09 14:20 +0100
csiph-web