Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.misc > #29722

Re: Reading a file from the back

Show all headers | View raw

with <eli$2004122102@qaz.wtf> Eli the Bearded wrote:
> In comp.os.linux.misc, Eric Pozharski  <whynot@pozharski.name> wrote:
>> Eli the Bearded wrote:

>>> perl -ne '$all .= $_; END{ print scalar reverse $all }' optimized
>>> the perl for readibility, not speed or code golf
>> perl's 'reverse' is pretty much speed optimized for many releases
>> now.  Unfoturnately I still don't have perl-zoo thus I can't show it.
> That program will suck big time if the input exceeds the size of
> memory and starts to swap.
>
> If there are multiple files, reading them one at a time and reversing
> those is probably a better idea.
>
> For large *files*, reading block by block backwards from the file
> itself and reversing those blocks is probably a huge win over filling
> all of memory. For non-files (eg read from STDIN), using a temp file
> to store data instead of using RAM is a good idea.
>
> For medium files with very short lines, growing $all in a tight loop
> is probably a lot worse than sucking the whole file in with a single
> `$\ = undef;` read.

Excellent analysis.  perl's reverse has nothing to do with it.  Except,
it's hard to kick memory in the nuts these days.

> Need I go on?

Yes, please continue.

> It has nothing to do with if reverse() itself is fast or not, it
> simply isn't optimized to deal with edge cases. 

Exactly:

% dd if=/dev/urandom of=`.,m.s foo` bs=1G count=1 
/home/whynot/foo.Es2bzr.foo
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 118.571 s, 9.1 MB/s
% time perl -wne '$aa.=$_; END { print reverse $aa }' >/dev/null <~/foo.Es2bzr.foo                 
perl -wne '$aa.=$_; END { print reverse $aa }' > /dev/null < ~/foo.Es2bzr.foo  4.11s user 1.25s system 67% cpu 7.899 total
% time perl -wne '$aa.=$_; END { print $aa }' >/dev/null <~/foo.Es2bzr.foo
perl -wne '$aa.=$_; END { print $aa }' > /dev/null < ~/foo.Es2bzr.foo  4.02s user 1.28s system 82% cpu 6.408 total

> did you reply specifically to show off perl knowledge?

Might be so, I should contemplate on this.  Also, 'scalar' is useless
here.


-- 
Torvalds' goal for Linux is very simple: World Domination
Stallman's goal for GNU is even simpler: Freedom

Back to comp.os.linux.misc | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

Reading a file from the back Dieter Britz <dieterhansbritz@gmail.com> - 2020-04-07 12:01 +0000
  Re: Reading a file from the back Vasco Costa <vasco.costa@invalid.invalid> - 2020-04-07 12:12 +0000
  Re: Reading a file from the back Fenris <fenris@invalid.invalid> - 2020-04-07 12:12 +0000
  Re: Reading a file from the back Rich <rich@example.invalid> - 2020-04-07 15:18 +0000
    Re: Reading a file from the back Jim Jackson <jj@franjam.org.uk> - 2020-04-10 19:21 +0000
      Re: Reading a file from the back Eli the Bearded <*@eli.users.panix.com> - 2020-04-10 20:37 +0000
        Re: Reading a file from the back Eric Pozharski <whynot@pozharski.name> - 2020-04-11 15:22 +0300
          Re: Reading a file from the back Eli the Bearded <*@eli.users.panix.com> - 2020-04-13 01:03 +0000
            Re: Reading a file from the back Eric Pozharski <whynot@pozharski.name> - 2020-04-13 11:16 +0300
              Re: Reading a file from the back Eli the Bearded <*@eli.users.panix.com> - 2020-04-13 20:01 +0000
                Re: Reading a file from the back Eric Pozharski <whynot@pozharski.name> - 2020-04-14 12:14 +0300
                Re: Reading a file from the back Eli the Bearded <*@eli.users.panix.com> - 2020-04-14 16:50 +0000
      Re: Reading a file from the back Rich <rich@example.invalid> - 2020-04-10 21:40 +0000
  Re: Reading a file from the back The Natural Philosopher <tnp@invalid.invalid> - 2020-04-08 08:55 +0100

csiph-web