Re: query: how many use PHP for linux scripts

From	The Natural Philosopher <tnp@invalid.invalid>
Newsgroups	alt.os.linux, alt.os.linux.ubuntu, comp.lang.php
Subject	Re: query: how many use PHP for linux scripts
Date	2014-05-15 17:48 +0100
Organization	albasani.net
Message-ID	<ll2r4f$7v7$1@news.albasani.net> (permalink)
References	(6 earlier) <lkuonh$6na$1@news.albasani.net> <0.93e9843bc1b5f085d0a5.20140514133906BST.87bnv0edqt.fsf@bsb.me.uk> <ll2c99$nr9$4@gonzo.reversiblemaps.ath.cx> <ll2gr1$hse$1@news.albasani.net> <ll2p3b$m7$1@dont-email.me>

Cross-posted to 3 groups.

Show all headers | View raw

On 15/05/14 17:13, crankypuss wrote:
> On 05/15/2014 07:52 AM, The Natural Philosopher wrote:
>> On 15/05/14 13:34, Jasen Betts wrote:
>>> including files with lines larger than  RAM + swap.
>>
>> THAT is challenge!
>>
>> Have to use temporary files..
>>
>>
>
> Whatever for?  You need enough "disk" space to contain the input file
> and the output file, you need enough memory to contain the two largest
> contiguous records

but in the case specified, you dont have that space.

  and the code to read-2-(swap-2)-write-2.  Whether the
> swap involves actual data movement or not is a nit, it boils down to a
> file-copy 2 lines at a time.  (Wouldn't surprise me if the "cp" command
> has some option for this kind of thing along with the rest of the
> kitchen sink... or is that "tar" that I'm thinking of...)
>
> Unless of course I'm missing something, I'm feeling stupider with each
> post about this exercise that I read. <g>

"*lines* larger than  RAM + swap" is the key.

you need to essentially read through the input file and tag where the 
line begins and ends are.

THEN start reading it sequentially from the tag marks and assembling the 
output a few megabytes at a time

i.e. you will end up wit at worst enough storage needed to hold an array 
of 'line start and end' offsets in the original file.

Or if space is REALLY tight you do it the way COBOL used to do things. 
Yuu scan the file to the first line end and then write thet second line 
out, too the new file, then store that location as line 3, and write 
line one to the tail of file 2, and so on

The point is that's fairly trivial with C. It doesn't ENFORCE a 
particular way of dealing with files on you.

And C with a good library can be really high level too.

BUT if that doesn't work for you, no problem in going near to the 
hardware either.

-- 
Ineptocracy

(in-ep-toc’-ra-cy) – a system of government where the least capable to 
lead are elected by the least capable of producing, and where the 
members of society least likely to sustain themselves or succeed, are 
rewarded with goods and services paid for by the confiscated wealth of a 
diminishing number of producers.

Thread

query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-13 02:17 -0600
  Re: query: how many use PHP for linux scripts Arno Welzel <usenet@arnowelzel.de> - 2014-05-13 10:37 +0200
  Re: query: how many use PHP for linux scripts Tim Streater <timstreater@greenbee.net> - 2014-05-13 11:52 +0100
  Re: query: how many use PHP for linux scripts "Auric__" <not.my.real@email.address> - 2014-05-13 13:28 +0000
    Re: query: how many use PHP for linux scripts "Jonathan N. Little" <lws4art@gmail.com> - 2014-05-13 10:29 -0400
  Re: query: how many use PHP for linux scripts Collin Pruitt <hellow+newsgroup@binenv.com> - 2014-05-13 11:00 -0400
    Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-13 10:05 -0600
      Re: query: how many use PHP for linux scripts Johnny <johnny@invalid.net> - 2014-05-13 11:32 -0500
        Re: query: how many use PHP for linux scripts The Natural Philosopher <tnp@invalid.invalid> - 2014-05-13 19:09 +0100
          Re: query: how many use PHP for linux scripts Tim Streater <timstreater@greenbee.net> - 2014-05-13 21:35 +0100
            Re: query: how many use PHP for linux scripts JEDIDIAH <jedi@nomad.mishnet> - 2014-05-13 16:10 -0500
              Re: query: how many use PHP for linux scripts Tim Streater <timstreater@greenbee.net> - 2014-05-13 23:03 +0100
                Re: query: how many use PHP for linux scripts Jerry Stuckle <jstucklex@attglobal.net> - 2014-05-13 19:55 -0400
                Re: query: how many use PHP for linux scripts Chris Ahlstrom <OFeem1987@teleworm.us> - 2014-05-13 20:26 -0400
                Re: query: how many use PHP for linux scripts Jerry Stuckle <jstucklex@attglobal.net> - 2014-05-13 20:37 -0400
                Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-14 03:18 -0600
                Re: query: how many use PHP for linux scripts Jerry Stuckle <jstucklex@attglobal.net> - 2014-05-14 08:20 -0400
                PHP Gui Toolkits (was: query: how many use PHP for linux scripts) Christoph Michael Becker <cmbecker69@arcor.de> - 2014-05-14 14:58 +0200
                Re: PHP Gui Toolkits crankypuss <crankypuss@nomail.invalid> - 2014-05-15 02:04 -0600
                Re: PHP Gui Toolkits Christoph Michael Becker <cmbecker69@arcor.de> - 2014-05-15 14:38 +0200
                Re: PHP Gui Toolkits The Natural Philosopher <tnp@invalid.invalid> - 2014-05-15 13:59 +0100
                Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-15 02:01 -0600
                Re: query: how many use PHP for linux scripts Chris Ahlstrom <OFeem1987@teleworm.us> - 2014-05-14 06:40 -0400
                Re: query: how many use PHP for linux scripts Jerry Stuckle <jstucklex@attglobal.net> - 2014-05-14 08:23 -0400
                Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-14 03:00 -0600
                Re: query: how many use PHP for linux scripts JEDIDIAH <jedi@nomad.mishnet> - 2014-05-14 14:49 -0500
              Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-14 02:44 -0600
                Re: query: how many use PHP for linux scripts The Natural Philosopher <tnp@invalid.invalid> - 2014-05-14 10:17 +0100
                Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-14 03:39 -0600
                Re: query: how many use PHP for linux scripts Tim Streater <timstreater@greenbee.net> - 2014-05-14 12:29 +0100
                Re: query: how many use PHP for linux scripts JEDIDIAH <jedi@nomad.mishnet> - 2014-05-14 14:54 -0500
                Re: query: how many use PHP for linux scripts The Natural Philosopher <tnp@invalid.invalid> - 2014-05-14 21:50 +0100
                Re: query: how many use PHP for linux scripts Jerry Stuckle <jstucklex@attglobal.net> - 2014-05-14 22:49 -0400
                Re: query: how many use PHP for linux scripts JEDIDIAH <jedi@nomad.mishnet> - 2014-05-15 09:45 -0500
                Re: query: how many use PHP for linux scripts The Natural Philosopher <tnp@invalid.invalid> - 2014-05-15 17:35 +0100
                Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-15 02:07 -0600
                Re: query: how many use PHP for linux scripts "Jonathan N. Little" <lws4art@gmail.com> - 2014-05-15 07:58 -0400
                Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-15 09:55 -0600
                Re: query: how many use PHP for linux scripts Jerry Stuckle <jstucklex@attglobal.net> - 2014-05-15 08:08 -0400
                Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-15 10:04 -0600
            Re: query: how many use PHP for linux scripts The Natural Philosopher <tnp@invalid.invalid> - 2014-05-14 04:42 +0100
              Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-14 03:21 -0600
                Re: query: how many use PHP for linux scripts The Natural Philosopher <tnp@invalid.invalid> - 2014-05-14 12:32 +0100
              Re: query: how many use PHP for linux scripts Ben Bacarisse <ben.usenet@bsb.me.uk> - 2014-05-14 13:39 +0100
                Re: query: how many use PHP for linux scripts The Natural Philosopher <tnp@invalid.invalid> - 2014-05-14 15:23 +0100
                Re: query: how many use PHP for linux scripts Christoph Michael Becker <cmbecker69@arcor.de> - 2014-05-14 20:00 +0200
                Re: query: how many use PHP for linux scripts The Natural Philosopher <tnp@invalid.invalid> - 2014-05-14 21:48 +0100
                Re: query: how many use PHP for linux scripts Ben Bacarisse <ben.usenet@bsb.me.uk> - 2014-05-14 22:18 +0100
                Re: query: how many use PHP for linux scripts The Natural Philosopher <tnp@invalid.invalid> - 2014-05-14 23:10 +0100
                Re: query: how many use PHP for linux scripts Ben Bacarisse <ben.usenet@bsb.me.uk> - 2014-05-14 22:05 +0100
                Re: query: how many use PHP for linux scripts The Natural Philosopher <tnp@invalid.invalid> - 2014-05-14 23:07 +0100
                Re: query: how many use PHP for linux scripts Jerry Stuckle <jstucklex@attglobal.net> - 2014-05-14 22:51 -0400
                Re: query: how many use PHP for linux scripts Jasen Betts <jasen@xnet.co.nz> - 2014-05-15 12:34 +0000
                Re: query: how many use PHP for linux scripts The Natural Philosopher <tnp@invalid.invalid> - 2014-05-15 14:52 +0100
                Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-15 10:13 -0600
                Re: query: how many use PHP for linux scripts The Natural Philosopher <tnp@invalid.invalid> - 2014-05-15 17:48 +0100
                Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-16 01:12 -0600
                Re: query: how many use PHP for linux scripts Jasen Betts <jasen@xnet.co.nz> - 2014-05-16 11:29 +0000
                Re: query: how many use PHP for linux scripts The Natural Philosopher <tnp@invalid.invalid> - 2014-05-16 13:03 +0100
                PHP integers (was: query: how many use PHP for linux scripts) Christoph Michael Becker <cmbecker69@arcor.de> - 2014-05-16 14:16 +0200
                Re: query: how many use PHP for linux scripts Jerry Stuckle <jstucklex@attglobal.net> - 2014-05-15 10:03 -0400
                Re: query: how many use PHP for linux scripts Ben Bacarisse <ben.usenet@bsb.me.uk> - 2014-05-15 16:26 +0100
                Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-15 10:07 -0600
                Re: query: how many use PHP for linux scripts The Natural Philosopher <tnp@invalid.invalid> - 2014-05-15 17:37 +0100
                Re: query: how many use PHP for linux scripts Christoph Michael Becker <cmbecker69@arcor.de> - 2014-05-15 19:14 +0200
                Re: query: how many use PHP for linux scripts Ben Bacarisse <ben.usenet@bsb.me.uk> - 2014-05-16 02:45 +0100
                Re: query: how many use PHP for linux scripts The Natural Philosopher <tnp@invalid.invalid> - 2014-05-16 09:25 +0100
                Re: query: how many use PHP for linux scripts Jerry Stuckle <jstucklex@attglobal.net> - 2014-05-16 09:14 -0400
                Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-16 01:17 -0600
                Re: query: how many use PHP for linux scripts Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2014-05-15 19:33 +0200
            Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-14 02:36 -0600
              Re: query: how many use PHP for linux scripts The Natural Philosopher <tnp@invalid.invalid> - 2014-05-14 10:09 +0100
                Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-14 03:58 -0600
          Re: query: how many use PHP for linux scripts Allodoxaphobia <knock_yourself_out@example.net> - 2014-05-13 21:23 +0000
            Re: query: how many use PHP for linux scripts Mike Yetto <unet.lighthouse@xoxy.net> - 2014-05-13 19:02 -0400
              Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-14 03:24 -0600
                Re: query: how many use PHP for linux scripts Mike Yetto <unet.lighthouse@xoxy.net> - 2014-05-14 17:23 -0400
                Re: query: how many use PHP for linux scripts Jerry Stuckle <jstucklex@attglobal.net> - 2014-05-14 22:53 -0400
                Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-15 02:13 -0600
                Re: query: how many use PHP for linux scripts Chris Ahlstrom <OFeem1987@teleworm.us> - 2014-05-15 06:09 -0400
                Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-15 10:16 -0600
                Re: query: how many use PHP for linux scripts Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2014-05-15 18:49 +0200
                Re: query: how many use PHP for linux scripts Jasen Betts <jasen@xnet.co.nz> - 2014-05-16 12:02 +0000
                Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-16 08:17 -0600
                init-system, bash, etc. (was: query: how many use PHP for linux scripts) Christoph Michael Becker <cmbecker69@arcor.de> - 2014-05-16 18:48 +0200
                Re: query: how many use PHP for linux scripts Mike Yetto <unet.lighthouse@xoxy.net> - 2014-05-15 08:02 -0400
                Re: query: how many use PHP for linux scripts Aragorn <thorongil@telenet.be.invalid> - 2014-05-15 03:16 +0200
        Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-13 12:20 -0600
          Re: query: how many use PHP for linux scripts Johnny <johnny@invalid.net> - 2014-05-13 13:35 -0500
            Re: query: how many use PHP for linux scripts Mike Yetto <unet.lighthouse@xoxy.net> - 2014-05-13 19:04 -0400
            Re: query: how many use PHP for linux scripts crankypuss <crankypuss@nomail.invalid> - 2014-05-14 03:26 -0600
              Re: query: how many use PHP for linux scripts Jerry Stuckle <jstucklex@attglobal.net> - 2014-05-14 08:28 -0400
  Re: query: how many use PHP for linux scripts The Natural Philosopher <tnp@invalid.invalid> - 2014-05-13 17:26 +0100

csiph-web