Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > gnu.bash.bug > #14491

Re: Unexpected delay in using arguments.

Path csiph.com!xmission!news.snarked.org!news.linkpendium.com!news.linkpendium.com!panix!usenet.stanford.edu!not-for-mail
From Bize Ma <binaryzebra@gmail.com>
Newsgroups gnu.bash.bug
Subject Re: Unexpected delay in using arguments.
Date Tue, 14 Aug 2018 23:17:55 -0400
Lines 42
Approved bug-bash@gnu.org
Message-ID <mailman.5124.1534303082.1292.bug-bash@gnu.org> (permalink)
References <CAFra36j=UuqHv2Spp26ebAW0dRW+KsgNS21hZJrwuku9QUayWw@mail.gmail.com> <20180814123113518879862@bob.proulx.com>
NNTP-Posting-Host lists.gnu.org
Mime-Version 1.0
Content-Type text/plain; charset="UTF-8"
Content-Transfer-Encoding quoted-printable
X-Trace usenet.stanford.edu 1534303083 21530 208.118.235.17 (15 Aug 2018 03:18:03 GMT)
X-Complaints-To action@cs.stanford.edu
Cc bug-bash <bug-bash@gnu.org>
To Bize Ma <binaryzebra@gmail.com>, Bob Proulx <bob@proulx.com>
Envelope-to bug-bash@gnu.org
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=a6+cylHTPW3IF3IStGAGN2Uwi1aLNvtQeYLbEMSzUL0=; b=WJn3m3gsPaRc9skmM8DiPwk9HHb9WTjD7GyjmlHa/27bUCjcZXqXCqMYBgbqvppDMG 98NVyPH4RTDuafay/6WC3hplNTdiO2fLO2btzNlULFh+RF1ddzah2iBGrpD6DN2fS56F c+dja0ScwR4OgxbUsSyBrcpnUSoILtqbJOSTx6/W2awZZLccQCABtBaRaCRsQc0NoBe6 63wySULdJEIg2C4t8QnsxPxRdIpfnzeog3yI5ZVEP6K3coRgMDnCSGhwyHTWdnU6C3Gv SEQbrerMjkp/X6kprpn4PXw545H0sbyHTecxfV+PIDE3CdRyqsb/bzhdC/f7MT/LUX64 GYjQ==
X-Google-DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=a6+cylHTPW3IF3IStGAGN2Uwi1aLNvtQeYLbEMSzUL0=; b=tuRv191AYRbaiy5aCc0ZWLIAJlwqwe9BbDVOyXfV8UHQiZivePyk/Me4KgrEPlC9Wr vTx+IkNcR57pu5PMzfodjtvhwdTTGpKvd/RkLQ0T3f68lZuFxVm1hdmk2B9jXF1bdk1V KmlIRZdewZpf2QOMLz1yj7ajn6pHAOUYxFppm6PNzozJaunudsR3JzdhH47m9Lezray/ m6bz0Vbn+bx2gl/NLObbjKS6x9U2I9bbiTcl9vDpa5w0TaHryHUpLTtYqZvRrN+bZzEP htS+V2mL91s8WbkaFPlno2pBc5fbG0sIY3KKKlJ+KHSmKa2EEgs/bYivziZzkaStyjfQ VlLw==
X-Gm-Message-State AOUpUlGIJJQUxzPMBQ2aonFhkBiE2ZYarEXN2OnxsZqA9fADK+3SgVPF M27xyopM/7+/HRlNIfrirUAdCX9LvxzXwyjxX9c=
X-Google-Smtp-Source AA+uWPzlNGOmhmt9l4WgzkdNQZ4+VrAUYjxSzn3f1l0bWUF9fQVtE9YUXahzWngWL+zcck7JVWwjhyFIrjoqfHbvF/k=
X-Received by 2002:aca:4fd1:: with SMTP id d200-v6mr26859829oib.224.1534303075979; Tue, 14 Aug 2018 20:17:55 -0700 (PDT)
In-Reply-To <20180814123113518879862@bob.proulx.com>
X-detected-operating-system by eggs.gnu.org: Genre and OS details not recognized.
X-Received-From 2607:f8b0:4003:c06::22d
X-Content-Filtered-By Mailman/MimeDel 2.1.21
X-BeenThere bug-bash@gnu.org
X-Mailman-Version 2.1.21
Precedence list
List-Id Bug reports for the GNU Bourne Again SHell <bug-bash.gnu.org>
List-Unsubscribe <https://lists.gnu.org/mailman/options/bug-bash>, <mailto:bug-bash-request@gnu.org?subject=unsubscribe>
List-Archive <http://lists.gnu.org/archive/html/bug-bash/>
List-Post <mailto:bug-bash@gnu.org>
List-Help <mailto:bug-bash-request@gnu.org?subject=help>
List-Subscribe <https://lists.gnu.org/mailman/listinfo/bug-bash>, <mailto:bug-bash-request@gnu.org?subject=subscribe>
Xref csiph.com gnu.bash.bug:14491

Show key headers only | View raw


> Of course I assume this is only a proxy simpler reproducer
> for the actual problem program

Of course this is a "reproducer" of the issue.

> but just the same it is almost always possible
> to refactor a program into a different algorithm that avoids the need
> to enumerate so many arguments in memory.

As you say: "almost".

Take a look at the Stéphane Chazelas example to convince yourself.



2018-08-14 17:50 GMT-04:00 Bob Proulx <bob@proulx.com>:

> I followed along through the script:
>
> >     m=20000
> ...
> >     test1() { ...
>
> Eventually I got to this line:
>
> >       set -- $(seq $m)
>
> At that line I see that bash must allocate enough memory to hold all
> of the numbers from 1 through 20000 in memory all at one time.  That
> is very inefficient.  That is at least 100K of memory.
>
>   $ seq 20000 | wc -c
>   108894
>
> Of course I assume this is only a proxy simpler reproducer for the
> actual problem program but just the same it is almost always possible
> to refactor a program into a different algorithm that avoids the need
> to enumerate so many arguments in memory.  I suggest refactoring the
> program to avoid the need for this memory stress.
>
> Bob
>

Back to gnu.bash.bug | Previous | Next | Find similar | Unroll thread


Thread

Re: Unexpected delay in using arguments. Bize Ma <binaryzebra@gmail.com> - 2018-08-14 23:17 -0400

csiph-web