Path: csiph.com!xmission!news.snarked.org!news.linkpendium.com!news.linkpendium.com!panix!usenet.stanford.edu!not-for-mail
From: Daniel Colascione <dancol@dancol.org>
Newsgroups: gnu.bash.bug
Subject: Re: Proposed new feature for bash:  unbuffered pipes
Date: Thu, 23 Apr 2020 17:26:56 -0700
Lines: 46
Approved: bug-bash@gnu.org
Message-ID: <mailman.1063.1587688023.3066.bug-bash@gnu.org>
References: <87mu73ugw8.fsf@hobgoblin.ariadne.com> <E1jRmBF-00035H-FD@dancol.org>
NNTP-Posting-Host: lists.gnu.org
Mime-Version: 1.0
Content-Type: text/plain; format=flowed; charset="us-ascii"
Content-Transfer-Encoding: 8bit
To: <worley@alum.mit.edu>, <bug-bash@gnu.org>
Envelope-to: bug-bash@gnu.org
In-Reply-To: <87mu73ugw8.fsf@hobgoblin.ariadne.com>
User-Agent: AquaMail/1.24.0-1572 (build: 102400003)
Received-SPF: pass client-ip=2600:3c01::f03c:91ff:fedf:adf3; envelope-from=dancol@dancol.org; helo=dancol.org
Precedence: list
Xref: csiph.com gnu.bash.bug:16236

On April 22, 2020 6:32:07 PM worley@alum.mit.edu (Dale R. Worley) wrote:

> The crux of the problem, IMHO, is to look at it from the right angle:
>
> Occasionally, the user desires that I/O through certain pipes should be
> unbuffered, that is, the stdio stream(s) that write into the pipe should
> be unbuffered, rather than the default block-buffered.  These are
> situations where the user can tell that it is desirable to sacrifice
> efficiency to gain low latency, whereas the writer of the program that
> the user is running could not predict the needs of this particular
> invocation.  (This is where Ulrich Drepper was incorrect; the writer of
> the program doesn't know when this is desirable.)
>
> This facility is best activated by a feature in bash, as the need arises
> from the relationship between two processes.
>
> The ideal way to implement this would be for the kernel to provide two
> flavors of pipe, one designated buffered and one designated unbuffered,
> and according to the user's instructions, bash would allocate an
> unbuffered pipe when it was needed.  The kernel itself would treat the
> two types of pipe in the same way, but stdio would create buffered
> streams for fd's that were opened on buffered pipes and it would create
> unbuffered streams for fd's that were opened on unbuffered pipes.
>
> However, getting a coordinated change into the kernel seems like it
> would be quite difficult.
>
> So an alternative implementation is to have a way for bash to notify
> processes it creates that certain pipes (identified by their dev/ino
> numbers) should have streams that are unbuffered.  Pretty much the only
> way to do this is to have a conventional environment variable for
> communicating this information.
>
> Of course, either of these solutions requires changes to glibc, but
> then, if you're going to modify buffering behavior of streams, you are
> going to have to modify glibc.

Given the current political realities in the relevant FOSS ecosystem, it 
might be easier to add a new pipe flavor or attribute to the kernel and 
then get libc implementations to support it as a fait accompli than to do 
something purely in libc. Also, an environment variable is semantically 
wrong: its scope is too  broad, well beyond the specific pipe you care 
about. Imagine the variable leaking into a daemon that lives forever.