Path: csiph.com!goblin2!goblin1!goblin.stu.neva.ru!usenet.stanford.edu!not-for-mail From: "Jason A. Donenfeld" Newsgroups: gnu.bash.bug Subject: Re: process substitution error handling Date: Thu, 6 Aug 2020 17:31:10 +0200 Lines: 93 Approved: bug-bash@gnu.org Message-ID: References: <20200420051508.GA2359844@zx2c4.com> <7496b183-2db3-6c03-6074-928adcd08f45@case.edu> <9b358a76-4867-41b9-5a3a-c1892c76b8ee@case.edu> NNTP-Posting-Host: lists.gnu.org Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Trace: usenet.stanford.edu 1596727893 4509 209.51.188.17 (6 Aug 2020 15:31:33 GMT) X-Complaints-To: action@cs.stanford.edu Cc: bug-bash@gnu.org, "Demi M. Obenour" To: Chester Ramey Envelope-to: bug-bash@gnu.org DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=zx2c4.com; h=mime-version :references:in-reply-to:from:date:message-id:subject:to:cc :content-type; s=mail; bh=xBeJO+tZhc5wSU9S50TzlfRHZYM=; b=byLWl2 0qjxuE8n0Bp9wSZQ8C2Sh0AUhmtbze3SJBikTCWeM+gxv2RwrTGn3gmrjIcwmehm M9TZy2XJXCdrWHjU3zjz1jpIaXgGjuo6GJ6ZhhT/2UsiVc/qqUW09D5DPGXjm1OZ LjeEU0/4hnV6Wacdrqcl+qOYJXEeEuTwaGmRu5uf0cSL2Lm4n6pfweE9xuQTdIsm FPuvOn+Kh7JKb8zWEwSV1B3NhbVyJQ11jBVBONxfKxBGGiBliApv6ZBZBqveFhqK gFhUwpEJVl/9BYRlka1bHK6bl1vbfz8J9e9Onv27cXEfh3x9joVaCYe9RPVrH1e1 IRq/AL/d6iXcolUg== X-Gm-Message-State: AOAM532O9mMD3NsvHvKDST2TFCeZaWc8TmK+i0QG4me7W2e/gMV7reLk A0fJItSrHCQ5RxjBsYqmzdoHe0OH57qCTH3dQ7Q= X-Google-Smtp-Source: ABdhPJz8Fm+b1m+W828JJr0xR7FqmuC5jP17TDmXK/d2gaHxtuX7oOTI0RWviSFuTPojJl32Gg0EuPkhz3S/ON6aoJ4= X-Received: by 2002:a92:bb92:: with SMTP id x18mr10641191ilk.64.1596727884112; Thu, 06 Aug 2020 08:31:24 -0700 (PDT) In-Reply-To: <9b358a76-4867-41b9-5a3a-c1892c76b8ee@case.edu> X-Gmail-Original-Message-ID: Received-SPF: pass client-ip=192.95.5.64; envelope-from=Jason@zx2c4.com; helo=mail.zx2c4.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/06 10:36:52 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: bug-bash@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Bug reports for the GNU Bourne Again SHell List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Mailman-Original-Message-ID: X-Mailman-Original-References: <20200420051508.GA2359844@zx2c4.com> <7496b183-2db3-6c03-6074-928adcd08f45@case.edu> <9b358a76-4867-41b9-5a3a-c1892c76b8ee@case.edu> Xref: csiph.com gnu.bash.bug:16719 On Thu, Aug 6, 2020 at 4:49 PM Chet Ramey wrote: > > On 8/6/20 10:36 AM, Jason A. Donenfeld wrote: > > Hi Chet, > > > > On Thu, Aug 6, 2020 at 4:30 PM Chet Ramey wrote: > >> > >> On 8/6/20 6:05 AM, Jason A. Donenfeld wrote: > >>> Hi, > >>> > >>> It may be a surprise to some that this code here winds up printing > >>> "done", always: > >>> > >>> $ cat a.bash > >>> set -e -o pipefail > >>> while read -r line; do > >>> echo "$line" > >>> done < <(echo 1; sleep 1; echo 2; sleep 1; false; exit 1) > >>> sleep 1 > >>> echo done > >>> > >>> $ bash a.bash > >>> 1 > >>> 2 > >>> done > >>> > >>> The reason for this is that process substitution right now does not > >>> propagate errors. It's sort of possible to almost make this better > >>> with `|| kill $$` or some variant, and trap handlers, but that's very > >>> clunky and fraught with its own problems. > >>> > >>> Therefore, I propose a `set -o substfail` option for the upcoming bash > >>> 5.1, which would cause process substitution to propagate its errors > >>> upwards, even if done asynchronously. > >>> > >>> Chet - thoughts? > >> > >> I don't like it, for two reasons: > >> > >> 1. Process substitution is a word expansion, and, with one exception, word > >> expansions don't contribute to a command's exit status and > >> consequently the behavior of errexit, and this proposal isn't compelling > >> enough to change that even with a new option; and > >> > >> 2. Process substitution is asynchronous. I can't think of how spontaneously > >> changing $? (and possibly exiting) at some random point in a script when > >> the shell reaps a process substitution will make scripts more reliable. > > > > Demi (CC'd) points out that there might be security dangers around > > patterns like: > > > > while read -r one two three; do > > add_critical_thing_for "$one" "$two" "$three" > > done < <(get_critical_things) > > > > If get_critical_things returns a few lines but then exits with a > > failure, the script will forget to call add_critical_thing_for, and > > some kind of door will be held wide open. This is problematic and > > arguably makes bash unsuitable for many of the sysadmin things that > > people use bash for. > > If this is a problem for a particular script, add the usual `wait $!' > idiom and react accordingly. If that's not feasible, you can always > use some construct other than process substitution (e.g., a file). > I don't see how this "makes bash unsuitable for many [...] sysadmin > things." > > > > > Perhaps another, clunkier, proposal would be to add `wait -s` so that > > the wait builtin also waits for process substitutions and returns > > their exit codes and changes $?. The downside would be that scripts > > now need to add a "wait" after all of above such loops, but on the > > upside, it's better than the current problematic situation. > > You can already do this. Since process substitution sets $!, you can > keep track of all of the process substitutions of interest and wait > for as many of them as you like. `wait' will return their statuses > and set $? for you. That doesn't always work: set -e while read -r line; do echo "$line" & done < <(echo 1; sleep 1; echo 2; sleep 1; exit 77) sleep 1 wait $! echo done Either way, tagging on `wait $!` everywhere, and hoping it works like I want feels pretty flimsy. Are you sure you're opposed to set -o procsuberr that would do the right thing for most common use cases?