Path: csiph.com!goblin2!goblin1!goblin.stu.neva.ru!usenet.stanford.edu!not-for-mail From: Piotr Grzybowski Newsgroups: gnu.bash.bug Subject: Re: Directing into a variable doesn't work Date: Thu, 28 Jun 2018 14:12:12 +0200 Lines: 263 Approved: bug-bash@gnu.org Message-ID: References: NNTP-Posting-Host: lists.gnu.org Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: multipart/mixed; boundary=Apple-Mail-195--22564486 X-Trace: usenet.stanford.edu 1530187947 2548 208.118.235.17 (28 Jun 2018 12:12:27 GMT) X-Complaints-To: action@cs.stanford.edu Cc: chet.ramey@case.edu, "Peter Passchier" , "Martijn Dekker" , bug-bash@gnu.org To: tetsujin@scope-eye.net Envelope-to: bug-bash@gnu.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:mime-version:from:in-reply-to:date:cc:message-id:references :to; bh=hTZEqDGxL0FrXZ27F7aUN98ambNNbr5ug3ck6jezgxc=; b=ifW17qpYZMwxhRw75GOxReXzLaCCLzviHWFKOmy+fTjMa0vxVMUaNcDKBCCXjshdJX zGMd8qKpigvimaVNRwDC6HGVfaum+TgfOFNN3wJSRighmWPPE4ETDF441X+xmyqPF1Z7 Xnkp9PHG2OY7D/kxuOSOBLD2xTjwN7XLgWLylB0X2wDalHDkpSZbWIdSRp/8q847D45v OTBHHXkrnYYxqT1W0c3DKAKzsw6vuR/Ss66hznwT4oa2nA3wYvvSHL/sZnafNNV9lEn0 tSn9TnZlSjzo59KzKjrwR9DgVABmav72PVk/ZsHk6HVdU5n21dl6JbryPc57AUhvrfTJ uMCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:mime-version:from:in-reply-to:date:cc :message-id:references:to; bh=hTZEqDGxL0FrXZ27F7aUN98ambNNbr5ug3ck6jezgxc=; b=fwnfe/7FGnQv4cP0sSc9Fi6WgvIEhMkY7IEcdbihdBwBv4GJXcwsmLsai43IxRTiJG 4NFQmDTY9kQYXdaoST4H+/+hcIFDJjK5dmIPVu6TAxjJog4hJBqg6c9e0uU1JVR3E1pB ymx+gS/dxlq6LBC9ezux8GIguSE/wrjWU7h90fGhGtaYE2t4B6wGteKiQcQkjqSiG4nX EhCqh38kvm8/224biA408yYFI7sk2GgYq6OhxaOFgwX56/lDRoV5tctNsVUxxmoqYgf3 K1ANy8ZaemM+7MjCSbAdgZCxOlZ6c1mafyNJlwNyDJEb4lND56Z8ec6gNv6Lix0Vd8B3 lGFQ== X-Gm-Message-State: APt69E0pZU2DgnLz9z49v7hkcSH2JniFmvIpwzZImjLK7zAPWNLIDU0o 6MhOuIze1IcJ7cQ+4SoNz/8= X-Google-Smtp-Source: ADUXVKIayi0VFfgsJsQ0bvn9e+qPXc1jb51fVAqm7rZ52Cvg2sPhm51AhJrZtFXTA/t9ZQfd6DNaWg== X-Received: by 2002:a2e:9887:: with SMTP id b7-v6mr6874372ljj.80.1530187936982; Thu, 28 Jun 2018 05:12:16 -0700 (PDT) In-Reply-To: X-Mailer: Apple Mail (2.1085) X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::243 X-BeenThere: bug-bash@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Bug reports for the GNU Bourne Again SHell List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Xref: csiph.com gnu.bash.bug:14285 --Apple-Mail-195--22564486 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii hello, I wanted to play around with this idea, and see how it works. Quick and = dirty patch (against devel@c181950a89c0f0ca4a3ae2480b783da2f7de9565) is = attached. Just modified parser and hardcoded tempfile = /tmp/v_save.bash.temp. result: bash-5.0$ echo 'hello a' >>> a; echo $a hello a=20 If you don't mind we can get it off the list, and see if we can workout = a better way to bind data from redirection with variable, and come up = with something. If not, maybe you will find something useful in this = patch, I have no idea if I will be able to present any decent version of = it. cheers, pg =20 --Apple-Mail-195--22564486 Content-Disposition: attachment; filename=store_result_in_varible_quick_and_dirty.diff.patch Content-Type: application/octet-stream; name="store_result_in_varible_quick_and_dirty.diff.patch" Content-Transfer-Encoding: quoted-printable =46rom=20bbe8e85b77e3942224857bfe4516628e90ba852d=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Piotr=20Grzybowski=20=0A= Date:=20Thu,=2028=20Jun=202018=2013:56:15=20+0200=0ASubject:=20[PATCH]=20= quick=20and=20dirty:=20store=20result=20in=20variable=0A=0A---=0A=20= command.h=20=20=20=20=20|=20=203=20++-=0A=20execute_cmd.c=20|=2052=20= ++++++++++++++++++++++++++++++++++++++++++++++++++++=0A=20make_cmd.c=20=20= =20=20|=20=201=20+=0A=20parse.y=20=20=20=20=20=20=20|=2016=20= ++++++++++++++--=0A=20redir.c=20=20=20=20=20=20=20|=20=207=20++++++-=0A=20= 5=20files=20changed,=2075=20insertions(+),=204=20deletions(-)=0A=0Adiff=20= --git=20a/command.h=20b/command.h=0Aindex=208562644..7519123=20100644=0A= ---=20a/command.h=0A+++=20b/command.h=0A@@=20-32,7=20+32,7=20@@=20enum=20= r_instruction=20{=0A=20=20=20r_close_this,=20r_err_and_out,=20= r_input_output,=20r_output_force,=0A=20=20=20r_duplicating_input_word,=20= r_duplicating_output_word,=0A=20=20=20r_move_input,=20r_move_output,=20= r_move_input_word,=20r_move_output_word,=0A-=20=20r_append_err_and_out=0A= +=20=20r_append_err_and_out,=20r_store_to_variable=0A=20};=0A=20=0A=20/*=20= Redirection=20flags;=20values=20for=20rflags=20*/=0A@@=20-149,6=20+149,7=20= @@=20typedef=20struct=20word_list=20{=0A=20typedef=20union=20{=0A=20=20=20= int=20dest;=09=09=09/*=20Place=20to=20redirect=20REDIRECTOR=20to,=20or=20= ...=20*/=0A=20=20=20WORD_DESC=20*filename;=09=09/*=20filename=20to=20= redirect=20to.=20*/=0A+=20=20WORD_DESC=20*variable_name;=09=09/*=20= variable=20to=20redirect=20to.=20*/=0A=20}=20REDIRECTEE;=0A=20=0A=20/*=20= Structure=20describing=20a=20redirection.=20=20If=20REDIRECTOR=20is=20= negative,=20the=20parser=0Adiff=20--git=20a/execute_cmd.c=20= b/execute_cmd.c=0Aindex=20126a54a..7ece25e=20100644=0A---=20= a/execute_cmd.c=0A+++=20b/execute_cmd.c=0A@@=20-5112,6=20+5112,48=20@@=20= execute_subshell_builtin_or_function=20(words,=20redirects,=20builtin,=20= var,=0A=20=20=20=20=20}=0A=20}=0A=20=0A+static=20long=20= get_file_size(const=20char=20*file_name)=20{=0A+=20int=20rc;=0A+=20= struct=20stat=20stats;=0A+=0A+=20if=20(file_name=3D=3DNULL)=20{=0A+=20=20= return=20-1;=0A+=20}=0A+=20rc=3Dstat(file_name,=20&stats);=0A+=20if=20= (rc=3D=3D0)=20{=0A+=20=20return=20stats.st_size;=0A+=20}=0A+=20return=20= -1;=0A+}=0A+=0A+static=20char=20*read_file(char=20*file_name)=20{=0A+=20= int=20rc;=0A+=20int=20fd;=0A+=20char=20*b;=0A+=20long=20= s=3Dget_file_size(file_name);=0A+=0A+=20if=20(s<=3D0)=20{=0A+=20=20= return=20NULL;=0A+=20}=0A+=0A+=20b=3Dxmalloc(s);=0A+=20if=20(b=3D=3DNULL)=20= {=0A+=20=20return=20NULL;=0A+=20}=0A+=0A+=20fd=3Dopen(file_name,=20= O_RDONLY);=0A+=20if=20(fd<0)=20{=0A+=20=20return=20NULL;=0A+=20}=0A+=20= rc=3Dread(fd,=20b,=20s);=0A+=20if=20(rc<0)=20{=0A+=20=20close(fd);=0A+=20= =20return=20NULL;=0A+=20}=0A+=20close(fd);=0A+=20return=20b;=0A+}=0A+=0A=20= /*=20Execute=20a=20builtin=20or=20function=20in=20the=20current=20shell=20= context.=20=20If=20BUILTIN=0A=20=20=20=20is=20non-null,=20it=20is=20the=20= builtin=20command=20to=20execute,=20otherwise=20VAR=20points=0A=20=20=20=20= to=20the=20body=20of=20a=20function.=20=20WORDS=20are=20the=20command's=20= arguments,=20REDIRECTS=0A@@=20-5131,12=20+5173,18=20@@=20= execute_builtin_or_function=20(words,=20builtin,=20var,=20redirects,=0A=20= =20=20=20=20=20int=20flags;=0A=20{=0A=20=20=20int=20result;=0A+=20=20= SHELL_VAR=20*result_variable=3DNULL;=0A=20=20=20REDIRECT=20= *saved_undo_list;=0A=20#if=20defined=20(PROCESS_SUBSTITUTION)=0A=20=20=20= int=20ofifo,=20nfifo,=20osize;=0A=20=20=20char=20*ofifo_list;=0A=20= #endif=0A=20=0A+=20=20if=20= (redirects&&redirects->instruction=3D=3Dr_store_to_variable)=20{=0A+=20=20= =20//expand_word(s)=20on=20redirects->redirectee.variable_name->word=20?=0A= +=20=20=20= result_variable=3Dbind_variable(redirects->redirectee.variable_name->word,= "result",0);=0A+=20=20}=0A+=0A=20#if=20defined=20(PROCESS_SUBSTITUTION)=0A= =20=20=20begin_unwind_frame=20("saved_fifos");=0A=20=20=20/*=20If=20we=20= return,=20we=20longjmp=20and=20don't=20get=20a=20chance=20to=20restore=20= the=20old=0A@@=20-5182,6=20+5230,10=20@@=20execute_builtin_or_function=20= (words,=20builtin,=20var,=20redirects,=0A=20=20=20else=0A=20=20=20=20=20= result=20=3D=20execute_function=20(var,=20words,=20flags,=20= fds_to_close,=200,=200);=0A=20=0A+=20=20if=20(result_variable)=20{=0A+=20= =20=20= bind_variable(result_variable->name,read_file("/tmp/v_save.bash.temp"),0);= =0A+=20=20}=0A+=0A=20=20=20/*=20We=20do=20this=20before=20undoing=20the=20= effects=20of=20any=20redirections.=20*/=0A=20=20=20fflush=20(stdout);=0A=20= =20=20fpurge=20(stdout);=0Adiff=20--git=20a/make_cmd.c=20b/make_cmd.c=0A= index=20ecbbfd6..9578f44=20100644=0A---=20a/make_cmd.c=0A+++=20= b/make_cmd.c=0A@@=20-697,6=20+697,7=20@@=20make_redirection=20(source,=20= instruction,=20dest_and_filename,=20flags)=0A=20=20=20switch=20= (instruction)=0A=20=20=20=20=20{=0A=20=0A+=20=20=20=20case=20= r_store_to_variable:=20/*=20>>>=20word=20*/=0A=20=20=20=20=20case=20= r_output_direction:=09=09/*=20>foo=20*/=0A=20=20=20=20=20case=20= r_output_force:=09=09/*=20>|=20foo=20*/=0A=20=20=20=20=20case=20= r_err_and_out:=09=09=09/*=20&>filename=20*/=0Adiff=20--git=20a/parse.y=20= b/parse.y=0Aindex=206457782..b15ec3b=20100644=0A---=20a/parse.y=0A+++=20= b/parse.y=0A@@=20-348,7=20+348,7=20@@=20static=20FILE=20*yyerrstream;=0A=20= %token=20=20NUMBER=0A=20%token=20=20ARITH_CMD=20= ARITH_FOR_EXPRS=0A=20%token=20=20COND_CMD=0A-%token=20AND_AND=20= OR_OR=20GREATER_GREATER=20LESS_LESS=20LESS_AND=20LESS_LESS_LESS=0A= +%token=20AND_AND=20OR_OR=20GREATER_GREATER=20LESS_LESS=20LESS_AND=20= LESS_LESS_LESS=20GREATER_GREATER_GREATER=0A=20%token=20GREATER_AND=20= SEMI_SEMI=20SEMI_AND=20SEMI_SEMI_AND=0A=20%token=20LESS_LESS_MINUS=20= AND_GREATER=20AND_GREATER_GREATER=20LESS_GREATER=0A=20%token=20= GREATER_BAR=20BAR_AND=0A@@=20-583,6=20+583,13=20@@=20redirection:=09'>'=20= WORD=0A=20=09=09=09=20=20redir.filename=20=3D=20$2;=0A=20=09=09=09=20=20= $$=20=3D=20make_redirection=20(source,=20r_reading_string,=20redir,=20= 0);=0A=20=09=09=09}=0A+=09|=09GREATER_GREATER_GREATER=20WORD=0A+=09=09=09= {=0A+=09=09=09=20source.dest=20=3D=201;=0A+=09=09=09=20redir.filename=20= =3D=20NULL;=0A+=20=20=20=20redir.variable_name=20=3D=20$2;=0A+=09=09=09=20= $$=20=3D=20make_redirection=20(source,=20r_store_to_variable,=20redir,=20= 0);=0A+=09=09=09}=0A=20=09|=09NUMBER=20LESS_LESS_LESS=20WORD=0A=20=09=09=09= {=0A=20=09=09=09=20=20source.dest=20=3D=20$1;=0A@@=20-2180,6=20+2187,7=20= @@=20STRING_INT_ALIST=20other_token_alist[]=20=3D=20{=0A=20=20=20{=20= ";;&",=20SEMI_SEMI_AND=20},=0A=20=20=20{=20"<<-",=20LESS_LESS_MINUS=20},=0A= =20=20=20{=20"<<<",=20LESS_LESS_LESS=20},=0A+=20=20{=20">>>",=20= GREATER_GREATER_GREATER=20},=0A=20=20=20{=20"&>",=20AND_GREATER=20},=0A=20= =20=20{=20"&>>",=20AND_GREATER_GREATER=20},=0A=20=20=20{=20"<>",=20= LESS_GREATER=20},=0A@@=20-2807,7=20+2815,7=20@@=20static=20int=20= open_brace_count;=0A=20=20=20(token=20=3D=3D=20'<'=20||=20token=20=3D=3D=20= '>'=20||=20\=0A=20=20=20=20token=20=3D=3D=20GREATER_GREATER=20||=20token=20= =3D=3D=20GREATER_BAR=20||=20\=0A=20=20=20=20token=20=3D=3D=20= LESS_GREATER=20||=20token=20=3D=3D=20LESS_LESS_MINUS=20||=20\=0A-=20=20=20= token=20=3D=3D=20LESS_LESS=20||=20token=20=3D=3D=20LESS_LESS_LESS=20||=20= \=0A+=20=20=20token=20=3D=3D=20LESS_LESS=20||=20token=20=3D=3D=20= LESS_LESS_LESS=20||=20token=20=3D=3D=20GREATER_GREATER_GREATER=20||=20\=0A= =20=20=20=20token=20=3D=3D=20LESS_AND=20||=20token=20=3D=3D=20= GREATER_AND=20||=20token=20=3D=3D=20AND_GREATER)=0A=20=0A=20/*=20Is=20= `token'=20one=20that=20will=20allow=20a=20WORD=20to=20be=20read=20in=20a=20= command=20position?=0A@@=20-3307,6=20+3315,10=20@@=20itrace("shell_getc:=20= bash_input.location.string=20=3D=20`%s'",=20bash_input.location.stri=0A=20= =09=09}=0A=20=0A=20=09=20=20=20=20case=20'>':=0A+=20=20=20=20=20=20=20= peek_char=20=3D=20shell_getc=20(1);=0A+=20=20=20=20=20=20=20if=20= MBTEST(peek_char=20=3D=3D=20'>')=20{=0A+=20=20=20=20=20=20=20=20return=20= (GREATER_GREATER_GREATER);=0A+=20=20=20=20=20=20=20}=0A=20=09=20=20=20=20= =20=20return=20(GREATER_GREATER);=0A=20=0A=20=09=20=20=20=20case=20';':=0A= diff=20--git=20a/redir.c=20b/redir.c=0Aindex=20f5276dc..6c49d1e=20100644=0A= ---=20a/redir.c=0A+++=20b/redir.c=0A@@=20-729,7=20+729,11=20@@=20= do_redirection_internal=20(redirect,=20flags)=0A=20=20=20REDIRECT=20= *new_redirect;=0A=20=20=20REDIRECTEE=20sd;=0A=20=0A-=20=20redirectee=20=3D= =20redirect->redirectee.filename;=0A+=20=20if=20= (redirect->instruction=3D=3Dr_store_to_variable)=20{=0A+=20=20=20= redirectee=20=3D=20make_bare_word("/tmp/v_save.bash.temp");=0A+=20=20}=20= else=20{=0A+=20=20=20redirectee=20=3D=20redirect->redirectee.filename;=0A= +=20=20}=0A=20=20=20redir_fd=20=3D=20redirect->redirectee.dest;=0A=20=20=20= redirector=20=3D=20redirect->redirector.dest;=0A=20=20=20ri=20=3D=20= redirect->instruction;=0A@@=20-823,6=20+827,7=20@@=20= do_redirection_internal=20(redirect,=20flags)=0A=20=0A=20=20=20switch=20= (ri)=0A=20=20=20=20=20{=0A+=20=20=20=20case=20r_store_to_variable:=0A=20=20= =20=20=20case=20r_output_direction:=0A=20=20=20=20=20case=20= r_appending_to:=0A=20=20=20=20=20case=20r_input_direction:=0A--=20=0A= 2.10.0=0A=0A= --Apple-Mail-195--22564486 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii On 27 Jun 2018, at 21:03, tetsujin@scope-eye.net wrote: > > > When I initially read this thread, I was concerned about the idea of > adding yet another mutation of the redirect syntax. > Like how far does this go? Would we introduce a ">>>>" someday for > some other bit of functionality? > > Ideally, I think it would be better if this could be done with pipe > syntax and builtins > (so that instead of introducing another redirect syntax for "capture > to variable", > we instead pipe to the already-existing "read" command) > This would also allow for using the built-ins to specify options like > delimiters, how much to read, allow for reading > into an array, and so on, and without diving deeper into increasingly > exotic syntax. > > However adding support for connecting separate pipes to stdout and > stderr and a generalized version of "lastpipe" > that can handle multiple builtins within the same job would be a > pretty big undertaking. It'd probably mean threading > and thread-safety, and of course some strategy for tying that into job > control. > > For what it's worth, I think I have a real use case for this proposed > syntax. > I'm working on loadable modules to do things like open Unix Domain > Sockets and so on. > The general form of these things (so far) is open the file > descriptors, and then store the FD numbers in a variable. > I think a better pattern would be "open the file descriptors, and > write their FD numbers to stdout" - and then capture them. > Of course in Bash this wouldn't work via command substitution: > > fd=$(connect_to_socket "$socket_path") # Command subst. is a > subshell, so file is opened but lost > > However, with a "capture-to-variable" redirect, this pattern would > work: > > connect_to_socket "$socket_path" >>> socket_fd_var > > More generally, it solves the problem of how to capture output from a > command that must also modify the > shell's environment. > > While it's essentially syntactic sugar (and despite my concerns about > further overloading the redirect operators) > I think I'm starting to see the value in it. I can't promise > anything at this point but I'd like to see what I can come > up with. > ----- Original Message ----- > From: chet.ramey@case.edu > To:"Peter Passchier" , "Martijn Dekker" > , > Cc: > Sent:Mon, 25 Jun 2018 09:11:45 -0400 > Subject:Re: Directing into a variable doesn't work > > On 6/24/18 11:26 AM, Peter Passchier wrote: >> Thank you for the feedback, very insightful. Yes, scratch that > first >> 'example'. Yay for the here-variable redirection! > > The answer is ultimately the same as it was last month: > > http://lists.gnu.org/archive/html/bug-bash/2018-05/msg00056.html > > (from link:) > > What you're asking for is syntactic sugar for: > > some-command > temp-file > echo '#' >> temp-file > variablename=$(< temp-file) > rm -f temp-file > variablename=${variablename%?} > > I would look at a sample implementation, possibly using mmap, if > someone > did one. > > --Apple-Mail-195--22564486--