Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #164216

Re: pipe a buffer to a program's stdin using popen

Show all headers | View raw

Kaz Kylheku <480-992-1380@kylheku.com> wrote:
> On 2022-01-01, John Forkosh <forkosh@panix.com> wrote:
>> Suppose I have a program that, in part, runs another program
>> using FILE *outptr=popen("some command string","r") and then
>> fread()'s its stdout from outptr. Now, if that other program
>> is otherpgm which reads input from its stdin, and I also want
>> that input to come from a file inputfile, then I could easily
>> just write  FILE *outptr=popen("cat inputfile|otherpgm","r");
> 
> That would be a useless use of cat; it could be done with:
>   FILE *outptr=popen("otherpgm < inputfile","r");

Thanks, Kaz:
Yeah, I'm not actually doing either; the chosen illustration
was basically just pipes versus redirection, and since I was
kind of guessing the ultimate answer more likely involves pipes,
I illustrated it along those lines.

>> But here's the rub: rather than some inputfile on disk, my
>> program has an internal  unsigned char buffer[9999]  which is
>> what I want piped to otherpgm, but which I don't want written
>> to disk first. How can I get buffer piped to otherpgm's stdin
>> using popen()? Or some other mechanism, as long as I can read
>> otherpgm's stdout from within my program.

> A program to which you send input while reading its output
> is a coprocess. The popen function is for simple uses; it doesn't
> support coprocesses. You have to cob those together yourself
> with fork, exec, and various other functions.

Yeah, I've done a little, but not much, of that. And, as
suggested in a preceding followup, I actually have a hardback
paper copy of Stevens' APUE, which I occasionally refer to.
But I couldn't quite figure out what to do from his Chapters
14 and 15, assuming that's where an answer's buried.

> With a coprocess, there is the risk of deadlock. It happens
> like this: you write output for the process to read, while
> it concurrently reads it and processes output. Because you're
> writing, you're not reading. The process blocks trying to
> write the data to you, and therefore stops reading. Since it
> stops reading, you block writing to it.

Not a problem in my case. The 'otherpgm' reads all its input
before it writes anything.

> In the general case, coprocesses require non-blocking I/O,
> polling for input and output simultaneously, or else threads.
> When the amount of data exchanged is small (so small that you can you
> can rely on it all fitting into the pipe size on the given system)
> deadlock may be avoided that way.
> 
> In the case of just wanting to feed a canned piece of data to
> the process from memory, while reading its output, we can use
> fork() to create a process which will do the feeding.

Feed how, exactly??? That had occurred to me (at least if I
correctly understand what you're saying), but as far as I could
tell that forked child is in exactly the same predicament as the
parent was in my original question: how does it pipe (or redirect
or whatever) a large unsigned char buffer[9999] (actually
malloc/realloc'ed, often larger than 20MB) from its memory to
the stdin of that 'otherpgm'?

> We let that run in the background. (Thus we have a multi-threaded
> solution via multi-process, and do not need to juggle non-blocking
> I/O and polling.)
> 
> Just like in your original popen above, you have a "cat inputfile |
> otherpgm" where cat is useless, a forked process is instantiated which
> "cats" the memory buffer.  That "memory cat" is not useless because a
> piece of memory will not serve itself as a file.

-- 
John Forkosh  ( mailto:  j@f.com  where j=john and f=forkosh )

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

pipe a buffer to a program's stdin using popen John Forkosh <forkosh@panix.com> - 2022-01-01 09:35 +0000
  Re: pipe a buffer to a program's stdin using popen Mark Bluemel <mark.bluemel@gmail.com> - 2022-01-01 03:22 -0800
    Re: pipe a buffer to a program's stdin using popen Meredith Montgomery <mmontgomery@levado.to> - 2022-01-01 11:21 -0300
    Re: pipe a buffer to a program's stdin using popen John Forkosh <forkosh@panix.com> - 2022-01-02 09:34 +0000
  Re: pipe a buffer to a program's stdin using popen Kaz Kylheku <480-992-1380@kylheku.com> - 2022-01-01 19:51 +0000
    Re: pipe a buffer to a program's stdin using popen gazelle@shell.xmission.com (Kenny McCormack) - 2022-01-01 21:00 +0000
      Re: pipe a buffer to a program's stdin using popen John Forkosh <forkosh@panix.com> - 2022-01-02 09:12 +0000
    Re: pipe a buffer to a program's stdin using popen John Forkosh <forkosh@panix.com> - 2022-01-02 09:30 +0000
      Re: pipe a buffer to a program's stdin using popen Kaz Kylheku <480-992-1380@kylheku.com> - 2022-01-03 00:03 +0000
        Re: pipe a buffer to a program's stdin using popen John Forkosh <forkosh@panix.com> - 2022-01-03 08:07 +0000
          Re: pipe a buffer to a program's stdin using popen Manfred <noname@add.invalid> - 2022-01-03 22:54 +0100
  Re: pipe a buffer to a program's stdin using popen Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2022-01-01 15:00 -0800
    Re: pipe a buffer to a program's stdin using popen gazelle@shell.xmission.com (Kenny McCormack) - 2022-01-02 00:01 +0000
  Re: pipe a buffer to a program's stdin using popen Lew Pitcher <lew.pitcher@digitalfreehold.ca> - 2022-01-02 12:54 +0000
    Re: pipe a buffer to a program's stdin using popen gazelle@shell.xmission.com (Kenny McCormack) - 2022-01-02 14:54 +0000
      Re: pipe a buffer to a program's stdin using popen Lew Pitcher <lew.pitcher@digitalfreehold.ca> - 2022-01-02 15:37 +0000
      Re: pipe a buffer to a program's stdin using popen John Forkosh <forkosh@panix.com> - 2022-01-02 18:00 +0000
        Re: pipe a buffer to a program's stdin using popen gazelle@shell.xmission.com (Kenny McCormack) - 2022-01-02 18:03 +0000
    Re: pipe a buffer to a program's stdin using popen John Forkosh <forkosh@panix.com> - 2022-01-02 17:51 +0000

csiph-web