Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #164227

Re: pipe a buffer to a program's stdin using popen

Show all headers | View raw

On 2022-01-02, John Forkosh <forkosh@panix.com> wrote:
> Kaz Kylheku <480-992-1380@kylheku.com> wrote:
>> On 2022-01-01, John Forkosh <forkosh@panix.com> wrote:
>>> Suppose I have a program that, in part, runs another program
>>> using FILE *outptr=popen("some command string","r") and then
>>> fread()'s its stdout from outptr. Now, if that other program
>>> is otherpgm which reads input from its stdin, and I also want
>>> that input to come from a file inputfile, then I could easily
>>> just write  FILE *outptr=popen("cat inputfile|otherpgm","r");
>> 
>> That would be a useless use of cat; it could be done with:
>>   FILE *outptr=popen("otherpgm < inputfile","r");
>
> Thanks, Kaz:
> Yeah, I'm not actually doing either; the chosen illustration
> was basically just pipes versus redirection, and since I was
> kind of guessing the ultimate answer more likely involves pipes,
> I illustrated it along those lines.
>
>>> But here's the rub: rather than some inputfile on disk, my
>>> program has an internal  unsigned char buffer[9999]  which is
>>> what I want piped to otherpgm, but which I don't want written
>>> to disk first. How can I get buffer piped to otherpgm's stdin
>>> using popen()? Or some other mechanism, as long as I can read
>>> otherpgm's stdout from within my program.
>
>> A program to which you send input while reading its output
>> is a coprocess. The popen function is for simple uses; it doesn't
>> support coprocesses. You have to cob those together yourself
>> with fork, exec, and various other functions.
>
> Yeah, I've done a little, but not much, of that. And, as
> suggested in a preceding followup, I actually have a hardback
> paper copy of Stevens' APUE, which I occasionally refer to.
> But I couldn't quite figure out what to do from his Chapters
> 14 and 15, assuming that's where an answer's buried.
>
>> With a coprocess, there is the risk of deadlock. It happens
>> like this: you write output for the process to read, while
>> it concurrently reads it and processes output. Because you're
>> writing, you're not reading. The process blocks trying to
>> write the data to you, and therefore stops reading. Since it
>> stops reading, you block writing to it.
>
> Not a problem in my case. The 'otherpgm' reads all its input
> before it writes anything.

That only exacerbates deadlock.

>
>> In the general case, coprocesses require non-blocking I/O,
>> polling for input and output simultaneously, or else threads.
>> When the amount of data exchanged is small (so small that you can you
>> can rely on it all fitting into the pipe size on the given system)
>> deadlock may be avoided that way.
>> 
>> In the case of just wanting to feed a canned piece of data to
>> the process from memory, while reading its output, we can use
>> fork() to create a process which will do the feeding.
>
> Feed how, exactly??? That had occurred to me (at least if I

The feeding process (forked child) just sits in a loop sending the bytes
to one end of a pipe. That pipe is installed as the standard input
of the coprocess.

> correctly understand what you're saying), but as far as I could
> tell that forked child is in exactly the same predicament as the
> parent was in my original question: how does it pipe (or redirect
> or whatever) a large unsigned char buffer[9999] (actually
> malloc/realloc'ed, often larger than 20MB) from its memory to
> the stdin of that 'otherpgm'?

Your shell can do it:

   cat_memory()
   {
      echo "string in shell's memory"
   }

   upcased=$(cat_memory | tr '[a-z]' '[A-Z]')

   # upcased is now "STRING IN SHELL'S MEMORY"

The shell forks a process which calls the function (i.e. does not exec
anything external). And it forks a process for the "tr" command. As it's
doing that, it hooks up the pipes in all the right ways and so it goes.

The shell itself goes into a loop reading the output of the pipe,
so that it can gather it into a string via the $(...) syntax, and then
assign that to a variable when it's done.

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

pipe a buffer to a program's stdin using popen John Forkosh <forkosh@panix.com> - 2022-01-01 09:35 +0000
  Re: pipe a buffer to a program's stdin using popen Mark Bluemel <mark.bluemel@gmail.com> - 2022-01-01 03:22 -0800
    Re: pipe a buffer to a program's stdin using popen Meredith Montgomery <mmontgomery@levado.to> - 2022-01-01 11:21 -0300
    Re: pipe a buffer to a program's stdin using popen John Forkosh <forkosh@panix.com> - 2022-01-02 09:34 +0000
  Re: pipe a buffer to a program's stdin using popen Kaz Kylheku <480-992-1380@kylheku.com> - 2022-01-01 19:51 +0000
    Re: pipe a buffer to a program's stdin using popen gazelle@shell.xmission.com (Kenny McCormack) - 2022-01-01 21:00 +0000
      Re: pipe a buffer to a program's stdin using popen John Forkosh <forkosh@panix.com> - 2022-01-02 09:12 +0000
    Re: pipe a buffer to a program's stdin using popen John Forkosh <forkosh@panix.com> - 2022-01-02 09:30 +0000
      Re: pipe a buffer to a program's stdin using popen Kaz Kylheku <480-992-1380@kylheku.com> - 2022-01-03 00:03 +0000
        Re: pipe a buffer to a program's stdin using popen John Forkosh <forkosh@panix.com> - 2022-01-03 08:07 +0000
          Re: pipe a buffer to a program's stdin using popen Manfred <noname@add.invalid> - 2022-01-03 22:54 +0100
  Re: pipe a buffer to a program's stdin using popen Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2022-01-01 15:00 -0800
    Re: pipe a buffer to a program's stdin using popen gazelle@shell.xmission.com (Kenny McCormack) - 2022-01-02 00:01 +0000
  Re: pipe a buffer to a program's stdin using popen Lew Pitcher <lew.pitcher@digitalfreehold.ca> - 2022-01-02 12:54 +0000
    Re: pipe a buffer to a program's stdin using popen gazelle@shell.xmission.com (Kenny McCormack) - 2022-01-02 14:54 +0000
      Re: pipe a buffer to a program's stdin using popen Lew Pitcher <lew.pitcher@digitalfreehold.ca> - 2022-01-02 15:37 +0000
      Re: pipe a buffer to a program's stdin using popen John Forkosh <forkosh@panix.com> - 2022-01-02 18:00 +0000
        Re: pipe a buffer to a program's stdin using popen gazelle@shell.xmission.com (Kenny McCormack) - 2022-01-02 18:03 +0000
    Re: pipe a buffer to a program's stdin using popen John Forkosh <forkosh@panix.com> - 2022-01-02 17:51 +0000

csiph-web