Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.ruby > #2377 > unrolled thread

capture the output of a grandchild

Back to article view | Back to comp.lang.ruby

Contents

  capture the output of a grandchild Chandan Bansal <chandan89@hotmail.com> - 2011-04-06 03:35 -0500
    Re: capture the output of a grandchild Brian Candler <b.candler@pobox.com> - 2011-04-06 03:43 -0500
      Re: capture the output of a grandchild Brian Candler <b.candler@pobox.com> - 2011-04-06 07:33 -0500
        Re: capture the output of a grandchild Chandan Bansal <chandan89@hotmail.com> - 2011-04-07 07:09 -0500
          Re: capture the output of a grandchild Brian Candler <b.candler@pobox.com> - 2011-04-07 07:24 -0500
            Re: capture the output of a grandchild Chandan Bansal <chandan89@hotmail.com> - 2011-04-08 00:40 -0500
              Sum of arrays getting slower as size increments Vicente Bosch Campos <vbosch@gmail.com> - 2011-04-08 01:48 -0500
                Re: Sum of arrays getting slower as size increments Vicente Bosch Campos <vbosch@gmail.com> - 2011-04-08 01:58 -0500
                Re: Sum of arrays getting slower as size increments Jesús Gabriel y Galán <jgabrielygalan@gmail.com> - 2011-04-08 02:00 -0500
                  Re: Sum of arrays getting slower as size increments Vicente Bosch Campos <vbosch@gmail.com> - 2011-04-08 02:06 -0500
                  Re: Sum of arrays getting slower as size increments Vicente Bosch Campos <vbosch@gmail.com> - 2011-04-08 02:10 -0500
    Re: capture the output of a grandchild Chandan Bansal <chandan89@hotmail.com> - 2011-04-07 06:49 -0500

#2377 — capture the output of a grandchild

hi

here is my problem :
  i started a process A through my ruby script using system() on my unix
os .The process A then started a process B. I want to capture the output
of the process B provided I cannot modify the source code of process A
and process B

I tried using named pipes and `` to capture the output but its not
working


thanks in advance  .

-- 
Posted via http://www.ruby-forum.com/.

[toc] | [next] | [standalone]

#2378

Chandan Bansal wrote in post #991188:
> I tried using named pipes and `` to capture the output but its not
> working

Show your code. Show what B and C are doing, or write replacements in 
ruby to demonstrate.

If you use IO.popen(...) then you will capture stdout of the child; if 
that child forks a grandchild, which inherits the same stdout, then you 
will capture that as well.

Maybe the output which you're failing to capture is on stderr not 
stdout. If so, look at open3.rb in the standard library; or do

IO.popen("/path/to/prog 2>&1") { |io| ... }

-- 
Posted via http://www.ruby-forum.com/.

[toc] | [prev] | [next] | [standalone]

#2382

Here's a simple test you can do:

ruby -e 'system("/path/to/prog")' >cap.out 2>cap.err

Does the output from the grandchild appear in cap.out or cap.err?

-- 
Posted via http://www.ruby-forum.com/.

[toc] | [prev] | [next] | [standalone]

#2450

Brian Candler wrote in post #991223:
> Here's a simple test you can do:
>
> ruby -e 'system("/path/to/prog")' >cap.out 2>cap.err
>
> Does the output from the grandchild appear in cap.out or cap.err?

thanks dude it worked finally

i called it from my ruby script like

system ("ruby -e \'system(\"/path/to/prog\")\' >output.file 2>&1")

-- 
Posted via http://www.ruby-forum.com/.

[toc] | [prev] | [next] | [standalone]

#2451

Chandan Bansal wrote in post #991446:
> thanks dude it worked finally
>
> i called it from my ruby script like
>
> system ("ruby -e \'system(\"/path/to/prog\")\' >output.file 2>&1")

In that case, all you should need is:

system("/path/to/prog >output.file 2>&1")

And if you don't want a temporary file, then you can use IO.popen 
instead.

Regards,

Brian.

-- 
Posted via http://www.ruby-forum.com/.

[toc] | [prev] | [next] | [standalone]

#2506

Brian Candler wrote in post #991449:
> In that case, all you should need is:
>
> system("/path/to/prog >output.file 2>&1")


ya u are right, i tried

system("(cd path/to/perl && ./myscript.pl) > output.file 2>&1")

the program that i am calling from my ruby script is a perl script

but there is one more problem, i am using this ruby script with screens
and this system function call just happens to capture output from some 
other screens as well.. dunno whats the reason..


Thanks,
Chandan.

-- 
Posted via http://www.ruby-forum.com/.

[toc] | [prev] | [next] | [standalone]

#2508 — Sum of arrays getting slower as size increments

Hi,

I am currently doing some code for a face detection algorithm. In order to generate the models I need to load 3 GB worth of data and do a K-means on it ( just to give some context of why I need all of the data loaded
at the same time).

At some point of the code I am retrieving a line of the file, turning the items into floats and dividing them in different set of arrays. 

Once a line is extracted and given the correct format I add that array to the one containing all lines

all_data += line_array

I have noticed that the + operation gets slower as the all_data array gets bigger but this should not be so at the end what the + operation is doing is inserting them at the back of the array and its always the same 
amount hence it should always take the same time. 

My ruby version:

MacBosch:~ vbosch$ ruby -v
ruby 1.9.2p180 (2011-02-18 revision 30909) [x86_64-darwin10.6.0]

Am I making a wrong assumption ? Is there a better performing way to do this ? 


I have coded following example to illustrate the point:

#! /usr/bin/env ruby

require 'gnuplot'
require 'benchmark'

b = Array.new  
x = Array.new
y = Array.new
  
1000.times do  
  
time=Benchmark.realtime{b+=Array.new(10000,1)}

puts "Length is #{b.length} last insertion took #{time}"

x.push(b.length)
y.push(time)

end

Gnuplot.open do |gp|
  Gnuplot::Plot.new( gp ) do |plot|

    plot.title  "Array insertion time vs Length"
    plot.ylabel "time"
    plot.xlabel "length"

    plot.data << Gnuplot::DataSet.new( [x, y] ) do |ds|
      ds.with = "lines"
      ds.notitle
    end
  end
end

[toc] | [prev] | [next] | [standalone]

#2510 — Re: Sum of arrays getting slower as size increments

After some much needed coffee  :-)   I changed from 

all_data += line_array

to 

all_data << line_array

Result is the same I have an array with the elements concatenated but so much faster. Solves my problem.

Anyone knows why + gets slower ? Because it makes a copy of the original ? 

Regards,
V.



On Apr 8, 2011, at 8:48 AM, Vicente Bosch Campos wrote:

> Hi,
> 
> I am currently doing some code for a face detection algorithm. In order to generate the models I need to load 3 GB worth of data and do a K-means on it ( just to give some context of why I need all of the data loaded
> at the same time).
> 
> At some point of the code I am retrieving a line of the file, turning the items into floats and dividing them in different set of arrays. 
> 
> Once a line is extracted and given the correct format I add that array to the one containing all lines
> 
> all_data += line_array
> 
> I have noticed that the + operation gets slower as the all_data array gets bigger but this should not be so at the end what the + operation is doing is inserting them at the back of the array and its always the same 
> amount hence it should always take the same time. 
> 
> My ruby version:
> 
> MacBosch:~ vbosch$ ruby -v
> ruby 1.9.2p180 (2011-02-18 revision 30909) [x86_64-darwin10.6.0]
> 
> Am I making a wrong assumption ? Is there a better performing way to do this ? 
> 
> 
> I have coded following example to illustrate the point:
> 
> #! /usr/bin/env ruby
> 
> require 'gnuplot'
> require 'benchmark'
> 
> b = Array.new  
> x = Array.new
> y = Array.new
> 
> 1000.times do  
> 
> time=Benchmark.realtime{b+=Array.new(10000,1)}
> 
> puts "Length is #{b.length} last insertion took #{time}"
> 
> x.push(b.length)
> y.push(time)
> 
> end
> 
> Gnuplot.open do |gp|
>  Gnuplot::Plot.new( gp ) do |plot|
> 
>    plot.title  "Array insertion time vs Length"
>    plot.ylabel "time"
>    plot.xlabel "length"
> 
>    plot.data << Gnuplot::DataSet.new( [x, y] ) do |ds|
>      ds.with = "lines"
>      ds.notitle
>    end
>  end
> end

[toc] | [prev] | [next] | [standalone]

#2511 — Re: Sum of arrays getting slower as size increments

On Fri, Apr 8, 2011 at 8:48 AM, Vicente Bosch Campos <vbosch@gmail.com> wrote:
> Hi,
>
> I am currently doing some code for a face detection algorithm. In order to generate the models I need to load 3 GB worth of data and do a K-means on it ( just to give some context of why I need all of the data loaded
> at the same time).
>
> At some point of the code I am retrieving a line of the file, turning the items into floats and dividing them in different set of arrays.
>
> Once a line is extracted and given the correct format I add that array to the one containing all lines
>
> all_data += line_array
>
> I have noticed that the + operation gets slower as the all_data array gets bigger but this should not be so at the end what the + operation is doing is inserting them at the back of the array and its always the same
> amount hence it should always take the same time.

The problem is that b += array is the same as b = b + array, which
creates a new array everytime, with the contents of both arrays. As b
gets bigger, creating this intermediate array gets more and more
expensive. You can use concat instead, so that you don't create
intermediate arrays, just modify b:

require 'benchmark'

TIMES = 500
Benchmark.bmbm do |x|
  x.report("+=") { b = [];  TIMES.times {b+=Array.new(10000,1)}}
  x.report("concat") { b = [];  TIMES.times {b.concat(Array.new(10000,1))}}
end



$ ruby bm_arrayconcat.rb
Rehearsal ------------------------------------------
+=      22.900000   3.500000  26.400000 ( 26.434147)
concat   1.010000   0.020000   1.030000 (  1.038697)
-------------------------------- total: 27.430000sec

             user     system      total        real
+=       5.090000   3.170000   8.260000 (  8.281054)
concat   0.520000   0.010000   0.530000 (  0.533721)

Hope this helps,

Jesus.

[toc] | [prev] | [next] | [standalone]

#2512 — Re: Sum of arrays getting slower as size increments

Thanks Jesús!!

On Apr 8, 2011, at 9:00 AM, Jesús Gabriel y Galán wrote:

> On Fri, Apr 8, 2011 at 8:48 AM, Vicente Bosch Campos <vbosch@gmail.com> wrote:
>> Hi,
>> 
>> I am currently doing some code for a face detection algorithm. In order to generate the models I need to load 3 GB worth of data and do a K-means on it ( just to give some context of why I need all of the data loaded
>> at the same time).
>> 
>> At some point of the code I am retrieving a line of the file, turning the items into floats and dividing them in different set of arrays.
>> 
>> Once a line is extracted and given the correct format I add that array to the one containing all lines
>> 
>> all_data += line_array
>> 
>> I have noticed that the + operation gets slower as the all_data array gets bigger but this should not be so at the end what the + operation is doing is inserting them at the back of the array and its always the same
>> amount hence it should always take the same time.
> 
> The problem is that b += array is the same as b = b + array, which
> creates a new array everytime, with the contents of both arrays. As b
> gets bigger, creating this intermediate array gets more and more
> expensive. You can use concat instead, so that you don't create
> intermediate arrays, just modify b:
> 
> require 'benchmark'
> 
> TIMES = 500
> Benchmark.bmbm do |x|
>  x.report("+=") { b = [];  TIMES.times {b+=Array.new(10000,1)}}
>  x.report("concat") { b = [];  TIMES.times {b.concat(Array.new(10000,1))}}
> end
> 
> 
> 
> $ ruby bm_arrayconcat.rb
> Rehearsal ------------------------------------------
> +=      22.900000   3.500000  26.400000 ( 26.434147)
> concat   1.010000   0.020000   1.030000 (  1.038697)
> -------------------------------- total: 27.430000sec
> 
>             user     system      total        real
> +=       5.090000   3.170000   8.260000 (  8.281054)
> concat   0.520000   0.010000   0.530000 (  0.533721)
> 
> Hope this helps,
> 
> Jesus.
> 

[toc] | [prev] | [next] | [standalone]

#2513 — Re: Sum of arrays getting slower as size increments

Your solution is much better than mine with << as that just adds the right hand side of the operation as an element and as I was adding an array of arrays the effect was not the intended.

I managed to solve it with a .flatten!(1) afterwards but its one more operation I do not need to do....

Thanks :-)

On Apr 8, 2011, at 9:00 AM, Jesús Gabriel y Galán wrote:

> On Fri, Apr 8, 2011 at 8:48 AM, Vicente Bosch Campos <vbosch@gmail.com> wrote:
>> Hi,
>> 
>> I am currently doing some code for a face detection algorithm. In order to generate the models I need to load 3 GB worth of data and do a K-means on it ( just to give some context of why I need all of the data loaded
>> at the same time).
>> 
>> At some point of the code I am retrieving a line of the file, turning the items into floats and dividing them in different set of arrays.
>> 
>> Once a line is extracted and given the correct format I add that array to the one containing all lines
>> 
>> all_data += line_array
>> 
>> I have noticed that the + operation gets slower as the all_data array gets bigger but this should not be so at the end what the + operation is doing is inserting them at the back of the array and its always the same
>> amount hence it should always take the same time.
> 
> The problem is that b += array is the same as b = b + array, which
> creates a new array everytime, with the contents of both arrays. As b
> gets bigger, creating this intermediate array gets more and more
> expensive. You can use concat instead, so that you don't create
> intermediate arrays, just modify b:
> 
> require 'benchmark'
> 
> TIMES = 500
> Benchmark.bmbm do |x|
>  x.report("+=") { b = [];  TIMES.times {b+=Array.new(10000,1)}}
>  x.report("concat") { b = [];  TIMES.times {b.concat(Array.new(10000,1))}}
> end
> 
> 
> 
> $ ruby bm_arrayconcat.rb
> Rehearsal ------------------------------------------
> +=      22.900000   3.500000  26.400000 ( 26.434147)
> concat   1.010000   0.020000   1.030000 (  1.038697)
> -------------------------------- total: 27.430000sec
> 
>             user     system      total        real
> +=       5.090000   3.170000   8.260000 (  8.281054)
> concat   0.520000   0.010000   0.530000 (  0.533721)
> 
> Hope this helps,
> 
> Jesus.
> 

[toc] | [prev] | [next] | [standalone]

#2448

thanks for the reply

ruby -e 'system("/path/to/prog")' >cap.out 2>cap.err

i tried this and yes the output from the grandchild was there in cap.out

-- 
Posted via http://www.ruby-forum.com/.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.ruby

csiph-web