Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.graphics.apps.gnuplot > #3244 > unrolled thread

chart disappears when i add xrange / yrange to binary array plot of radio spectrogram data

Back to article view | Back to comp.graphics.apps.gnuplot

Contents

  chart disappears when i add xrange / yrange to binary array plot of radio spectrogram data Mario Cannistrà <mariocannistra@gmail.com> - 2016-03-01 05:58 -0800
    Re: chart disappears when i add xrange / yrange to binary array plot of radio spectrogram data Mario Cannistrà <mariocannistra@gmail.com> - 2016-03-01 06:07 -0800
    Re: chart disappears when i add xrange / yrange to binary array plot of radio spectrogram data Hans-Bernhard Bröker <HBBroeker@t-online.de> - 2016-03-01 20:54 +0100
      Re: chart disappears when i add xrange / yrange to binary array plot of radio spectrogram data Mario Cannistrà <mariocannistra@gmail.com> - 2016-03-01 13:07 -0800
        Re: chart disappears when i add xrange / yrange to binary array plot of radio spectrogram data Hans-Bernhard Bröker <HBBroeker@t-online.de> - 2016-03-01 23:29 +0100
          Re: chart disappears when i add xrange / yrange to binary array plot of radio spectrogram data Mario Cannistrà <mariocannistra@gmail.com> - 2016-03-02 01:08 -0800
    Re: chart disappears when i add xrange / yrange to binary array plot of radio spectrogram data Karl Ratzsch <mail.kfr@gmx.net> - 2016-03-02 00:33 +0100
      Re: chart disappears when i add xrange / yrange to binary array plot of radio spectrogram data Mario Cannistrà <mariocannistra@gmail.com> - 2016-03-02 01:13 -0800

#3244 — chart disappears when i add xrange / yrange to binary array plot of radio spectrogram data

Hello.
I need to plot datafiles in binary format with radio spectrogram data.

They are created as a sequence of float values and I'm successfully reading them as array=8000x79   (for this test) where 8000 are the frequency bins and 79 are the scans made and recorded.
Real files will reach at least 100 MB size and this is why they are in binary format and not in csv. For the same reason I'm storing only z values and NO x,y (timestamps, frequencies).

Please find below the gnuplot file.

When I try to uncomment the xrange and yrange statements, i get the axis labels correctly represented but the data area of the plot simply disappears.

Please see  https://www.dropbox.com/s/n022h60lyvf57t1/plot-showing-spectrogram.png?dl=0  for the plot with data and  https://www.dropbox.com/s/c8c9yhoy1tkqbhq/plot-showing-labels.png?dl=0  for the plot with labels only.

The binary input file is here:  https://www.dropbox.com/s/qrd6wu779pxh2fg/UTC20160228082732-Jupiter-88.000M-96.000M-4000-n100-g22-e1m.bin?dl=0

The gnuplot script should be run with the following command line:

gnuplot -e "scanstart='160228082733'" -e "scanend='160228082833'" -e fstart=88000000 -e fend=95999000 -e fstep=1000 -e singlescandur=0.576923 -e "matorg='(8000,79)'" -e "outname='UTC20160228082732-Jupiter-88.000M-96.000M-4000-n100-g22-e1m'" -e globmin=0.000000 -e globmax=0.000000 plotbin.gnu

Could someone here in the group please help me out ?  I've been fighting against this problem for at least 2 days now...
Kind regards,
Mario


gnuplot script below (also available at  https://www.dropbox.com/s/q91nkiwmcvocbfu/plotbin.gnu?dl=0  )


#load './default.plt'
#load './parula.pal'
#set palette model RGB rgbformulae 7,5,15
#set palette model XYZ rgbformulae 7,5,15

#set palette rgb 7,5,15 #"traditional pm3d\n(black-blue-red-yellow)"; splot g(x)

#set palette rgb 3,11,6 #"green-red-violet"; splot g(x)

#set palette rgb 23,28,3 #"ocean (green-blue-white)\ntry also other permutations"; splot g(x)

set palette rgb 21,22,23 #"hot (black-red-yellow-white)"; splot g(x)

#set palette rgb 30,31,32 #"color printable on gray\n(black-blue-violet-yellow-white)"; splot g(x)
#set palette rgb 33,13,10 #"rainbow (blue-green-yellow-red)"; splot g(x)
#set palette rgb 34,35,36 #"AFM hot (black-red-yellow-white)"; splot g(x)
#set palette model HSV #
#set palette rgb 3,2,2 #"HSV model\n(red-yellow-green-cyan-blue-magenta-red)"; splot g(x)
#set pal gray #"gray palette"; splot g(x)

#set grid lc rgbcolor "#BBBBBB"

unset key
mytitle = "scan: ".outname

set title mytitle

set xlabel 'UTC time'
set ylabel 'Frequency (MHz)'
set cblabel 'Signal power'

set term png size 3661, 3000 font "arial,36"

set tics out

set autoscale fix

rangeonx = '["'.scanstart.'":"'.scanend.'"]'

set xdata time
set timefmt '%y%m%d%H%M%S'
set format x "%H:%M:%S"
#set xrange @rangeonx
set xtics 5 nomirror rotate

set format y "%.1s"
#set yrange [fstart:fend]
set ytics nomirror

set output outname.".png"

plot outname.".bin" binary array=@matorg rotation=-90d format='%float' with image

[toc] | [next] | [standalone]

#3245

forgot to mention that I'm using Gnuplot:

 Version 5.0 patchlevel 1    last modified 2015-06-07 

on Debian Jessie

[toc] | [prev] | [next] | [standalone]

#3246

Am 01.03.2016 um 14:58 schrieb Mario Cannistrà:

> For the same reason I'm storing only z values
> and NO x,y (timestamps, frequencies).

I suspect that's your problem right there.  Without the time stamps in 
the data, where are the x coordinates you want to limit the range to 
supposed to come from?

> When I try to uncomment the xrange and yrange statements, i get the
> axis labels correctly represented but the data area of the plot
> simply disappears.

That would be case, as-is, your data file does not actually hold any 
data inside the ranges you specified.

> Please see
> https://www.dropbox.com/s/n022h60lyvf57t1/plot-showing-spectrogram.png?dl=0
> for the plot with data and

If you take a good hard look at that, you may notice that the x axis is 
from timestamps 01:06:00 to about 01:07:18, on some unknown day.  That 
doesn't overlap with the range you pass to gnuplot:

> gnuplot -e "scanstart='160228082733'" -e "scanend='160228082833'"
[...]
> rangeonx = '["'.scanstart.'":"'.scanend.'"]'
[...]
> set format x "%H:%M:%S"
> set xrange @rangeonx

There are no data around 08:28 h in that file.

There is nothing in that plot script that would move the actual 
(implicit) data range of [0:7999] into the realm of time stamps you 
specify as your xrange.

Binary data in %float format wouldn't be able to hold the kind of 
resolution you're attempting, anyway: you're specifying a range of 60 
seconds, 16 years away from gnuplot's epoch of 2000-01-01.

	60 seconds / 16 years

only about twice as large as FLT_EPS, i.e. there are only three 
different single-precision %float numbers in that entire x range.

[toc] | [prev] | [next] | [standalone]

#3247

Thank you Hans-Bernhard for your kind answer.
If I correctly understand, xrange is actually a filter that I'm applying on x axis values (values that I should keep in the file, currently absent).

Actually I don't understand where the x values 01:06:00 to 01:07:18 are coming from, since i intentionally stored only the Z values in the file.

About my next tentative: should I change the binary file structure to contain all info (x,y,z = time, frequencies,z), which format should I use for time ? would be ok a 32 bit integer ? I remember of binary "matrix" format I've read about, would that be fine and able to contain integers for x,y and float for z ?

As an alternative to all that: would it be possible to just produce all the labels with some loop, avoiding usage of "xrange" so that i can get the plot with the current binary file structure? For me it's really important to keep file size at a minimum... that sample is 2.5 MB for 60 seconds of scan and I should work with about 20 files - 30 minutes each, something that i can plot in acceptable time (already tested).

Kind regards,
Mario

[toc] | [prev] | [next] | [standalone]

#3248

Am 01.03.2016 um 22:07 schrieb Mario Cannistrà:

> If I correctly understand, xrange is actually a filter that I'm
> applying on x axis values (values that I should keep in the file,
> currently absent).

Yes.  A fixed axis range defines the range of the plot itself, and 
thereby filters the corresponding coordinate of the data and functions 
being plotted.

> Actually I don't understand where the x values 01:06:00 to 01:07:18
> are coming from, since i intentionally stored only the Z values in
> the file.

I suspect the fact that 01:06:00 h equals 4000 seconds past midnight, 
which just so happends to be half the number of samples in one dimension 
of your dataset, might have something do do with that.

> About my next tentative: should I change the binary file structure to
> contain all info (x,y,z = time, frequencies,z), which format should I
> use for time ? would be ok a 32 bit integer ? I remember of binary
> "matrix" format I've read about, would that be fine and able to
> contain integers for x,y and float for z ?

No.  You would have to do a double-precision matrix, with %double for 
all values.  See "help binary" for the gory details.

> As an alternative to all that: would it be possible to just produce
> all the labels with some loop, avoiding usage of "xrange" so that i
> can get the plot with the current binary file structure?

The much simpler approach would probably be to apply "help using".

[toc] | [prev] | [next] | [standalone]

#3250

I did some check and i cannot double the size of files so i will have to find a different approach.

About the "using" keyword: I leveraged that one when i was using the csv format and "matrix" instead of "array". The statement was:

plot outname.".mat" matrix using (strptime('%H:%M:%S', scanstart) + singlescandur * $2):( fstart + fstep * $1):3 with image

With "binary array" i get error about "too many using specified" for obvious reasons. In addition to this there's also the time taken to plot in that way compared to the time taken with binary (really really fast) that I would prefer.

Please consider that I have to run all this on a Raspberry PI.  It runs well and fast but I have some obvious constraints.

I will go through the pdf doc once again to see if there's something i did not try already.

Thank you,
Mario

[toc] | [prev] | [next] | [standalone]

#3249

Am 01.03.2016 um 14:58 schrieb Mario Cannistrà:
> They are created as a sequence of float values and I'm
> successfully reading them as array=8000x79   (for this test)
> where 8000 are the frequency bins and 79 are the scans made and
> recorded. Real files will reach at least 100 MB size and this is
> why they are in binary format and not in csv. For the same reason
> I'm storing only z values and NO x,y (timestamps, frequencies).
> 

Gnuplot cannot magically deduce the time & frequency from the range
settings. You need to calculate them from the indices $1 and $2 of
your matrix via a "using" statement:

  plot datafile binary matrix=... using (t($1)):(f($2)):3 with image

[toc] | [prev] | [next] | [standalone]

#3251

Hello Karl.
I already leveraged the using keyword in that way, please see my previous answer for details. I think the problem is the different file structure/size and options used. Seems i cannot have all the benefits together :-)
I will investigate further for other approaches.
Thank you,
Mario

[toc] | [prev] | [standalone]

Back to top | Article view | comp.graphics.apps.gnuplot

csiph-web