Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.graphics.apps.gnuplot > #1960 > unrolled thread

histogram : without using commanade "smooth freq"

Back to article view | Back to comp.graphics.apps.gnuplot

Contents

  histogram :  without using commanade "smooth freq" "D.GH" <dariush.ghorbanzadeh@gmail.com> - 2013-08-11 16:29 -0700
    Re: histogram :  without using commanade "smooth freq" Karl <mail.kfr@gmx.net> - 2013-08-12 08:10 +0200
      Re: histogram :  without using commanade "smooth freq" "D.GH" <dariush.ghorbanzadeh@gmail.com> - 2013-08-12 15:35 -0700
        Re: histogram :  without using commanade "smooth freq" Karl <ratzsch@polymer.uka.de> - 2013-08-13 09:37 +0200
          Re: histogram :  without using commanade "smooth freq" "D.GH" <dariush.ghorbanzadeh@gmail.com> - 2013-08-13 06:45 -0700
          Re: histogram :  without using commanade "smooth freq" "D.GH" <dariush.ghorbanzadeh@gmail.com> - 2013-08-13 10:48 -0700
            Re: histogram :  without using commanade "smooth freq" Karl <mail.kfr@gmx.net> - 2013-08-14 16:27 +0200
              Re: histogram :  without using commanade "smooth freq" sfeam <sfeam@users.sourceforge.net> - 2013-08-14 08:46 -0700
                Re: histogram :  without using commanade "smooth freq" Karl <mail.kfr@gmx.net> - 2013-08-14 18:48 +0200
                Re: histogram :  without using commanade "smooth freq" Karl <mail.kfr@gmx.net> - 2013-08-14 18:54 +0200
                  Re: histogram :  without using commanade "smooth freq" sfeam <sfeam@users.sourceforge.net> - 2013-08-14 18:05 -0700
                    Re: histogram :  without using commanade "smooth freq" "D.GH" <dariush.ghorbanzadeh@gmail.com> - 2013-08-15 16:00 -0700
                      Re: histogram :  without using commanade "smooth freq" sfeam <sfeam@users.sourceforge.net> - 2013-08-15 17:18 -0700
        Re: histogram :  without using commanade "smooth freq" Hans-Bernhard Bröker <HBBroeker@t-online.de> - 2013-08-13 22:35 +0200

#1960 — histogram : without using commanade "smooth freq"

Hello,
 
I have a data file that contains one column like this:
 
1120
1160
963
....

I would like to build the histogram (without using commanade "smooth freq" of GNUPLOT) of these data with a number of intervals (n_bins) that varies.

I used the command "stats" of gunplot for the corresponding data statistics

stats 'mydata.dat'  u 1  name "XX"

To construct the histogram, the data are divided into n_bins intervals as follows:

            XX_min+((XX_max-XX_min)*k)/n_bins

where k belongs to the set {0,1, ... , n_bins}.

This method can be (n_bins+1) Extremities of  intervals I_1,I_2, ..., I_{n_bins} .

My problem N°1: I do not know how to define what k in {0,1, ... , N_bins} vector (with specific commands to GNUPLOT).

 My problem  N°2: How to count the data that is in each interval Ij (j = 1, ... n_bins)
 
Any ideas ?
 
Thank you

[toc] | [next] | [standalone]

#1961

Am 12.08.2013 01:29, schrieb D.GH:
> I would like to build the histogram (without using commanade "smooth freq" of GNUPLOT) of these data with a number of intervals (n_bins) that varies.
> 

So you want to be able to change the bin size, so that more than one
number lands in the same bin, right?

Why don´t you construct a function that converts your values to bin
numbers, something like

bin(x) = int(x/bs)

with bs being (xmax-xmin) / bin_number,
and then

plot fname using (bin($1)):xticlabels($1) smooth freq

? (Hope the thing with the labels works, haven´t tried now.)

  Karl

[toc] | [prev] | [next] | [standalone]

#1962

On Monday, August 12, 2013 8:10:38 AM UTC+2, Karl wrote:
> Am 12.08.2013 01:29, schrieb D.GH:
> 
> > I would like to build the histogram (without using commanade "smooth freq" of GNUPLOT) of these data with a number of intervals (n_bins) that varies.
> 
> > 
> 
> 
> 
> So you want to be able to change the bin size, so that more than one
> 
> number lands in the same bin, right?
> 
> 
> 
> Why don´t you construct a function that converts your values to bin
> 
> numbers, something like
> 
> bin(x) = int(x/bs)
> 
> 
> 
> with bs being (xmax-xmin) / bin_number,
> 
> and then
> 
> 
> 
> plot fname using (bin($1)):xticlabels($1) smooth freq
> ? (Hope the thing with the labels works, haven´t tried now.)
>   Karl


Here is my data : 'mydata.dat'

1120
1160
963 
1546
258
964
1128
1169
1007
259
179
1408
1207
1389
308
1179
147
508
1148
1308

here my code :

reset
stats 'mydata.dat'  u 1  name "XX" 

bin_number=4
bs=(XX_max-XX_min) / bin_number
bin(x) =bs*int(x/bs) 

set table 'out1.dat'
plot 'mydata.dat' using (bin($1)):(1.0) smooth freq with boxes
unset table 

the output file: 'out1.dat'

# Curve 0 of 1, 6 points
# x y xlow xhigh type
 0  5  0  0  i
 349.75  1  349.75  349.75  i
 699.5  3  699.5  699.5  i
 1049.25  9  1049.25  1049.25  i
 1399  2  1399  1399  i
 349.75  1  349.75  349.75  u   ***(I do not understand why there is this last line!)

#######
We note that the (left) end of the first interval should be the minimum 'mydata' = 147, whereas here ('out1.dat' = 0). the (right) end of the last interval should be maximum 'mydata' = 1546, whereas here ('out1.dat' = 1399).

This is why I asked if there was another way to construct the histogram of intervals.

[toc] | [prev] | [next] | [standalone]

#1963

On 13.08.2013 00:35, D.GH wrote:
> On Monday, August 12, 2013 8:10:38 AM UTC+2, Karl wrote:
>> Am 12.08.2013 01:29, schrieb D.GH:
> 
> # Curve 0 of 1, 6 points
> # x y xlow xhigh type
>  0  5  0  0  i
>  349.75  1  349.75  349.75  i
>  699.5  3  699.5  699.5  i
>  1049.25  9  1049.25  1049.25  i
>  1399  2  1399  1399  i
>  349.75  1  349.75  349.75  u   ***(I do not understand why there is this last line!)
> 

Do you have the latest version of gnuplot? I remember there was a
bug that adds invalid lines to a table when there is an empty line
in a dataset.

Otherwise this might just be another bug of the table output in
conjunction with "boxes" or with "smooth freq". Probably a very
rarely used combination.

> #######
> We note that the (left) end of the first interval should be the minimum 'mydata' = 147, whereas here ('out1.dat' = 0). the (right) end of the last interval should be maximum 'mydata' = 1546, whereas here ('out1.dat' = 1399).
> 
> This is why I asked if there was another way to construct the histogram of intervals.
> 

Well, you´ll just have to make the binning function a bit more
elaborate. You can get the minimum/maximum of your dataset with the
"stats" command before plotting.

Best,
  Karl

[toc] | [prev] | [next] | [standalone]

#1964

On Tuesday, August 13, 2013 9:37:25 AM UTC+2, Karl wrote:
> On 13.08.2013 00:35, D.GH wrote:
> 
> > On Monday, August 12, 2013 8:10:38 AM UTC+2, Karl wrote:
> 
> >> Am 12.08.2013 01:29, schrieb D.GH:
> 
> > 
> 
> > # Curve 0 of 1, 6 points
> 
> > # x y xlow xhigh type
> 
> >  0  5  0  0  i
> 
> >  349.75  1  349.75  349.75  i
> 
> >  699.5  3  699.5  699.5  i
> 
> >  1049.25  9  1049.25  1049.25  i
> 
> >  1399  2  1399  1399  i
> 
> >  349.75  1  349.75  349.75  u   ***(I do not understand why there is this last line!)
> 
> > 
> 
> 
> 
> Do you have the latest version of gnuplot? I remember there was a
> 
> bug that adds invalid lines to a table when there is an empty line
> 
> in a dataset.
> 
> 
> 
> Otherwise this might just be another bug of the table output in
> 
> conjunction with "boxes" or with "smooth freq". Probably a very
> 
> rarely used combination.
> 
> 
> 
> > #######
> 
> > We note that the (left) end of the first interval should be the minimum 'mydata' = 147, whereas here ('out1.dat' = 0). the (right) end of the last interval should be maximum 'mydata' = 1546, whereas here ('out1.dat' = 1399).
> 
> > 
> 
> > This is why I asked if there was another way to construct the histogram of intervals.
> 
> > 
> 
> 
> 
> Well, you´ll just have to make the binning function a bit more
> 
> elaborate. You can get the minimum/maximum of your dataset with the
> 
> "stats" command before plotting.
> 
> 
> 
> Best,
> 
>   Karl



On Tuesday, August 13, 2013 9:37:25 AM UTC+2, Karl wrote:
> On 13.08.2013 00:35, D.GH wrote:
> 
> > On Monday, August 12, 2013 8:10:38 AM UTC+2, Karl wrote:
> 
> >> Am 12.08.2013 01:29, schrieb D.GH:
> 
> > 
> 
> > # Curve 0 of 1, 6 points
> 
> > # x y xlow xhigh type
> 
> >  0  5  0  0  i
> 
> >  349.75  1  349.75  349.75  i
> 
> >  699.5  3  699.5  699.5  i
> 
> >  1049.25  9  1049.25  1049.25  i
> 
> >  1399  2  1399  1399  i
> 
> >  349.75  1  349.75  349.75  u   ***(I do not understand why there is this last line!)
> 
> > 
> 
> 
> 
> Do you have the latest version of gnuplot? I remember there was a
> 
> bug that adds invalid lines to a table when there is an empty line
> 
> in a dataset.
> 
> 
> 
> Otherwise this might just be another bug of the table output in
> 
> conjunction with "boxes" or with "smooth freq". Probably a very
> 
> rarely used combination.
> 
> 
> 
> > #######
> 
> > We note that the (left) end of the first interval should be the minimum 'mydata' = 147, whereas here ('out1.dat' = 0). the (right) end of the last interval should be maximum 'mydata' = 1546, whereas here ('out1.dat' = 1399).
> 
> > 
> 
> > This is why I asked if there was another way to construct the histogram of intervals.
> 
> > 
> 
> 
> 
> Well, you´ll just have to make the binning function a bit more
> 
> elaborate. You can get the minimum/maximum of your dataset with the
> 
> "stats" command before plotting.
> 
> 
> 
> Best,
> 
>   Karl

> Do you have the latest version of gnuplot? 

I have Version 4.6. (for windows)

[toc] | [prev] | [next] | [standalone]

#1965

I try to understand how it works in the ' smooth freq '

with the following code: 

reset
stats 'mydata.dat'  u 1  name "XX" 

bin_number=4
bs=(XX_max-XX_min) / bin_number
bin(x) =XX_min+bs*int(x/bs) 

set table 'out1.dat'
plot 'mydata.dat' using (bin($1)):(1.0) smooth freq  w boxes
unset table 

the output file 'out1.dat' :

# Curve 0 of 1, 6 points
# Curve title: "'mydata.dat' using (bin($1)):(1.0)"
# x       y  xlow     xhigh      type
 147      5  147      147          i
 496.75   1  496.75   496.75       i
 846.5    3  846.5    846.5        i
 1196.25  9  1196.25  1196.25      i
 1546     2  1546     1546         i
 496.75   1  496.75   496.75       u


I do not understand why xlow  = xhigh ?

If we break the data into bin_number (here bin_number = 4), we must have

xlow       xhigh   y(=number of data in the range)

147       496.75         5
496.75    846.5          1
846.5     1196.3         9
1196.3    1546           5

[toc] | [prev] | [next] | [standalone]

#1967

Am 13.08.2013 19:48, schrieb D.GH:
> bin(x) =XX_min+bs*int(x/bs) 

> 
> I do not understand why xlow  = xhigh ?

Somehow the combination of "smooth freq" and "with boxes" doesn´t work
correctly. The width of the boxes cannot be changed :

===
set yrange [0:]

boxwidth=0.2

plot '-' using ($1):(1):(boxwidth) smooth freq w boxes
1
2
2
3
e
===

They always stretch up to the next box, and the table output then prints
nonsense too.



But xlow and xhigh would in any case only contain the average of the
centers of adjacent boxes. Any information on the actual values that
contribute to the individual bins is cut off by the int() function in
your binning.


  Karl

[toc] | [prev] | [next] | [standalone]

#1968

Karl wrote:

> Am 13.08.2013 19:48, schrieb D.GH:
>> bin(x) =XX_min+bs*int(x/bs)
> 
>> 
>> I do not understand why xlow  = xhigh ?
> 
> Somehow the combination of "smooth freq" and "with boxes" doesn´t work
> correctly. The width of the boxes cannot be changed :
> 
> ===
> set yrange [0:]
> 
> boxwidth=0.2

The command you want is
   set boxwidth 0.2 absolute




> 
> plot '-' using ($1):(1):(boxwidth) smooth freq w boxes
> 1
> 2
> 2
> 3
> e
> ===
> 
> They always stretch up to the next box, and the table output then
> prints nonsense too.
> 
> 
> 
> But xlow and xhigh would in any case only contain the average of the
> centers of adjacent boxes. Any information on the actual values that
> contribute to the individual bins is cut off by the int() function in
> your binning.
> 
> 
>   Karl

[toc] | [prev] | [next] | [standalone]

#1969

Am 14.08.2013 17:46, schrieb sfeam:
> 
> The command you want is
>    set boxwidth 0.2 absolute

OK, that works, but giving the boxwidth as a third column in the "using"
statment should also work, right?


Funnily, the boxwidth in the table output is unaffected by "set
boxwidth" case :


===
set boxwidth 0.2 absolute
set table
plot '-' using ($1):(1) smooth freq w boxes
1
2
2
3
e
unset table
===

It only turns up in the additional invalid last row (see also

https://sourceforge.net/p/gnuplot/bugs/1274/

)


Best,
  Karl

[toc] | [prev] | [next] | [standalone]

#1970

answer went out too early, sorry.

Am 14.08.2013 17:46, schrieb sfeam:
>
> The command you want is
>    set boxwidth 0.2 absolute

OK, that works, but giving the boxwidth as a third column in the "using"
statment should also work, right?


Funnily, the boxwidth in the table output is unaffected by "set
boxwidth"  :

===
set boxwidth 0.2 absolute
set table
plot '-' using ($1):(1) smooth freq w boxes
1
2
2
3
e
unset table
===

gives

===
# Curve 0 of 1, 4 points
# Curve title: "'-' using ($1):(1)"
# x y xlow xhigh type
 1  1  1  1  i
 2  2  2  2  i
 3  1  3  3  i
 3  1  2.9  3.1  u
===

It only turns up in the additional invalid last row (see also

https://sourceforge.net/p/gnuplot/bugs/1274/

)


Best,
  Karl

[toc] | [prev] | [next] | [standalone]

#1971

Karl wrote:

> answer went out too early, sorry.
> 
> Am 14.08.2013 17:46, schrieb sfeam:
>>
>> The command you want is
>>    set boxwidth 0.2 absolute
> 
> OK, that works, but giving the boxwidth as a third column in the
> "using" statment should also work, right?

The "smooth frequency" operation replaces your original data
with one column of x coords and one column of summed y.
There is no third column.

[toc] | [prev] | [next] | [standalone]

#1975

With the following code:

set boxwidth 0.2 absolute 
set table 'out1.dat'
plot 'mydata.dat' using ($1):(1) smooth freq  w boxes
unset table 

I get:

# Curve 0 of 1, 20 points
# Curve title: "'mydata.dat' using ($1):(1)"
# x y xlow xhigh type
 147  1  147  147  i
 179  1  179  179  i
 258  1  258  258  i
 259  1  259  259  i
 308  1  308  308  i
 508  1  508  508  i
 963  1  963  963  i
 964  1  964  964  i
 1007  1  1007  1007  i
 1120  1  1120  1120  i
 1128  1  1128  1128  i
 1148  1  1148  1148  i
 1160  1  1160  1160  i
 1169  1  1169  1169  i
 1179  1  1179  1179  i
 1207  1  1207  1207  i
 1308  1  1308  1308  i
 1389  1  1389  1389  i
 1408  1  1408  1408  i
 1546  1  1546  1546  i


By this method we did not have control over the number of intervals for the histogram to present.

[toc] | [prev] | [next] | [standalone]

#1976

D.GH wrote:

> With the following code:
> 
> set boxwidth 0.2 absolute
> set table 'out1.dat'
> plot 'mydata.dat' using ($1):(1) smooth freq  w boxes
> unset table

> By this method we did not have control over the number of intervals for
> the histogram to present.

Huh?

"smooth frequency" can be used to generate a histogram of binned values,
but that is not what the command intrinsically does.

If you do not  provide an explicit binning function with defined intervals,
then the data itself is the only source of x-coordinates.
You get one output entry for each unique input x-coord.

"help smooth freq" says:

 The `frequency` option makes the data monotonic in x; points with the same
 x-value are replaced by a single point having the summed y-values.
 To plot a histogram of the number of data values in equal size bins,
 set the y-value to 1.0 so that the sum is a count of occurances in that bin:
 Example:
      binwidth = <something>  # set width of x values in each bin
      bin(val) = binwidth * floor(val/binwidth)
      plot "datafile" using (bin(column(1))):(1.0) smooth frequency

[toc] | [prev] | [next] | [standalone]

#1966

On 13.08.2013 00:35, D.GH wrote:

> We note that the (left) end of the first interval should be the
> minimum 'mydata' = 147, whereas here ('out1.dat' = 0).

I don't know who this "we" is you're referring to, but I demand not to 
be included in that set.

No, I don't note that the left end of the histogram should automatically 
be the left end of the data --- not using that binning function, anyway.

To see what actually happens, plot your data modified by your binning 
function, but without "smooth freq":

# Curve 0 of 1, 20 points
# Curve title: "'mydata.dat' u 1:(bin($1))"
# x y type
  1120  1049.25  i
  1160  1049.25  i
  963  699.5  i
  1546  1399  i
  258  0  i
  964  699.5  i
  1128  1049.25  i
  1169  1049.25  i
  1007  699.5  i
  259  0  i
  179  0  i
  1408  1399  i
  1207  1049.25  i
  1389  1049.25  i
  308  0  i
  1179  1049.25  i
  147  0  i
  508  349.75  i
  1148  1049.25  i
  1308  1049.25  i

Note how the actual lower end of those data is not 147, like you think, 
but 0.  Same goes for the upper end.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.graphics.apps.gnuplot

csiph-web