Path: csiph.com!news.swapon.de!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: "John L. Males" <gnuplot.open.source@softhome.ca>
Newsgroups: comp.graphics.apps.gnuplot
Subject: Question: How to Plot Data From One File In Defined Time Slices in One Graph
Date: Wed, 3 Jul 2024 18:07:30 +0000
Organization: Toronto, Ontario, Canada
Lines: 240
Message-ID: <20240703180730.23264515936b587c89686d42@softhome.ca>
Reply-To: gnuplot_open_source@softhome_ca
MIME-Version: 1.0
Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg="PGP-SHA512"; boundary="Signature=_Wed__3_Jul_2024_18_07_30_+0000_2rbsSFq.eA1uh7Po"
Injection-Date: Wed, 03 Jul 2024 20:07:57 +0200 (CEST)
Injection-Info: dont-email.me; posting-host="5066c7f93a903ab3d246b9e0c745ae1a"; logging-data="2427159"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX18JYaU+Z0RPuJEiIuKtF/hSizPKFj5K1vA="
Cancel-Lock: sha1:OJnaxtgXYlSiYwH8A1YdCciYbXM=
X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu)
Return-Receipt-To: gnuplot.open.source@softhome.ca
Disposition-Notification-To: gnuplot.open.source@softhome.ca
X-Newsreader: Sylpheed 3.7.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu)
X-Compose-Start: %F-%T%z%Z-%s
Xref: csiph.com comp.graphics.apps.gnuplot:4667

--Signature=_Wed__3_Jul_2024_18_07_30_+0000_2rbsSFq.eA1uh7Po
Content-Type: text/plain; charset=US-ASCII
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Hello,

Any tips, insight, pointers, et al would be most appreciated.  I
have spent weeks of hundreds of ideas (mine, research, or
combination of) would be most appreciate.

In essence what I am doing is offsetting each line of time using
x:y+TimePerLine*yoffset via a scaling factor of yoffset to plot in
one graph no multiplot for each hour TimePerLine (3600 seconds) to
start. Hence the two plot lines I am trying to find way to plot
with goal of seismic helicorder with gnuplot:

plot '<cat' using ($1)-TimeSecondsGraphStart :($2)+
(VerticalSensitivity/24*16)+Offset with lines linestyle 9

plot '<cat' using ($1)-TimeSecondsGraphStart-3600 :($2)+
(VerticalSensitivity/24*12)+Offset with lines linestyle 9 set

The reason for the "($1)-TimeSecondsGraphStart" is because "set
xrange [0:TimeSecondsDurationGraphLine]" as the input data
timestamp in "%S" is actual time of data. I did not pre-process the
data timestamps of data sent to gnuplot relative to zero.  Data is
mostly less than 0.003, often less than 0.0001.

I am skipping all but the two lines (two plot statements) of
gnuplot code in my gnuplot source as these are the two lines at
heart of my challenges I am working with test in order to achieve a
helicorder with gnuplot.

If both lines are as above the one, the first plot will plot as
expected, and second will say "....  line foo: warning: Skipping
data file with no valid points".

Same "....  line foo: warning: Skipping data file with no valid
points"" occurs if I continue first plot line to second line
without using "plot" on second line.

If I comment out one of these plot statements then each respective
plot statement creates a graph with the correct data at the correct
vertical displacement.  In this case on the 16 or 12 vertical
displacement.  For the moment I am using hard coded 3600 for
seconds per each vertical line, 24 for hours of graph period, and
12/16 as respective vertical line displacement position.  The code
already has variables for these specific or calculated from base
values, but I hardcoded to ensure the time segment calculation is
correct and to ensure the vertical calculation position is very
clear for this two line plot test before I generate likely via
for/do for the variable specified graph duration, time per line,
vertical displacement.


Background
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

I have been using gnuplot for years to plot various csv data code I
write generates on 24/7/365 basis.  I am stumped in how to achieve
with gnuplot how to create what is known in seismic domain as a
helicorder.  I have tried many many ideas of my own and many
variations of ideas or variations of those ideas via weeks of
searching wide variety of sources and search venues.

Before I attempted this I did the really simple plot of 24 hours of
data as single line for full 24 hours on graph that is easy to do.
With that single line graph for purposes of testing I add a fixed
value to the y and depending on of plus or minus value the graph
would shift towards top or bottom of graph.  I also did tests on
same single plot by adding +- value to x and graph shifted left or
right per value with of course no data plotted on side of data
opposite shift value as would expect.=20

I will state upfront many will suggest multiplot, but other than
one special case of multiplot I will expalin that I tried is not
appropriate as the data of each multiplot is contained within the
limits of that multiplot, i.e. the graph data cannot exceed the
borders unless I have missed in my weeks of searching not to be
case.

One link with various examples of helicorders and why as far as I
can determine multiplot will not allow a gnuplot like helicorder to
transcend many horizontal (time) segments:

<https://earthquake.usgs.gov/monitoring/seismograms/examples.php>

For the moment my concern is how to graph multiple lines such that
will look like the example helicorders.

For those that want to have copy of data bear in mind the data for
a 24 hour helicorder in generally 30-60 million million lines of
data depending on if 100 or 300 samples a second for three axis of
Z, N, and E for about 0.5TB data before compressing.  The data
format in simplest form for one channel is in format of "%s.%N,
%e".  Unlike most seismic recording systems that output the ADC
counts for the input ADC voltage my ADC code outputs voltage.  I
use voltage so if the ADC changes (and will as I work on code for
better ADCs) the same instrument will record in the data file same
data such that one does not need to "scale" count data due to
different +- input sensitivity range ratings, and/or PGA settings
ADCs have (varies from +-0.6V to +-10V) then layered difference if
ADC is fully differential, bipolar differential, or single ended.
There are other variables related to AC and instrument, but noted
are most common.

The ADC currently being used is set to +-0.256V as that is most
sensitive for ADC.  ADC code is outputting voltage based on ADC
sensitivity.  The counts from ADC is converted by code for ADC with
respect to ADC sensitivity.

What I am trying to do is plot one data file of 24 hours of data
into 24 lines of one hour plots to start.  Once I have this
accomplished then I will use core variables such as period of time
in hours (common for 24, and 12, but some use 36) helicorder is to
cover, time in minutes for each line of helicorder (typically 60,
30, and 15 are used all quite commonly).  One side of helicorder
will have minute start for data line and maybe date as well.  Other
side of graph may have local time, end time in munutes of line, or
vertical scaling (such a nm of movement information). For the
purposes of this question I am only interested in how to plot one
data file split by uniform time length segment of data across
multiple lines of time, i.e. ignore how to address vertical
labelling of vertical part of helicorder.=20

To cover the one special case of multiplot some of my research
results suggested if I placed "set multiplot" before and "set
nomultiplot" after the set of plot statements that would achieve
the desired result.  Sadly the exact results as stated without the
related multiplot statements occurred.  I cannot recall how many
ideas I tried that were simply not acceptable to gnuplot at all,
but tried just in case even if to me made no sense or would not
work just to be sure.



John L. Males
Toronto, Ontario, Canada
03 July 2024 18:07 +0000 UTC
03 July 2024 14:07 -0400 EDT


__________________________________________________________________

V01.06.00
2024-07-03 17:11:37.710639223+0000-UTC Time: 1720026697.710639223
PC/System time

2024-07-03 17:11:37.819672 (+0000) +0.000006 +/- 0.000054
localhost ::1 s2 no-leap

2024-07-03 17:11:38.66783 (+0000) -0.001678 +/- 0.018325
1.ca.pool.ntp.org 216.197.228.230 s1 no-leap

Linux 5.10.0-23-amd64 #1 SMP Debian 5.10.179-1 (2023-05-12)

Debian GNU/Linux 11 (bullseye)
CustomLiveJLMAdvancedWithuC-MX-21.3_x64 Wildflower
20230709-0256+0000-UTC cat /proc/cpuinfo (Selected):

model name	: Intel(R) Pentium(R) CPU G4560T @ 2.90GHz

vmstat -s :

     32757020 K total memory
     13972428 K used memory
      9149956 K active memory
     16965800 K inactive memory
      3408380 K free memory
            4 K buffer memory
     15376208 K swap cache
            0 K total swap
            0 K used swap
            0 K free swap
     10205965 non-nice user cpu ticks
    120469430 nice user cpu ticks
     41361092 system cpu ticks
    118919900 idle cpu ticks
        36736 IO-wait cpu ticks
            0 IRQ cpu ticks
       814678 softirq cpu ticks
            0 stolen cpu ticks
     27256401 pages paged in
      3998951 pages paged out
            0 pages swapped in
            0 pages swapped out
   3539710811 interrupts
   1966169522 CPU context switches
   1717501612 boot time
       798709 forks

/proc/vmstat (Selected) :

pgpgin 27256401
pgpgout 3998951
pswpin 0
pswpout 0
pgfree 11380358641
pgfault 6146587069
pgmajfault 92199

/proc/meminfo (Selected) :

Mlocked:           17368 kB
VmallocTotal:   34359738367 kB
VmallocChunk:          0 kB
HugePages_Total:       0

ps -A (Selected) :

%CPU  START   TIME  C CLS COMMAND             TIME  NI     PID POL
PRI    SZ   RSS    VSZ  SIZE  MAJFL  MINFL SCH STAT     TIME WCHAN

 3.4 Jun  4 1453:05  3 TS kswapd0         1-00:13:05 0      70 TS
19     0     0      0     0      0      0   0 S    1-00:13:05 -

__________________________________________________________________

--Signature=_Wed__3_Jul_2024_18_07_30_+0000_2rbsSFq.eA1uh7Po
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEELO+C3LyeAem0WQtdprsqY7y+VzUFAmaFk2IACgkQprsqY7y+
VzWp6xAAlcZ/YX7pEkdItzbItzeT+5Mo3/gnF5r6JCaW02kzM/6Hmhz5yxX11Qep
MFnzP2/4jNKHm2phme9/z1zAuDkVqL41G12r9XK2NX45mW3fpy+GD+X0xJr0xWyV
+ZdAlGkftjW3mnxqkrxaIXttV+dY5BanL7bfjGJk/A0C+LMZ9y+mFDBSh5pmp2Vv
uXNdwxYQph8vnY98jUy3y6khE45anuQUBtPPRtL2w2DgL5FId8h0Tn+1YCJltL4R
SIS4dxzpMRkSZLCwNoMBR1ookaMeH5tzLGjajocYZPAF8kkmwUBYuqcmUEt+baHg
MJX9I/+usD/q9NIRmNV7yHCN7G5LwElPmWavdeGi34wDvIgSILpNn2r5wqQRZwd7
7KOQbDF9Hju3Jy1cJeB9QC+uy4EUrIPFcG6RZop6O30I668lrBAAI4joLhHaB+/l
5aoOZqUlA5TUm6j58kWbnQat9B/be2m4xdk5ZSnfzWMMRmnrj1geRldOuVDPCzqV
wxf5Y+V4iRBrckf9hF+7u36P04ep6zU4OqfP4oqu1EUUalw8KXh4LTilg9c7qUok
W8AGQIuFoTtt3NdxCJSw0bNGO4yzIt/AF8340472SXiV4MQmzKxk3AjytcYMEAS/
zX3KGKlmYlnClS4/9SGRYMDw4aI6wo4hijRy3WU/v3ylMi9Ei/A=
=clda
-----END PGP SIGNATURE-----

--Signature=_Wed__3_Jul_2024_18_07_30_+0000_2rbsSFq.eA1uh7Po--