Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.arch.embedded > #12917 > unrolled thread

Comparing phase of physically distant signals

Back to article view | Back to comp.arch.embedded

Contents

  Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 00:45 -0700
    Re: Comparing phase of physically distant signals Jan Panteltje <pNaonStpealmtje@yahoo.com> - 2013-08-03 07:51 +0000
    Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 09:58 +0100
      Re: Comparing phase of physically distant signals upsidedown@downunder.com - 2013-08-03 13:21 +0300
        Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 12:16 +0100
        Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 11:56 -0700
          Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 20:17 +0100
            Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 13:41 -0700
              Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 22:46 +0100
                Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 14:57 -0700
                  Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 23:09 +0100
      Re: Comparing phase of physically distant signals upsidedown@downunder.com - 2013-08-03 13:58 +0300
        Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 12:30 +0100
      Re: Comparing phase of physically distant signals upsidedown@downunder.com - 2013-08-03 16:08 +0300
        Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 14:22 +0100
          Re: Comparing phase of physically distant signals upsidedown@downunder.com - 2013-08-03 16:38 +0300
            Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 10:46 -0700
        Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 10:40 -0700
          Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 20:04 +0100
          Re: Comparing phase of physically distant signals Hans-Bernhard Bröker <HBBroeker@t-online.de> - 2013-08-04 15:55 +0200
            Re: Comparing phase of physically distant signals upsidedown@downunder.com - 2013-08-04 19:42 +0300
              Re: Comparing phase of physically distant signals Hans-Bernhard Bröker <HBBroeker@t-online.de> - 2013-08-04 21:08 +0200
                Re: Comparing phase of physically distant signals upsidedown@downunder.com - 2013-08-05 15:28 +0300
                  Re: Comparing phase of physically distant signals Hans-Bernhard Bröker <HBBroeker@t-online.de> - 2013-08-05 22:17 +0200
                    Re: Comparing phase of physically distant signals Richard Damon <Richard@Damon-Family.org> - 2013-08-05 23:20 -0400
      Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 14:26 +0100
      Re: Comparing phase of physically distant signals George Neuner <gneuner2@comcast.net> - 2013-08-03 14:47 -0400
    Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-03 08:17 -0700
      Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 12:08 -0700
        Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 20:37 +0100
          Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 12:57 -0700
            Re: Comparing phase of physically distant signals Hans-Bernhard Bröker <HBBroeker@t-online.de> - 2013-08-04 15:58 +0200
        Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-03 12:44 -0700
          Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 18:36 -0700
            Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-04 09:56 -0700
              Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-05 14:53 -0700
                Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-05 15:38 -0700
                  Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-07 01:25 -0700
                    Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-07 08:06 -0700
                      Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-07 09:21 -0700
    Re: Comparing phase of physically distant signals josephkk <joseph_barrett@sbcglobal.net> - 2013-08-03 15:40 +0000
      Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-03 10:05 -0700
        Re: Comparing phase of physically distant signals John S <Sophi.2@invalid.org> - 2013-08-03 12:13 -0500
          Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-03 10:26 -0700
            Re: Comparing phase of physically distant signals John S <Sophi.2@invalid.org> - 2013-08-03 12:51 -0500
              Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-03 11:54 -0700
                Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 20:27 +0100
                  Re: Comparing phase of physically distant signals George Neuner <gneuner2@comcast.net> - 2013-08-05 03:26 -0400
                    Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-05 10:34 +0100
                    Re: Comparing phase of physically distant signals Richard Damon <Richard@Damon-Family.org> - 2013-08-05 23:29 -0400
                      Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-06 08:53 +0100
            Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 12:49 -0700
          Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 12:34 -0700
        Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 12:18 -0700
          Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-03 12:52 -0700
            Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 13:21 -0700
              Re: Comparing phase of physically distant signals Jeff Liebermann <jeffl@cruzio.com> - 2013-08-03 13:31 -0700
                Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 14:46 -0700
                  Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-03 15:03 -0700
                  Re: Comparing phase of physically distant signals Jeff Liebermann <jeffl@cruzio.com> - 2013-08-03 17:25 -0700
              Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-03 14:50 -0700
      Re: Comparing phase of physically distant signals "Paul E. Bennett" <Paul_E.Bennett@topmail.co.uk> - 2013-08-04 11:06 +0100
        Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-04 12:19 -0700
    Re: Comparing phase of physically distant signals John Larkin <jjlarkin@highNOTlandTHIStechnologyPART.com> - 2013-08-03 11:47 -0700
    Re: Comparing phase of physically distant signals Richard Damon <Richard@Damon-Family.org> - 2013-08-03 18:23 -0400
    Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-05 16:53 +0100
      Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-05 13:19 -0700
        Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-06 00:27 +0100
          Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-05 20:09 -0700
            Re: Comparing phase of physically distant signals Paul Rubin <no.email@nospam.invalid> - 2013-08-05 22:12 -0700
              Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-06 09:12 +0100
              Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-06 01:47 -0700
                Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-06 10:08 +0100
                  Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-06 02:25 -0700
                    Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-06 02:48 -0700
                    Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-06 10:49 +0100
                      Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-06 03:06 -0700
                        Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-06 11:57 +0100
                          Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-06 10:28 -0700
                            Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-06 21:52 +0100
                              Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-07 01:34 -0700
                                Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-07 09:46 +0100
                Re: Comparing phase of physically distant signals Les Cargill <lcargill99@comcast.com> - 2013-08-06 07:22 -0500
                  Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-06 10:42 -0700
                    Re: Comparing phase of physically distant signals Les Cargill <lcargill99@comcast.com> - 2013-08-06 23:04 -0500
                      Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-06 23:50 -0700
                        Re: Comparing phase of physically distant signals Les Cargill <lcargill99@comcast.com> - 2013-08-07 13:05 -0500
                          Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-07 18:22 -0700
                            Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-08 09:35 +0100
                Re: Comparing phase of physically distant signals Paul Rubin <no.email@nospam.invalid> - 2013-08-06 22:18 -0700
                  Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-07 03:09 -0700
                    Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-07 12:39 +0100
                    Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-07 13:18 +0100
            Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-06 09:23 +0100
              Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-06 02:02 -0700
            Re: Comparing phase of physically distant signals "Paul E. Bennett" <Paul_E.Bennett@topmail.co.uk> - 2013-08-06 09:49 +0100
              Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-06 02:17 -0700
    Re: Comparing phase of physically distant signals Paul E Bennett <Paul_E.Bennett@topmail.co.uk> - 2013-08-16 21:07 +0100
      Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-16 14:50 -0700
        Re: Comparing phase of physically distant signals Paul E Bennett <Paul_E.Bennett@topmail.co.uk> - 2013-08-18 19:55 +0100
          Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-19 11:43 -0700
            Re: Comparing phase of physically distant signals Paul E Bennett <Paul_E.Bennett@topmail.co.uk> - 2013-08-20 21:43 +0100
              Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-20 14:49 -0700

Page 4 of 6 — ← Prev page 1 2 3 [4] 5 6  Next page →

#12957

Don Y wrote:
> Hi Joerg,
> 
> On 8/3/2013 12:52 PM, Joerg wrote:

[...]


>>> I can easily get synchronization to < 1us.  How much better
>>> depends on the resources I bring to bear on this.  I would
>>> like to come up with a practical scheme for verifying and
>>> troubleshooting synchronization problems on *deployed*
>>> hardware (vs. "on the bench" where I can put things in
>>> convenient places)
>>
>> But in your other post you said you don't want the radios in the BOM.
>>
>> <scratching head>
> 
> Correct.  We're conflating two different discussions:  my
> time sync MEASUREMENT requirements and LORAN's capabilities.
> 
> I.e., to be more verbose:
> 
> My software  can easily achieve synchronization between nodes to
> a level better than 1us -- without doing anything fancy.  Recall
> you don't just have two radios talking to each other or two
> wired devices -- there are also bits in the middle that contribute
> to the uncertainty (e.g., network switch, length of interconnect
> cable, "software", etc.)
> 
> I have modified that software to toggle an output pin at specific
> "times" (i.e., I'm not just locking frequency and phase of a
> timebase but also it's notion of "absolute" time).  More colloquially,
> each device can toggle this output pin at *it's* notion of "12 noon".
> I want to be able to *measure* (from the outside) how far off any
> two devices might be.  I.e., if device #1 generates its pulse
> *now*, then device #2 must generate it's pulse within << 1us of
> "now".
> 
> [I.e., I'm not trying to compare two X Hz oscillators and get them
> "in sync".  Rather, I am trying to compare two TIMEBASES and ensure
> they are in sync -- they both agree that it is 12:00:00.000000
> *now* -- by emitting a signal "often enough" that a measurement
> device can provide adequate feedback to a human being regarding
> any discrepancies in the devices' notion of "now"]
> 

That can be done via RF or cable. With RF you can send out a pulse train
modulated onto a carrier whenever unit A says it is high noon. Then send
a similar pulse train when unit B thinks it's high noon, but this one
has to be on another carrier frequency so they can occur simultaneously.

The simplest form of comparing the transmission times would be
zero-crossers for the demodulated signal. Complex FFT and correlation
are other methods. All you need to know is the phase difference but you
want to know it quite precisely.

I have recently done something similar but using the sound card of a PC.
 It was possible to reliably detect phase shifts in the milli-degree
range. You have to make sure that the bandwidth after procesing is low,
to knock the noise down far enough. No big deal, because you can let
each unit transmit long enough.


>> Either way, geting down to the 100nsec ballpark or less should not be a
>> problem if the units are in the same building. The RF link ToF can be
>> measured and calculated out.
> 
> Actually, that gives me a *different* idea!
> 
> What if I sat at the "reference point" and injected a signal onto the
> network cable.  A "ping" of sorts.  Prior to (or coincident with!)
> this, I use a TDR to determine the (temporal!) length of the cable.
> 
> Now, at each node/device, I measure the time difference between the
> arrival of this ping (from which I can determine when it *departed*
> the point of origination) and the "pulse output".
> 
> Do that for each node and normalize the results?
> 
> This assumes a ping can transit from the reference to each of these
> points over a single piece of "cable".  And, that the propagation
> speed will be the same from one cable to another??
> 

I don't see why that would not work. Provided that any hardware between
pinger and "pingee" is not flopping around to much in terms of phase noise.

-- 
Regards, Joerg

http://www.analogconsultants.com/

[toc] | [prev] | [next] | [standalone]

#12965

josephkk wrote:

> On Sat, 03 Aug 2013 00:45:41 -0700, Don Y wrote:

[%X]

>> Of course, I would like to minimize the recurring cost of any solution
>> as it is just present for system qualification (and troubleshooting) and
>> offers no run-time advantage to the design.
>> 
>> Thx,
>> --don
> 
> Sounds like a use case for NTP (network time protocol) which already
> keeps millions of computers synchronized to a few milliseconds across the
> whole planet.

Don didn't mention any numbers with specific required tolerances but the NTP 
solution may work to an extent. If Don wanted the finer time difference 
granularity then he should look at LXI as a time protocol method (used in 
finely timed instrumentation) which can, I believe, attain better than 40ns.

--
********************************************************************
Paul E. Bennett...............<email://Paul_E.Bennett@topmail.co.uk>
Forth based HIDECS Consultancy
Mob: +44 (0)7811-639972
Tel: +44 (0)1235-510979
Going Forth Safely ..... EBA. www.electric-boat-association.org.uk..
********************************************************************

[toc] | [prev] | [next] | [standalone]

#12973

Greetings Paul,

On 8/4/2013 3:06 AM, Paul E. Bennett wrote:

>>> Of course, I would like to minimize the recurring cost of any solution
>>> as it is just present for system qualification (and troubleshooting) and
>>> offers no run-time advantage to the design.
>>
>> Sounds like a use case for NTP (network time protocol) which already
>> keeps millions of computers synchronized to a few milliseconds across the
>> whole planet.
>
> Don didn't mention any numbers with specific required tolerances but the NTP
> solution may work to an extent.

NTP is too coarse.  Great for "people time" but not for "event time".

> If Don wanted the finer time difference
> granularity then he should look at LXI as a time protocol method (used in
> finely timed instrumentation) which can, I believe, attain better than 40ns.

Yes, this is effectively what I have already implemented.  I just
cut some corners that pertain to interoperability as well as
exploit other invariants that my system imposes so I don't have
to add all the hardware that 1588 would "typically" require.

But, the problem I am trying to address falls more in the
category of verification/troubleshooting.

I.e., system is in place.  Nodes are physically separated (once
deployed).  Now, how do you verify that the "local clock"
(sense of time) on node A is "closely synchronized" (frequency,
phase and reference time) with the local clock on node B?

When the two nodes are sitting side by side on a bench, you can
compare synthesized "output pulses".  I.e., create some
arbitrary even that happens often enough that you aren't
waiting FOREVER for the next one to occur; yet not too often
that confusion can arise as to whether the event/pulse you
are observing from node A corresponds, wrt the reference time,
to the same pulse you are observing from node B.  (i.e.,
that the pulse sequence from one node isn't 360+ degrees
out of phase wrt the other node)

You need some way to get the two "signals" to a point where
they are physically close enough together to connect to *a*
piece of test gear.  Or, a way of comparing each, independently,
to some reference (that is stable over the test period).

E.g., the naive approach of is two equal lengths of cable
that can be used to "extend" the signals from the two nodes
to some common place.  You can then trim the lengths of the
cables to match their propagation delays to a tolerance
exceeding that required for your measurement.  Easy to
achieve a ~1.5 ft match between cable lengths and, thus,
be able to ignore the differences in the cable lengths as a
meaningful error source in your measurement.

OTOH, had you run *one* "extension cable" from node A over to
node B, then the difference in signal path easily swamps the
signal (time difference) being measured.  E.g., at ~60 ft
you're already up to 100ns.  So, if side-by-side, node A
and node B appeared to be within 100ns of each other, they
now appear to be perfectly coincident *or* 200ns apart
(depending on where the delay has been introduced).

Using radio as the "signal extender" has similar problems
regarding propagation delay (different media paths) as well
as requiring in situ calibration (wire can be cut to length
ahead of time)

Beyond "verification", there are issues for troubleshooting, etc.
E.g., how do you know that time *is* synchronized to the degree
The System thinks it is?  If something is broke, it is just as
likely to be the servo that does the synchronization ("I think
everything is perfectly synchronized, *now*, so I will take
no action to tweak the servo loop")

--don

[toc] | [prev] | [next] | [standalone]

#12937

On Sat, 03 Aug 2013 00:45:41 -0700, Don Y <this@isnotme.com> wrote:

>Hi,
>
>[posted to S.E.D in the hope that a hardware analog might exist]
>
>I synchronize the "clocks" on physically distributed processors
>such that two or more different machines can have a very finely
>defined sense of "synchronized time" between themselves.
>
>During development, I would measure this time skew (among other
>factors) by locating these devices side-by-side on a workbench
>interconnected by "unquantified" cable.  Then, measuring the
>time difference between to "pulse outputs" that I artificially
>generate on each board.
>
>So, I could introduce a disturbance to the system and watch to
>see how quickly -- and accurately -- the "clocks" (think FLL and
>PLL) come back into sync.
>
>How do I practically do this when the devices are *deployed*
>and physically distant (vs. "electrically distant" as in my
>test case)?
>
>Two ideas come to mind:
>1) two equal length cables to connect the "pulse outputs"
>from their respective originating devices to the test gear.

That will work, if you can get cables or fibers long enough. The propagation
delays can be measured, but tempcos will matter if you're concerned about
nanoseconds or less.

What are your numbers? Distance? Accuracy expectation?


>2) two *radios* to do the same thing -- after accounting
>for different flight times
>
>[Though I wonder how hard it is to qualify two different
>radios to have the same delay, etc.  Far easier to trim
>two long lengths of wire to the same length!]
>
>Of course, I would like to minimize the recurring cost of
>any solution as it is just present for system qualification
>(and troubleshooting) and offers no run-time advantage to
>the design.
>
>Thx,
>--don

There is a technique for absolute-time-syncing clocks that measures the
round-trip cable (or fiber) delay in both directions and does the math to take
it out. I don't recall what it's called.

Can't GPS deliver absolute time ticks? Wasn't that used in the recent bogus FTL
neutrino experiment?

You can physically transport an atomic clock from site to site to align absolute
times. You need relativity compensation to get really good. 

http://en.wikipedia.org/wiki/Hafele%E2%80%93Keating_experiment


-- 

John Larkin                  Highland Technology Inc
www.highlandtechnology.com   jlarkin at highlandtechnology dot com   

Precision electronic instrumentation
Picosecond-resolution Digital Delay and Pulse generators
Custom timing and laser controllers
Photonics and fiberoptic TTL data links
VME  analog, thermocouple, LVDT, synchro, tachometer
Multichannel arbitrary waveform generators

[toc] | [prev] | [next] | [standalone]

#12961

On 8/3/13 3:45 AM, Don Y wrote:
> Hi,
> 
> [posted to S.E.D in the hope that a hardware analog might exist]
> 
> I synchronize the "clocks" on physically distributed processors
> such that two or more different machines can have a very finely
> defined sense of "synchronized time" between themselves.
> 
> During development, I would measure this time skew (among other
> factors) by locating these devices side-by-side on a workbench
> interconnected by "unquantified" cable.  Then, measuring the
> time difference between to "pulse outputs" that I artificially
> generate on each board.
> 
> So, I could introduce a disturbance to the system and watch to
> see how quickly -- and accurately -- the "clocks" (think FLL and
> PLL) come back into sync.
> 
> How do I practically do this when the devices are *deployed*
> and physically distant (vs. "electrically distant" as in my
> test case)?
> 
> Two ideas come to mind:
> 1) two equal length cables to connect the "pulse outputs"
> from their respective originating devices to the test gear.
> 2) two *radios* to do the same thing -- after accounting
> for different flight times
> 
> [Though I wonder how hard it is to qualify two different
> radios to have the same delay, etc.  Far easier to trim
> two long lengths of wire to the same length!]
> 
> Of course, I would like to minimize the recurring cost of
> any solution as it is just present for system qualification
> (and troubleshooting) and offers no run-time advantage to
> the design.
> 
> Thx,
> --don

You might want to look into how NTP (the Network Time Protocol) works.
It is based on having a fairly consistent delay (or making multiple
measurements to average out the variations).

As I remember, the basic technique is to start with the two units A and
B having their own idea of what time is, time is advancing at the same
rate for the two units, but there may be a skew between them.

Unit A sends a message to B, with the time stamp of when it sent the
message.
Unit B then receives the message, marks it with the received time stamp,
and then resends it to A, adding again the time stamp of the
transmission, and when A get the answer back, it adds this time as the
forth and final time stamp to the message.

From the 4 time stamps, you can calculate the time it took for a message
to get between A and B, and the difference in their clocks, allowing A
to adjust his clock to be in synchrony with B. The math assumes that the
flight time in each directions is the same, any difference contributes
in an increase resultant skew in clocks.

[toc] | [prev] | [next] | [standalone]

#12980

Since you are building a distributed system, can I suggest
that it would be beneficial to you if you are aware of the
theory and practice that has been developed over the past
40 years. That might save you time by avoiding going down
blind alleys.

In particular you will find that you probably *don't need*
*to synchronise time*, if all you are interested in is
which event occurred first.

Most work stems from Lamport's seminal 1978 paper "Time,
clocks, and the ordering of events in a distributed system"
which is easily locatable via a google search, e.g.
http://www.stanford.edu/class/cs240/readings/lamport.pdf

I've used this book, which is sufficiently well-regarded
that it has been reprinted four times since 1994.
http://www.amazon.co.uk/Distributed-Systems-George-Coulouris/dp/0273760599
In particular the chapter 10 "Time and Coordination" has
useful info on the general problem, synchronisation,
logical time and logical clocks.

No doubt other distributed system text books cover similar
ground.

On 03/08/13 08:45, Don Y wrote:
> Hi,
>
> [posted to S.E.D in the hope that a hardware analog might exist]
>
> I synchronize the "clocks" on physically distributed processors
> such that two or more different machines can have a very finely
> defined sense of "synchronized time" between themselves.
>
> During development, I would measure this time skew (among other
> factors) by locating these devices side-by-side on a workbench
> interconnected by "unquantified" cable.  Then, measuring the
> time difference between to "pulse outputs" that I artificially
> generate on each board.
>
> So, I could introduce a disturbance to the system and watch to
> see how quickly -- and accurately -- the "clocks" (think FLL and
> PLL) come back into sync.
>
> How do I practically do this when the devices are *deployed*
> and physically distant (vs. "electrically distant" as in my
> test case)?
>
> Two ideas come to mind:
> 1) two equal length cables to connect the "pulse outputs"
> from their respective originating devices to the test gear.
> 2) two *radios* to do the same thing -- after accounting
> for different flight times
>
> [Though I wonder how hard it is to qualify two different
> radios to have the same delay, etc.  Far easier to trim
> two long lengths of wire to the same length!]
>
> Of course, I would like to minimize the recurring cost of
> any solution as it is just present for system qualification
> (and troubleshooting) and offers no run-time advantage to
> the design.
>
> Thx,
> --don

[toc] | [prev] | [next] | [standalone]

#12982

Hi Tom,

On 8/5/2013 8:53 AM, Tom Gardner wrote:
> Since you are building a distributed system, can I suggest
> that it would be beneficial to you if you are aware of the
> theory and practice that has been developed over the past
> 40 years. That might save you time by avoiding going down
> blind alleys.
>
> In particular you will find that you probably *don't need*
> *to synchronise time*, if all you are interested in is
> which event occurred first.

Who claimed that "all (I am) interested in is which
event occurred first"?

Is that the only use you have for time in a *uniprocessor*
system?  Why would you *only* have that need in a *distributed*
system?

My system is physically distributed because it is far more
economical and practical to design it in that way.  But, that
doesn't mean I should not avail myself of the same sorts of
capabilities that would have existed had it NOT been
distributed.

How do you cause two effectors to be engaged "simultaneously"
(or, with *any* known *fine* timing relationship to each other)
unless you have a common sense of time?

How do you triangulate the location of an event if you don't
know your positions accurately *and* a common reference
against which to operate?

> Most work stems from Lamport's seminal 1978 paper "Time,
> clocks, and the ordering of events in a distributed system"
> which is easily locatable via a google search, e.g.
> http://www.stanford.edu/class/cs240/readings/lamport.pdf
>
> I've used this book, which is sufficiently well-regarded
> that it has been reprinted four times since 1994.
> http://www.amazon.co.uk/Distributed-Systems-George-Coulouris/dp/0273760599
> In particular the chapter 10 "Time and Coordination" has
> useful info on the general problem, synchronisation,
> logical time and logical clocks.

Note that the issue isn't *synchronizing* the clocks.  There
are well-known technologies that will do this.

The question I posed was how to measure/verify/validate/troubleshoot
such a system ONCE DEPLOYED.

E.g., you can measure/verify/validate/troubleshoot how well
a *hardware* PLL is working with a pair of probes on a bench
(choice of test equipment can vary).  But, in a physically
distributed system where the distances are "longer than the
test leads", just connecting to the signals of interest
becomes a problem in itself!  When does the "verification device"
become a variable as well?

> No doubt other distributed system text books cover similar
> ground.

[toc] | [prev] | [next] | [standalone]

#12985

On 05/08/13 21:19, Don Y wrote:
> Hi Tom,
>
> On 8/5/2013 8:53 AM, Tom Gardner wrote:
>> Since you are building a distributed system, can I suggest
>> that it would be beneficial to you if you are aware of the
>> theory and practice that has been developed over the past
>> 40 years. That might save you time by avoiding going down
>> blind alleys.
>>
>> In particular you will find that you probably *don't need*
>> *to synchronise time*, if all you are interested in is
>> which event occurred first.
>
> Who claimed that "all (I am) interested in is which
> event occurred first"?

Because that's all you can realise, in theory and in practice.
If you want more and achieve it, please do publish: you'll
be famous (or infamous).


> Is that the only use you have for time in a *uniprocessor*
> system?  Why would you *only* have that need in a *distributed*
> system?

I'm sorry, I don't understand the question; but I don't
that that is important.

> My system is physically distributed because it is far more
> economical and practical to design it in that way.

Fine.

> But, that
> doesn't mean I should not avail myself of the same sorts of
> capabilities that would have existed had it NOT been
> distributed.

Because the mere act of physical distribution brings
constraints that are not present in non-distributed
systems. Sorry; welcome to this universe.

In general it is a mistake to create a non-distributed
system with the assumption that, later on, it can be
split into distributed components with no changes. In
my experience the /ability/ to split has to be
architected into the design right from the start.


> How do you cause two effectors to be engaged "simultaneously"
> (or, with *any* known *fine* timing relationship to each other)
> unless you have a common sense of time?

Read and understand the references I gave you.

The most relevant concept to your issues may be "logical clocks"
which aren't directly coupled to wallclock time.


> How do you triangulate the location of an event if you don't
> know your positions accurately *and* a common reference
> against which to operate?
>
>> Most work stems from Lamport's seminal 1978 paper "Time,
>> clocks, and the ordering of events in a distributed system"
>> which is easily locatable via a google search, e.g.
>> http://www.stanford.edu/class/cs240/readings/lamport.pdf
>>
>> I've used this book, which is sufficiently well-regarded
>> that it has been reprinted four times since 1994.
>> http://www.amazon.co.uk/Distributed-Systems-George-Coulouris/dp/0273760599
>> In particular the chapter 10 "Time and Coordination" has
>> useful info on the general problem, synchronisation,
>> logical time and logical clocks.
>
> Note that the issue isn't *synchronizing* the clocks.  There
> are well-known technologies that will do this.

You asked for a means of synchronising time so that
you could do X, when that is neither possible nor
necessary to achieve X.


> The question I posed was how to measure/verify/validate/troubleshoot
> such a system ONCE DEPLOYED.

Having synchronised time is irrelevant to that.

Understanding "logical clocks" [ibid] may help you.


> E.g., you can measure/verify/validate/troubleshoot how well
> a *hardware* PLL is working with a pair of probes on a bench
> (choice of test equipment can vary).  But, in a physically
> distributed system where the distances are "longer than the
> test leads", just connecting to the signals of interest
> becomes a problem in itself!

Even if you could still have long elastic test leads, the
phase (which is equivalent to time) of the PLL will
change as the elastic lead is elongated.
Now consider three nodes arranged in a triangle so that
each node is attached to two test leads. If only one lead
is lengthened, should the PLL phase change or not?


> When does the "verification device" become a variable as well?

I've spent a significant part of my professional life in
metrology and distributed systems, and the test equipment
and harnesses are *always* part of the system.

I strongly suspect you can achieve your requirements
without having synchronised time; some form of "logical
clock" should be sufficient.

[toc] | [prev] | [next] | [standalone]

#12986

Hi Tom,

On 8/5/2013 4:27 PM, Tom Gardner wrote:
> On 05/08/13 21:19, Don Y wrote:
>> On 8/5/2013 8:53 AM, Tom Gardner wrote:
>>> Since you are building a distributed system, can I suggest
>>> that it would be beneficial to you if you are aware of the
>>> theory and practice that has been developed over the past
>>> 40 years. That might save you time by avoiding going down
>>> blind alleys.
>>>
>>> In particular you will find that you probably *don't need*
>>> *to synchronise time*, if all you are interested in is
>>> which event occurred first.
>>
>> Who claimed that "all (I am) interested in is which
>> event occurred first"?
>
> Because that's all you can realise, in theory and in practice.

Nonsense.  Do you mean there is NO WAY for two physically
separated systems to have a common notion of "the passage of time"?
That a "second" (pick your favorite unit) somehow differs when
experienced by one system and another system "nearby" or "remote"?
That a chunk of quartz magically vibrates differently because
it's in one place and not another?

[Let's not be pedantic.  Assume STP and non-astronomical distances]

I.e., do the hands of Big Ben revolve at a different rate than the
clock in Times Square?

We rely on time (in "systems") to:
- provide a calibrated sense of elapsed time
- to note the order in which observations occurred
- to delay actions wrt "events"
- to have a common sense of "now" and "then"
etc.

> If you want more and achieve it, please do publish: you'll
> be famous (or infamous).

It's already been done, products sold with these capabilities,
industries *relying* on these capabilities, etc.

>> Is that the only use you have for time in a *uniprocessor*
>> system?  Why would you *only* have that need in a *distributed*
>> system?
>
> I'm sorry, I don't understand the question; but I don't
> that that is important.

See above.  Do you claim that a nondistributed system *can't*
have an idea of "passing time"?  Delays?  etc.

>> My system is physically distributed because it is far more
>> economical and practical to design it in that way.
>
> Fine.
>
>> But, that
>> doesn't mean I should not avail myself of the same sorts of
>> capabilities that would have existed had it NOT been
>> distributed.
>
> Because the mere act of physical distribution brings
> constraints that are not present in non-distributed
> systems. Sorry; welcome to this universe.

Sure!  Every physical system imposes constraints!  You can't
resolve a femtosecond on a modern PC.  "Oh, my!  The sky is
falling!!"  So what?  You don't apply a modern PC to a
problem that would *require* that capability.

> In general it is a mistake to create a non-distributed
> system with the assumption that, later on, it can be
> split into distributed components with no changes. In
> my experience the /ability/ to split has to be
> architected into the design right from the start.

Exactly.  Hence designing in this capability FROM THE START.

Node A sends a message to node B:  "This is message 1.
Be prepared to begin the time synchronization protocol"
This happens at some "real" (physical) time as well as
some *local* (wrt A) time.  (It also will have happened
at some local time wrt B but B might not have received
it, yet.)

Node B receives the message AFTER it was sent (since the
sending of the message was the causal event).  Because
the system was designed knowing that being able to observe
the local (i.e., wrt B) clock is important IN A TIMELY,
DETERMINISTIC FASHION, node B makes a note of the time at
which the message arrived (i.e., B's time).

[This can be done with or without hardware assist
depending on how precise your timing requirements are]

Some time later, node A sends a followup message saying
"I noticed that *my* local clock indicated XXX when that
message hit the wire (i.e., cleared the network protocol
stack, was copied into the NIC's output buffer and
eventually pushed out through the PHY onto the wire)"

[Again, this can be done with or without hardware assist]

Because node B is ready for this exchange (he has parsed the
previous message in a timely fashion), when B receives the
second message, it initiates an exchange with node A
and, some time later, knows *when* that message "hits the
wire".

Node A, like node B, knows when it received B's message
(wrt A's local clock).

With these 4 timestamps, we know the transmission delay
between node A and node B along with the "skew" between
A's and B's local clocks.  Each exchange is strictly causal.

A and B can share a common notion of "what time is it"
to a resolution no worse than the round-trip time
from A to B and, in practice, no worse than the one-way
transit time.

[Note that this can be done in any number of equivalent
ways; I've just chosen to illustrate PTP's approach]

>> How do you cause two effectors to be engaged "simultaneously"
>> (or, with *any* known *fine* timing relationship to each other)
>> unless you have a common sense of time?
>
> Read and understand the references I gave you.

Old news.

> The most relevant concept to your issues may be "logical clocks"
> which aren't directly coupled to wallclock time.

You can relate a *physical* clock to wall time in exactly the
same way.

The performance of this sort of algorithm depends on lots
of implementation details:
- how stable are the local oscillators over time
- how much jitter your "timestamping mechanism" exhibits
- how stable the reference oscillator is
- how aggressively you apply corrections to each local clock
   (i.e., time can *never* go backwards!)
- how consistent transport delays are (do they vary over time;
   are they always routed the same way)
- how consistent individual message delays are (is the transport
   time in one direction different from the other direction;
   is the return path different from the forward path)
etc.

But, it's relatively easy to get times (synchronization
"tightness") O(1us).  With care in controlling the traffic
on the network/nodes *and* characterizing the NIC's
involved along with other bits of network fabric, you can
hit the ~200ns range without breaking a sweat.

Beyond that, hardware timestamps in NICs are necessary.
Boundary clocks in switch fabric.  etc.

>> How do you triangulate the location of an event if you don't
>> know your positions accurately *and* a common reference
>> against which to operate?
>>
>>> Most work stems from Lamport's seminal 1978 paper "Time,
>>> clocks, and the ordering of events in a distributed system"
>>> which is easily locatable via a google search, e.g.
>>> http://www.stanford.edu/class/cs240/readings/lamport.pdf

You will note that neither NTP nor PTP violate the concepts he
sets forth.  Rather, they go beyond and allow you to put hard
numbers on actual implementations of "time synchronization".

>>> I've used this book, which is sufficiently well-regarded
>>> that it has been reprinted four times since 1994.
>>> http://www.amazon.co.uk/Distributed-Systems-George-Coulouris/dp/0273760599
>>>
>>> In particular the chapter 10 "Time and Coordination" has
>>> useful info on the general problem, synchronisation,
>>> logical time and logical clocks.
>>
>> Note that the issue isn't *synchronizing* the clocks.  There
>> are well-known technologies that will do this.
>
> You asked for a means of synchronising time so that
> you could do X, when that is neither possible nor
> necessary to achieve X.

No.  Note that I stated (original post) that I *already* do this:

     "I synchronize the 'clocks' on physically distributed processors
     such that two or more different machines can have a very finely
     defined sense of 'synchronized time' between themselves."

I stated how I *measure* the goodness of my implementation:

     "During development, I would measure this time skew (among other
     factors) by locating these devices side-by-side on a workbench
     interconnected by "unquantified" cable.  Then, measuring the
     time difference between to "pulse outputs" that I artificially
     generate on each board."

drawing attention to the fact that my implementation cares not
about how much interconnect cable is used -- where a foot of
such wire could account for a skew of ~1.5ns!

And, how I exercise my implementation to certify its performance
when challenged:

      "So, I could introduce a disturbance to the system and watch to
      see how quickly -- and accurately -- the "clocks" (think FLL and
      PLL) come back into sync."

*Then*, I *ask*:

      "How do I practically do this when the devices are *deployed*
      and physically distant (vs. "electrically distant" as in my
      test case)?"

I.e., electrically, nothing changes between deployment and the
"test bench" -- the same amount of cable can be stretched out
to allow the two nodes to be physically separated (by a distance
not exceeding the length of that cable).  But, I now can't
*practically* verify the relationship of the "clocks" between
the two nodes because "my arms aren't long enough" (nor are
the test probes that I was using!)

>> The question I posed was how to measure/verify/validate/troubleshoot
>> such a system ONCE DEPLOYED.
>
> Having synchronised time is irrelevant to that.

How so?  You are claiming node A and node B don't need a common
sense of time.  I am claiming they *do* -- hence the question.

E.g., the physical oscillators on each node may operate at
harmonically unrelated frequencies.  The timebases in each
device may similarly operate at different frequencies
(and resolutions) -- one might treat time as 2ms quanta
while the other treats it as 3ms quanta/

But, they both think a second is a second.  And, that
"now" is "now" on both devices.

> Understanding "logical clocks" [ibid] may help you.
>
>> E.g., you can measure/verify/validate/troubleshoot how well
>> a *hardware* PLL is working with a pair of probes on a bench
>> (choice of test equipment can vary).  But, in a physically
>> distributed system where the distances are "longer than the
>> test leads", just connecting to the signals of interest
>> becomes a problem in itself!
>
> Even if you could still have long elastic test leads, the
> phase (which is equivalent to time) of the PLL will
> change as the elastic lead is elongated.
> Now consider three nodes arranged in a triangle so that
> each node is attached to two test leads. If only one lead
> is lengthened, should the PLL phase change or not?

You don't lengthen *one* -- you lengthen *both*.  I.e., both
signals experience the same delay in transit... the phase
relationship is retained at the "extended" tips of the
cables (which COULD be different than at the *source* of
leads)

>> When does the "verification device" become a variable as well?
>
> I've spent a significant part of my professional life in
> metrology and distributed systems, and the test equipment
> and harnesses are *always* part of the system.
>
> I strongly suspect you can achieve your requirements
> without having synchronised time; some form of "logical
> clock" should be sufficient.

How do I ensure that the audio signal from one loudspeaker
is "synchronized" with the audio signal emitted by another
(served by a different device)?  That the audio emitted
by both of these is synchronized to the *video* being
displayed by a third device?

If I have a node detonate an explosive charge, how do I
note the precise times at which the shock wave arrives
at two *other* nodes physically separated from each other
and the first node?  (i.e., what if the medium encountered
from detonator to first node is different than medium
from detonator to second)

Getting precise time synchronization is *inexpensive*
so that not taking advantage of it only makes many tasks
unnecessarily harder.

[toc] | [prev] | [next] | [standalone]

#12989

Don Y <this@isnotme.com> writes:
> This happens at some "real" (physical) time as well as
> some *local* (wrt A) time. 

Pop science tells us there is no such thing as '"real" (physical) time'.
Whether A happens before B depends on the location of the observer.

Instead of all this philosophizing, could you instead say what practical
problem you are trying to solve?  Does it still involve lawn sprinklers?
If not, then what does it involve?  Supplying this info will surely
bring some helpful clarity.  Thanks.

[toc] | [prev] | [next] | [standalone]

#12991

On 06/08/13 06:12, Paul Rubin wrote:
> Don Y <this@isnotme.com> writes:
>> This happens at some "real" (physical) time as well as
>> some *local* (wrt A) time.
>
> Pop science tells us there is no such thing as '"real" (physical) time'.
> Whether A happens before B depends on the location of the observer.

Precisely. I've tried to give simple examples that would
lead to that understanding, but have failed. (The last
one in my previous message was about three nodes + three
wires in a triangle, but he read it too quickly and replied
for two nodes + two wires).


> Instead of all this philosophizing, could you instead say what practical
> problem you are trying to solve?  Does it still involve lawn sprinklers?
> If not, then what does it involve?  Supplying this info will surely
> bring some helpful clarity.  Thanks.

I agree, but... his statement of his practical
requirements always includes statements about the
solution.

It would be more helpful if he could define /minimal/
"use cases" stating /only/ what he needs to /achieve/
by observations at specified nodes. If he did that he might
be able to understand known techniques and solutions.

[toc] | [prev] | [next] | [standalone]

#12993

Hi Paul,

On 8/5/2013 10:12 PM, Paul Rubin wrote:
> Don Y <this@isnotme.com> writes:
>> This happens at some "real" (physical) time as well as
>> some *local* (wrt A) time.
>
> Pop science tells us there is no such thing as '"real" (physical) time'.
> Whether A happens before B depends on the location of the observer.
>
> Instead of all this philosophizing, could you instead say what practical
> problem you are trying to solve?

It's not "philosophizing" -- it's a statement of an actual
(standardized) implementation.

The problem I am trying to solve is:

*verifying* that an EXISTING, WORKING IMPLEMENTATION that synchronizes
local clocks on physically (electrically) separated devices to BETTER
THAN MICROSECOND LEVEL tolerances is *still* achieving that level of
performance AFTER DEPLOYMENT... when the devices in question are no
longer *physically* close enough together to be conveniently "probed"
at the same time (as they are when examined on a test bench)

To state this, again, in a way that you can *physically* imagine:

I have two physical, tangible devices that consume power, generate
heat, occupy space, etc.  I have a wire from a digital output
(under processor control) on each device.  The software on each
device will pulse this output periodically (a duty that serves no
actual purpose in the operation of the device) when the local clock
maintained by the software on the device thinks it is "time X"
(X being an actual number).  The crystal oscillators powering
the two devices operate at different nominal frequencies.  The
two devices are interconnected by an ethernet (CAT5) cable
that is >100 ft long (but of uncalibrated length).

When I turn both devices on and wait a while (depends on how I
have the time constants for the capture loop set) and then
connect the two digital outputs to a two channel scope, the
difference in absolute time between the two pulses is O(<1us).
Regardless of how often a new value of X is inserted in the
algorithm (e.g., every second, 5 seconds, 4 hours, etc.).

If I lengthen the cable between the two devices and repeat the
experiment, the results do not change.  I.e., the devices
retain the same degree of synchronization.

Now, when I pick up those two devices and stretch the cable out
so the devices are *really* separated by that 100+ feet, I still
*know* the two pulses are being emitted at the "same" time
(within a microsecond of each other).  But, I can't *show*
that (or verify it!) because my scope probes aren't long enough
to reach both devices at the same time.

If the two devices are deployed at opposite ends of a football
stadium, I have no easy/inexpensive way of verifying that this
part of the devices' functionality is still operating correctly.
Or, of verifying its performance under challenges.

> Does it still involve lawn sprinklers?
> If not, then what does it involve?  Supplying this info will surely
> bring some helpful clarity.  Thanks.

*I* use it to synchronize (control the timing relationship between):
- audio emitted from different "speakers"
- audio from said speakers with video from a "display device"
- audio received from a microphone with video from a camera
- determining the location of a moving target within an arena
- partitioning of resources (in time) across the system

I.e., the same way you use the "system clock" in a single processor
system (if these I/O's were all attached to that *single* system)

I've already engaged in conversations with colleagues and clients
regarding uses to:
- control phased ultrasonic arrays (emit & detect)
- deploy audio in "stadium scale" venues
- geological probes (emit & detect)
- industrial (process) control systems

[When a "capable" processor, memory, ethernet MAC/PHY plus
application specific I/O's can be deployed for a few dollars,
a few square inches of board space and a few watts, the
idea of tackling big projects with *lots* of processors
is inevitable.  (i.e., if you had a tough time dealing with
the *illusion* of parallelism, your head's going to spin
when you have to deal with *true* parallelism!!)]

Of course, you could also use this as a "better NTP".  And, other
industries have numerous applications for the same technology
(which is why it has been *standardized* and vendors are offering
COMMERCIAL PRODUCTS that implement this mechanism).  Use your
favorite search engine to research "PTP" or "1588" or "1588v2"
for more varied applications.

Common application domains include:
- automation technology (gasp!)
- Motion Control (think servo controls)
- Process Control
- Robotics
- CIPsync (DeviceNet)
- PROFInet
- Ethernet Powerlink
- LXI (instrumentation)
- Energy distribution and monitoring (SmartGrid)
- Geoscience
- Telecom
- A/V (there's a formal standard that eludes me at the moment)
- Military (gasp!)

Having a common, shared, synchronized notion of "time" across
physically separate devices is a powerful mechanism!  It affords
the system designer with capabilities that wouldn't be practical
(or economical) otherwise!

Some folks have taken this to an even *finer* degree of precision.
For example, those silly folks at CERN use a (greatly) enhanced PTP
implementation ("White Rabbit") that delivers sub-NANOsecond
(i.e., 10-9 seconds lest you didn't catch that on first read)
synchronization over distances of KILOMETERS.  (and here *I* was
aiming to do sub-MICROseconds over a CITY BLOCK!  :-/  Lack of
ambition??)

Someone should lecture those CERN folks (academics??) as to the
RELATIVISTIC IMPLICATIONS of their hair-brained scheme.  Don't they
*know* it *CAN'T WORK*???  Gotta wonder what those crazy europeans
are *smoking* to come up with such a silly idea!!  Maybe they need
a theoretical physicist on their staff...

Likewise, tell the folks at Cisco, Juniper, etc. that this is
a fool's goal before their stockholders discover they've spent
all that NRE money developing REAL PRODUCTS that implement these
*impossible* PTP mechanisms!!  There's NO MARKET FOR THEM!!!
You *can't* synchronize time over a distance!!  (or, so I'm told)

(Really??!)

To play Tom's card:  find the references.  Read them.  UNDERSTAND
THEM.  *Then*, come back and point out the FLAWS in their
implementations!  (because you obviously are not convinced that
*I* have already done this!)  Unless you can point out why
1588 *can't* work or why White Rabbit is pure fantasy, you
won't be able to offer me any useful comment on the problem
I'm addressing (because I *see* 1588 working in front of my eyes
so regard comments that it "can't work" as in sharp contrast
with "reality")

Again, to be specific:  tell me how I can verify/measure the
time relationship between those two pulse outputs WHEN THEY ARE
ACTUALLY SEPARATED BY HUNDREDS OF FEET.

--don

[toc] | [prev] | [next] | [standalone]

#12996

On 06/08/13 09:47, Don Y wrote:
> Hi Paul,
>
> On 8/5/2013 10:12 PM, Paul Rubin wrote:
>> Don Y <this@isnotme.com> writes:
>>> This happens at some "real" (physical) time as well as
>>> some *local* (wrt A) time.
>>
>> Pop science tells us there is no such thing as '"real" (physical) time'.
>> Whether A happens before B depends on the location of the observer.
>>
>> Instead of all this philosophizing, could you instead say what practical
>> problem you are trying to solve?
>
> It's not "philosophizing" -- it's a statement of an actual
> (standardized) implementation.
>
> The problem I am trying to solve is:
>
> *verifying* that an EXISTING, WORKING IMPLEMENTATION

I'm confused. I wrote:
  > In general it is a mistake to create a non-distributed
  > system with the assumption that, later on, it can be
  > split into distributed components with no changes. In
  > my experience the /ability/ to split has to be
  > architected into the design right from the start.
and you replied
  > Exactly.  Hence designing in this capability FROM THE START.

If you had built in the capability from the start, you
wouldn't have to be asking these questions.


> Some folks have taken this to an even *finer* degree of precision.
> For example, those silly folks at CERN use a (greatly) enhanced PTP
> implementation ("White Rabbit") that delivers sub-NANOsecond
> (i.e., 10-9 seconds lest you didn't catch that on first read)
> synchronization over distances of KILOMETERS.  (and here *I* was
> aiming to do sub-MICROseconds over a CITY BLOCK!  :-/  Lack of
> ambition??)

I think you'll find they aren't doing what you think they
are doing :)

[toc] | [prev] | [next] | [standalone]

#12998

Hi Tom,

On 8/6/2013 2:08 AM, Tom Gardner wrote:
> On 06/08/13 09:47, Don Y wrote:
>>> Pop science tells us there is no such thing as '"real" (physical) time'.
>>> Whether A happens before B depends on the location of the observer.
>>>
>>> Instead of all this philosophizing, could you instead say what practical
>>> problem you are trying to solve?
>>
>> It's not "philosophizing" -- it's a statement of an actual
>> (standardized) implementation.
>>
>> The problem I am trying to solve is:
>>
>> *verifying* that an EXISTING, WORKING IMPLEMENTATION
>
> I'm confused. I wrote:
>   > In general it is a mistake to create a non-distributed
>   > system with the assumption that, later on, it can be
>   > split into distributed components with no changes. In
>   > my experience the /ability/ to split has to be
>   > architected into the design right from the start.
> and you replied
>   > Exactly.  Hence designing in this capability FROM THE START.
>
> If you had built in the capability from the start, you
> wouldn't have to be asking these questions.

I can prove that the design and implementation works.
But, I can't ECONOMICALLY troubleshoot it after deployment.
That doesn't mean I can't take advantage of a closely
synchronized timebase over physically distributed devices.
It just means that any problems suspected of being caused
by dissynchrony would be more expensive to troubleshoot
than I would like.  E.g., embelish the existing run-time
diagnostics to provide more detailed information and
design a special piece of test equipment and possibly require
special training to use it.  (of course, if the subsystem
*doesn't* fail in an invisible way, then this isn't necessary)

>> Some folks have taken this to an even *finer* degree of precision.
>> For example, those silly folks at CERN use a (greatly) enhanced PTP
>> implementation ("White Rabbit") that delivers sub-NANOsecond
>> (i.e., 10-9 seconds lest you didn't catch that on first read)
>> synchronization over distances of KILOMETERS.  (and here *I* was
>> aiming to do sub-MICROseconds over a CITY BLOCK!  :-/  Lack of
>> ambition??)
>
> I think you'll find they aren't doing what you think they
> are doing :)

Great!  Then you should be more than capable of telling us *why*
this ISN'T the case!  I can provide URLs if you'd like...

[toc] | [prev] | [next] | [standalone]

#12999

Hi Tom,

On 8/6/2013 2:25 AM, Don Y wrote:
>> I think you'll find they aren't doing what you think they
>> are doing :)
>
> Great!  Then you should be more than capable of telling us *why*
> this ISN'T the case!  I can provide URLs if you'd like...

For example:

<http://www.ntp-servers.com/file_upl/PDF/Seminaria/Elproma%20CERN%20(White_Rabbit).pdf>

     "WR combines IEEE1588-v2 timestamping with a synchronous
     physical layer based on Synchronous Ethernet. Sub-nanosecond
     accuracy is achieved by actively compensating the link delay
     using phase measurements done with Digital Dual Mixer Time
     Difference (DDMTD) phase detectors and on-line calibration
     of the gigabit transceivers' latencies. The measurements
     presented in the article show a long-term master-to-slave
     offset below 1 ns, while the precision is better than 20 ps
     rms for a 5 km single mode fiber connection."

Yeah, they use Gb fiber instead of my 100Mb copper but the
idea of synchronizing (the devices' sense of) "time" to this
level of precision is the same.

In the section entitled "Applications" (uncluttered by the
technical description of the implementation):

     "One field of applications of WR are distributed data
     acquisition systems, for example the OASIS system at
     CERN, which acts as a huge oscilloscope measuring thousands
     of signals coming from sources distant by several kilometers.
     WR can be used to accurately time tag the blocks of samples
     at the ADC cards and produce nanosecond-accurate time tags
     for the external trigger signals. Having the sample blocks
     with associated time tags, one can reconstruct the original
     time relations between the signals and the triggers in
     software and present the measurements to the operator as if
     it was displayed on a typical oscilloscope."

You will note the illustration (Fig 13) accompanying the text
clearly shows physically distributed "ADC cards", trigger inputs,
etc. *all* connected to the "White Rabbit Network" which provides
the "synchronized (nanosecond level) notion of time" to all
of these devices.  "Thousands of signals" (no doubt from a
large accelerator?) "coming from sources distant by several
kilometers".

Sure *sounds* like what I'm trying to do (on a much smaller
physical scale and coarser sense of time).

Time for me to get some *real* work done...  Enjoy your reading!

--don

[toc] | [prev] | [next] | [standalone]

#13000

On 06/08/13 10:25, Don Y wrote:
> Hi Tom,
>
> On 8/6/2013 2:08 AM, Tom Gardner wrote:
>> On 06/08/13 09:47, Don Y wrote:
>>>> Pop science tells us there is no such thing as '"real" (physical) time'.
>>>> Whether A happens before B depends on the location of the observer.
>>>>
>>>> Instead of all this philosophizing, could you instead say what practical
>>>> problem you are trying to solve?
>>>
>>> It's not "philosophizing" -- it's a statement of an actual
>>> (standardized) implementation.
>>>
>>> The problem I am trying to solve is:
>>>
>>> *verifying* that an EXISTING, WORKING IMPLEMENTATION
>>
>> I'm confused. I wrote:
>>   > In general it is a mistake to create a non-distributed
>>   > system with the assumption that, later on, it can be
>>   > split into distributed components with no changes. In
>>   > my experience the /ability/ to split has to be
>>   > architected into the design right from the start.
>> and you replied
>>   > Exactly.  Hence designing in this capability FROM THE START.
>>
>> If you had built in the capability from the start, you
>> wouldn't have to be asking these questions.
>
> I can prove that the design and implementation works.

... as a non-distributed system.

> But, I can't ECONOMICALLY troubleshoot it after deployment.

So you didn't architect it and design it (and then
implement it) as a distributed system.

But maybe distributed deployment has been added as a
requirement at a relative late stage.

[toc] | [prev] | [next] | [standalone]

#13001

Hi Tom,

On 8/6/2013 2:49 AM, Tom Gardner wrote:
> On 06/08/13 10:25, Don Y wrote:
>> Hi Tom,
>>
>> On 8/6/2013 2:08 AM, Tom Gardner wrote:
>>> On 06/08/13 09:47, Don Y wrote:
>>>>> Pop science tells us there is no such thing as '"real" (physical)
>>>>> time'.
>>>>> Whether A happens before B depends on the location of the observer.
>>>>>
>>>>> Instead of all this philosophizing, could you instead say what
>>>>> practical
>>>>> problem you are trying to solve?
>>>>
>>>> It's not "philosophizing" -- it's a statement of an actual
>>>> (standardized) implementation.
>>>>
>>>> The problem I am trying to solve is:
>>>>
>>>> *verifying* that an EXISTING, WORKING IMPLEMENTATION
>>>
>>> I'm confused. I wrote:
>>>   > In general it is a mistake to create a non-distributed
>>>   > system with the assumption that, later on, it can be
>>>   > split into distributed components with no changes. In
>>>   > my experience the /ability/ to split has to be
>>>   > architected into the design right from the start.
>>> and you replied
>>>   > Exactly.  Hence designing in this capability FROM THE START.
>>>
>>> If you had built in the capability from the start, you
>>> wouldn't have to be asking these questions.
>>
>> I can prove that the design and implementation works.
>
> .... as a non-distributed system.

No, as a distributed system.  I can insert hundreds of feet of
wire between nodes and *prove* it works.  I can then stretch out
that wire and demonstrate that it *still* works (because the
only thing that has changed is the physical locations of the
individual nodes).

>> But, I can't ECONOMICALLY troubleshoot it after deployment.
>
> So you didn't architect it and design it (and then
> implement it) as a distributed system.

How does that follow from my statements?  If I use an expensive piece
of test equipment, I can verify it operates in it's physically
distributed state.  I can use an *inexpensive* piece of test
equipment to prove it works when electrically connected in the
EXACT SAME MANNER (there are no "wireless" paths by which data
are exchanged between these nodes).

I would like to be able to use a similar piece of inexpensive
test equipment to troubleshoot WHILE DEPLOYED (and, in fact, I
*can* use the same piece of equipment but it requires bringing
a long spool of wire along)

> But maybe distributed deployment has been added as a
> requirement at a relative late stage.

You *claim* to have been involved in the design of distributed
systems.  Did you design and test by locating one device at one
end of a football field and another at the other end -- running
back and forth throughout the days, weeks, months that it took
to develop the software and hardware?  Or, did you *simulate*
a deployed configuration ELECTRICALLY, on a bench, so you could
sit, comfortably, and interact with every device in the system
without leaving your chair?  Devising tests in that simulated
(though electrically and functionally equivalent) environment
to prove that your design works and applying those tests to
the system *before* undertaking the cost of deployment?

Sure sounds like you feel backed into a corner and are trying
to avoid defending your EXPLICIT *claims* regarding the
original question.  That's OK.  We can wait for your
clarification.

Until you have something pertinent to add...  (note:  the
email addresses of the CERN folks are probably available if
you'd like to lecture them on why their idea won't work...)

[toc] | [prev] | [next] | [standalone]

#13002

On 06/08/13 11:06, Don Y wrote:
> You *claim* to have been involved in the design of distributed
> systems.  Did you design and test by locating one device at one
> end of a football field and another at the other end -- running
> back and forth throughout the days, weeks, months that it took
> to develop the software and hardware?  Or, did you *simulate*
> a deployed configuration ELECTRICALLY, on a bench, so you could
> sit, comfortably, and interact with every device in the system
> without leaving your chair?  Devising tests in that simulated
> (though electrically and functionally equivalent) environment
> to prove that your design works and applying those tests to
> the system *before* undertaking the cost of deployment?

I architected designed and implemented a global
order fulfillment system with lax time constraints other
than happens-before at a single global synchronisation
location.

I architected designed and implemented telco
systems that have soft real-time timing constraints, are
completely reliant on happens-before for correct operation,
and, by definition, there is no possibility of a
single system-wide global time. However, time differentials
were directly translated into money: that was the whole
point of the system.

Great care was taken to ensure high availability, which
implies remote[1] logging/diagnosis, remote debugging, remote
control/recovery and remote upgrading. In addition, I insisted
on building in tracing techniques that were able to prove
that our system behaved correctly and that faults lay in
other companies' systems that were attached to our
system - which was commercially highly beneficial.

I suspect that would be more than sufficient for
your requirements.

[1] Remote implies in a different country (or
even continent), some I wouldn't visit voluntarily.

[toc] | [prev] | [next] | [standalone]

#13006

On 8/6/2013 3:57 AM, Tom Gardner wrote:
> On 06/08/13 11:06, Don Y wrote:
>> You *claim* to have been involved in the design of distributed
>> systems.  Did you design and test by locating one device at one
>> end of a football field and another at the other end -- running
>> back and forth throughout the days, weeks, months that it took
>> to develop the software and hardware?  Or, did you *simulate*
>> a deployed configuration ELECTRICALLY, on a bench, so you could
>> sit, comfortably, and interact with every device in the system
>> without leaving your chair?  Devising tests in that simulated
>> (though electrically and functionally equivalent) environment
>> to prove that your design works and applying those tests to
>> the system *before* undertaking the cost of deployment?
>
> I architected designed and implemented a global
> order fulfillment system with lax time constraints other
> than happens-before at a single global synchronisation
> location.
>
> I architected designed and implemented telco
> systems that have soft real-time timing constraints, are
> completely reliant on happens-before for correct operation,
> and, by definition, there is no possibility of a
> single system-wide global time. However, time differentials
> were directly translated into money: that was the whole
> point of the system.
>
> Great care was taken to ensure high availability, which
> implies remote[1] logging/diagnosis, remote debugging, remote
> control/recovery and remote upgrading. In addition, I insisted
> on building in tracing techniques that were able to prove
> that our system behaved correctly and that faults lay in
> other companies' systems that were attached to our
> system - which was commercially highly beneficial.
>
> I suspect that would be more than sufficient for
> your requirements.

And, you designed and developed these systems PHYSICALLY
DISTRIBUTED (i.e., they were in geographically different
locations WHILE YOU WERE DEVELOPING THE HARDWARE AND SOFTWARE)?
I.e., you didn't take the OBVIOUS shortcut of having multiple
nodes "in the same building, lab, workbench WHILE DEVELOPING,
VERIFYING and CERTIFYING their performance -- which is what you
frowned on *me* doing??)

> [1] Remote implies in a different country (or
> even continent), some I wouldn't visit voluntarily.

So, on day 0, you *deployed* the node (yet to be fabricated)
to that remote location.  Then, started working on the software
to make it work?

C'mon, stop squirming and read what you *accused* me of doing.
Then, tell your maker that you *didn't* do exactly he same
thing.  Yet, you claim *your* system was "designed as a distributed
system" and IMPLY that mine wasn't?

Still waiting for that refutation of the CERN system, Tom.

Hey, if you don't believe what people write (especially a
bunch a physicists), perhaps you can purchase a 1588 reference
design and see for yourself that it works.  Then, you can
explain why it *doesn't*!

Here are some from Freescale:
<http://www.freescale.com/webapp/sps/site/application.jsp?code=APLIEEE_1588>

A third-party design on similar hardware:
<http://pcsemicon.blogspot.com/2010/06/symmetricom-announces-ieee-1588-ptp.html>

Here's an (old) reference to a design from NatSemi:
<http://www.electronicproducts.com/Software/Switch_design_follows_IEEE_1588_timing.aspx>

Lots of pointers to design data for TI's 1588 current offerings:
<http://www.ti.com/ww/en/analog/interface/ethernet.shtml>

I'm sure with all this to choose from (plus what you can
undoubtedly find with *your* search engine) you'll be
able to identify the common "deception" that all of these
things MUST be relying on!

I'll be waiting for that "pertinent information"...

[toc] | [prev] | [next] | [standalone]

#13011

On 06/08/13 18:28, Don Y wrote:
> On 8/6/2013 3:57 AM, Tom Gardner wrote:
>> On 06/08/13 11:06, Don Y wrote:
>>> You *claim* to have been involved in the design of distributed
>>> systems.  Did you design and test by locating one device at one
>>> end of a football field and another at the other end -- running
>>> back and forth throughout the days, weeks, months that it took
>>> to develop the software and hardware?  Or, did you *simulate*
>>> a deployed configuration ELECTRICALLY, on a bench, so you could
>>> sit, comfortably, and interact with every device in the system
>>> without leaving your chair?  Devising tests in that simulated
>>> (though electrically and functionally equivalent) environment
>>> to prove that your design works and applying those tests to
>>> the system *before* undertaking the cost of deployment?
>>
>> I architected designed and implemented a global
>> order fulfillment system with lax time constraints other
>> than happens-before at a single global synchronisation
>> location.
>>
>> I architected designed and implemented telco
>> systems that have soft real-time timing constraints, are
>> completely reliant on happens-before for correct operation,
>> and, by definition, there is no possibility of a
>> single system-wide global time. However, time differentials
>> were directly translated into money: that was the whole
>> point of the system.
>>
>> Great care was taken to ensure high availability, which
>> implies remote[1] logging/diagnosis, remote debugging, remote
>> control/recovery and remote upgrading. In addition, I insisted
>> on building in tracing techniques that were able to prove
>> that our system behaved correctly and that faults lay in
>> other companies' systems that were attached to our
>> system - which was commercially highly beneficial.
>>
>> I suspect that would be more than sufficient for
>> your requirements.
>
> And, you designed and developed these systems PHYSICALLY
> DISTRIBUTED (i.e., they were in geographically different
> locations WHILE YOU WERE DEVELOPING THE HARDWARE AND SOFTWARE)?

Yes. Just so. Of course.

Because that's the way to uncover hidden assumptions
and find unpleasant surprises when there is time
to fix them easily.

During initial feasibility tests different parts of the
system (owned by different companies) were in different
countries.

Once initial feasibility tests had shown basic functionality
we proceeded to full elaboration of the functionality,
occasionally re-testing the interworking during development.

In addition, before we even decided whether or not to
use some components we tested their properties when
different parts of the system were terminated with
extreme prejudice, e.g. killing processes, yanking
network and power cables. We didn't bother to synchronise
clocks and time since we had made sure that it
wasn't a necessary precondition for correct operation.

Such testing on known flaky hardware caused a
component manufacturer to determine they had
a problem and to fix it before anybody noticed.
They were most grateful to us for saving them
potential embarrassment!


> I.e., you didn't take the OBVIOUS shortcut of having multiple
> nodes "in the same building, lab, workbench WHILE DEVELOPING,
> VERIFYING and CERTIFYING their performance --

Sure we did, sometimes. But we never bothered to ensure
clock nor time synchronisation any more than we bothered to
ensure identical hardware or paint colour schemes - because
the system was architected to be immune to such differences.


> which is what you frowned on *me* doing??)

No, I didn't. I referred to architecting and designing
(so the problems aren't there when implementing, verifying
and certifying).


>> [1] Remote implies in a different country (or
>> even continent), some I wouldn't visit voluntarily.
>
> So, on day 0, you *deployed* the node (yet to be fabricated)
> to that remote location.  Then, started working on the software
> to make it work?

No, that would have been unnecessarily risky.

We first deployed trivial functionality on a
distributed system on month -6, as outlined above.


> C'mon, stop squirming and read what you *accused* me of doing.
> Then, tell your maker that you *didn't* do exactly he same
> thing.  Yet, you claim *your* system was "designed as a distributed
> system" and IMPLY that mine wasn't?

Well, from what you say you appear to be unsure how
to assure that your system works when distributed.
We didn't have such issues because of the architecture
and design, supported by early verification.


> Still waiting for that refutation of the CERN system, Tom.

That would require
  - my understanding the detail of what CERN was *and was not*
    assuming and doing
  - my understanding the details of your system, the
    assumptions implicitly buried within it, plus
    your misunderstandings
  - my trying to reconcile all of that lot

Sorry. My life is too short for me to solve _your_ problems!
I'm happy to give you pointers to how such issues have been
successfully dealt with over the decades - but you have to
solve your problems.

[toc] | [prev] | [next] | [standalone]

Page 4 of 6 — ← Prev page 1 2 3 [4] 5 6  Next page →

Back to top | Article view | comp.arch.embedded

csiph-web