Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.arch.embedded > #12986
| From | Don Y <this@isnotme.com> |
|---|---|
| Newsgroups | comp.arch.embedded, comp.realtime |
| Subject | Re: Comparing phase of physically distant signals |
| Date | 2013-08-05 20:09 -0700 |
| Organization | Aioe.org NNTP Server |
| Message-ID | <ktppcd$3vf$1@speranza.aioe.org> (permalink) |
| References | <kticf7$hg3$1@speranza.aioe.org> <WTPLt.1755$3z5.266@fx35.am4> <ktp1ci$dsj$1@speranza.aioe.org> <jxWLt.346$Yd5.283@fx30.am4> |
Cross-posted to 2 groups.
Hi Tom,
On 8/5/2013 4:27 PM, Tom Gardner wrote:
> On 05/08/13 21:19, Don Y wrote:
>> On 8/5/2013 8:53 AM, Tom Gardner wrote:
>>> Since you are building a distributed system, can I suggest
>>> that it would be beneficial to you if you are aware of the
>>> theory and practice that has been developed over the past
>>> 40 years. That might save you time by avoiding going down
>>> blind alleys.
>>>
>>> In particular you will find that you probably *don't need*
>>> *to synchronise time*, if all you are interested in is
>>> which event occurred first.
>>
>> Who claimed that "all (I am) interested in is which
>> event occurred first"?
>
> Because that's all you can realise, in theory and in practice.
Nonsense. Do you mean there is NO WAY for two physically
separated systems to have a common notion of "the passage of time"?
That a "second" (pick your favorite unit) somehow differs when
experienced by one system and another system "nearby" or "remote"?
That a chunk of quartz magically vibrates differently because
it's in one place and not another?
[Let's not be pedantic. Assume STP and non-astronomical distances]
I.e., do the hands of Big Ben revolve at a different rate than the
clock in Times Square?
We rely on time (in "systems") to:
- provide a calibrated sense of elapsed time
- to note the order in which observations occurred
- to delay actions wrt "events"
- to have a common sense of "now" and "then"
etc.
> If you want more and achieve it, please do publish: you'll
> be famous (or infamous).
It's already been done, products sold with these capabilities,
industries *relying* on these capabilities, etc.
>> Is that the only use you have for time in a *uniprocessor*
>> system? Why would you *only* have that need in a *distributed*
>> system?
>
> I'm sorry, I don't understand the question; but I don't
> that that is important.
See above. Do you claim that a nondistributed system *can't*
have an idea of "passing time"? Delays? etc.
>> My system is physically distributed because it is far more
>> economical and practical to design it in that way.
>
> Fine.
>
>> But, that
>> doesn't mean I should not avail myself of the same sorts of
>> capabilities that would have existed had it NOT been
>> distributed.
>
> Because the mere act of physical distribution brings
> constraints that are not present in non-distributed
> systems. Sorry; welcome to this universe.
Sure! Every physical system imposes constraints! You can't
resolve a femtosecond on a modern PC. "Oh, my! The sky is
falling!!" So what? You don't apply a modern PC to a
problem that would *require* that capability.
> In general it is a mistake to create a non-distributed
> system with the assumption that, later on, it can be
> split into distributed components with no changes. In
> my experience the /ability/ to split has to be
> architected into the design right from the start.
Exactly. Hence designing in this capability FROM THE START.
Node A sends a message to node B: "This is message 1.
Be prepared to begin the time synchronization protocol"
This happens at some "real" (physical) time as well as
some *local* (wrt A) time. (It also will have happened
at some local time wrt B but B might not have received
it, yet.)
Node B receives the message AFTER it was sent (since the
sending of the message was the causal event). Because
the system was designed knowing that being able to observe
the local (i.e., wrt B) clock is important IN A TIMELY,
DETERMINISTIC FASHION, node B makes a note of the time at
which the message arrived (i.e., B's time).
[This can be done with or without hardware assist
depending on how precise your timing requirements are]
Some time later, node A sends a followup message saying
"I noticed that *my* local clock indicated XXX when that
message hit the wire (i.e., cleared the network protocol
stack, was copied into the NIC's output buffer and
eventually pushed out through the PHY onto the wire)"
[Again, this can be done with or without hardware assist]
Because node B is ready for this exchange (he has parsed the
previous message in a timely fashion), when B receives the
second message, it initiates an exchange with node A
and, some time later, knows *when* that message "hits the
wire".
Node A, like node B, knows when it received B's message
(wrt A's local clock).
With these 4 timestamps, we know the transmission delay
between node A and node B along with the "skew" between
A's and B's local clocks. Each exchange is strictly causal.
A and B can share a common notion of "what time is it"
to a resolution no worse than the round-trip time
from A to B and, in practice, no worse than the one-way
transit time.
[Note that this can be done in any number of equivalent
ways; I've just chosen to illustrate PTP's approach]
>> How do you cause two effectors to be engaged "simultaneously"
>> (or, with *any* known *fine* timing relationship to each other)
>> unless you have a common sense of time?
>
> Read and understand the references I gave you.
Old news.
> The most relevant concept to your issues may be "logical clocks"
> which aren't directly coupled to wallclock time.
You can relate a *physical* clock to wall time in exactly the
same way.
The performance of this sort of algorithm depends on lots
of implementation details:
- how stable are the local oscillators over time
- how much jitter your "timestamping mechanism" exhibits
- how stable the reference oscillator is
- how aggressively you apply corrections to each local clock
(i.e., time can *never* go backwards!)
- how consistent transport delays are (do they vary over time;
are they always routed the same way)
- how consistent individual message delays are (is the transport
time in one direction different from the other direction;
is the return path different from the forward path)
etc.
But, it's relatively easy to get times (synchronization
"tightness") O(1us). With care in controlling the traffic
on the network/nodes *and* characterizing the NIC's
involved along with other bits of network fabric, you can
hit the ~200ns range without breaking a sweat.
Beyond that, hardware timestamps in NICs are necessary.
Boundary clocks in switch fabric. etc.
>> How do you triangulate the location of an event if you don't
>> know your positions accurately *and* a common reference
>> against which to operate?
>>
>>> Most work stems from Lamport's seminal 1978 paper "Time,
>>> clocks, and the ordering of events in a distributed system"
>>> which is easily locatable via a google search, e.g.
>>> http://www.stanford.edu/class/cs240/readings/lamport.pdf
You will note that neither NTP nor PTP violate the concepts he
sets forth. Rather, they go beyond and allow you to put hard
numbers on actual implementations of "time synchronization".
>>> I've used this book, which is sufficiently well-regarded
>>> that it has been reprinted four times since 1994.
>>> http://www.amazon.co.uk/Distributed-Systems-George-Coulouris/dp/0273760599
>>>
>>> In particular the chapter 10 "Time and Coordination" has
>>> useful info on the general problem, synchronisation,
>>> logical time and logical clocks.
>>
>> Note that the issue isn't *synchronizing* the clocks. There
>> are well-known technologies that will do this.
>
> You asked for a means of synchronising time so that
> you could do X, when that is neither possible nor
> necessary to achieve X.
No. Note that I stated (original post) that I *already* do this:
"I synchronize the 'clocks' on physically distributed processors
such that two or more different machines can have a very finely
defined sense of 'synchronized time' between themselves."
I stated how I *measure* the goodness of my implementation:
"During development, I would measure this time skew (among other
factors) by locating these devices side-by-side on a workbench
interconnected by "unquantified" cable. Then, measuring the
time difference between to "pulse outputs" that I artificially
generate on each board."
drawing attention to the fact that my implementation cares not
about how much interconnect cable is used -- where a foot of
such wire could account for a skew of ~1.5ns!
And, how I exercise my implementation to certify its performance
when challenged:
"So, I could introduce a disturbance to the system and watch to
see how quickly -- and accurately -- the "clocks" (think FLL and
PLL) come back into sync."
*Then*, I *ask*:
"How do I practically do this when the devices are *deployed*
and physically distant (vs. "electrically distant" as in my
test case)?"
I.e., electrically, nothing changes between deployment and the
"test bench" -- the same amount of cable can be stretched out
to allow the two nodes to be physically separated (by a distance
not exceeding the length of that cable). But, I now can't
*practically* verify the relationship of the "clocks" between
the two nodes because "my arms aren't long enough" (nor are
the test probes that I was using!)
>> The question I posed was how to measure/verify/validate/troubleshoot
>> such a system ONCE DEPLOYED.
>
> Having synchronised time is irrelevant to that.
How so? You are claiming node A and node B don't need a common
sense of time. I am claiming they *do* -- hence the question.
E.g., the physical oscillators on each node may operate at
harmonically unrelated frequencies. The timebases in each
device may similarly operate at different frequencies
(and resolutions) -- one might treat time as 2ms quanta
while the other treats it as 3ms quanta/
But, they both think a second is a second. And, that
"now" is "now" on both devices.
> Understanding "logical clocks" [ibid] may help you.
>
>> E.g., you can measure/verify/validate/troubleshoot how well
>> a *hardware* PLL is working with a pair of probes on a bench
>> (choice of test equipment can vary). But, in a physically
>> distributed system where the distances are "longer than the
>> test leads", just connecting to the signals of interest
>> becomes a problem in itself!
>
> Even if you could still have long elastic test leads, the
> phase (which is equivalent to time) of the PLL will
> change as the elastic lead is elongated.
> Now consider three nodes arranged in a triangle so that
> each node is attached to two test leads. If only one lead
> is lengthened, should the PLL phase change or not?
You don't lengthen *one* -- you lengthen *both*. I.e., both
signals experience the same delay in transit... the phase
relationship is retained at the "extended" tips of the
cables (which COULD be different than at the *source* of
leads)
>> When does the "verification device" become a variable as well?
>
> I've spent a significant part of my professional life in
> metrology and distributed systems, and the test equipment
> and harnesses are *always* part of the system.
>
> I strongly suspect you can achieve your requirements
> without having synchronised time; some form of "logical
> clock" should be sufficient.
How do I ensure that the audio signal from one loudspeaker
is "synchronized" with the audio signal emitted by another
(served by a different device)? That the audio emitted
by both of these is synchronized to the *video* being
displayed by a third device?
If I have a node detonate an explosive charge, how do I
note the precise times at which the shock wave arrives
at two *other* nodes physically separated from each other
and the first node? (i.e., what if the medium encountered
from detonator to first node is different than medium
from detonator to second)
Getting precise time synchronization is *inexpensive*
so that not taking advantage of it only makes many tasks
unnecessarily harder.
Back to comp.arch.embedded | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 00:45 -0700
Re: Comparing phase of physically distant signals Jan Panteltje <pNaonStpealmtje@yahoo.com> - 2013-08-03 07:51 +0000
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 09:58 +0100
Re: Comparing phase of physically distant signals upsidedown@downunder.com - 2013-08-03 13:21 +0300
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 12:16 +0100
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 11:56 -0700
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 20:17 +0100
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 13:41 -0700
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 22:46 +0100
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 14:57 -0700
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 23:09 +0100
Re: Comparing phase of physically distant signals upsidedown@downunder.com - 2013-08-03 13:58 +0300
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 12:30 +0100
Re: Comparing phase of physically distant signals upsidedown@downunder.com - 2013-08-03 16:08 +0300
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 14:22 +0100
Re: Comparing phase of physically distant signals upsidedown@downunder.com - 2013-08-03 16:38 +0300
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 10:46 -0700
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 10:40 -0700
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 20:04 +0100
Re: Comparing phase of physically distant signals Hans-Bernhard Bröker <HBBroeker@t-online.de> - 2013-08-04 15:55 +0200
Re: Comparing phase of physically distant signals upsidedown@downunder.com - 2013-08-04 19:42 +0300
Re: Comparing phase of physically distant signals Hans-Bernhard Bröker <HBBroeker@t-online.de> - 2013-08-04 21:08 +0200
Re: Comparing phase of physically distant signals upsidedown@downunder.com - 2013-08-05 15:28 +0300
Re: Comparing phase of physically distant signals Hans-Bernhard Bröker <HBBroeker@t-online.de> - 2013-08-05 22:17 +0200
Re: Comparing phase of physically distant signals Richard Damon <Richard@Damon-Family.org> - 2013-08-05 23:20 -0400
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 14:26 +0100
Re: Comparing phase of physically distant signals George Neuner <gneuner2@comcast.net> - 2013-08-03 14:47 -0400
Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-03 08:17 -0700
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 12:08 -0700
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 20:37 +0100
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 12:57 -0700
Re: Comparing phase of physically distant signals Hans-Bernhard Bröker <HBBroeker@t-online.de> - 2013-08-04 15:58 +0200
Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-03 12:44 -0700
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 18:36 -0700
Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-04 09:56 -0700
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-05 14:53 -0700
Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-05 15:38 -0700
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-07 01:25 -0700
Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-07 08:06 -0700
Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-07 09:21 -0700
Re: Comparing phase of physically distant signals josephkk <joseph_barrett@sbcglobal.net> - 2013-08-03 15:40 +0000
Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-03 10:05 -0700
Re: Comparing phase of physically distant signals John S <Sophi.2@invalid.org> - 2013-08-03 12:13 -0500
Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-03 10:26 -0700
Re: Comparing phase of physically distant signals John S <Sophi.2@invalid.org> - 2013-08-03 12:51 -0500
Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-03 11:54 -0700
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-03 20:27 +0100
Re: Comparing phase of physically distant signals George Neuner <gneuner2@comcast.net> - 2013-08-05 03:26 -0400
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-05 10:34 +0100
Re: Comparing phase of physically distant signals Richard Damon <Richard@Damon-Family.org> - 2013-08-05 23:29 -0400
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-06 08:53 +0100
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 12:49 -0700
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 12:34 -0700
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 12:18 -0700
Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-03 12:52 -0700
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 13:21 -0700
Re: Comparing phase of physically distant signals Jeff Liebermann <jeffl@cruzio.com> - 2013-08-03 13:31 -0700
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-03 14:46 -0700
Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-03 15:03 -0700
Re: Comparing phase of physically distant signals Jeff Liebermann <jeffl@cruzio.com> - 2013-08-03 17:25 -0700
Re: Comparing phase of physically distant signals Joerg <invalid@invalid.invalid> - 2013-08-03 14:50 -0700
Re: Comparing phase of physically distant signals "Paul E. Bennett" <Paul_E.Bennett@topmail.co.uk> - 2013-08-04 11:06 +0100
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-04 12:19 -0700
Re: Comparing phase of physically distant signals John Larkin <jjlarkin@highNOTlandTHIStechnologyPART.com> - 2013-08-03 11:47 -0700
Re: Comparing phase of physically distant signals Richard Damon <Richard@Damon-Family.org> - 2013-08-03 18:23 -0400
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-05 16:53 +0100
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-05 13:19 -0700
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-06 00:27 +0100
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-05 20:09 -0700
Re: Comparing phase of physically distant signals Paul Rubin <no.email@nospam.invalid> - 2013-08-05 22:12 -0700
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-06 09:12 +0100
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-06 01:47 -0700
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-06 10:08 +0100
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-06 02:25 -0700
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-06 02:48 -0700
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-06 10:49 +0100
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-06 03:06 -0700
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-06 11:57 +0100
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-06 10:28 -0700
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-06 21:52 +0100
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-07 01:34 -0700
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-07 09:46 +0100
Re: Comparing phase of physically distant signals Les Cargill <lcargill99@comcast.com> - 2013-08-06 07:22 -0500
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-06 10:42 -0700
Re: Comparing phase of physically distant signals Les Cargill <lcargill99@comcast.com> - 2013-08-06 23:04 -0500
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-06 23:50 -0700
Re: Comparing phase of physically distant signals Les Cargill <lcargill99@comcast.com> - 2013-08-07 13:05 -0500
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-07 18:22 -0700
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-08 09:35 +0100
Re: Comparing phase of physically distant signals Paul Rubin <no.email@nospam.invalid> - 2013-08-06 22:18 -0700
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-07 03:09 -0700
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-07 12:39 +0100
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-07 13:18 +0100
Re: Comparing phase of physically distant signals Tom Gardner <spamjunk@blueyonder.co.uk> - 2013-08-06 09:23 +0100
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-06 02:02 -0700
Re: Comparing phase of physically distant signals "Paul E. Bennett" <Paul_E.Bennett@topmail.co.uk> - 2013-08-06 09:49 +0100
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-06 02:17 -0700
Re: Comparing phase of physically distant signals Paul E Bennett <Paul_E.Bennett@topmail.co.uk> - 2013-08-16 21:07 +0100
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-16 14:50 -0700
Re: Comparing phase of physically distant signals Paul E Bennett <Paul_E.Bennett@topmail.co.uk> - 2013-08-18 19:55 +0100
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-19 11:43 -0700
Re: Comparing phase of physically distant signals Paul E Bennett <Paul_E.Bennett@topmail.co.uk> - 2013-08-20 21:43 +0100
Re: Comparing phase of physically distant signals Don Y <this@isnotme.com> - 2013-08-20 14:49 -0700
csiph-web