Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #37469 > unrolled thread

Is time an illusion?

Back to article view | Back to comp.lang.java.programmer

Contents

  Is time an illusion? Daniele Futtorovic <da.futt.news@laposte.net> - 2017-09-25 22:30 +0200
    Re: Is time an illusion? Eric Sosman <esosman@comcast-dot-net.invalid> - 2017-09-25 18:02 -0400
      Re: Is time an illusion? Daniele Futtorovic <da.futt.news@laposte.net> - 2017-09-26 00:16 +0200
        Re: Is time an illusion? Jerry Stuckle <jstucklex@attglobal.net> - 2017-09-25 19:18 -0400
        Re: Is time an illusion? Eric Sosman <esosman@comcast-dot-net.invalid> - 2017-09-26 15:37 -0400
          Re: Is time an illusion? Daniele Futtorovic <da.futt.news@laposte.net> - 2017-09-27 01:13 +0200
            Re: Is time an illusion? Jerry Stuckle <jstucklex@attglobal.net> - 2017-09-26 21:00 -0400
              Re: Is time an illusion? Daniele Futtorovic <da.futt.news@laposte.net> - 2017-09-27 04:26 +0200
                Re: Is time an illusion? Jerry Stuckle <jstucklex@attglobal.net> - 2017-09-26 23:32 -0400
                  Re: Is time an illusion? Daniele Futtorovic <da.futt.news@laposte.net> - 2017-09-28 10:58 +0200
                    Re: Is time an illusion? Jerry Stuckle <jstucklex@attglobal.net> - 2017-09-28 12:50 -0400
              Re: Is time an illusion? Eric Douglas <e.d.programmer@gmail.com> - 2017-09-27 05:25 -0700
              Re: Is time an illusion? Arne Vajhøj <arne@vajhoej.dk> - 2017-09-30 22:57 -0400
    Re: Is time an illusion? Arne Vajhøj <arne@vajhoej.dk> - 2017-09-25 20:22 -0400
    Re: Is time an illusion? Marcel Mueller <news.5.maazl@spamgourmet.org> - 2017-09-26 20:18 +0200
      Re: Is time an illusion? Daniele Futtorovic <da.futt.news@laposte.net> - 2017-09-27 01:30 +0200
        Re: Is time an illusion? lewbloch@gmail.com - 2017-09-28 10:52 -0700
    Re: Is time an illusion? Eric Douglas <e.d.programmer@gmail.com> - 2017-09-26 12:59 -0700
      Re: Is time an illusion? Daniele Futtorovic <da.futt.news@laposte.net> - 2017-09-27 01:42 +0200
    Re: Is time an illusion? lewbloch@gmail.com - 2017-09-28 10:49 -0700
      Re: Is time an illusion? Robert Klemme <shortcutter@googlemail.com> - 2017-09-28 22:33 +0200

Page 1 of 2  [1] 2  Next page →

#37469 — Is time an illusion?

Cheers,

I'm trying to synchronise a process between two machines (actual, not
virtual). I require sub-millisecond precision.

I initialise clocks backed by System#nanoTime() on each end and
synchronise them somehow (there'll be some deviation at this point.
Doesn't matter). Subsequently, I time my process on each end relative to
this starting point.

I have the impression that I'm seeing time drift in this process.
Disregarding measurement errors, the differences between the clocks seem
to increase over time.

Q: Is this to be expected? I've been trying to read up on it a bit
(Oracle memory-holing all the sun.com blogs doesn't help), but wanted
some confirmation: is time really an illusion? Are 10^9 nanoseconds on
box A possibly not 10^9 nanoseconds on box B?

If so, seriously, what the heck? I'm aware I might have stumbled on a
very elementary truth here after so many years in the business, but I
must say, the idea itself feels deeply unsettling.

Existential musings aside, would any of you have suggestions as to how
tackle the issue?

-- 
DF.

[toc] | [next] | [standalone]

#37472

On 9/25/2017 4:30 PM, Daniele Futtorovic wrote:
> Cheers,
> 
> I'm trying to synchronise a process between two machines (actual, not
> virtual). I require sub-millisecond precision.
> 
> I initialise clocks backed by System#nanoTime() on each end and
> synchronise them somehow (there'll be some deviation at this point.
> Doesn't matter). Subsequently, I time my process on each end relative to
> this starting point.
> 
> I have the impression that I'm seeing time drift in this process.
> Disregarding measurement errors, the differences between the clocks seem
> to increase over time.
> 
> Q: Is this to be expected? I've been trying to read up on it a bit
> (Oracle memory-holing all the sun.com blogs doesn't help), but wanted
> some confirmation: is time really an illusion? Are 10^9 nanoseconds on
> box A possibly not 10^9 nanoseconds on box B?

     It's expected.  If the oscillators underlying the two machines'
clocks run at rates that differ by only 0.001% -- one part in a
hundred thousand -- the clock readings will diverge by 860-odd
milliseconds per day.

> If so, seriously, what the heck? I'm aware I might have stumbled on a
> very elementary truth here after so many years in the business, but I
> must say, the idea itself feels deeply unsettling.
> 
> Existential musings aside, would any of you have suggestions as to how
> tackle the issue?

	http://www.ntp.org/

     "Serious" systems already have NTP implementations, so it may just
be a matter of enabling the service on both machines and making sure
firewalls and such don't prevent them from reaching public time servers.

     I'm no expert on this, but I think that if the two machines run
NTP independently they may adjust themselves at different moments,
leading to disagreements of a different sort.  The solution (I'd
imagine, as a non-expert) would be to have one machine sync externally
and then turn around and act as the time server for the other.

-- 
esosman@comcast-dot-net.invalid
Twelve hundred thirteen days to go.

[toc] | [prev] | [next] | [standalone]

#37473

On 2017-09-26 00:02, Eric Sosman wrote:
> On 9/25/2017 4:30 PM, Daniele Futtorovic wrote:
>> Cheers,
>>
>> I'm trying to synchronise a process between two machines (actual, not
>> virtual). I require sub-millisecond precision.
>>
>> I initialise clocks backed by System#nanoTime() on each end and
>> synchronise them somehow (there'll be some deviation at this point.
>> Doesn't matter). Subsequently, I time my process on each end relative to
>> this starting point.
>>
>> I have the impression that I'm seeing time drift in this process.
>> Disregarding measurement errors, the differences between the clocks seem
>> to increase over time.
>>
>> Q: Is this to be expected? I've been trying to read up on it a bit
>> (Oracle memory-holing all the sun.com blogs doesn't help), but wanted
>> some confirmation: is time really an illusion? Are 10^9 nanoseconds on
>> box A possibly not 10^9 nanoseconds on box B?
> 
>     It's expected.  If the oscillators underlying the two machines'
> clocks run at rates that differ by only 0.001% -- one part in a
> hundred thousand -- the clock readings will diverge by 860-odd
> milliseconds per day.
> 
>> If so, seriously, what the heck? I'm aware I might have stumbled on a
>> very elementary truth here after so many years in the business, but I
>> must say, the idea itself feels deeply unsettling.
>>
>> Existential musings aside, would any of you have suggestions as to how
>> tackle the issue?
> 
>     http://www.ntp.org/
> 
>     "Serious" systems already have NTP implementations, so it may just
> be a matter of enabling the service on both machines and making sure
> firewalls and such don't prevent them from reaching public time servers.
> 
>     I'm no expert on this, but I think that if the two machines run
> NTP independently they may adjust themselves at different moments,
> leading to disagreements of a different sort.  The solution (I'd
> imagine, as a non-expert) would be to have one machine sync externally
> and then turn around and act as the time server for the other.

I don't think NTP would be a good fit for me. If I have to adjust clocks
periodically anyway, I can do that within my setup; I just need to elect
one end as the master and go from there. But is there perhaps something
I could tap, even if I have to go native, or do I really have to accept
that I just can't ever have congruent time intervals across potentially
heterogeneous hardware?

Also, if there is a drift, can I expect that drift to be constant? That
is, if I observe that box A ticks, say, 1% faster than box B over a
given interval of time, can I assume that it will do so for /any/ given
interval of time? I guess not, right?

-- 
DF.

[toc] | [prev] | [next] | [standalone]

#37474

On 9/25/2017 6:16 PM, Daniele Futtorovic wrote:
> On 2017-09-26 00:02, Eric Sosman wrote:
>> On 9/25/2017 4:30 PM, Daniele Futtorovic wrote:
>>> Cheers,
>>>
>>> I'm trying to synchronise a process between two machines (actual, not
>>> virtual). I require sub-millisecond precision.
>>>
>>> I initialise clocks backed by System#nanoTime() on each end and
>>> synchronise them somehow (there'll be some deviation at this point.
>>> Doesn't matter). Subsequently, I time my process on each end relative to
>>> this starting point.
>>>
>>> I have the impression that I'm seeing time drift in this process.
>>> Disregarding measurement errors, the differences between the clocks seem
>>> to increase over time.
>>>
>>> Q: Is this to be expected? I've been trying to read up on it a bit
>>> (Oracle memory-holing all the sun.com blogs doesn't help), but wanted
>>> some confirmation: is time really an illusion? Are 10^9 nanoseconds on
>>> box A possibly not 10^9 nanoseconds on box B?
>>
>>      It's expected.  If the oscillators underlying the two machines'
>> clocks run at rates that differ by only 0.001% -- one part in a
>> hundred thousand -- the clock readings will diverge by 860-odd
>> milliseconds per day.
>>
>>> If so, seriously, what the heck? I'm aware I might have stumbled on a
>>> very elementary truth here after so many years in the business, but I
>>> must say, the idea itself feels deeply unsettling.
>>>
>>> Existential musings aside, would any of you have suggestions as to how
>>> tackle the issue?
>>
>>      http://www.ntp.org/
>>
>>      "Serious" systems already have NTP implementations, so it may just
>> be a matter of enabling the service on both machines and making sure
>> firewalls and such don't prevent them from reaching public time servers.
>>
>>      I'm no expert on this, but I think that if the two machines run
>> NTP independently they may adjust themselves at different moments,
>> leading to disagreements of a different sort.  The solution (I'd
>> imagine, as a non-expert) would be to have one machine sync externally
>> and then turn around and act as the time server for the other.
> 
> I don't think NTP would be a good fit for me. If I have to adjust clocks
> periodically anyway, I can do that within my setup; I just need to elect
> one end as the master and go from there. But is there perhaps something
> I could tap, even if I have to go native, or do I really have to accept
> that I just can't ever have congruent time intervals across potentially
> heterogeneous hardware?
> 
> Also, if there is a drift, can I expect that drift to be constant? That
> is, if I observe that box A ticks, say, 1% faster than box B over a
> given interval of time, can I assume that it will do so for /any/ given
> interval of time? I guess not, right?
> 

The only way you're going to get the precision you need is for both 
machines to use the same external hardware clock reference.  If you need 
extreme accuracy, you'll need to use a hardware interface - ethernet or 
usb will not provide it.

-- 
==================
Remove the "x" from my email address
Jerry Stuckle
jstucklex@attglobal.net
==================

[toc] | [prev] | [next] | [standalone]

#37481

On 9/25/2017 6:16 PM, Daniele Futtorovic wrote:
> On 2017-09-26 00:02, Eric Sosman wrote:
>> On 9/25/2017 4:30 PM, Daniele Futtorovic wrote:
>>> Cheers,
>>>
>>> I'm trying to synchronise a process between two machines (actual, not
>>> virtual). I require sub-millisecond precision.
>>>
>>> I initialise clocks backed by System#nanoTime() on each end and
>>> synchronise them somehow (there'll be some deviation at this point.
>>> Doesn't matter). Subsequently, I time my process on each end relative to
>>> this starting point.
>>>
>>> I have the impression that I'm seeing time drift in this process.
>>> Disregarding measurement errors, the differences between the clocks seem
>>> to increase over time.
>>>
>>> Q: Is this to be expected? I've been trying to read up on it a bit
>>> (Oracle memory-holing all the sun.com blogs doesn't help), but wanted
>>> some confirmation: is time really an illusion? Are 10^9 nanoseconds on
>>> box A possibly not 10^9 nanoseconds on box B?
>>
>>      It's expected.  If the oscillators underlying the two machines'
>> clocks run at rates that differ by only 0.001% -- one part in a
>> hundred thousand -- the clock readings will diverge by 860-odd
>> milliseconds per day.
>>
>>> If so, seriously, what the heck? I'm aware I might have stumbled on a
>>> very elementary truth here after so many years in the business, but I
>>> must say, the idea itself feels deeply unsettling.
>>>
>>> Existential musings aside, would any of you have suggestions as to how
>>> tackle the issue?
>>
>>      http://www.ntp.org/
>>
>>      "Serious" systems already have NTP implementations, so it may just
>> be a matter of enabling the service on both machines and making sure
>> firewalls and such don't prevent them from reaching public time servers.
>>
>>      I'm no expert on this, but I think that if the two machines run
>> NTP independently they may adjust themselves at different moments,
>> leading to disagreements of a different sort.  The solution (I'd
>> imagine, as a non-expert) would be to have one machine sync externally
>> and then turn around and act as the time server for the other.
> 
> I don't think NTP would be a good fit for me. If I have to adjust clocks
> periodically anyway, I can do that within my setup; I just need to elect
> one end as the master and go from there. But is there perhaps something
> I could tap, even if I have to go native, or do I really have to accept
> that I just can't ever have congruent time intervals across potentially
> heterogeneous hardware?
> 
> Also, if there is a drift, can I expect that drift to be constant? That
> is, if I observe that box A ticks, say, 1% faster than box B over a
> given interval of time, can I assume that it will do so for /any/ given
> interval of time? I guess not, right?

     I can't think of any reason to expect constant drift.  The machines
don't operate at constant temperature, nor at constant voltage, nor at
constant age ...  In my (very brief) look at the NTP documentation I
found a glossary of terms.  There's a word for the first derivative
of a clock's tick rate; the protocol anticipates that the rate is not
a constant.  There's even a word for the *second* derivative of the
tick rate; not even the rate of change of rate is considered constant!

     I'm somewhat curious about why you need such lock-step agreement.
What are you trying to accomplish?  Why do you need to measure the
same interval on two independent stopwatches?

-- 
esosman@comcast-dot-net.invalid
Twelve hundred twelve days to go.

[toc] | [prev] | [next] | [standalone]

#37483

On 2017-09-26 21:37, Eric Sosman wrote:
> On 9/25/2017 6:16 PM, Daniele Futtorovic wrote:
>> Also, if there is a drift, can I expect that drift to be constant? That
>> is, if I observe that box A ticks, say, 1% faster than box B over a
>> given interval of time, can I assume that it will do so for /any/ given
>> interval of time? I guess not, right?
> 
>     I can't think of any reason to expect constant drift.  The machines
> don't operate at constant temperature, nor at constant voltage, nor at
> constant age ...  In my (very brief) look at the NTP documentation I
> found a glossary of terms.  There's a word for the first derivative
> of a clock's tick rate; the protocol anticipates that the rate is not
> a constant.  There's even a word for the *second* derivative of the
> tick rate; not even the rate of change of rate is considered constant!

Makes sense once you accept the basic premise that a CPU doesn't "tick"
at a perfectly constant or perfectly known rate, but still, bloody hell.
Gives a whole new meaning to the idea that solar flares impact our lives. ;)

>     I'm somewhat curious about why you need such lock-step agreement.
> What are you trying to accomplish?  Why do you need to measure the
> same interval on two independent stopwatches?

I'm rendering audio over a network, in real-time, or as close to
real-time as I can get. It's not just a matter of "render as much of
this as you can, when you can"; I want to be able tightly to control the
rendering, as well as have accurate measurement of the resulting latency
-- possibly over multiple endpoints.

The upside is that I'm not constrained by absolute time; I just need
relative time. Also, the human ear is sufficiently forgiving that I can
afford to be a couple milliseconds off -- but not too many (I think).

Specialised equipment is out of the question for me, as the software is
intended for general consumption. But based on the feedback here, I've
incorporated clock synchronisation into my protocol, and it seems to
work well enough, at least at low latencies. It's still an open question
how well it will work when they get higher, but I'll burn that bridge
when I get there.

Thanks a lot for the assistance.

-- 
DF.

[toc] | [prev] | [next] | [standalone]

#37486

On 9/26/2017 7:13 PM, Daniele Futtorovic wrote:
> On 2017-09-26 21:37, Eric Sosman wrote:
>> On 9/25/2017 6:16 PM, Daniele Futtorovic wrote:
>>> Also, if there is a drift, can I expect that drift to be constant? That
>>> is, if I observe that box A ticks, say, 1% faster than box B over a
>>> given interval of time, can I assume that it will do so for /any/ given
>>> interval of time? I guess not, right?
>>
>>      I can't think of any reason to expect constant drift.  The machines
>> don't operate at constant temperature, nor at constant voltage, nor at
>> constant age ...  In my (very brief) look at the NTP documentation I
>> found a glossary of terms.  There's a word for the first derivative
>> of a clock's tick rate; the protocol anticipates that the rate is not
>> a constant.  There's even a word for the *second* derivative of the
>> tick rate; not even the rate of change of rate is considered constant!
> 
> Makes sense once you accept the basic premise that a CPU doesn't "tick"
> at a perfectly constant or perfectly known rate, but still, bloody hell.
> Gives a whole new meaning to the idea that solar flares impact our lives. ;)
> 
>>      I'm somewhat curious about why you need such lock-step agreement.
>> What are you trying to accomplish?  Why do you need to measure the
>> same interval on two independent stopwatches?
> 
> I'm rendering audio over a network, in real-time, or as close to
> real-time as I can get. It's not just a matter of "render as much of
> this as you can, when you can"; I want to be able tightly to control the
> rendering, as well as have accurate measurement of the resulting latency
> -- possibly over multiple endpoints.
> 
> The upside is that I'm not constrained by absolute time; I just need
> relative time. Also, the human ear is sufficiently forgiving that I can
> afford to be a couple milliseconds off -- but not too many (I think).
> 
> Specialised equipment is out of the question for me, as the software is
> intended for general consumption. But based on the feedback here, I've
> incorporated clock synchronisation into my protocol, and it seems to
> work well enough, at least at low latencies. It's still an open question
> how well it will work when they get higher, but I'll burn that bridge
> when I get there.
> 
> Thanks a lot for the assistance.
> 

If you're trying to synchronize audio with video, anything less than 
100ms is generally considered sufficient and regular updates with ntp 
should work fine.  However, if you're trying to synchronize two 
different audio signals (i.e. left and right channels), you have to be 
less than around 50ms (1 cycle at 20 Khz), which you won't get reliably 
without additional hardware.

The problem here is that CPU clocks, even if they are crystal 
controlled, are not perfectly accurate.  A clock on a PC can easily be 
off by a minute or two per week.

Here's a test you can do easily on either a Windows or Linux machine to 
see what happens.  Synchronize the system clock with the internet. 
Disable all synchronization for a week.  It won't matter whether the 
computer is on or off - it's real time clock runs at the same speed.

At the end of a week, check the time against the internet (or 
synchronize it, paying attention to how much the adjustment is).  You 
will find a difference of at least several seconds, and possibly as much 
as a couple of minutes.

-- 
==================
Remove the "x" from my email address
Jerry Stuckle
jstucklex@attglobal.net
==================

[toc] | [prev] | [next] | [standalone]

#37487

On 2017-09-27 03:00, Jerry Stuckle wrote:
> On 9/26/2017 7:13 PM, Daniele Futtorovic wrote:
>> On 2017-09-26 21:37, Eric Sosman wrote:
>>>      I'm somewhat curious about why you need such lock-step agreement.
>>> What are you trying to accomplish?  Why do you need to measure the
>>> same interval on two independent stopwatches?
>>
>> I'm rendering audio over a network, in real-time, or as close to
>> real-time as I can get. It's not just a matter of "render as much of
>> this as you can, when you can"; I want to be able tightly to control the
>> rendering, as well as have accurate measurement of the resulting latency
>> -- possibly over multiple endpoints.
>>
>> The upside is that I'm not constrained by absolute time; I just need
>> relative time. Also, the human ear is sufficiently forgiving that I can
>> afford to be a couple milliseconds off -- but not too many (I think).
>>
>> Specialised equipment is out of the question for me, as the software is
>> intended for general consumption. But based on the feedback here, I've
>> incorporated clock synchronisation into my protocol, and it seems to
>> work well enough, at least at low latencies. It's still an open question
>> how well it will work when they get higher, but I'll burn that bridge
>> when I get there.
>>
>> Thanks a lot for the assistance.
>>
> 
> If you're trying to synchronize audio with video, anything less than
> 100ms is generally considered sufficient and regular updates with ntp
> should work fine.  However, if you're trying to synchronize two
> different audio signals (i.e. left and right channels), you have to be
> less than around 50ms (1 cycle at 20 Khz), which you won't get reliably
> without additional hardware.

I don't think I want to sync audio signals, but... what makes you say I
wouldn't get better than 50ms resolution?

The way I see it, provided I get my clocks synched, the only other
problem is the drift; otherwise I can do whatever I want. Therefore if I
find a workable time interval over the length of which drift is
negligible (say, <1ms), synchronisation is the only problem.

In my current dev setup with random hardware, I see a drift of about 1ms
per minute. I can work with and correct that. And as for the clock
synching, as I understand it, it's a matter of the symmetry between the
upward and downward leg of the packet roundtrip between both ends -- the
more symmetric, the more accurate. And that difference is itself
constrained by the roundtrip time, i.e., the error can never be greater
than the overall roundtrip time.

Again, I'm targeting a LAN. Possibly a WLAN. Roundtrip in such
environments should be magnitudes smaller than 50ms, especially if I go
pure UDP for the synching. Wouldn't you agree?

Also, can you tell me how you arrive at the 20Khz figure?

> The problem here is that CPU clocks, even if they are crystal
> controlled, are not perfectly accurate.  A clock on a PC can easily be
> off by a minute or two per week.

Indeed, and that was the big revelation of these last few days. I feel
silly for not having thought of it before -- it's kinda obvious, when
you look at it -- but, ah well.

-- 
DF.

[toc] | [prev] | [next] | [standalone]

#37488

On 9/26/2017 10:26 PM, Daniele Futtorovic wrote:
> On 2017-09-27 03:00, Jerry Stuckle wrote:
>> On 9/26/2017 7:13 PM, Daniele Futtorovic wrote:
>>> On 2017-09-26 21:37, Eric Sosman wrote:
>>>>       I'm somewhat curious about why you need such lock-step agreement.
>>>> What are you trying to accomplish?  Why do you need to measure the
>>>> same interval on two independent stopwatches?
>>>
>>> I'm rendering audio over a network, in real-time, or as close to
>>> real-time as I can get. It's not just a matter of "render as much of
>>> this as you can, when you can"; I want to be able tightly to control the
>>> rendering, as well as have accurate measurement of the resulting latency
>>> -- possibly over multiple endpoints.
>>>
>>> The upside is that I'm not constrained by absolute time; I just need
>>> relative time. Also, the human ear is sufficiently forgiving that I can
>>> afford to be a couple milliseconds off -- but not too many (I think).
>>>
>>> Specialised equipment is out of the question for me, as the software is
>>> intended for general consumption. But based on the feedback here, I've
>>> incorporated clock synchronisation into my protocol, and it seems to
>>> work well enough, at least at low latencies. It's still an open question
>>> how well it will work when they get higher, but I'll burn that bridge
>>> when I get there.
>>>
>>> Thanks a lot for the assistance.
>>>
>>
>> If you're trying to synchronize audio with video, anything less than
>> 100ms is generally considered sufficient and regular updates with ntp
>> should work fine.  However, if you're trying to synchronize two
>> different audio signals (i.e. left and right channels), you have to be
>> less than around 50ms (1 cycle at 20 Khz), which you won't get reliably
>> without additional hardware.
> 
> I don't think I want to sync audio signals, but... what makes you say I
> wouldn't get better than 50ms resolution?
>

Sorry, that should have been 50 microseconds, not 50 ms.  That's what 
you would need as a minimum to synchronize audio signals because the ear 
can easily hear phase differences between signals.

> The way I see it, provided I get my clocks synched, the only other
> problem is the drift; otherwise I can do whatever I want. Therefore if I
> find a workable time interval over the length of which drift is
> negligible (say, <1ms), synchronisation is the only problem.
> 

It all depends on the computers themselves.  There is no way to 
guarantee drift on its own will be < 1ms, and no internet-based 
synchronization will keep them that close.

> In my current dev setup with random hardware, I see a drift of about 1ms
> per minute. I can work with and correct that. And as for the clock
> synching, as I understand it, it's a matter of the symmetry between the
> upward and downward leg of the packet roundtrip between both ends -- the
> more symmetric, the more accurate. And that difference is itself
> constrained by the roundtrip time, i.e., the error can never be greater
> than the overall roundtrip time.
> 

That's with your current hardware.  Different hardware can be or or less.

> Again, I'm targeting a LAN. Possibly a WLAN. Roundtrip in such
> environments should be magnitudes smaller than 50ms, especially if I go
> pure UDP for the synching. Wouldn't you agree?
> 
> Also, can you tell me how you arrive at the 20Khz figure?
> 

It's generally considered about the minimum for synchronization for 
non-high fidelity audio (i.e. voice and general sounds) - up to around 7 
kHz or so.  The human ear can hear up to 15 kHz or so so you need 
something higher than that.  For high quality audio you're looking at 
around 45 kHz (3x maximum audible frequency).

>> The problem here is that CPU clocks, even if they are crystal
>> controlled, are not perfectly accurate.  A clock on a PC can easily be
>> off by a minute or two per week.
> 
> Indeed, and that was the big revelation of these last few days. I feel
> silly for not having thought of it before -- it's kinda obvious, when
> you look at it -- but, ah well.
> 

What is so important about keeping the time down to < 1ms?  You can have 
more variation than that even in must processing a signal due to the 
multiprocessing nature of the computers.

-- 
==================
Remove the "x" from my email address
Jerry Stuckle
jstucklex@attglobal.net
==================

[toc] | [prev] | [next] | [standalone]

#37503

On 2017-09-27 05:32, Jerry Stuckle wrote:
> On 9/26/2017 10:26 PM, Daniele Futtorovic wrote:
>> On 2017-09-27 03:00, Jerry Stuckle wrote:
>>> If you're trying to synchronize audio with video, anything less
>>> than 100ms is generally considered sufficient and regular updates
>>> with ntp should work fine.  However, if you're trying to
>>> synchronize two different audio signals (i.e. left and right
>>> channels), you have to be less than around 50ms (1 cycle at 20
>>> Khz), which you won't get reliably without additional hardware.
>> 
>> I don't think I want to sync audio signals, but... what makes you
>> say I wouldn't get better than 50ms resolution?
>> 
> 
> Sorry, that should have been 50 microseconds, not 50 ms.  That's
> what you would need as a minimum to synchronize audio signals because
> the ear can easily hear phase differences between signals.
> 
>> Also, can you tell me how you arrive at the 20Khz figure?
>> 
> 
> It's generally considered about the minimum for synchronization for 
> non-high fidelity audio (i.e. voice and general sounds) - up to
> around 7 kHz or so.  The human ear can hear up to 15 kHz or so so you
> need something higher than that.  For high quality audio you're
> looking at around 45 kHz (3x maximum audible frequency).

Makes sense. Thanks a lot for the figures.

>> The way I see it, provided I get my clocks synched, the only other 
>> problem is the drift; otherwise I can do whatever I want. Therefore
>> if I find a workable time interval over the length of which drift
>> is negligible (say, <1ms), synchronisation is the only problem.
>> 
> It all depends on the computers themselves.  There is no way to 
> guarantee drift on its own will be < 1ms, and no internet-based 
> synchronization will keep them that close.
> 
> What is so important about keeping the time down to < 1ms?  You can
> have more variation than that even in must processing a signal due to
> the multiprocessing nature of the computers.

*Shrug* I'm just trying to be as accurate as I can afford to be. The
basic premise isn't signal mixing, but the transport of PCM audio at the
standard 44.1/48kHz sample rates. If I want flexibility with respects to
the lower end of sizes of the packets I divide the data in (which does
impact latency, which I want to minimize), I need to measure
sub-millisecond. If then the whole thing can also be used for signal
mixing over the network, all the better -- although from what you told
me, it doesn't sound like I'll be able to achieve that, at least not
without specialised clocks. But basically, for what it's worth, I'm just
trying to make something that'll be useful as possible.

For the record, I started this whole thing by using JACK
(<http://jackaudio.org>) with its net driver -- maybe you're familiar
with it -- but then found I had to roll my own, because NetJack only
uses UDP, and I want this to work over wireless, which turned out to be
a no-no. I've got both TCP and UDP going for now, and am planning an
SCTP
(<https://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol>)
implementation as well.

-- 
DF.

[toc] | [prev] | [next] | [standalone]

#37506

On 9/28/2017 4:58 AM, Daniele Futtorovic wrote:
> On 2017-09-27 05:32, Jerry Stuckle wrote:
>> On 9/26/2017 10:26 PM, Daniele Futtorovic wrote:
>>> On 2017-09-27 03:00, Jerry Stuckle wrote:
>>>> If you're trying to synchronize audio with video, anything less
>>>> than 100ms is generally considered sufficient and regular updates
>>>> with ntp should work fine.  However, if you're trying to
>>>> synchronize two different audio signals (i.e. left and right
>>>> channels), you have to be less than around 50ms (1 cycle at 20
>>>> Khz), which you won't get reliably without additional hardware.
>>>
>>> I don't think I want to sync audio signals, but... what makes you
>>> say I wouldn't get better than 50ms resolution?
>>>
>>
>> Sorry, that should have been 50 microseconds, not 50 ms.  That's
>> what you would need as a minimum to synchronize audio signals because
>> the ear can easily hear phase differences between signals.
>>
>>> Also, can you tell me how you arrive at the 20Khz figure?
>>>
>>
>> It's generally considered about the minimum for synchronization for
>> non-high fidelity audio (i.e. voice and general sounds) - up to
>> around 7 kHz or so.  The human ear can hear up to 15 kHz or so so you
>> need something higher than that.  For high quality audio you're
>> looking at around 45 kHz (3x maximum audible frequency).
> 
> Makes sense. Thanks a lot for the figures.
> 
>>> The way I see it, provided I get my clocks synched, the only other
>>> problem is the drift; otherwise I can do whatever I want. Therefore
>>> if I find a workable time interval over the length of which drift
>>> is negligible (say, <1ms), synchronisation is the only problem.
>>>
>> It all depends on the computers themselves.  There is no way to
>> guarantee drift on its own will be < 1ms, and no internet-based
>> synchronization will keep them that close.
>>
>> What is so important about keeping the time down to < 1ms?  You can
>> have more variation than that even in must processing a signal due to
>> the multiprocessing nature of the computers.
> 
> *Shrug* I'm just trying to be as accurate as I can afford to be. The
> basic premise isn't signal mixing, but the transport of PCM audio at the
> standard 44.1/48kHz sample rates. If I want flexibility with respects to
> the lower end of sizes of the packets I divide the data in (which does
> impact latency, which I want to minimize), I need to measure
> sub-millisecond. If then the whole thing can also be used for signal
> mixing over the network, all the better -- although from what you told
> me, it doesn't sound like I'll be able to achieve that, at least not
> without specialised clocks. But basically, for what it's worth, I'm just
> trying to make something that'll be useful as possible.
> 
> For the record, I started this whole thing by using JACK
> (<http://jackaudio.org>) with its net driver -- maybe you're familiar
> with it -- but then found I had to roll my own, because NetJack only
> uses UDP, and I want this to work over wireless, which turned out to be
> a no-no. I've got both TCP and UDP going for now, and am planning an
> SCTP
> (<https://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol>)
> implementation as well.
> 

No, you won't be able to do this without specialized clocks.  Even a 1 
minute gain or loss per day in the clock ends up with a skew rate of 
.695 ms per minute.  And even with a perfectly aligned clock your 
packets can have several ms (or more) variation in how quickly they are 
handled, depending on what else is going on in both machines (i.e. other 
programs, hardware interrupts, etc.).  Other network traffic can affect 
the timing even more.

The bottom line here is TCP/IP is pretty good at getting things from one 
point to another.  But not good at doing it within a certain timeframe.

-- 
==================
Remove the "x" from my email address
Jerry Stuckle
jstucklex@attglobal.net
==================

[toc] | [prev] | [next] | [standalone]

#37489

On Tuesday, September 26, 2017 at 9:01:03 PM UTC-4, Jerry Stuckle wrote:

> At the end of a week, check the time against the internet (or 
> synchronize it, paying attention to how much the adjustment is).  You 
> will find a difference of at least several seconds, and possibly as much 
> as a couple of minutes.

The difference primarily depends on the quality and the age of the hardware.  We once had a PC in the office that wasn't syncing time all day long that was off by 5 minutes or more by the end of a day if we set it correct first thing in the morning.  I find that a 5 year old PC doesn't seem to run as fast as when it was new even with a clean reformat (though formatting should help).

I always wonder how accurately you can sync a machine.  Just going to another machine on the same network depends on the speed of their network cards and cables and switches.  Going out to the internet adds a lot more variables, which is why estimates of time for downloading files tend to vary widely.  The first thing you want to do is connect as smoothly as possible.  If processes have to run on different machines, do they have to go through a switch or can they connect directly?  Can you use 10 gigabit?

One possible workaround is to timestamp.  If each machine can just march to it's own beat and note where those beats are they can be adjusted.  If the full recording on one machine comes out 5 minutes longer than the other, can you just squish it after the fact to match?

[toc] | [prev] | [next] | [standalone]

#37516

On 9/26/2017 9:00 PM, Jerry Stuckle wrote:
> The problem here is that CPU clocks, even if they are crystal 
> controlled, are not perfectly accurate.  A clock on a PC can easily be 
> off by a minute or two per week.
> 
> Here's a test you can do easily on either a Windows or Linux machine to 
> see what happens.  Synchronize the system clock with the internet. 
> Disable all synchronization for a week.  It won't matter whether the 
> computer is on or off - it's real time clock runs at the same speed.
> 
> At the end of a week, check the time against the internet (or 
> synchronize it, paying attention to how much the adjustment is).  You 
> will find a difference of at least several seconds, and possibly as much 
> as a couple of minutes.

If doing this experiment one should be aware that there are
two clocks on the system RTC and system clock.

If the system is down most of the week then the difference
will show RTC drift.

If the system is up most of the week then most commands
and tools will show drift in system clock, but hwclock
will show drift in RTC.

The application logic will most likely be based on system
clock.

Both RTC and system clock will drift, but not necessarily the
same.

Arne

[toc] | [prev] | [next] | [standalone]

#37475

On 9/25/2017 4:30 PM, Daniele Futtorovic wrote:
> I'm trying to synchronise a process between two machines (actual, not
> virtual). I require sub-millisecond precision.
> 
> I initialise clocks backed by System#nanoTime() on each end and
> synchronise them somehow (there'll be some deviation at this point.
> Doesn't matter). Subsequently, I time my process on each end relative to
> this starting point.
> 
> I have the impression that I'm seeing time drift in this process.
> Disregarding measurement errors, the differences between the clocks seem
> to increase over time.
> 
> Q: Is this to be expected? I've been trying to read up on it a bit
> (Oracle memory-holing all the sun.com blogs doesn't help), but wanted
> some confirmation: is time really an illusion? Are 10^9 nanoseconds on
> box A possibly not 10^9 nanoseconds on box B?
> 
> If so, seriously, what the heck? I'm aware I might have stumbled on a
> very elementary truth here after so many years in the business, but I
> must say, the idea itself feels deeply unsettling.
> 
> Existential musings aside, would any of you have suggestions as to how
> tackle the issue?

Clocks on computers tend to drift slightly.

By using an external time source you can keep time reasonable in sync.

Using a NTP server over the internet is common, but sub millisecond
precision is very hard to achieve with that.

Getting time from a GPS receiver should be better and may
accomplish your goals.

Arne

[toc] | [prev] | [next] | [standalone]

#37480

On 25.09.17 22.30, Daniele Futtorovic wrote:
> I'm trying to synchronise a process between two machines (actual, not
> virtual). I require sub-millisecond precision.

What is the distance between these two machines?

If they are far away there is not even a physical definition of the term 
"the same time" within the required precision. The theory of relativity 
prohibits this. The maximum theoretical distance for let's say 0,1ms is 
30km.
However, you can still make more precise assumptions about time 
differences under certain conditions.


> I have the impression that I'm seeing time drift in this process.
> Disregarding measurement errors, the differences between the clocks seem
> to increase over time.
>
> Q: Is this to be expected?

Yes.

> Are 10^9 nanoseconds on box A possibly not 10^9 nanoseconds on box B?

Exactly. They are not even the same when box A resides on a different 
location on earth.
But anything else depends on what you accept as "the same".

> If so, seriously, what the heck? I'm aware I might have stumbled on a
> very elementary truth here after so many years in the business, but I
> must say, the idea itself feels deeply unsettling.

Welcome to real world. ;-)

> Existential musings aside, would any of you have suggestions as to how
> tackle the issue?

Supply both computers with an atomic clock.
Reasonable Rubidium clocks are not that expensive. The most challenging 
task is to synchronize this clocks and, of course, avoid power failures 
after that.

A GPS clock might also be an alternative. They have very good long term 
precision - basically limited by the Caesium clocks of the sattelites -, 
but may suffer short term stability because of atmospheric effects or 
interference.

ntp, as recommended, is /not/ sufficient to synchronize within 
microsecond precision.


Marcel

[toc] | [prev] | [next] | [standalone]

#37484

On 2017-09-26 20:18, Marcel Mueller wrote:
> On 25.09.17 22.30, Daniele Futtorovic wrote:
>> I'm trying to synchronise a process between two machines (actual, not
>> virtual). I require sub-millisecond precision.
> 
> What is the distance between these two machines?
> 
> If they are far away there is not even a physical definition of the term
> "the same time" within the required precision. The theory of relativity
> prohibits this. The maximum theoretical distance for let's say 0,1ms is
> 30km.
> However, you can still make more precise assumptions about time
> differences under certain conditions.

The problem space can be, at least as a working assumption, restricted
to a LAN.

I get your explanation; let me point out though that for my purposes, I
do not need the concept of "same time" -- only of same /interval/.
Which, from what has been said here and that I've read up on in the
meantime, indeed appears to be something I have to give up on, or at
least that I can't rely on perfectly (barring specialised equipment, as
you mention further on).

>> If so, seriously, what the heck? I'm aware I might have stumbled on a
>> very elementary truth here after so many years in the business, but I
>> must say, the idea itself feels deeply unsettling.
> 
> Welcome to real world. ;-)

... shattering the foundations of the virtual one. ;)

>> Existential musings aside, would any of you have suggestions as to how
>> tackle the issue?
> 
> Supply both computers with an atomic clock.
> Reasonable Rubidium clocks are not that expensive. The most challenging
> task is to synchronize this clocks and, of course, avoid power failures
> after that.
> 
> A GPS clock might also be an alternative. They have very good long term
> precision - basically limited by the Caesium clocks of the satelites -,
> but may suffer short term stability because of atmospheric effects or
> interference.
> 
> ntp, as recommended, is /not/ sufficient to synchronize within
> microsecond precision.

Yeah... no. The need for specialised equipment would be prohibitive for
the intended uses. It's good to know it would be an option,
theoretically speaking, but in this case, that's definitely out of the
question.

As mentioned elsethreads, I have for now fixed my problem by
synchronising my clocks periodically (every 30s for now). It seems to
work well enough for the usage, at least at low latencies. Like any
network-based synchronisation protocol, its accuracy will depend on the
symmetry between the upward and downward leg of the round-trip, but I'll
have to see how bad it can get.

-- 
DF.

[toc] | [prev] | [next] | [standalone]

#37508

On Tuesday, September 26, 2017 at 4:31:29 PM UTC-7, Daniele Futtorovic wrote:
> On 2017-09-26 20:18, Marcel Mueller wrote:
> > On 25.09.17 22.30, Daniele Futtorovic wrote:
> >> I'm trying to synchronise a process between two machines (actual, not
> >> virtual). I require sub-millisecond precision.
> > 
> > What is the distance between these two machines?
> > 
> > If they are far away there is not even a physical definition of the term
> > "the same time" within the required precision. The theory of relativity
> > prohibits this. The maximum theoretical distance for let's say 0,1ms is
> > 30km.
> > However, you can still make more precise assumptions about time
> > differences under certain conditions.
> 
> The problem space can be, at least as a working assumption, restricted
> to a LAN.

LANs have latency. Latency kills synchronization. LAN latency is largely unpredictable, especially with Ethernet with it's "Fail? Abort, Retry" protocol.

-- 
Lew

[toc] | [prev] | [next] | [standalone]

#37482

On Monday, September 25, 2017 at 4:31:11 PM UTC-4, Daniele Futtorovic wrote:
> Cheers,
> 
> I'm trying to synchronise a process between two machines (actual, not
> virtual). I require sub-millisecond precision.
> 
> I initialise clocks backed by System#nanoTime() on each end and
> synchronise them somehow (there'll be some deviation at this point.
> Doesn't matter). Subsequently, I time my process on each end relative to
> this starting point.
> 
> I have the impression that I'm seeing time drift in this process.
> Disregarding measurement errors, the differences between the clocks seem
> to increase over time.
> 
> Q: Is this to be expected? I've been trying to read up on it a bit
> (Oracle memory-holing all the sun.com blogs doesn't help), but wanted
> some confirmation: is time really an illusion? Are 10^9 nanoseconds on
> box A possibly not 10^9 nanoseconds on box B?
> 
> If so, seriously, what the heck? I'm aware I might have stumbled on a
> very elementary truth here after so many years in the business, but I
> must say, the idea itself feels deeply unsettling.
> 
> Existential musings aside, would any of you have suggestions as to how
> tackle the issue?
> 
> -- 
> DF.

First, have you read about time?
https://en.wikipedia.org/wiki/System_time

Second, are the 2 machines physically beside each other?  That does make a difference in measuring real Earth time.
https://en.wikipedia.org/wiki/Gravitational_time_dilation

[toc] | [prev] | [next] | [standalone]

#37485

On 2017-09-26 21:59, Eric Douglas wrote:
> On Monday, September 25, 2017 at 4:31:11 PM UTC-4, Daniele Futtorovic wrote:
>> Cheers,
>>
>> I'm trying to synchronise a process between two machines (actual, not
>> virtual). I require sub-millisecond precision.
>>
>> I initialise clocks backed by System#nanoTime() on each end and
>> synchronise them somehow (there'll be some deviation at this point.
>> Doesn't matter). Subsequently, I time my process on each end relative to
>> this starting point.
>>
>> I have the impression that I'm seeing time drift in this process.
>> Disregarding measurement errors, the differences between the clocks seem
>> to increase over time.
>>
>> Q: Is this to be expected? I've been trying to read up on it a bit
>> (Oracle memory-holing all the sun.com blogs doesn't help), but wanted
>> some confirmation: is time really an illusion? Are 10^9 nanoseconds on
>> box A possibly not 10^9 nanoseconds on box B?
>>
>> If so, seriously, what the heck? I'm aware I might have stumbled on a
>> very elementary truth here after so many years in the business, but I
>> must say, the idea itself feels deeply unsettling.
>>
>> Existential musings aside, would any of you have suggestions as to how
>> tackle the issue?
>>
> First, have you read about time?
> https://en.wikipedia.org/wiki/System_time

I hadn't, but now that I have, I'm not sure what it would have changed.
Unless I've missed something, that article doesn't seem to mention the
central fact that CPUs cannot deterministically measure time (whatever
time is).

> Second, are the 2 machines physically beside each other?  That does make a difference in measuring real Earth time.
> https://en.wikipedia.org/wiki/Gravitational_time_dilation

All right, but that's not relativity that's at work here, is it? That
article for instance mentions differences in the trillionth for clocks
set a couple of kilometres apart; we're talking about differences in the
thousandth or ten-thousandth for boxes standing in the same room.

As others have weighed in, voltage fluctuations (and perhaps material
imperfections?) seem to be a perfectly reasonable -- if unsettling --
explanation, and those aren't due to relativity, are they?

-- 
DF.

[toc] | [prev] | [next] | [standalone]

#37507

Daniele Futtorovic wrote:
> I'm trying to synchronise a process between two machines (actual, not
> virtual). I require sub-millisecond precision.
> 
> I initialise clocks backed by System#nanoTime() on each end and
> synchronise them somehow (there'll be some deviation at this point.
> Doesn't matter). Subsequently, I time my process on each end relative to
> this starting point.
> 
> I have the impression that I'm seeing time drift in this process.
> Disregarding measurement errors, the differences between the clocks seem
> to increase over time.
> 
> Q: Is this to be expected? I've been trying to read up on it a bit
> (Oracle memory-holing all the sun.com blogs doesn't help), but wanted
> some confirmation: is time really an illusion? Are 10^9 nanoseconds on
> box A possibly not 10^9 nanoseconds on box B?
> 
> If so, seriously, what the heck? I'm aware I might have stumbled on a
> very elementary truth here after so many years in the business, but I
> must say, the idea itself feels deeply unsettling.
> 
> Existential musings aside, would any of you have suggestions as to how
> tackle the issue?

Look into https://en.wikipedia.org/wiki/Precision_Time_Protocol

Very few systems run at nanosecond precision. Yours almost certainly don't.

Even measurements on the same system will be off by some fraction. IIRC the precision on Windows systems is about 10 ms. Maybe it's improved since I heard that figure. Add in the delay to service a `nanoTime` call and you might as call it `wishTime`.

When you think about it, it's obvious that synchronization is all but impossible. But the Precision Time Protocol has ways to deal with that.

It's a deep topic, like date/time issues in software generally. It's not for the faint of heart, nor the lazy.

And yes, time is an illusion. 

'Time,' he said, 'is what keeps everything from happening at once.' - Ray Cummings

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.java.programmer

csiph-web