Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.python > #70713
| From | Roy Smith <roy@panix.com> |
|---|---|
| Newsgroups | comp.lang.python |
| Subject | Re: Significant digits in a float? |
| Date | 2014-04-29 09:38 -0400 |
| Organization | PANIX Public Access Internet and UNIX, NYC |
| Message-ID | <roy-DA9493.09383329042014@news.panix.com> (permalink) |
| References | <mailman.9549.1398700831.18130.python-list@python.org> <535f0f9f$0$29965$c3e8da3$5496439d@news.astraweb.com> |
In article <535f0f9f$0$29965$c3e8da3$5496439d@news.astraweb.com>,
Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:
> On Mon, 28 Apr 2014 12:00:23 -0400, Roy Smith wrote:
>
> [...]
> > Fundamentally, these numbers have between 0 and 4 decimal digits of
> > precision,
>
> I'm surprised that you have a source of data with variable precision,
> especially one that varies by a factor of TEN THOUSAND.
OK, you're surprised.
> I don't know what justification you have for combining such a
> mix of data sources.
Because that's the data that was given to me. Real life data is messy.
> One possible interpretation of your post is that you have a source of
> floats, where all the numbers are actually measured to the same
> precision, and you've simply misinterpreted the fact that some of them
> look like they have less precision.
Another possibility is that they're latitude/longitude coordinates, some
of which are given to the whole degree, some of which are given to
greater precision, all the way down to the ten-thousandth of a degree.
> What reason do you have to think that something recorded to 14
> decimal places was only intended to have been recorded to 4?
Because I understand the physical measurement these numbers represent.
Sometimes, Steve, you have to assume that when somebody asks a question,
they actually have asked the question then intended to ask.
> Perhaps you need to explain why you're doing this, as it seems
> numerically broken.
These are latitude and longitude coordinates of locations. Some
locations are known to a specific street address. Some are known to a
city. Some are only known to the country. So, for example, the 38.0
value represents the latitude, to the nearest whole degree, of the
geographic center of the contiguous United States.
> I really think you need to go back to the source. Trying to infer the
> precision of the measurements from the accident of the string formatting
> seems pretty dubious to me.
Sure it is. But, like I said, real-life data is messy. You can wring
your hands and say, "this data sucks, I can't use it", or you can figure
out some way to deal with it. Which is the whole point of my post. The
best I've come up with is inferring something from the string formatting
and I'm hoping there might be something better I might do.
> But I suppose if you wanted to infer the number of digits after the
> decimal place, excluding trailing zeroes (why, I do not understand), up
> to a maximum of four digits, then you could do:
>
> s = "%.4f" % number # rounds to four decimal places
> s = s.rstrip("0") # ignore trailing zeroes, whether significant or not
> count = len(s.split(".")[1])
This at least seems a little more robust than just calling str(). Thank
you :-)
> Assuming all the numbers fit in the range where they are shown in non-
> exponential format.
They're latitude/longitude, so they all fall into [-180, 180].
> Perhaps you ought to be using Decimal rather than float.
Like I said, "The numbers are given to me as Python floats; I have no
control over that".
> > I'm willing to accept that fact that I won't be able to differentiate
> > between float("38.0") and float("38.0000"). Both of those map to 1,
> > which is OK for my purposes.
>
> That seems... well, "bizarre and wrong" are the only words that come to
> mind.
I'm trying to intuit, from the values I've been given, which coordinates
are likely to be accurate to within a few miles. I'm willing to accept
a few false negatives. If the number is float("38"), I'm willing to
accept that it might actually be float("38.0000"), and I might be
throwing out a good data point that I don't need to.
For the purpose I'm using the data for, excluding the occasional good
data point won't hurt me. Including the occasional bad one, will.
> By the way, you contradict yourself here. Earlier, you described 38.0 as
> having zero decimal places (which is wrong). Here you describe it as
> having one, which is correct, and then in a later post you describe it as
> having zero decimal places again.
I was sloppy there. I was copy-pasting data from my program output.
Observe:
>>> print float("38")
38.0
In standard engineering parlance, the string "38" represents a number
with a precision of +/- 1 unit. Unfortunately, Python's default str()
representation turns this into "38.0", which implies +/- 0.1 unit.
Floats represented as strings (at least in some disciplines, such as
engineering) include more information than just the value. By the
number of trailing zeros, they also include information about the
precision of the measurement. That information is lost when the string
is converted to a IEEE float. I'm trying to intuit that information
back, and as I mentioned earlier, am willing to accept that the
intuiting process will be imperfect. There is real-life value in
imperfect processes.
Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Significant digits in a float? Roy Smith <roy@panix.com> - 2014-04-28 12:00 -0400
Re: Significant digits in a float? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-04-29 02:34 +0000
Re: Significant digits in a float? Ben Finney <ben@benfinney.id.au> - 2014-04-29 13:23 +1000
Re: Significant digits in a float? Steven D'Aprano <steve@pearwood.info> - 2014-04-29 05:43 +0000
Re: Significant digits in a float? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-04-29 20:33 -0400
Re: Significant digits in a float? Roy Smith <roy@panix.com> - 2014-04-29 20:42 -0400
Re: Significant digits in a float? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-04-30 20:02 -0400
Re: Significant digits in a float? Mark H Harris <harrismh777@gmail.com> - 2014-04-30 22:54 -0500
Re: Significant digits in a float? Paul Rubin <no.email@nospam.invalid> - 2014-04-30 20:56 -0700
Re: Significant digits in a float? Mark H Harris <harrismh777@gmail.com> - 2014-04-30 23:16 -0500
Re: Significant digits in a float? William Ray Wing <wrw@mac.com> - 2014-05-01 11:53 -0400
Re: Significant digits in a float? Mark H Harris <harrismh777@gmail.com> - 2014-05-06 11:29 -0500
Re: Significant digits in a float? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-05-01 22:06 -0400
Re: Significant digits in a float? Mark H Harris <harrismh777@gmail.com> - 2014-05-06 09:51 -0500
Re: Significant digits in a float? alister <alister.nospam.ware@ntlworld.com> - 2014-05-06 15:18 +0000
Re: Significant digits in a float? Joel Goldstick <joel.goldstick@gmail.com> - 2014-05-06 12:14 -0400
Re: Significant digits in a float? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-05-06 19:48 -0400
Re: Significant digits in a float? Roy Smith <roy@panix.com> - 2014-05-01 08:56 -0400
Re: Significant digits in a float? Larry Hudson <orgnut@yahoo.com> - 2014-05-01 12:51 -0700
Re: Significant digits in a float? Adam Funk <a24061@ducksburg.com> - 2014-05-01 22:05 +0100
Re: Significant digits in a float? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-05-01 21:47 -0400
Re: Significant digits in a float? Mark H Harris <harrismh777@gmail.com> - 2014-05-06 09:59 -0500
[OT] Silde rules [was Re: Significant digits in a float?] Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-05-06 16:10 +0000
Re: [OT] Silde rules Ben Finney <ben@benfinney.id.au> - 2014-05-07 10:01 +1000
Re: Significant digits in a float? Adam Funk <a24061@ducksburg.com> - 2014-05-01 21:55 +0100
Re: Significant digits in a float? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-05-01 22:26 -0400
Re: Significant digits in a float? Adam Funk <a24061@ducksburg.com> - 2014-05-08 20:58 +0100
Re: Significant digits in a float? Roy Smith <roy@panix.com> - 2014-04-29 09:38 -0400
Re: Significant digits in a float? Chris Angelico <rosuav@gmail.com> - 2014-04-30 02:30 +1000
Re: Significant digits in a float? Roy Smith <roy@panix.com> - 2014-04-29 19:53 -0400
Re: Significant digits in a float? Chris Angelico <rosuav@gmail.com> - 2014-04-30 10:13 +1000
Re: Significant digits in a float? Ben Finney <ben@benfinney.id.au> - 2014-04-30 10:13 +1000
Re: Significant digits in a float? Chris Angelico <rosuav@gmail.com> - 2014-04-30 10:17 +1000
Re: Significant digits in a float? Ben Finney <ben@benfinney.id.au> - 2014-04-30 10:20 +1000
Re: Significant digits in a float? Grant Edwards <invalid@invalid.invalid> - 2014-04-30 18:19 +0000
Re: Significant digits in a float? Ned Batchelder <ned@nedbatchelder.com> - 2014-04-29 12:47 -0400
Re: Significant digits in a float? Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-04-30 09:45 +1200
Re: Significant digits in a float? Chris Angelico <rosuav@gmail.com> - 2014-04-30 02:59 +1000
Re: Significant digits in a float? Adam Funk <a24061@ducksburg.com> - 2014-04-29 21:16 +0100
Re: Significant digits in a float? Mark H Harris <harrismh777@gmail.com> - 2014-04-29 15:39 -0500
Re: Significant digits in a float? Chris Angelico <rosuav@gmail.com> - 2014-04-30 07:15 +1000
Re: Significant digits in a float? Ryan Hiebert <ryan@ryanhiebert.com> - 2014-04-29 15:42 -0500
Re: Significant digits in a float? emile <emile@fenx.com> - 2014-04-29 15:42 -0700
Re: Significant digits in a float? alister <alister.nospam.ware@ntlworld.com> - 2014-04-30 09:03 +0000
Re: Significant digits in a float? Grant Edwards <invalid@invalid.invalid> - 2014-04-30 18:21 +0000
Re: Significant digits in a float? emile <emile@fenx.com> - 2014-05-01 09:34 -0700
Re: Significant digits in a float? alister <alister.nospam.ware@ntlworld.com> - 2014-05-01 17:13 +0000
Re: Significant digits in a float? Chris Angelico <rosuav@gmail.com> - 2014-04-30 08:51 +1000
Re: Significant digits in a float? Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-04-30 20:14 +1200
Re: Significant digits in a float? Chris Angelico <rosuav@gmail.com> - 2014-04-30 18:27 +1000
Re: Significant digits in a float? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-04-30 14:05 +0100
Re: Significant digits in a float? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-04-30 00:34 +0100
Re: Significant digits in a float? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-04-29 20:37 -0400
Re: Significant digits in a float? Roy Smith <roy@panix.com> - 2014-04-29 20:45 -0400
Re: Significant digits in a float? Chris Angelico <rosuav@gmail.com> - 2014-04-30 10:45 +1000
Re: Significant digits in a float? Roy Smith <roy@panix.com> - 2014-04-29 20:48 -0400
Re: Significant digits in a float? Ethan Furman <ethan@stoneleaf.us> - 2014-04-29 19:31 -0700
Re: Significant digits in a float? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-04-30 02:59 +0000
Re: Significant digits in a float? Roy Smith <roy@panix.com> - 2014-04-29 23:30 -0400
Re: Significant digits in a float? Chris Angelico <rosuav@gmail.com> - 2014-04-30 13:40 +1000
Re: Significant digits in a float? Chris Angelico <rosuav@gmail.com> - 2014-04-30 13:03 +1000
Re: Significant digits in a float? Roy Smith <roy@panix.com> - 2014-04-30 08:01 -0400
Off-topic circumnavigating the earth in a mile or less [was Re: Significant digits in a float?] Ethan Furman <ethan@stoneleaf.us> - 2014-04-30 06:14 -0700
Re: Off-topic circumnavigating the earth in a mile or less [was Re: Significant digits in a float?] Ethan Furman <ethan@stoneleaf.us> - 2014-04-30 07:02 -0700
Re: Off-topic circumnavigating the earth in a mile or less Adam Funk <a24061@ducksburg.com> - 2014-05-01 21:52 +0100
Re: Off-topic circumnavigating the earth in a mile or less [was Re: Significant digits in a float?] Chris Angelico <rosuav@gmail.com> - 2014-05-01 00:27 +1000
Re: Off-topic circumnavigating the earth in a mile or less [was Re: Significant digits in a float?] Chris Angelico <rosuav@gmail.com> - 2014-05-01 05:10 +1000
Re: Off-topic circumnavigating the earth in a mile or less [was Re: Significant digits in a float?] Ryan Hiebert <ryan@ryanhiebert.com> - 2014-04-30 10:24 -0500
Re: Off-topic circumnavigating the earth in a mile or less [was Re: Significant digits in a float?] Ian Kelly <ian.g.kelly@gmail.com> - 2014-04-30 17:46 -0600
Re: Off-topic circumnavigating the earth in a mile or less [was Re: Significant digits in a float?] Terry Reedy <tjreedy@udel.edu> - 2014-04-30 22:20 -0400
Re: Off-topic circumnavigating the earth in a mile or less [was Re: Significant digits in a float?] Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-05-01 18:16 +1200
Re: Off-topic circumnavigating the earth in a mile or less Adam Funk <a24061@ducksburg.com> - 2014-05-01 21:57 +0100
Re: Off-topic circumnavigating the earth in a mile or less Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-05-01 22:49 +0100
Re: Off-topic circumnavigating the earth in a mile or less alister <alister.nospam.ware@ntlworld.com> - 2014-05-02 08:49 +0000
Re: Off-topic circumnavigating the earth in a mile or less [was Re: Significant digits in a float?] Chris Angelico <rosuav@gmail.com> - 2014-05-01 11:57 +1000
Re: Off-topic circumnavigating the earth in a mile or less [was Re: Significant digits in a float?] Vlastimil Brom <vlastimil.brom@gmail.com> - 2014-05-01 09:49 +0200
Re: Significant digits in a float? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-04-30 02:50 +0000
Re: Significant digits in a float? Grant Edwards <invalid@invalid.invalid> - 2014-04-30 18:18 +0000
csiph-web