Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #65533 > unrolled thread

Handling numeric types in interpreter

Back to article view | Back to comp.lang.c

Contents

  Handling numeric types in interpreter luser droog <luser.droog@gmail.com> - 2015-07-13 19:33 -0700
    Re: Handling numeric types in interpreter jacobnavia <jacob@jacob.remcomp.fr> - 2015-07-14 07:31 +0200
    Handling numeric types in interpreter asetofsymbols@gmail.com - 2015-07-14 00:35 -0700
    Re: Handling numeric types in interpreter David Kleinecke <dkleinecke@gmail.com> - 2015-07-14 08:55 -0700
    Re: Handling numeric types in interpreter Bartc <bc@freeuk.com> - 2015-07-15 23:18 +0100
      Re: Handling numeric types in interpreter Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-07-24 00:21 -0700
        Re: Handling numeric types in interpreter Keith Thompson <kst-u@mib.org> - 2015-07-24 08:36 -0700
          Re: Handling numeric types in interpreter Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-07-24 11:04 -0700
            Re: Handling numeric types in interpreter Keith Thompson <kst-u@mib.org> - 2015-07-24 12:25 -0700
              Re: Handling numeric types in interpreter James Kuyper <jameskuyper@verizon.net> - 2015-07-24 22:21 -0400
                Re: Handling numeric types in interpreter Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-07-25 02:28 -0700
                  Re: Handling numeric types in interpreter Bartc <bc@freeuk.com> - 2015-07-25 15:16 +0100
                  Re: Handling numeric types in interpreter Richard Damon <Richard@Damon-Family.org> - 2015-07-25 11:03 -0400
                  Re: Handling numeric types in interpreter Keith Thompson <kst-u@mib.org> - 2015-07-25 10:05 -0700
                Re: Handling numeric types in interpreter gazelle@shell.xmission.com (Kenny McCormack) - 2015-07-25 12:32 +0000
                  Re: Handling numeric types in interpreter Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-07-25 06:12 -0700
                    Re: Handling numeric types in interpreter Keith Thompson <kst-u@mib.org> - 2015-07-25 10:08 -0700
                      Re: Handling numeric types in interpreter ais523 <ais523@nethack4.org> - 2015-07-28 17:50 +0000
                        Re: Handling numeric types in interpreter Keith Thompson <kst-u@mib.org> - 2015-07-28 11:52 -0700
                        Re: Handling numeric types in interpreter Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-07-28 12:25 -0700
                        Re: Handling numeric types in interpreter Richard Heathfield <rjh@cpax.org.uk> - 2015-07-28 20:52 +0100
                          Re: Handling numeric types in interpreter jacobnavia <jacob@jacob.remcomp.fr> - 2015-07-28 23:01 +0200
                            Re: Handling numeric types in interpreter Keith Thompson <kst-u@mib.org> - 2015-07-28 14:44 -0700
                              Re: Handling numeric types in interpreter Ian Collins <ian-news@hotmail.com> - 2015-07-29 09:43 +1200
                              Re: Handling numeric types in interpreter glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2015-07-28 23:10 +0000
                                Re: Handling numeric types in interpreter Ian Collins <ian-news@hotmail.com> - 2015-07-29 12:04 +1200
                                  Re: Handling numeric types in interpreter Melzzzzz <mel@zzzzz.com> - 2015-07-29 02:39 +0200
                                Re: Handling numeric types in interpreter jacobnavia <jacob@jacob.remcomp.fr> - 2015-07-29 12:04 +0200
                                  Re: Handling numeric types in interpreter glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2015-07-29 14:58 +0000
                                Re: Handling numeric types in interpreter jacobnavia <jacob@jacob.remcomp.fr> - 2015-07-29 14:14 +0200
                                  Re: Handling numeric types in interpreter Bartc <bc@freeuk.com> - 2015-07-29 13:29 +0100
                                    Re: Handling numeric types in interpreter Keith Thompson <kst-u@mib.org> - 2015-07-29 08:48 -0700
                                  Re: Handling numeric types in interpreter Keith Thompson <kst-u@mib.org> - 2015-07-29 08:45 -0700
                              Re: Handling numeric types in interpreter Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-07-28 16:28 -0700
                              Re: Handling numeric types in interpreter jacobnavia <jacob@jacob.remcomp.fr> - 2015-07-29 12:02 +0200
                              Re: Handling numeric types in interpreter Keith Thompson <kst-u@mib.org> - 2015-07-29 08:51 -0700
                                Re: Handling numeric types in interpreter jacobnavia <jacob@jacob.remcomp.fr> - 2015-07-29 18:35 +0200
                                  Re: Handling numeric types in interpreter Keith Thompson <kst-u@mib.org> - 2015-07-29 10:47 -0700
                    Re: Handling numeric types in interpreter Robert Wessel <robertwessel2@yahoo.com> - 2015-07-25 15:46 -0500
                      Re: Handling numeric types in interpreter Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-07-25 14:08 -0700
                        Re: Handling numeric types in interpreter Keith Thompson <kst-u@mib.org> - 2015-07-25 15:42 -0700
                          Re: Handling numeric types in interpreter Robert Wessel <robertwessel2@yahoo.com> - 2015-07-26 00:31 -0500
                      Re: Handling numeric types in interpreter James Kuyper <jameskuyper@verizon.net> - 2015-07-25 17:48 -0400
                        Re: Handling numeric types in interpreter Robert Wessel <robertwessel2@yahoo.com> - 2015-07-25 17:31 -0500
                          Re: Handling numeric types in interpreter Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-07-25 15:57 -0700
                            Re: Handling numeric types in interpreter Bartc <bc@freeuk.com> - 2015-07-26 01:07 +0100
                              Re: Handling numeric types in interpreter Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-07-25 17:26 -0700
                                Re: Handling numeric types in interpreter Richard Damon <Richard@Damon-Family.org> - 2015-07-25 22:25 -0400
                          Re: Handling numeric types in interpreter James Kuyper <jameskuyper@verizon.net> - 2015-07-25 19:04 -0400
                            Re: Handling numeric types in interpreter Robert Wessel <robertwessel2@yahoo.com> - 2015-07-26 00:23 -0500
                              Re: Handling numeric types in interpreter James Kuyper <jameskuyper@verizon.net> - 2015-07-26 08:24 -0400
                                Re: Handling numeric types in interpreter Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-07-26 06:36 -0700
                                  Re: Handling numeric types in interpreter Robert Wessel <robertwessel2@yahoo.com> - 2015-07-26 16:30 -0500
                                  Re: Handling numeric types in interpreter Keith Thompson <kst-u@mib.org> - 2015-07-26 15:42 -0700
                                Re: Handling numeric types in interpreter Robert Wessel <robertwessel2@yahoo.com> - 2015-07-26 16:28 -0500
                                  Re: Handling numeric types in interpreter James Kuyper <jameskuyper@verizon.net> - 2015-07-26 18:57 -0400
                                    Re: Handling numeric types in interpreter Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-07-26 16:20 -0700
                        Re: Handling numeric types in interpreter Keith Thompson <kst-u@mib.org> - 2015-07-25 15:58 -0700
                Re: Handling numeric types in interpreter James Kuyper <jameskuyper@verizon.net> - 2015-07-25 12:56 -0400
                  Re: Handling numeric types in interpreter Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-07-25 14:18 -0700
        Re: Handling numeric types in interpreter glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2015-07-24 18:06 +0000
          Re: Handling numeric types in interpreter Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-07-24 11:16 -0700
            Re: Handling numeric types in interpreter glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2015-07-24 23:06 +0000
              Re: Handling numeric types in interpreter Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-07-24 16:21 -0700
    Re: Handling numeric types in interpreter Michael Angelo Ravera <maravera@prodigy.net> - 2015-07-15 19:13 -0700
      Re: Handling numeric types in interpreter luser droog <luser.droog@gmail.com> - 2015-07-23 08:36 -0700

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

#66367

Malcolm McLean <malcolm.mclean5@btinternet.com> writes:
[...]
> TIFF files allow values to be entered as 32 bit longs. A byte buffer
> could easily use most of this range (the standard suggests limiting
> buffers to about 8K, but this isn't enforced). The realistic risk
> is that we'll need to multiply two 32 values, and end up with
> a valid seeming but actually wrong answer, and a buffer overflow
> attack.
> Doubles help because the result should still be representable.
> It won't be accurate, but it will be obviously out of the
> sensible range, and thus easier to reject.

The product of two 32-bit values will fit in a long long.

If you need to deal with compilers that don't support long long
(or another 64-bit integer type), you might consider writing two
alternative implementation, a simpler one that just uses integer
arithmetic and another that uses floating-point.  As support
for 64-bit integers becomes more widespread (it's probably close
to universal now, with pre-C99 compilers supporting them as an
extension), you can deprecate the other version.

> But making the reader robust to any attack whatsoever is a major
> challenge when writing it in C. Lib Tiff has suffered from exactly
> that problem. 

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#66395

On Sat, 25 Jul 2015 15:42:48 -0700, Keith Thompson <kst-u@mib.org>
wrote:

>Malcolm McLean <malcolm.mclean5@btinternet.com> writes:
>[...]
>> TIFF files allow values to be entered as 32 bit longs. A byte buffer
>> could easily use most of this range (the standard suggests limiting
>> buffers to about 8K, but this isn't enforced). The realistic risk
>> is that we'll need to multiply two 32 values, and end up with
>> a valid seeming but actually wrong answer, and a buffer overflow
>> attack.
>> Doubles help because the result should still be representable.
>> It won't be accurate, but it will be obviously out of the
>> sensible range, and thus easier to reject.
>
>The product of two 32-bit values will fit in a long long.
>
>If you need to deal with compilers that don't support long long
>(or another 64-bit integer type), you might consider writing two
>alternative implementation, a simpler one that just uses integer
>arithmetic and another that uses floating-point.  As support
>for 64-bit integers becomes more widespread (it's probably close
>to universal now, with pre-C99 compilers supporting them as an
>extension), you can deprecate the other version.


Well, as opposed to having both an integer and FP version, a small
bignum implementation (pretty trivial for just 64-bit ints), would do
nicely.  While in C you can't overload operators, and you'd have to do
all the arithmetic with function calls, you could still have that
build two ways, one where the individual functions are just operations
on long longs (perhaps embedded in a struct for consistency), the
other on a struct representing a bignum 64-bit item.  And there are
important platforms where there is a 64-bit type, but it's not long
long (__INT64 on MSVC, for example) - that could be accommodated in
that scheme as well.

Having both an int and FP version just seems like a good way for
allowing a lot of off-by-one errors to slip in, depending on the mode.

[toc] | [prev] | [next] | [standalone]

#66362

On 07/25/2015 04:46 PM, Robert Wessel wrote:
> On Sat, 25 Jul 2015 06:12:16 -0700 (PDT), Malcolm McLean
> <malcolm.mclean5@btinternet.com> wrote:
> 
>> On Saturday, July 25, 2015 at 1:32:37 PM UTC+1, Kenny McCormack wrote:
>>> In article <mourp8$mf$1@dont-email.me>,
>>> James Kuyper  <jameskuyper@verizon.net> wrote:
>>>> Malcolm McLean <malcolm.mclean5@btinternet.com> writes:
>>>> ...
>>>>> It's more likely that the tiff reader will be run on a machine
>>>>> that doesn't support long double ...
>>>>
>>>> How many implementations of C do you know of that fail to conform to the
>>>> C standard by not supporting long double?
>>>
>>> Now, *that* is a rhetorical question.
>>>
>>> Seriously, are you really looking for a numerical answer?

I'll accept 'None' or 'Many' if that's the best answer he can give me.
However, if it's a non-zero number, I expect it to be small enough that
he can reasonably be expected to give me the exact value - it is, after,
the number of such implementations *known to him*, not the total number
of such implementations, which would be much harder to count.

I expected the number to be 0, in which case I would criticize his
decision to worry about the issue, so in that sense the question is
rhetorical - it's intended primarily to remind him how ridiculous it is
to worry about that possibility.
However, if he were to give me a non-zero answer, I'd find that
interesting, and inquire further, so in that sense it's not a rhetorical
question.

If I knew for certain that the number of such implementations was 0 (I
can't imagine how I could justify such certainty - but most people find
unjustified certainty easy to come by), then I would not have asked a
rhetorical question, I would have stated it as a fact. Would you prefer
that I lied by pretending that I was certain that the number was 0?

>> But it's a fair enough point. If the strategy is to put all scalars
>> into a common format, then the question of whether we can rely on
>> long double being present or not is important. Actually legitimate
>> values in a tiff file for a tiff reader can't go over 2^50 as far as 
>> I know, so double is OK.
> 
> 
> Doubles are required to have 2**50 in their range, but are not

I'm curious - where is this specified? I went searching for a
specification of the TIFF format, and found
<http://partners.adobe.com/public/developer/tiff/index.html>. I couldn't
find it in any of the documents listed there. Did I find the wrong
documents?

[toc] | [prev] | [next] | [standalone]

#66366

On Sat, 25 Jul 2015 17:48:59 -0400, James Kuyper
<jameskuyper@verizon.net> wrote:

>On 07/25/2015 04:46 PM, Robert Wessel wrote:
>> On Sat, 25 Jul 2015 06:12:16 -0700 (PDT), Malcolm McLean
>> <malcolm.mclean5@btinternet.com> wrote:
>> 
>>> On Saturday, July 25, 2015 at 1:32:37 PM UTC+1, Kenny McCormack wrote:
>>>> In article <mourp8$mf$1@dont-email.me>,
>>>> James Kuyper  <jameskuyper@verizon.net> wrote:
>>>>> Malcolm McLean <malcolm.mclean5@btinternet.com> writes:
>>>>> ...
>>>>>> It's more likely that the tiff reader will be run on a machine
>>>>>> that doesn't support long double ...
>>>>>
>>>>> How many implementations of C do you know of that fail to conform to the
>>>>> C standard by not supporting long double?
>>>>
>>>> Now, *that* is a rhetorical question.
>>>>
>>>> Seriously, are you really looking for a numerical answer?
>
>I'll accept 'None' or 'Many' if that's the best answer he can give me.
>However, if it's a non-zero number, I expect it to be small enough that
>he can reasonably be expected to give me the exact value - it is, after,
>the number of such implementations *known to him*, not the total number
>of such implementations, which would be much harder to count.
>
>I expected the number to be 0, in which case I would criticize his
>decision to worry about the issue, so in that sense the question is
>rhetorical - it's intended primarily to remind him how ridiculous it is
>to worry about that possibility.
>However, if he were to give me a non-zero answer, I'd find that
>interesting, and inquire further, so in that sense it's not a rhetorical
>question.
>
>If I knew for certain that the number of such implementations was 0 (I
>can't imagine how I could justify such certainty - but most people find
>unjustified certainty easy to come by), then I would not have asked a
>rhetorical question, I would have stated it as a fact. Would you prefer
>that I lied by pretending that I was certain that the number was 0?
>
>>> But it's a fair enough point. If the strategy is to put all scalars
>>> into a common format, then the question of whether we can rely on
>>> long double being present or not is important. Actually legitimate
>>> values in a tiff file for a tiff reader can't go over 2^50 as far as 
>>> I know, so double is OK.
>> 
>> 
>> Doubles are required to have 2**50 in their range, but are not
>
>I'm curious - where is this specified? I went searching for a
>specification of the TIFF format, and found
><http://partners.adobe.com/public/developer/tiff/index.html>. I couldn't
>find it in any of the documents listed there. Did I find the wrong
>documents?


I was referring to the C requirements for doubles, which require a
range of 10**-37..10**37 and 10 decimal digit of precision for a
double.  The first requirement accommodates 2**50, the second does
not.  The TIFF spec is that floats and double are IEEE format (which
has a 11 bit exponent and a 53 bit significand).

[toc] | [prev] | [next] | [standalone]

#66368

On Saturday, July 25, 2015 at 11:31:01 PM UTC+1, robert...@yahoo.com wrote:
> 
> I was referring to the C requirements for doubles, which require a
> range of 10**-37..10**37 and 10 decimal digit of precision for a
> double.  The first requirement accommodates 2**50, the second does
> not.  The TIFF spec is that floats and double are IEEE format (which
> has a 11 bit exponent and a 53 bit significand).
>
The numbers in a TIFF file have to represent something. It's
hard to say how big a byte buffer can legitimately be, because
not all TIFF images are intended for human viewing, some of them
are microarray data or medical images. But they still can't have
more than 2^50 bytes of data in them with current technology.
 

[toc] | [prev] | [next] | [standalone]

#66380

On 25/07/2015 23:57, Malcolm McLean wrote:
> On Saturday, July 25, 2015 at 11:31:01 PM UTC+1, robert...@yahoo.com wrote:
>>
>> I was referring to the C requirements for doubles, which require a
>> range of 10**-37..10**37 and 10 decimal digit of precision for a
>> double.  The first requirement accommodates 2**50, the second does
>> not.  The TIFF spec is that floats and double are IEEE format (which
>> has a 11 bit exponent and a 53 bit significand).
>>
> The numbers in a TIFF file have to represent something. It's
> hard to say how big a byte buffer can legitimately be, because
> not all TIFF images are intended for human viewing, some of them
> are microarray data or medical images. But they still can't have
> more than 2^50 bytes of data in them with current technology.

TIFF is one of those complicated file formats that you would have hoped 
had died out. But apparently it's still going strong!

(I can't remember if I ever wrote a loader for TIFF, but if I did, it 
would have been for something simple such as a bi-level image, and for 
the first image in the file.)

What would it be needing floats for anyway? Image data is usually 
integers. It would also be unwise for someone to store data in a file 
that relies on an ill-defined 'long double' (which might be more precise 
than a double, or it might be the same; so why not just use double?)

-- 
Bartc

[toc] | [prev] | [next] | [standalone]

#66382

On Sunday, July 26, 2015 at 1:07:26 AM UTC+1, Bart wrote:
> On 25/07/2015 23:57, Malcolm McLean wrote:
> >
> > The numbers in a TIFF file have to represent something. It's
> > hard to say how big a byte buffer can legitimately be, because
> > not all TIFF images are intended for human viewing, some of them
> > are microarray data or medical images. But they still can't have
> > more than 2^50 bytes of data in them with current technology.
> 
> TIFF is one of those complicated file formats that you would have hoped 
> had died out. But apparently it's still going strong!
>
It's not really a file format, its's more a system of tags that
tell the reader what format the data is in. Then there are
several sub-formats embedded in it, some quite trivial, like raw
8 bit rgb samples, and some quite complex, like sub-sampled
YCbCr with zlib compression and predictive coding, arranged
in tiles.
>
> (I can't remember if I ever wrote a loader for TIFF, but if I did, it 
> would have been for something simple such as a bi-level image, and for 
> the first image in the file.)
>
Ha. Bi-level images aren't simple. It took several days to get
the fax codecs right. The 2D compression is very hard to debug 
because the system appears to be working fine for a while after
it has failed. I was blowing up images to pixel level, trying to
work out the first pixel that was different, and back tracking into
the code to find the point at which it went wrong.
>
> What would it be needing floats for anyway? Image data is usually 
> integers. It would also be unwise for someone to store data in a file 
> that relies on an ill-defined 'long double' (which might be more precise 
> than a double, or it might be the same; so why not just use double?)
> 
Not high-end image data. 24 bit rgb is what most monitors handle,
and you only get a few artifacts because of unrepresentable colours.
But that doesn't mean that the underlying image can be adequately
represented by 8 bit channels. When you print it, or try to correct
it for display on a  monitor with different physical colour 
characteristics, you can notice a subtle but, for professional
graphics, important loss of quality.
TIFF files really come into their own for high end image data.
A general purpose reader inherently can't support that, but I'm
hoping the code will be simple enough for anyone needing a 
specialised reader to modify as a short, one programmer job.
 

[toc] | [prev] | [next] | [standalone]

#66386

On 7/25/15 8:26 PM, Malcolm McLean wrote:
> Not high-end image data. 24 bit rgb is what most monitors handle,
> and you only get a few artifacts because of unrepresentable colours.
> But that doesn't mean that the underlying image can be adequately
> represented by 8 bit channels. When you print it, or try to correct
> it for display on a  monitor with different physical colour
> characteristics, you can notice a subtle but, for professional
> graphics, important loss of quality.
> TIFF files really come into their own for high end image data.
> A general purpose reader inherently can't support that, but I'm
> hoping the code will be simple enough for anyone needing a
> specialised reader to modify as a short, one programmer job.
>
>

Actually, a lot of the protocols will handle 36 bit RGB (12 bits per color).

I have also run into a number of 16 bit gray scale images, and a few 
which expressed the image as an array of single precision floats because 
the underlying data had an extreme dynamic range.

Normally these sort of images are just directly displayed, but are 
processed and the results displayed.

[toc] | [prev] | [next] | [standalone]

#66373

On 07/25/2015 06:31 PM, Robert Wessel wrote:
> On Sat, 25 Jul 2015 17:48:59 -0400, James Kuyper
> <jameskuyper@verizon.net> wrote:
> 
>> On 07/25/2015 04:46 PM, Robert Wessel wrote:
>>> On Sat, 25 Jul 2015 06:12:16 -0700 (PDT), Malcolm McLean
>>> <malcolm.mclean5@btinternet.com> wrote:
...
>>>> long double being present or not is important. Actually legitimate
>>>> values in a tiff file for a tiff reader can't go over 2^50 as far as 
>>>> I know, so double is OK.
>>>
>>>
>>> Doubles are required to have 2**50 in their range, but are not
>>
>> I'm curious - where is this specified? I went searching for a
>> specification of the TIFF format, and found
>> <http://partners.adobe.com/public/developer/tiff/index.html>. I couldn't
>> find it in any of the documents listed there. Did I find the wrong
>> documents?
> 
> 
> I was referring to the C requirements for doubles, which require a
> range of 10**-37..10**37 and 10 decimal digit of precision for a
> double.  The first requirement accommodates 2**50, the second does
> not.  The TIFF spec is that floats and double are IEEE format (which
> has a 11 bit exponent and a 53 bit significand).

In that case, the limit that Malcolm should be using should be derived
from the IEEE double specification, and is therefore
0X1.fffffffffffffP1023, not 2^50.

[toc] | [prev] | [next] | [standalone]

#66394

On Sat, 25 Jul 2015 19:04:34 -0400, James Kuyper
<jameskuyper@verizon.net> wrote:

>On 07/25/2015 06:31 PM, Robert Wessel wrote:
>> On Sat, 25 Jul 2015 17:48:59 -0400, James Kuyper
>> <jameskuyper@verizon.net> wrote:
>> 
>>> On 07/25/2015 04:46 PM, Robert Wessel wrote:
>>>> On Sat, 25 Jul 2015 06:12:16 -0700 (PDT), Malcolm McLean
>>>> <malcolm.mclean5@btinternet.com> wrote:
>...
>>>>> long double being present or not is important. Actually legitimate
>>>>> values in a tiff file for a tiff reader can't go over 2^50 as far as 
>>>>> I know, so double is OK.
>>>>
>>>>
>>>> Doubles are required to have 2**50 in their range, but are not
>>>
>>> I'm curious - where is this specified? I went searching for a
>>> specification of the TIFF format, and found
>>> <http://partners.adobe.com/public/developer/tiff/index.html>. I couldn't
>>> find it in any of the documents listed there. Did I find the wrong
>>> documents?
>> 
>> 
>> I was referring to the C requirements for doubles, which require a
>> range of 10**-37..10**37 and 10 decimal digit of precision for a
>> double.  The first requirement accommodates 2**50, the second does
>> not.  The TIFF spec is that floats and double are IEEE format (which
>> has a 11 bit exponent and a 53 bit significand).
>
>In that case, the limit that Malcolm should be using should be derived
>from the IEEE double specification, and is therefore
>0X1.fffffffffffffP1023, not 2^50.


No, he has to be able to process all values that can *validly* occur
in a TIFF file, or at least in a TIFF file his code purports to
handle.  If an invalid value is found (and they all ned to be checked)
he needs to return INVALID_TIFF_FILE.  I'm not commenting on what the
actual range of valid values are.

[toc] | [prev] | [next] | [standalone]

#66402

On 07/26/2015 01:23 AM, Robert Wessel wrote:
> On Sat, 25 Jul 2015 19:04:34 -0400, James Kuyper
> <jameskuyper@verizon.net> wrote:
> 
>> On 07/25/2015 06:31 PM, Robert Wessel wrote:
>>> On Sat, 25 Jul 2015 17:48:59 -0400, James Kuyper
>>> <jameskuyper@verizon.net> wrote:
>>>
>>>> On 07/25/2015 04:46 PM, Robert Wessel wrote:
>>>>> On Sat, 25 Jul 2015 06:12:16 -0700 (PDT), Malcolm McLean
>>>>> <malcolm.mclean5@btinternet.com> wrote:
>> ...
>>>>>> long double being present or not is important. Actually legitimate
>>>>>> values in a tiff file for a tiff reader can't go over 2^50 as far as 
>>>>>> I know, so double is OK.
>>>>>
>>>>>
>>>>> Doubles are required to have 2**50 in their range, but are not
>>>>
>>>> I'm curious - where is this specified? I went searching for a
>>>> specification of the TIFF format, and found
>>>> <http://partners.adobe.com/public/developer/tiff/index.html>. I couldn't
>>>> find it in any of the documents listed there. Did I find the wrong
>>>> documents?
>>>
>>>
>>> I was referring to the C requirements for doubles, which require a
>>> range of 10**-37..10**37 and 10 decimal digit of precision for a
>>> double.  The first requirement accommodates 2**50, the second does
>>> not.  The TIFF spec is that floats and double are IEEE format (which
>>> has a 11 bit exponent and a 53 bit significand).
>>
>> In that case, the limit that Malcolm should be using should be derived
>>from the IEEE double specification, and is therefore
>> 0X1.fffffffffffffP1023, not 2^50.
> 
> 
> No, he has to be able to process all values that can *validly* occur
> in a TIFF file, or at least in a TIFF file his code purports to
> handle.  If an invalid value is found (and they all ned to be checked)
> he needs to return INVALID_TIFF_FILE.  I'm not commenting on what the
> actual range of valid values are.

Well, unlike you, I was commenting on the actual range of valid values.
He seems to think the upper limit is 2^50, and I was asking about the
basis for that limit. His wording imply that it's a limit that applies
to all TIFF files, not just the particular TIFF files his reader is
intended to process. As far as I can tell, since the TIFF specification
says that DOUBLE is supposed to have IEEE format (and presumably,
specifically the IEEE format for double precision),
0X1.fffffffffffffP1023 is the correct upper limit. As a result, on some
machines 'double' might not have enough range: DBL_MAX is allowed to be
as low as 1e37. Of course, LDBL_MAX is also allowed to be as low as 1e37
- but it will never be smaller than DBL_MAX, so using 'long double' is
marginally safer. He's free to assume that all machines this code will
ever be ported to will have IEEE doubles, but if so, he should state
that as a requirement rather than silently failing to address the issue.
-- 
James Kuyper

[toc] | [prev] | [next] | [standalone]

#66405

On Sunday, July 26, 2015 at 1:25:09 PM UTC+1, James Kuyper wrote:
> On 07/26/2015 01:23 AM, Robert Wessel wrote:
> 
> Well, unlike you, I was commenting on the actual range of valid values.
> He seems to think the upper limit is 2^50, and I was asking about the
> basis for that limit. His wording imply that it's a limit that applies
> to all TIFF files, not just the particular TIFF files his reader is
> intended to process. 
>
The value needs to represent something. So what would a value of over
2^50 represent, in the context of a TIFF file? I've been coding the reader
for a couple of weeks now in my spare time, and I haven't come
across an example of a value that could go above 2^50 in a legitimate
file.

[toc] | [prev] | [next] | [standalone]

#66419

On Sun, 26 Jul 2015 06:36:33 -0700 (PDT), Malcolm McLean
<malcolm.mclean5@btinternet.com> wrote:

>On Sunday, July 26, 2015 at 1:25:09 PM UTC+1, James Kuyper wrote:
>> On 07/26/2015 01:23 AM, Robert Wessel wrote:
>> 
>> Well, unlike you, I was commenting on the actual range of valid values.
>> He seems to think the upper limit is 2^50, and I was asking about the
>> basis for that limit. His wording imply that it's a limit that applies
>> to all TIFF files, not just the particular TIFF files his reader is
>> intended to process. 
>>
>The value needs to represent something. So what would a value of over
>2^50 represent, in the context of a TIFF file? I've been coding the reader
>for a couple of weeks now in my spare time, and I haven't come
>across an example of a value that could go above 2^50 in a legitimate
>file.


So I'm curious, what actually uses floats (of either size) in a TIFF
file?  I know the types are defined, but I don't actually know of any
tags using them.

[toc] | [prev] | [next] | [standalone]

#66425

Malcolm McLean <malcolm.mclean5@btinternet.com> writes:
> On Sunday, July 26, 2015 at 1:25:09 PM UTC+1, James Kuyper wrote:
>> Well, unlike you, I was commenting on the actual range of valid values.
>> He seems to think the upper limit is 2^50, and I was asking about the
>> basis for that limit. His wording imply that it's a limit that applies
>> to all TIFF files, not just the particular TIFF files his reader is
>> intended to process. 
>>
> The value needs to represent something. So what would a value of over
> 2^50 represent, in the context of a TIFF file? I've been coding the reader
> for a couple of weeks now in my spare time, and I haven't come
> across an example of a value that could go above 2^50 in a legitimate
> file.

Answering that would require understanding the TIFF format, which
I don't, and I'm sure I'm not the only one.

If the value represents the size of an image, then it's reasonable
to assume that an image file won't be over a quadrillion pixels
across -- or over a quadrillion pixels total, or over a quadrillion
bytes or bits in size.  If it represents the distance in meters
to an object in the image, such a limit would exclude photos of
any objects outside the solar system.  I don't know what kind of
metadata are included.

I *think* you picked 2^50 as an arbitrary limit, so that no
plausible image would be that big.  I think some others are assuming
it there's a specific reason you chose that particular value.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#66418

On Sun, 26 Jul 2015 08:24:50 -0400, James Kuyper
<jameskuyper@verizon.net> wrote:

>On 07/26/2015 01:23 AM, Robert Wessel wrote:
>> On Sat, 25 Jul 2015 19:04:34 -0400, James Kuyper
>> <jameskuyper@verizon.net> wrote:
>> 
>>> On 07/25/2015 06:31 PM, Robert Wessel wrote:
>>>> On Sat, 25 Jul 2015 17:48:59 -0400, James Kuyper
>>>> <jameskuyper@verizon.net> wrote:
>>>>
>>>>> On 07/25/2015 04:46 PM, Robert Wessel wrote:
>>>>>> On Sat, 25 Jul 2015 06:12:16 -0700 (PDT), Malcolm McLean
>>>>>> <malcolm.mclean5@btinternet.com> wrote:
>>> ...
>>>>>>> long double being present or not is important. Actually legitimate
>>>>>>> values in a tiff file for a tiff reader can't go over 2^50 as far as 
>>>>>>> I know, so double is OK.
>>>>>>
>>>>>>
>>>>>> Doubles are required to have 2**50 in their range, but are not
>>>>>
>>>>> I'm curious - where is this specified? I went searching for a
>>>>> specification of the TIFF format, and found
>>>>> <http://partners.adobe.com/public/developer/tiff/index.html>. I couldn't
>>>>> find it in any of the documents listed there. Did I find the wrong
>>>>> documents?
>>>>
>>>>
>>>> I was referring to the C requirements for doubles, which require a
>>>> range of 10**-37..10**37 and 10 decimal digit of precision for a
>>>> double.  The first requirement accommodates 2**50, the second does
>>>> not.  The TIFF spec is that floats and double are IEEE format (which
>>>> has a 11 bit exponent and a 53 bit significand).
>>>
>>> In that case, the limit that Malcolm should be using should be derived
>>>from the IEEE double specification, and is therefore
>>> 0X1.fffffffffffffP1023, not 2^50.
>> 
>> 
>> No, he has to be able to process all values that can *validly* occur
>> in a TIFF file, or at least in a TIFF file his code purports to
>> handle.  If an invalid value is found (and they all ned to be checked)
>> he needs to return INVALID_TIFF_FILE.  I'm not commenting on what the
>> actual range of valid values are.
>
>Well, unlike you, I was commenting on the actual range of valid values.
>He seems to think the upper limit is 2^50, and I was asking about the
>basis for that limit. His wording imply that it's a limit that applies
>to all TIFF files, not just the particular TIFF files his reader is
>intended to process. As far as I can tell, since the TIFF specification
>says that DOUBLE is supposed to have IEEE format (and presumably,
>specifically the IEEE format for double precision),
>0X1.fffffffffffffP1023 is the correct upper limit. As a result, on some
>machines 'double' might not have enough range: DBL_MAX is allowed to be
>as low as 1e37. Of course, LDBL_MAX is also allowed to be as low as 1e37
>- but it will never be smaller than DBL_MAX, so using 'long double' is
>marginally safer. He's free to assume that all machines this code will
>ever be ported to will have IEEE doubles, but if so, he should state
>that as a requirement rather than silently failing to address the issue.


Just because a value is stored as an IEEE float doesn't mean that all
values possible in that representation are meaningful or valid.
Consider a medical application that stored a human's mass and height
in floats - no one would expect it to produce valid results if it
encountered a record specifying a 20,000kg or 40m human.  OTOH, it's
perfectly reasonable to expect that the application would (safely)
refuse to process such a record with an appropriate error message.

Again, I'm declining to comment on whether or not 2**50 is a
reasonable limit for TIFF files, and even if that is the limit,
whether or not C doubles are adequate to handle them.

[toc] | [prev] | [next] | [standalone]

#66426

On 07/26/2015 05:28 PM, Robert Wessel wrote:
> On Sun, 26 Jul 2015 08:24:50 -0400, James Kuyper
> <jameskuyper@verizon.net> wrote:
> 
>> On 07/26/2015 01:23 AM, Robert Wessel wrote:
>>> On Sat, 25 Jul 2015 19:04:34 -0400, James Kuyper
>>> <jameskuyper@verizon.net> wrote:
>>>
>>>> On 07/25/2015 06:31 PM, Robert Wessel wrote:
>>>>> On Sat, 25 Jul 2015 17:48:59 -0400, James Kuyper
>>>>> <jameskuyper@verizon.net> wrote:
>>>>>
>>>>>> On 07/25/2015 04:46 PM, Robert Wessel wrote:
>>>>>>> On Sat, 25 Jul 2015 06:12:16 -0700 (PDT), Malcolm McLean
>>>>>>> <malcolm.mclean5@btinternet.com> wrote:
>>>> ...
>>>>>>>> long double being present or not is important. Actually legitimate
>>>>>>>> values in a tiff file for a tiff reader can't go over 2^50 as far as 
>>>>>>>> I know, so double is OK.
>>>>>>>
>>>>>>>
>>>>>>> Doubles are required to have 2**50 in their range, but are not
>>>>>>
>>>>>> I'm curious - where is this specified? I went searching for a
>>>>>> specification of the TIFF format, and found
>>>>>> <http://partners.adobe.com/public/developer/tiff/index.html>. I couldn't
>>>>>> find it in any of the documents listed there. Did I find the wrong
>>>>>> documents?
>>>>>
>>>>>
>>>>> I was referring to the C requirements for doubles, which require a
>>>>> range of 10**-37..10**37 and 10 decimal digit of precision for a
>>>>> double.  The first requirement accommodates 2**50, the second does
>>>>> not.  The TIFF spec is that floats and double are IEEE format (which
>>>>> has a 11 bit exponent and a 53 bit significand).
>>>>
>>>> In that case, the limit that Malcolm should be using should be derived
>>> >from the IEEE double specification, and is therefore
>>>> 0X1.fffffffffffffP1023, not 2^50.
>>>
>>>
>>> No, he has to be able to process all values that can *validly* occur
>>> in a TIFF file, or at least in a TIFF file his code purports to
>>> handle.  If an invalid value is found (and they all ned to be checked)
>>> he needs to return INVALID_TIFF_FILE.  I'm not commenting on what the
>>> actual range of valid values are.
>>
>> Well, unlike you, I was commenting on the actual range of valid values.
>> He seems to think the upper limit is 2^50, and I was asking about the
>> basis for that limit. His wording imply that it's a limit that applies
>> to all TIFF files, not just the particular TIFF files his reader is
>> intended to process. As far as I can tell, since the TIFF specification
>> says that DOUBLE is supposed to have IEEE format (and presumably,
>> specifically the IEEE format for double precision),
>> 0X1.fffffffffffffP1023 is the correct upper limit. As a result, on some
>> machines 'double' might not have enough range: DBL_MAX is allowed to be
>> as low as 1e37. Of course, LDBL_MAX is also allowed to be as low as 1e37
>> - but it will never be smaller than DBL_MAX, so using 'long double' is
>> marginally safer. He's free to assume that all machines this code will
>> ever be ported to will have IEEE doubles, but if so, he should state
>> that as a requirement rather than silently failing to address the issue.
> 
> 
> Just because a value is stored as an IEEE float doesn't mean that all
> values possible in that representation are meaningful or valid.
> Consider a medical application that stored a human's mass and height
> in floats - no one would expect it to produce valid results if it
> encountered a record specifying a 20,000kg or 40m human.  OTOH, it's
> perfectly reasonable to expect that the application would (safely)
> refuse to process such a record with an appropriate error message.

Again, that's application-specific. His wording implied a reason
specific to TIFF for a limit that all TIFF readers can rely upon TIFF
data not to exceed. Your argument suggests that it's a completely
arbitrary limit based upon "nobody needs to store numbers that big".

It's called "scientific notation" for a reason - I'd expect that any
TIFF file that contains data pushing the limits on the range of double
would contain scientific data. I can assure you that TIFF is at least
occasionally used to store floating point scientific data - I've seen it
used to store MODIS (the NASA project I work on) radiance data, for
instance.

The larger a number is, the fewer are the contexts in which it's likely
to come up, and even I have a great deal of trouble coming up with a
plausible example of a reason to store a number that big. The most
likely candidate I can think of for such numbers is numerical modeling
of black hole collisions - but the designers of such simulations would
presumable go to a great deal of trouble to design them so they don't
have to involve numbers that big - in order to ensure that intermediate
calculations neither overflow nor underflow nor suffer a large loss of
significance.

But if it's to be a generic limit that applies to all TIFF readers, I
don't see how he can justify assuming that no one will ever use more
than a tiny fraction of the permitted range of a supported data type.

[toc] | [prev] | [next] | [standalone]

#66429

On Sunday, July 26, 2015 at 11:57:21 PM UTC+1, James Kuyper wrote:
> On 07/26/2015 05:28 PM, Robert Wessel wrote:
> 
> It's called "scientific notation" for a reason - I'd expect that any
> TIFF file that contains data pushing the limits on the range of double
> would contain scientific data. I can assure you that TIFF is at least
> occasionally used to store floating point scientific data - I've seen it
> used to store MODIS (the NASA project I work on) radiance data, for
> instance.
> 
The reader is a general purpose reader that returns the image as 
a buffer of 8 bit RGBA values. So if an image actually represents
2D radiance data, you'll get back an RGB interpretation of that
data, which might be useful for casual visualisation, but is
unlikely to be useful for detailed calculations.

[toc] | [prev] | [next] | [standalone]

#66369

James Kuyper <jameskuyper@verizon.net> writes:
[...]
> I expected the number to be 0, in which case I would criticize his
> decision to worry about the issue, so in that sense the question is
> rhetorical - it's intended primarily to remind him how ridiculous it is
> to worry about that possibility.
> However, if he were to give me a non-zero answer, I'd find that
> interesting, and inquire further, so in that sense it's not a rhetorical
> question.
[...]

So the question is rhetorical or not, depending on what the answer is.

This is what happens when you spend too much time reading language
standards.

8-)}

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#66339

> On Saturday, July 25, 2015 at 3:22:13 AM UTC+1, James Kuyper wrote:
>> Malcolm McLean <malcolm...@btinternet.com> writes:
>> ...
>> > It's more likely that the tiff reader will be run on a machine
>> > that doesn't support long double ...
>>
>> How many implementations of C do you know of that fail to conform to the
>> C standard by not supporting long double?
>
> Don't know of any modern ones.
> But my parallel compiler I used for my PhD rejected slash slash comments.

There's a big difference there: // comments were added in C99; long
double dates all the way back to C89. There's many compilers that never
fully conformed to C99 (though just about any compiler that implemented
any C99 feature implemented // comments). However, compilers that don't
fully conform to at least C89 are quite rare.

> My attitude is that you seldom err by being too conservative. 

I've seen many errors committed by being too conservative - including
missed opportunities and unnecessarily complicated code.
My attitude is that I choose the version of the C standard that I'm
targeting, and then I make sure I don't violate any provision of that
standard, and avoid writing code that violates later standards, but I
don't worry about whether I violate any provisions of earlier standards.

Which standard version are you targeting? If it's K&R C, you can't even
use function prototypes; and that's too painful to even think about. If
it's C89 or later, you should feel free to use long double.

[toc] | [prev] | [next] | [standalone]

#66357

On Saturday, July 25, 2015 at 5:57:02 PM UTC+1, James Kuyper wrote:
> 
> Which standard version are you targeting? If it's K&R C, you can't even
> use function prototypes; and that's too painful to even think about. If
> it's C89 or later, you should feel free to use long double.
>
I want the code to be useful now and into the distant future.
Obviously things like Microsoft's deprecation of the standard
library are hard to foresee and plan for. But anyone with what
constitutes a C compiler and a "big" machine (we can write a tight
little reader that will handle most types of tiff file, but only
by sacrificing other features) ought to be able to drop the
source file into his code, and take in a tiff image.

[toc] | [prev] | [next] | [standalone]

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

Back to top | Article view | comp.lang.c

csiph-web