Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.clipper.visual-objects > #11155 > unrolled thread

Re: Strange Cannot Solve, Help Needed...

Started byRobert Pope <cooeee@gmail.com>
First post2017-11-21 00:25 -0800
Last post2017-11-21 10:31 +0100
Articles 2 — 2 participants

Back to article view | Back to comp.lang.clipper.visual-objects

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.


Contents

  Re: Strange Cannot Solve, Help Needed... Robert Pope <cooeee@gmail.com> - 2017-11-21 00:25 -0800
    Re: Strange Cannot Solve, Help Needed... Arne Ortlinghaus <Arne.Ortlinghaus@acs.it> - 2017-11-21 10:31 +0100

#11155 — Re: Strange Cannot Solve, Help Needed...

FromRobert Pope <cooeee@gmail.com>
Date2017-11-21 00:25 -0800
SubjectRe: Strange Cannot Solve, Help Needed...
Message-ID<58708111-4d37-4c24-a8a5-3def4fb4a1ec@googlegroups.com>
Posting to this old thread as it helped me find my issue. Hopefully this might help someone else one day.

There seems to be a bug in Val() affecting numbers within a certain range. It appears to affect numbers larger than 32bit, or 2^32. After a certain point beyond that, it seems to start working again.

I was working with Global Trade Item Numbers - the barcodes you see on almost every product. These have a maximum length of 14 digits, and can be zero padded. I have been trying to covert these from string to numeric values.

The problem with Val() seems to occur on certain numbers with leading zeros and spaces at the end. What happens is that the bits above 32bit are lost, so you end up with a smaller number than you began with. The easy fix to this problem is to AllTrim() the values before handing it off to Val().

The specific value I was having trouble with was "0021363150003 ".

Code that reproduces this issue:


FUNCTION Start(p)

    LOCAL sCmdLine := Psz2String(_GetCmdLine()) AS STRING
    LOCAL oCon AS Console
    
    LOCAL cTemp     AS STRING
    LOCAL cTemp1    AS STRING

    oCon := Console{}

    // Bad  
    cTemp   := "0021363150003"
    cTemp1  := "0021363150003 "
    
    oCon:WriteLine(cTemp + ": => " + NTrim(FloatFormat(Val(cTemp),14,0)))
    oCon:WriteLine(cTemp1 + ": => " + NTrim(FloatFormat(Val(cTemp1),14,0)))     // -- Incorrect Result - 4183280819 !! 04 F9 57 CC B3 became 00 F9 57 CC B3.

    // Good
    cTemp   := "0011"
    cTemp1  := "0011 "
    
    oCon:WriteLine(cTemp + ": => " + NTrim(FloatFormat(Val(cTemp),14,0)))
    oCon:WriteLine(cTemp1 + ": => " + NTrim(FloatFormat(Val(cTemp1),14,0)))

    // Good 
    cTemp   := "1121363150003"
    cTemp1  := "1121363150003 "
    
    oCon:WriteLine(cTemp + ": => " + NTrim(FloatFormat(Val(cTemp),14,0)))
    oCon:WriteLine(cTemp1 + ": => " + NTrim(FloatFormat(Val(cTemp1),14,0)))

    // Good 
    cTemp   := "0121363150003"
    cTemp1  := "0121363150003 "
    
    oCon:WriteLine(cTemp + ": => " + NTrim(FloatFormat(Val(cTemp),14,0)))
    oCon:WriteLine(cTemp1 + ": => " + NTrim(FloatFormat(Val(cTemp1),14,0)))
    
    RETURN NIL

I'm going to mention the term 64 bit here in case anyone searches these groups for this problem in the future thinking it is a 64 bit number support issue.

[toc] | [next] | [standalone]


#11156

FromArne Ortlinghaus <Arne.Ortlinghaus@acs.it>
Date2017-11-21 10:31 +0100
Message-ID<ov0rpd$1v8q$1@gioia.aioe.org>
In reply to#11155
Hi Robert,

for conversion processes it is better to write an own function/class 
that does for example the alltrim of a value before trying to make a val.

But you should not try to convert the "global trade item number" into a 
number although the name seem to suggest this. Instead it is a type of 
code consisting of number digits. Always try to save it as strings 
including also the leading zeros. The leading zeros are part of the code.

What is the internal problem with big numbers:
VO tries to create a longint 4 byte variable in case of missing 
decimals. The biggest longint value is about 2000000000.

The value can only fit into a float variable (float/double/real8). But a 
float variable has also only a limited precision of 12-14 digits, not 
regarding possible rounding errors.

Probably with bigger numbers VO tries to convert it into a float, but 
this is nothing you should try to rely on.

Arne Ortlinghaus




Robert Pope wrote:
> Posting to this old thread as it helped me find my issue. Hopefully this might help someone else one day.
> 
> There seems to be a bug in Val() affecting numbers within a certain range. It appears to affect numbers larger than 32bit, or 2^32. After a certain point beyond that, it seems to start working again.
> 
> I was working with Global Trade Item Numbers - the barcodes you see on almost every product. These have a maximum length of 14 digits, and can be zero padded. I have been trying to covert these from string to numeric values.
> 
> The problem with Val() seems to occur on certain numbers with leading zeros and spaces at the end. What happens is that the bits above 32bit are lost, so you end up with a smaller number than you began with. The easy fix to this problem is to AllTrim() the values before handing it off to Val().
> 
> The specific value I was having trouble with was "0021363150003 ".
> 
> Code that reproduces this issue:
> 
> 
> FUNCTION Start(p)
> 
>      LOCAL sCmdLine := Psz2String(_GetCmdLine()) AS STRING
>      LOCAL oCon AS Console
>      
>      LOCAL cTemp     AS STRING
>      LOCAL cTemp1    AS STRING
> 
>      oCon := Console{}
> 
>      // Bad
>      cTemp   := "0021363150003"
>      cTemp1  := "0021363150003 "
>      
>      oCon:WriteLine(cTemp + ": => " + NTrim(FloatFormat(Val(cTemp),14,0)))
>      oCon:WriteLine(cTemp1 + ": => " + NTrim(FloatFormat(Val(cTemp1),14,0)))     // -- Incorrect Result - 4183280819 !! 04 F9 57 CC B3 became 00 F9 57 CC B3.
> 
>      // Good
>      cTemp   := "0011"
>      cTemp1  := "0011 "
>      
>      oCon:WriteLine(cTemp + ": => " + NTrim(FloatFormat(Val(cTemp),14,0)))
>      oCon:WriteLine(cTemp1 + ": => " + NTrim(FloatFormat(Val(cTemp1),14,0)))
> 
>      // Good
>      cTemp   := "1121363150003"
>      cTemp1  := "1121363150003 "
>      
>      oCon:WriteLine(cTemp + ": => " + NTrim(FloatFormat(Val(cTemp),14,0)))
>      oCon:WriteLine(cTemp1 + ": => " + NTrim(FloatFormat(Val(cTemp1),14,0)))
> 
>      // Good
>      cTemp   := "0121363150003"
>      cTemp1  := "0121363150003 "
>      
>      oCon:WriteLine(cTemp + ": => " + NTrim(FloatFormat(Val(cTemp),14,0)))
>      oCon:WriteLine(cTemp1 + ": => " + NTrim(FloatFormat(Val(cTemp1),14,0)))
>      
>      RETURN NIL
> 
> I'm going to mention the term 64 bit here in case anyone searches these groups for this problem in the future thinking it is a 64 bit number support issue.
> 

[toc] | [prev] | [standalone]


Back to top | Article view | comp.lang.clipper.visual-objects


csiph-web