Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.clipper.visual-objects > #11156
| From | Arne Ortlinghaus <Arne.Ortlinghaus@acs.it> |
|---|---|
| Newsgroups | comp.lang.clipper.visual-objects |
| Subject | Re: Strange Cannot Solve, Help Needed... |
| Date | 2017-11-21 10:31 +0100 |
| Organization | Aioe.org NNTP Server |
| Message-ID | <ov0rpd$1v8q$1@gioia.aioe.org> (permalink) |
| References | <ipqiea$lfa$1@speranza.aioe.org> <ipt3qa$v5l$1@speranza.aioe.org> <d15f1991-3482-4b17-8147-9dd65d26c8fb@18g2000prd.googlegroups.com> <58708111-4d37-4c24-a8a5-3def4fb4a1ec@googlegroups.com> |
Hi Robert,
for conversion processes it is better to write an own function/class
that does for example the alltrim of a value before trying to make a val.
But you should not try to convert the "global trade item number" into a
number although the name seem to suggest this. Instead it is a type of
code consisting of number digits. Always try to save it as strings
including also the leading zeros. The leading zeros are part of the code.
What is the internal problem with big numbers:
VO tries to create a longint 4 byte variable in case of missing
decimals. The biggest longint value is about 2000000000.
The value can only fit into a float variable (float/double/real8). But a
float variable has also only a limited precision of 12-14 digits, not
regarding possible rounding errors.
Probably with bigger numbers VO tries to convert it into a float, but
this is nothing you should try to rely on.
Arne Ortlinghaus
Robert Pope wrote:
> Posting to this old thread as it helped me find my issue. Hopefully this might help someone else one day.
>
> There seems to be a bug in Val() affecting numbers within a certain range. It appears to affect numbers larger than 32bit, or 2^32. After a certain point beyond that, it seems to start working again.
>
> I was working with Global Trade Item Numbers - the barcodes you see on almost every product. These have a maximum length of 14 digits, and can be zero padded. I have been trying to covert these from string to numeric values.
>
> The problem with Val() seems to occur on certain numbers with leading zeros and spaces at the end. What happens is that the bits above 32bit are lost, so you end up with a smaller number than you began with. The easy fix to this problem is to AllTrim() the values before handing it off to Val().
>
> The specific value I was having trouble with was "0021363150003 ".
>
> Code that reproduces this issue:
>
>
> FUNCTION Start(p)
>
> LOCAL sCmdLine := Psz2String(_GetCmdLine()) AS STRING
> LOCAL oCon AS Console
>
> LOCAL cTemp AS STRING
> LOCAL cTemp1 AS STRING
>
> oCon := Console{}
>
> // Bad
> cTemp := "0021363150003"
> cTemp1 := "0021363150003 "
>
> oCon:WriteLine(cTemp + ": => " + NTrim(FloatFormat(Val(cTemp),14,0)))
> oCon:WriteLine(cTemp1 + ": => " + NTrim(FloatFormat(Val(cTemp1),14,0))) // -- Incorrect Result - 4183280819 !! 04 F9 57 CC B3 became 00 F9 57 CC B3.
>
> // Good
> cTemp := "0011"
> cTemp1 := "0011 "
>
> oCon:WriteLine(cTemp + ": => " + NTrim(FloatFormat(Val(cTemp),14,0)))
> oCon:WriteLine(cTemp1 + ": => " + NTrim(FloatFormat(Val(cTemp1),14,0)))
>
> // Good
> cTemp := "1121363150003"
> cTemp1 := "1121363150003 "
>
> oCon:WriteLine(cTemp + ": => " + NTrim(FloatFormat(Val(cTemp),14,0)))
> oCon:WriteLine(cTemp1 + ": => " + NTrim(FloatFormat(Val(cTemp1),14,0)))
>
> // Good
> cTemp := "0121363150003"
> cTemp1 := "0121363150003 "
>
> oCon:WriteLine(cTemp + ": => " + NTrim(FloatFormat(Val(cTemp),14,0)))
> oCon:WriteLine(cTemp1 + ": => " + NTrim(FloatFormat(Val(cTemp1),14,0)))
>
> RETURN NIL
>
> I'm going to mention the term 64 bit here in case anyone searches these groups for this problem in the future thinking it is a 64 bit number support issue.
>
Back to comp.lang.clipper.visual-objects | Previous | Next — Previous in thread | Find similar | Unroll thread
Re: Strange Cannot Solve, Help Needed... Robert Pope <cooeee@gmail.com> - 2017-11-21 00:25 -0800 Re: Strange Cannot Solve, Help Needed... Arne Ortlinghaus <Arne.Ortlinghaus@acs.it> - 2017-11-21 10:31 +0100
csiph-web