Re: Updated C2N and CharToNum procedures

[boldt@xxxxxxxxxx]

Leif wrote:
>>  >   **   NumVal = atof('1234.56') // result is NumVal=1234.56
>> Bob:  I don't mean to nitpick, but the statement
>> "NumVal = atof('1234.56')" returns the value 1234.55, not 1234.56.
>
>-----
>You got to be kidding. Explain the rationale for that one.

Not kidding.  Try it yourself.  The inaccuracies (and
associated dangers) involved with floating point data
type are well-known to those who use floating point.

In this case, atof('1234.56') returns a long float
value of +1.234559999999999E+003.  When this value is
converted to packed decimal, you get 1234.55.

If you want a more useful answer, use half-adjust when
assigning the float value to a decimal variable.

Also be aware that long float only gives you about 16
digits of real precision.  If you want to move a large
float value to a large packed variable with more than
16 digits, the lower-order digits in the packed
variable cannot be trusted.

The lesson is:  Be very careful when using float.  In
particular, avoid float for monetary values.  Normally
avoiding float is no big deal, but note that the **
operator in RPG IV returns a float value.  Also, any
time you introduce float data type in an expression,
subsequent operations in that expression happen in
float format.  Since ** is useful in certain monetary
calculations, be careful how you structure your code.

(Floating point is commonly used for scientific
calculations (especially in physics) where you're
doing good if you have 3 or 4 significant digits.)

Cheers!  Hans

Hans Boldt, ILE RPG Development, IBM Toronto Lab, boldt@ca.ibm.com


+---
| This is the Midrange System Mailing List!
| To submit a new message, send your mail to MIDRANGE-L@midrange.com.
| To subscribe to this list send email to MIDRANGE-L-SUB@midrange.com.
| To unsubscribe from this list send email to MIDRANGE-L-UNSUB@midrange.com.
| Questions should be directed to the list owner/operator: david@midrange.com
+---