Re: Updated C2N and CharToNum procedures

[Don <dr2@xxxxxxxxxxxx>]

UH, Hans,

I've done many years of eco and financials sims on many languages and to
have roundoff in the HUNDRETHS position is just unforgivable.  I can see
it way down  at .00000n for a non-dble-precision, but geezuz, for a simple
F7.2 number to have that kind of rounding error is something that needs to
be looked at...  Or, just  use dblpre for everything...

Don



On Tue, 4 Jul 2000 boldt@ca.ibm.com wrote:

> 
> 
> Leif wrote:
> >>  >   **   NumVal = atof('1234.56') // result is NumVal=1234.56
> >> Bob:  I don't mean to nitpick, but the statement
> >> "NumVal = atof('1234.56')" returns the value 1234.55, not 1234.56.
> >
> >-----
> >You got to be kidding. Explain the rationale for that one.
> 
> Not kidding.  Try it yourself.  The inaccuracies (and
> associated dangers) involved with floating point data
> type are well-known to those who use floating point.
> 
> In this case, atof('1234.56') returns a long float
> value of +1.234559999999999E+003.  When this value is
> converted to packed decimal, you get 1234.55.
> 
> If you want a more useful answer, use half-adjust when
> assigning the float value to a decimal variable.
> 
> Also be aware that long float only gives you about 16
> digits of real precision.  If you want to move a large
> float value to a large packed variable with more than
> 16 digits, the lower-order digits in the packed
> variable cannot be trusted.
> 
> The lesson is:  Be very careful when using float.  In
> particular, avoid float for monetary values.  Normally
> avoiding float is no big deal, but note that the **
> operator in RPG IV returns a float value.  Also, any
> time you introduce float data type in an expression,
> subsequent operations in that expression happen in
> float format.  Since ** is useful in certain monetary
> calculations, be careful how you structure your code.
> 
> (Floating point is commonly used for scientific
> calculations (especially in physics) where you're
> doing good if you have 3 or 4 significant digits.)
> 
> Cheers!  Hans
> 
> Hans Boldt, ILE RPG Development, IBM Toronto Lab, boldt@ca.ibm.com
> 
> 
> +---
> | This is the Midrange System Mailing List!
> | To submit a new message, send your mail to MIDRANGE-L@midrange.com.
> | To subscribe to this list send email to MIDRANGE-L-SUB@midrange.com.
> | To unsubscribe from this list send email to MIDRANGE-L-UNSUB@midrange.com.
> | Questions should be directed to the list owner/operator: david@midrange.com
> +---
> 

+---
| This is the Midrange System Mailing List!
| To submit a new message, send your mail to MIDRANGE-L@midrange.com.
| To subscribe to this list send email to MIDRANGE-L-SUB@midrange.com.
| To unsubscribe from this list send email to MIDRANGE-L-UNSUB@midrange.com.
| Questions should be directed to the list owner/operator: david@midrange.com
+---