Re: Floating point, was: Have you read this

[Chuck Lewis <clewis@xxxxxxxxxx>]

Actually, Hans posted this reply back on 12/1/1999 under the subject "Float
numbers in RPG IV (was: RE: Procedue with *varsize parm)"

=========================================================

What you're seeing is normal behaviour for floating point numbers.  Using float
numbers, you have a vast range of possible values, but the numbers are never
completely precise.  At best, using 8-byte values, you never have better than
about 16 digits of precision.  And very often, like you see here, inaccuracy in
the least significant digit skews the value downwards, resulting in that string 
of
nines.

Floating point is ideally suited for modelling physical processes, such as
computing satellite orbits.  But since physicists deal with measured values, 
their
numbers are
never fully precise anyways.

Generally, the RPG programmer really shouldn't worry too much about floating
point.  However, there are a couple of places where you need to understand
floating point. For example, the ** operator returns a floating point value.
Furthermore, SQRT is done in long float format.

If you're dealing with numeric amounts that absolutely must be accurate to the
last digit, such as amounts of money, avoid floating point.  If your monetary
calculation
involves using the ** operator (useful for things like calculating interest), 
put
the ** computation in a separate expression, and store the result in a decimal
variable.

Cheers!  Hans

Hans Boldt, ILE RPG Development, IBM Toronto Lab, boldt@ca.ibm.com

=========================================================

So it sounds like HE might be able to shed some light on this ! Hans ? :-)

Chuck

"Shaw, David" wrote:

> -----Original Message-----
> From: leif@attglobal.net [mailto:leif@attglobal.net]
>
> > Don't know what to tell you other than I think it IS enabled on 530's and
> 53S's
> > (pages 43-40 in the book) :-)
>
> I was really appealing to some of the IBM lurkers on this list to set us
> straight
> (there are several) but the silence is deafening. Speaks volumes maybe?
>
> --------------
>
> I'm not sure, but I think all of our IBM folks are software people who aren't
> intimately involved with the hardware.  I would expect the FPU software switch
> to be set in the MULIC code, so that the marketing folk could change it on a
> model-by-model basis on whatever whim they might have about it.  I'll be
> surprised if we have any lurkers from whichever group does that stuff.
>
> Dave Shaw
> +---
> | This is the Midrange System Mailing List!
> | To submit a new message, send your mail to MIDRANGE-L@midrange.com.
> | To subscribe to this list send email to MIDRANGE-L-SUB@midrange.com.
> | To unsubscribe from this list send email to MIDRANGE-L-UNSUB@midrange.com.
> | Questions should be directed to the list owner/operator: david@midrange.com
> +---

+---
| This is the Midrange System Mailing List!
| To submit a new message, send your mail to MIDRANGE-L@midrange.com.
| To subscribe to this list send email to MIDRANGE-L-SUB@midrange.com.
| To unsubscribe from this list send email to MIDRANGE-L-UNSUB@midrange.com.
| Questions should be directed to the list owner/operator: david@midrange.com
+---