RE: Exponential

[Buck Calabro <buck.calabro@xxxxxxxxxxxxxxxxx>]

> Patrick Conner wrote:
> 
> >I'm getting the wrong answer on an eval 
> >statement. Why?
> >
> >D#ExPo         S              1P 0   Inz(0)
> >D#Base         S              4P 2   Inz(18.10)
> >D#Result       S              5P 2
> >
> >Eval      #Result = #Base + (1 / (10 ** #Expo))
> >
> >     Expected answer:    19.10
> >     Actual answer:      19.09
> >
> >We are on V4R3.
> >
> >If I break it down, it works.
> >Eval #CalcExpo = 10 ** (#Expo)
> >Eval #Result = #Base + (1/#CalcExpo)
> 
> Peter Connell gave a very good answer (intermediate results can result in
> loss of least significant digits).  I just wanted to add that the ILE RPG
> Reference has a section entitled "Precision Rules for Numeric Operations"
> that every RPG programmer should read before using EVAL.  
> 
> Lest someone start a calumny against IBM documentation, the section on
> EVAL specifically refers to the precision rules:
> V3R2:
> http://publib.boulder.ibm.com:80/cgi-bin/bookmgr/BOOKS/QBKAQE01/4.4.34
> V4R1:
> http://publib.boulder.ibm.com:80/cgi-bin/bookmgr/BOOKS/QBJAQE01/4.4.36
> V4R2:
> http://publib.boulder.ibm.com:80/cgi-bin/bookmgr/BOOKS/QB3AGZ00/4.4.36
> V4R3:
> http://publib.boulder.ibm.com:80/cgi-bin/bookmgr/BOOKS/QB3AGZ01/4.4.36
> V4R4:
> http://publib.boulder.ibm.com:80/cgi-bin/bookmgr/BOOKS/QB3AGZ03/4.4.36
> V4R5:
> http://publib.boulder.ibm.com:80/cgi-bin/bookmgr/BOOKS/QB3AGZ03/4.4.36
> 
> Buck Calabro
> Aptis; Albany, NY
> "We are what we repeatedly do.
>  Excellence, then, is not an act, but a habit." --Aristotle
> 
> 
Billing Concepts Corp., a NASDAQ Listed Company, Symbol: BILL
+---
| This is the RPG/400 Mailing List!
| To submit a new message, send your mail to RPG400-L@midrange.com.
| To subscribe to this list send email to RPG400-L-SUB@midrange.com.
| To unsubscribe from this list send email to RPG400-L-UNSUB@midrange.com.
| Questions should be directed to the list owner/operator: david@midrange.com
+---