× The internal search function is temporarily non-functional. The current search engine is no longer viable and we are researching alternatives.
As a stop gap measure, we are using Google's custom search engine service.
If you know of an easy to use, open source, search engine ... please contact support@midrange.com.



> Patrick Conner wrote:
> 
> >I'm getting the wrong answer on an eval 
> >statement. Why?
> >
> >D#ExPo         S              1P 0   Inz(0)
> >D#Base         S              4P 2   Inz(18.10)
> >D#Result       S              5P 2
> >
> >Eval      #Result = #Base + (1 / (10 ** #Expo))
> >
> >     Expected answer:    19.10
> >     Actual answer:      19.09
> >
> >We are on V4R3.
> >
> >If I break it down, it works.
> >Eval #CalcExpo = 10 ** (#Expo)
> >Eval #Result = #Base + (1/#CalcExpo)
> 
> Peter Connell gave a very good answer (intermediate results can result in
> loss of least significant digits).  I just wanted to add that the ILE RPG
> Reference has a section entitled "Precision Rules for Numeric Operations"
> that every RPG programmer should read before using EVAL.  
> 
> Lest someone start a calumny against IBM documentation, the section on
> EVAL specifically refers to the precision rules:
> V3R2:
> http://publib.boulder.ibm.com:80/cgi-bin/bookmgr/BOOKS/QBKAQE01/4.4.34
> V4R1:
> http://publib.boulder.ibm.com:80/cgi-bin/bookmgr/BOOKS/QBJAQE01/4.4.36
> V4R2:
> http://publib.boulder.ibm.com:80/cgi-bin/bookmgr/BOOKS/QB3AGZ00/4.4.36
> V4R3:
> http://publib.boulder.ibm.com:80/cgi-bin/bookmgr/BOOKS/QB3AGZ01/4.4.36
> V4R4:
> http://publib.boulder.ibm.com:80/cgi-bin/bookmgr/BOOKS/QB3AGZ03/4.4.36
> V4R5:
> http://publib.boulder.ibm.com:80/cgi-bin/bookmgr/BOOKS/QB3AGZ03/4.4.36
> 
> Buck Calabro
> Aptis; Albany, NY
> "We are what we repeatedly do.
>  Excellence, then, is not an act, but a habit." --Aristotle
> 
> 
Billing Concepts Corp., a NASDAQ Listed Company, Symbol: BILL
+---
| This is the RPG/400 Mailing List!
| To submit a new message, send your mail to RPG400-L@midrange.com.
| To subscribe to this list send email to RPG400-L-SUB@midrange.com.
| To unsubscribe from this list send email to RPG400-L-UNSUB@midrange.com.
| Questions should be directed to the list owner/operator: david@midrange.com
+---

As an Amazon Associate we earn from qualifying purchases.

This thread ...


Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.