× The internal search function is temporarily non-functional. The current search engine is no longer viable and we are researching alternatives.
As a stop gap measure, we are using Google's custom search engine service.
If you know of an easy to use, open source, search engine ... please contact support@midrange.com.



Patrick Conner wrote:

>I'm getting the wrong answer on an eval 
>statement. Why?
>
>D#ExPo         S              1P 0   Inz(0)
>D#Base         S              4P 2   Inz(18.10)
>D#Result       S              5P 2
>
>Eval      #Result = #Base + (1 / (10 ** #Expo))
>
>     Expected answer:    19.10
>     Actual answer:      19.09
>
>We are on V4R3.
>
>If I break it down, it works.
>Eval #CalcExpo = 10 ** (#Expo)
>Eval #Result = #Base + (1/#CalcExpo)

Peter Connell gave a very good answer (intermediate results can result in
loss of least significant digits).  I just wanted to add that the ILE RPG
Reference has a section entitled "Precision Rules for Numeric Operations"
that every RPG programmer should read before using EVAL.  

Lest someone start a calumny against IBM documentation, the section on EVAL
specifically refers to the precision rules:
V3R2: http://publib.boulder.ibm.com:80/cgi-bin/bookmgr/BOOKS/QBKAQE01/4.4.34
V4R1: http://publib.boulder.ibm.com:80/cgi-bin/bookmgr/BOOKS/QBJAQE01/4.4.36
V4R2: http://publib.boulder.ibm.com:80/cgi-bin/bookmgr/BOOKS/QB3AGZ00/4.4.36
V4R3: http://publib.boulder.ibm.com:80/cgi-bin/bookmgr/BOOKS/QB3AGZ01/4.4.36
V4R4: http://publib.boulder.ibm.com:80/cgi-bin/bookmgr/BOOKS/QB3AGZ03/4.4.36
V4R5: http://publib.boulder.ibm.com:80/cgi-bin/bookmgr/BOOKS/QB3AGZ03/4.4.36

Buck Calabro
Aptis; Albany, NY
"We are what we repeatedly do.
 Excellence, then, is not an act, but a habit." --Aristotle


Billing Concepts Corp., a NASDAQ Listed Company, Symbol: BILL
+---
| This is the RPG/400 Mailing List!
| To submit a new message, send your mail to RPG400-L@midrange.com.
| To subscribe to this list send email to RPG400-L-SUB@midrange.com.
| To unsubscribe from this list send email to RPG400-L-UNSUB@midrange.com.
| Questions should be directed to the list owner/operator: david@midrange.com
+---

As an Amazon Associate we earn from qualifying purchases.

This thread ...


Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.