|
Hi all, This just bit me today and I thought I'd share it. I've always assumed that the extender (R) on the Eval statement meant Round the result. Don't know why I thought that, I guess I just jumped to a logical conclusion. My problem today was: Eval(R) wkdsc1 = (tmdisc * odmtot) 8.34 = .020 * 417.30 Real answer is 8.346, from a calculator wkdsc1 should be 8.35 Eval(H) wkdsc1 = (tmdisc * odmtot) this one gets 8.35, as expected above........ >From the RPG reference: ---------------------------------------------------- There are two sets of precision rules that you can use to control the sizes of intermediate values: The default rules give you intermediate results that are as large as possible in order to minimize the possibility of numeric overflow. Unfortunately, in certain cases, this may yield results with zero decimal places if the result is very large. The "Result Decimal Positions" precision rule works the same as the default rule except that if the statement involves an assignment to a numeric variable or a conversion to a specific decimal precision, the number of decimal positions of any intermediate result is never reduced below the desired result decimal places. ---------------------------------------------------- This is on a V4R4 machine, at first I thought it was a compiler error which hopefully is fixed on a subsequent release. To me, this makes the (R) extender less than worthless and I cannot imagine who on the Rochester development team thought this was a good idea, since the Eval with no extender (default rules) should give this same result except with more accurate intermediate work variables used. Or is it just me???? Maybe I'll feel better about this when the blood on my head dries (from beating my head against the wall trying to figure out what the problem was). AS/Resources, Inc. William A.(Tony) Corbett IBM Certified Specialist - AS/400 Developer http://www.asresources.com corbett@asresources.com 770-587-4812 (office) 678-935-5006 (mobile) 404-663-4737 (fax)
As an Amazon Associate we earn from qualifying purchases.
This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].
Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.