×
The internal search function is temporarily non-functional. The current search engine is no longer viable and we are researching alternatives.
As a stop gap measure, we are using Google's custom search engine service.
If you know of an easy to use, open source, search engine ... please contact support@midrange.com.
Searching the v5r4 InfoCenter on the following tokens, yields the
quoted text that follows:
exponentiation rpg
"VisualAge RPG Language Reference
Precision of Intermediate Results
Table 52 describes the default precision rules in more detail.
Table 52. Precision of Intermediate Results
Operation Result Precision
Note:
The following operations produce a numeric result. Ln is the length
of the operand in digits where n is either r for result or a numeric
representing the operand. Dn is the number of digits to the right of the
decimal point where n is either r for result or a numeric representing
the operand. T is the temporary value.
Note that if any operand has a floating point representation (for
example, it is *the result of the exponentiation operator* ), the result
also is a floating point value and the precision rules no longer apply.
A floating point value has the precision available from double precision
floating point representation.
...
"
Regards, Chuck
On 02 Jul 2012 15:45, Anderson, Kurt wrote:
IBM i 7.1
There is a situation where we have an integer sent to us carrying an
amount value. Another field contains the number of implied decimals
in that integer.
Here is what I'm seeing:
dividend = 418
divisor = 10 to the power of 4 (implied decimal value) = 10000.
quotient = .041799 (but I'm expecting .041800)
C = a / (10 ** ds_Inp.RevDec);
// C = .041799
I have tried with and without the H-spec ExPrOpts( *ResDecPos ).
When I break down the code into two lines, it does work. And 2 lines
is fine, b/c this is still better than the 19 lines of code we used
to have, but I was would like to understand why this is happening.
Example of how to get .041800.
B = (10 ** ds_Inp.RevDec);
C = a / b;
// C = .041800
As an Amazon Associate we earn from qualifying purchases.
This mailing list archive is Copyright 1997-2025 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact
[javascript protected email address].
Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.