Re: Unexpected division quotient

[CRPence <CRPbottle@xxxxxxxxx>]

  Searching the v5r4 InfoCenter on the following tokens, yields the 
quoted text that follows:
    exponentiation rpg

"VisualAge RPG Language Reference

Precision of Intermediate Results

Table 52 describes the default precision rules in more detail.

Table 52. Precision of Intermediate Results
Operation 	Result Precision

Note:
    The following operations produce a numeric result. Ln is the length 
of the operand in digits where n is either r for result or a numeric 
representing the operand. Dn is the number of digits to the right of the 
decimal point where n is either r for result or a numeric representing 
the operand. T is the temporary value.

    Note that if any operand has a floating point representation (for 
example, it is *the result of the exponentiation operator* ), the result 
also is a floating point value and the precision rules no longer apply. 
A floating point value has the precision available from double precision 
floating point representation.

 ...
"

Regards, Chuck

On 02 Jul 2012 15:45, Anderson, Kurt wrote:
> IBM i 7.1
>
> There is a situation where we have an integer sent to us carrying an
> amount value. Another field contains the number of implied decimals
> in that integer.
>
> Here is what I'm seeing:
>
> dividend = 418
> divisor = 10 to the power of 4 (implied decimal value) = 10000.
> quotient = .041799  (but I'm expecting .041800)
>
> C = a / (10 ** ds_Inp.RevDec);
> // C = .041799
>
>
> I have tried with and without the H-spec ExPrOpts( *ResDecPos ).
>
> When I break down the code into two lines, it does work. And 2 lines
> is fine, b/c this is still better than the 19 lines of code we used
> to have, but I was would like to understand why this is happening.
>
> Example of how to get .041800.
> B = (10 ** ds_Inp.RevDec);
> C = a / b;
> // C = .041800