Re: Unexpected division quotient

Kurt,

Just curious, How are C and A defined?

Regards,

Luis Rodriguez
IBM Certified Systems Expert — eServer i5 iSeries
--




On Mon, Jul 2, 2012 at 4:15 PM, Anderson, Kurt <KAnderson@xxxxxxxxxxxx>wrote:

IBM i 7.1

There is a situation where we have an integer sent to us carrying an
amount value. Another field contains the number of implied decimals in
that integer.

Here is what I'm seeing:

dividend = 418
divisor = 10 to the power of 4 (implied decimal value) = 10000.
quotient = .041799 (but I'm expecting .041800)

C = a / (10 ** ds_Inp.RevDec);
// C = .041799


I have tried with and without the H-spec ExPrOpts( *ResDecPos ).

When I break down the code into two lines, it does work. And 2 lines is
fine, b/c this is still better than the 19 lines of code we used to have,
but I was would like to understand why this is happening.

Example of how to get .041800.
B = (10 ** ds_Inp.RevDec);
C = a / b;
// C = .041800

Thanks,
Kurt Anderson
Sr. Programmer/Analyst
CustomCall Data Systems, a division of Enghouse Ltd.
--
This is the RPG programming on the IBM i / System i (RPG400-L) mailing list
To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.

--
This is the RPG programming on the IBM i / System i (RPG400-L) mailing list
To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.