Re: Strange Result

John,
The ** (exponentiation) operator outputs data type 8F (8 byte aka 64-bit
aka "double precision" floating point), so unfortunately it will lose
precision just as floating point numbers do. This is explained here in
the ILE RPG reference manual:
https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_74/rzasd/prectdf.htm

Sorry to be the bearer of bad news :-( I was quite unhappy when I
first learned of this years ago.

-SK


On 10/21/2020 9:58 AM, john erps wrote:

Hi,

I have the following program.

ctl-opt actgrp(*new);

dcl-s bignum1 packed(30);
dcl-s bignum2 packed(30:10);

*inLR = *on;

bignum1 = 777777777777777777777777777777;
bignum2 = bignum1 / 10 ** 10;

dsply (%char(bignum2));

Return;

Could anybody explain the following result?

DSPLY 77777777777777786880.0000000000

Apparently due to intermediate precision some digits are lost.
But i thought that the default behavior would be keeping as much digits

as

possible.

When i change the division to bignum1 / 10000000000 i get the correct
result.

We're on 7.3.

--
This is the RPG programming on IBM i (RPG400-L) mailing list
To post a message email: RPG400-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/rpg400-l.

Please contact support@xxxxxxxxxxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com