× The internal search function is temporarily non-functional. The current search engine is no longer viable and we are researching alternatives.
As a stop gap measure, we are using Google's custom search engine service.
If you know of an easy to use, open source, search engine ... please contact support@midrange.com.


  1.  Home
  2. RPG400-L
  3. October 2020

Strange Result

Hi,

I have the following program.

ctl-opt actgrp(*new);

dcl-s bignum1 packed(30);
dcl-s bignum2 packed(30:10);

*inLR = *on;

bignum1 = 777777777777777777777777777777;
bignum2 = bignum1 / 10 ** 10;

dsply (%char(bignum2));

Return;

Could anybody explain the following result?

DSPLY 77777777777777786880.0000000000

Apparently due to intermediate precision some digits are lost.
But i thought that the default behavior would be keeping as much digits as
possible.

When i change the division to bignum1 / 10000000000 i get the correct
result.

We're on 7.3.

As an Amazon Associate we earn from qualifying purchases.

This thread ...
Follow-Ups:
Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.