× The internal search function is temporarily non-functional. The current search engine is no longer viable and we are researching alternatives.
As a stop gap measure, we are using Google's custom search engine service.
If you know of an easy to use, open source, search engine ... please contact support@midrange.com.


  1.  Home
  2. RPG400-L
  3. February 2004

Re: Strange result with exponents...

> But even allowing for the inaccuracy of floating-point
> the fact the releases 440 to 510 give result of
> 1152921504606846976 and 520 gives
> 1152921504606846980 suggests a change in
> rounding during conversion to decimal.

> Has anyone tried the example on 520 in a language
> other than RPG IV to see if the behaviour is with
> RPG or the machine?

V5R2M0 -

C:
printf("%f\n", pow(16,15));
1152921504606847000.000000
printf("%f\n", (pow(16,14)*16));
1152921504606847000.000000

ReXX:
numeric digits 31
say 16**15
1152921504606846976
say ((16**14) * 16)
1152921504606846976
numeric digits 16
say ((16**14) * 16)
1.152921504606847E+18

  --buck

As an Amazon Associate we earn from qualifying purchases.

This thread ...
Follow-Ups:
Replies:
Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.