c400-l-bounces+seurer=us.ibm.com@xxxxxxxxxxxx wrote on 06/01/2011 11:17:23
AM:

Printf(“%f\n”,(double)(2482884 * 1024)); shows -1752494080.000000

Printf(“%f\n”,(2482884.0 * 1024.0)); shows 2542473216.000000

Printf(“%f\n”,((double)2482884 * (double)1024)); shows 2542473216.000000

It seems like the conversion to double does not work in the first
instance.
Am I doing something wrong? Shouldn’t the cast of the result make the
number correct?

2482884 * 1024 is 2,542,473,216 or in hex 0x978B1000. That is larger than
the largest positive value that can fit in a signed integer (0x7FFFFFFF) so
the value "wraps" and becomes negative.

-Bill

This thread ...

Follow-Ups:
Replies:

Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2020 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].