× The internal search function is temporarily non-functional. The current search engine is no longer viable and we are researching alternatives.
As a stop gap measure, we are using Google's custom search engine service.
If you know of an easy to use, open source, search engine ... please contact support@midrange.com.


  1.  Home
  2. RPG400-L
  3. August 2006

RE: SQL ROUND function on a FLOAT

I did a little research and I should actually know this.  ;-)

Just like you can't represent 1/3 in base 10 with 100% accuracy, you can't 
represent  3.42 in IEEE-754 format with 100% accuracy.

After reading some docs on the IEEE-754 float standard, some conversion 
routines will add a one to the significand to account for this issue.

Floats give you a lot of precision with little storage if you pay 
attention to significant digits(hence significand).  I just didn't realize 
that you have to cast it to a zoned or packed before you rounded it to 
those significant digits.  ;-)

Mark

As an Amazon Associate we earn from qualifying purchases.

This thread ...
Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.