Re: Decimal rounding of a float data type in Java.

I *think* the only way that you can completely eliminate floating point
math rounding errors is to not use floating point numbers.  Isn't this the
original rational for database NUMERIC and DECIMAL data types?  Using
something like java.math.BigDecimal is a pain (and a performance drain)
but it should eliminate these kinds of problems.

Of course, you should be able to accomplish the same thing with floats by
doing a little math to truncate the fractional values and counter the
founding error (needing only two digits of accuracy, you should be able to
avoid any gross rounding errors that could happen from, say repeated
multiplication or things like that).

Regards,

Richard D. Dettinger
AS/400 Java Data Access Team

"Biologists have a special word for stability -- dead"

                Larry Wall
                Open Source Developers Journal
                Issue 1, Jan  2000


"Sherlock Holmes" <ediam@visto.com>@midrange.com on 03/01/2001 04:59:24 AM

Please respond to JAVA400-L@midrange.com

Sent by:  owner-java400-l@midrange.com


To:   JAVA400-L@midrange.com
cc:   vijosh@systsoft.com
Subject:  Decimal rounding of a float data type in Java.



I need to display a float value on the screen with 2 decimal points and
later use that value to perform validations. The problem is that the float
value which calculated has 6plus decimal precision which is totally
unwarranted. for e.g.
   3.00 - 1.70 - 0.20 - 0.20 = 0.89999999... instead of 0.90
Since my validations are based on 0.90 there is a mismatch.

Is there any way in which I can calculate the float value without
unnecessary precision or at least display the right value of 0.90

Mangesh

___________________________________________________________________________
Visit http://www.visto.com/info, your free web-based communications center.
Visto.com. Life on the Dot.

+---
| This is the JAVA/400 Mailing List!
| To submit a new message, send your mail to JAVA400-L@midrange.com.
| To subscribe to this list send email to JAVA400-L-SUB@midrange.com.
| To unsubscribe from this list send email to JAVA400-L-UNSUB@midrange.com.
| Questions should be directed to the list owner: joe@zappie.net
+---



+---
| This is the JAVA/400 Mailing List!
| To submit a new message, send your mail to JAVA400-L@midrange.com.
| To subscribe to this list send email to JAVA400-L-SUB@midrange.com.
| To unsubscribe from this list send email to JAVA400-L-UNSUB@midrange.com.
| Questions should be directed to the list owner: joe@zappie.net
+---