× The internal search function is temporarily non-functional. The current search engine is no longer viable and we are researching alternatives.
As a stop gap measure, we are using Google's custom search engine service.
If you know of an easy to use, open source, search engine ... please contact support@midrange.com.



A floating point value has a "problem" with all values, in theory.  A floating 
point value
is stored in a number of bytes (in old days was 2, then 4, sometimes 8).  There 
are 3
parts to a floating point value, the fraction, the exponent and the sign.  In 
an 4 byte IEEE
floating value the fraction takes up 23 bits, the exponent takes up 8 bits and 
the sign
takes up one bit.

In 23 bits (1 bit shy of 3 bytes) you can represent a number up to 4,194,304 
(roughly,
I might be off a power of 2)  then with 8 bits we can represet up to 255.  So, 
in theory
the largest number we could show would be .4194304 and now move the decimal 
place
255 times to the right.  If our fraction was 23, our exponent was 2 I believe 
our number
would be 23 (might be 2.3).

Okay, say you want to represet the number .4194308, how are you goign to do 
that?
You can't, obviously, since you can't store that number in the binary 
representation.  It
would get rounded.  The problem also comes in that there are some other things 
that
come into play that complicate matters.  If you want a detailes explaination 
that is
accurate (as mine is not by any means) try this url:

http://www.sns.ias.edu/Main/computing/compilers_html/common-tools/numerical_comp_guide/ncg_math.doc.html

Regards,

Jim Langston

James David Rich wrote:

> On Tue, 8 Aug 2000, Jim Langston wrote:
>
> > I just want to add that in all fairness to IBM this is not really an RPG
> > or AS/400 issue, but an issue with how floating point values are stored
> > in computers.  You could expect similar types of results in other
> > langauges and platforms using floating point.
>
> Could someone please explain why binary representation has a problem with
> floating point?  I know I have heard this somewhere before but I can't
> remember the reason.
>
> James Rich
> james@dansfoods.com
>
> +---
> | This is the RPG/400 Mailing List!
> | To submit a new message, send your mail to RPG400-L@midrange.com.
> | To subscribe to this list send email to RPG400-L-SUB@midrange.com.
> | To unsubscribe from this list send email to RPG400-L-UNSUB@midrange.com.
> | Questions should be directed to the list owner/operator: david@midrange.com
> +---

+---
| This is the RPG/400 Mailing List!
| To submit a new message, send your mail to RPG400-L@midrange.com.
| To subscribe to this list send email to RPG400-L-SUB@midrange.com.
| To unsubscribe from this list send email to RPG400-L-UNSUB@midrange.com.
| Questions should be directed to the list owner/operator: david@midrange.com
+---

As an Amazon Associate we earn from qualifying purchases.

This thread ...

Replies:

Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.