|
A floating point value has a "problem" with all values, in theory. A floating point value is stored in a number of bytes (in old days was 2, then 4, sometimes 8). There are 3 parts to a floating point value, the fraction, the exponent and the sign. In an 4 byte IEEE floating value the fraction takes up 23 bits, the exponent takes up 8 bits and the sign takes up one bit. In 23 bits (1 bit shy of 3 bytes) you can represent a number up to 4,194,304 (roughly, I might be off a power of 2) then with 8 bits we can represet up to 255. So, in theory the largest number we could show would be .4194304 and now move the decimal place 255 times to the right. If our fraction was 23, our exponent was 2 I believe our number would be 23 (might be 2.3). Okay, say you want to represet the number .4194308, how are you goign to do that? You can't, obviously, since you can't store that number in the binary representation. It would get rounded. The problem also comes in that there are some other things that come into play that complicate matters. If you want a detailes explaination that is accurate (as mine is not by any means) try this url: http://www.sns.ias.edu/Main/computing/compilers_html/common-tools/numerical_comp_guide/ncg_math.doc.html Regards, Jim Langston James David Rich wrote: > On Tue, 8 Aug 2000, Jim Langston wrote: > > > I just want to add that in all fairness to IBM this is not really an RPG > > or AS/400 issue, but an issue with how floating point values are stored > > in computers. You could expect similar types of results in other > > langauges and platforms using floating point. > > Could someone please explain why binary representation has a problem with > floating point? I know I have heard this somewhere before but I can't > remember the reason. > > James Rich > james@dansfoods.com > > +--- > | This is the RPG/400 Mailing List! > | To submit a new message, send your mail to RPG400-L@midrange.com. > | To subscribe to this list send email to RPG400-L-SUB@midrange.com. > | To unsubscribe from this list send email to RPG400-L-UNSUB@midrange.com. > | Questions should be directed to the list owner/operator: david@midrange.com > +--- +--- | This is the RPG/400 Mailing List! | To submit a new message, send your mail to RPG400-L@midrange.com. | To subscribe to this list send email to RPG400-L-SUB@midrange.com. | To unsubscribe from this list send email to RPG400-L-UNSUB@midrange.com. | Questions should be directed to the list owner/operator: david@midrange.com +---
As an Amazon Associate we earn from qualifying purchases.
This mailing list archive is Copyright 1997-2025 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].
Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.