|
On 9/12/2012 5:53 PM, Dennis Lovelady wrote:
I didn't see anything in the original post that implied the kinds of
precision "issues" you are discussing will be a problem. ...
I think you're right about the float-precision limit of around 18
digits not being a problem, given that the values seemed to be quite
small.
But it's always important to use EVAL(H) when assigning float to
decimal, since even with small values it can happen that the binary
float representation will be a tiny bit less than the real value,
rather than a tiny bit more.
--
This is the RPG programming on the IBM i / System i (RPG400-L) mailing
list To post a message email: RPG400-L@xxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives at
http://archive.midrange.com/rpg400-l.
As an Amazon Associate we earn from qualifying purchases.
This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].
Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.