On Tue, Mar 19, 2013 at 4:30 PM, Jeff Young <jyoung0950@xxxxxxxxx> wrote:
The requirement is to take a currency conversion rate and convert it to a
price value.
The rate in question is 70.758.
The calculation is 1 / Rate (70.758) which yields .0141326.
I need the result to be .0142, not .0141.
Who came up with the "always round up" requirement? And what method
to *they* use to do the rounding?
I'm not challenging whether the requirement makes sense. The point of
asking these things is: If you're working with numbers other than
whole numbers, most likely you're working with floats. And if you're
doing that, you're inherently introducing funky, unintuitive numerical
behaviors. Now, whoever came up with the requirement, and whoever
will be verifying the results, will no doubt have to do some kind of
calculation themselves. You want to do the calculation the same way
they do (and with the same data types), to get the same results.
What do I mean by funky, unintuitive numerical behaviors? Well,
because floats are finite and binary, they can't exactly represent
most decimal numbers. Doesn't matter how many bits you've got,
there's no such thing as an exact binary representation of 0.2, for
example.
What's 3 * 0.2? Well, if we have an EXACT 0.2, then of course it's
exactly 0.6. But if we're dealing with double-precision floats (say),
then 3 * 0.2 is actually slightly MORE than 0.6 (about
0.6000000000000001). Depending on how the "always round up to the
nearest ten-thousandth" rule is applied, this could become 0.6001
instead of 0.6000.
Which may or may not be fine, but it depends on the exact algorithm
the authority/verifier is using. And that's why I'd like to know what
they are using.
John
midrange.com
