|
Well, adding 0.00005 and then doing a normal round (half-adjust) is
clearly wrong:
If the true value to be rounded happens to be 0.0142 exactly, then
your algorithm produces 0.0143.
The suggestions to add whatever is JUST smaller than 0.00005 and then
round normally; or to add JUST less than 0.0001 and then truncate; are
the right ones.
But what is JUST smaller than 0.00005? It depends on the number of
decimal places in the intermediate result (in this case, the
reciprocal of the rate).
For example, say the rate is exactly 399.999 (I'm using 3 decimal
places because your example rate has 3 decimal places). Then the
reciprocal is slightly higher than 0.0025, and thus should be rounded
up to 0.0026. This is in fact what Excel does, when using the
algorithm you describe.
But if you're doing this in RPG, and you're storing the intermediate
result in (for example) a field with 7 decimal places (and I'm
choosing this because you quoted your example value as 0.0141326),
then you're going to get 0.0025000. If you add just under 0.00005 to
that, you get 0.0025499, and when you round that to 4 places, you'll
only wind up with 0.0025, which is wrong.
(Don't get confused: You don't "fix" that by adding exactly 0.00005
before rounding, as you currently plan to do. If the rate happens to
be exactly 400.000, then the true reciprocal should be 0.0025 and your
method will yield 0.0026.)
Incidentally, your 7-digit intermediate value uses truncation. The
reciprocal of 70.758 is actually closer to 0.0141327 (which you would
get with normal rounding on 7 decimal places).
John
On Wed, Mar 20, 2013 at 12:04 PM, Jeff Young <jyoung0950@xxxxxxxxx> wrote:
All,that
The Round Up requirement is based on a formula in an EXCEL worksheet that
was used to compute the value I am using.
In essence, what I am being given is a currency exchange rate that I need
to convert to its reciprocal value as a fixed 11.4 field.
This is then used to convert a value in foreign currency to the native
currency.
Ie; Rate = 70.758
Amount = 396.13
1. I need to convert the 396.13 to the native currency. This is done by
multiplying the 396.13 by 70.758. No problems there.
2. I need to have the rate of 70.758 convert to the 11.4 field and use
to extend the value from step 1.ROUNDUP
In the EXCEL worksheet, the formula to get the 11.4 field uses the
function resulting in .0142. The value without rounding is .0141.the
I have taken the suggestion from some of the responses to add .00005 to
field and then use Eval(H) to get the result.--
Thanks to everyone who responded.
This is the RPG programming on the IBM i (AS/400 and iSeries) (RPG400-L)
mailing list
To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.
As an Amazon Associate we earn from qualifying purchases.
This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].
Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.