|
Hi Greg
Did I do something wrong ?
I'm afraid so. As Adam said, you have to use "integer division" for it to work, so in your example of: Say I want to round 1461 up to the nearest thousand, which would be 2000. So N is 1461 and S = 1000 Temp = 1461 + 1000/2 Temp = 1461 + 500 Temp = 1961 Result = (1961/1000) * 1000 Result = 1961 ? You need to have Result calculated as: Result = Int(1961/1000) * 1000 Result = 1000 If N had been 1641 then you would have had: Temp = 1641 + 1000/2 Temp = 1641 + 500 Temp = 2141 Result = Int(2141/1000) * 1000 Result = 2000 The half adjustment works on the mid boundary of the scale so anything below gets rounded down while anything equal to or above gets rounded up. All the best Jonathan -----Original Message----- From: rpg400-l-bounces@xxxxxxxxxxxx [mailto:rpg400-l-bounces@xxxxxxxxxxxx] On Behalf Of Fleming, Greg (ED) Sent: 10 November 2006 19:47 To: RPG programming on the AS400 / iSeries Subject: RE: Rounding up to nearest 1000 |The rounding algorithm goes something like this: | |Let N be the number to round, S be the scale of the rounding (in this |example S = 1000) |1) temp = N + S/2 |2) result = (temp / S) * S | |I think this has to use 'integer division' to work. | |This is essentially what the H extender is about - hence the name - |"(H)alf add". | |HTH, |Adam Adam, I'm trying that out using my trusty calculator, and I'm not ending up with a round number. Say I want to round 1461 up to the nearest thousand, which would be 2000. So N is 1461 and S = 1000 Temp = 1461 + 1000/2 Temp = 1461 + 500 Temp = 1961 Result = (1961/1000) * 1000 Result = 1961 ? Did I do something wrong ? Greg
As an Amazon Associate we earn from qualifying purchases.
This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].
Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.