RPG IV decimal precision -- RPG400-L

I was a bit confused by the results of my last program change - losing decimals. Searching the archives resulted in a thread < http://archive.midrange.com/rpg400-l/200001/msg00484.html > on the very subject which made me look at the compiler listing which showed that it would lose a decimal place. (I guess I need to read warnings instead of just errors!)

Anyway, I just wanted to see if I understand the rules correctly.

I am adding 24 fields of 9,2 into a result field that is 15,2. I know that if each field were 9,999,999.99 the total would be 239,999,999.76 and can never have more than 2 decimal positions.

Given the formula from the manual:

N1+N2 T=min (max (L1-D1, L2-D2)+1, 31)
Dr=min (max (D1,D2), 31-t)
Lr=t+Dr

I see  T=min(max(9-2, 9-2)+1, 31) = 8
and   Dr=min(max(2,2),31-8) = 2
and   Lr=8+2 = 10

I'm guessing that adding N3...N24 to the formula causes each result to be calculated with the next N until we get to N24.

LrDr+N3 T=min(max(10-2,9-2)+1, 31) = 9
       Dr=min(max(2,2), 31-9) = 2
       Lr=9+2 = 11

So +N4=12,2
   +N5=13,2
...
   +N23=31,2
   +N24=31,1

And I fix the whole thing by using Eval(R) or EXPROPTS(*RESDECPOS)

Is that about right?

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.