Re: Calculation Truncation Problem

"Weber, Richard" wrote:
> 
> I'm having trouble with the following line of code:
> 
> Sum_x_PJS = SumOProjD * P1_PjSlsDol;
> 
> Where the input variables are:
> 
> P1_PJSLSDOL = 0000000000000.01
> SUMOPROJD = 0000000000000.0005
> ...
> D  Sum_x_PJS      S             20P 7

P1_PjSlsDol is (15,2) and SumOProjD is (17,4).  The natural precision of
their product is 32,6 (add the lengths and the decimals of the
operands).  The compiler has to adjust the length of the actual
temporary used to hold the product down to 31 (v5r2) or 30 (v5r1 and
before)*.  By default it adjusts the number of decimal places to
minimize the possibility of overflow, so it uses a temporary of 31,5. 
Your product is actually .000005, but it gets truncated to .00000 in the
temporary.  Using the (R) extender would fix you up, because then the
temporary would be guaranteed to have at least as many decimals (7) as
your result field.

* The maximum is 63 for v5r3, so no adjustment is necessary, and you
wouldn't have this problem with this particular expression.