Re: The target for a numeric operation is too small to hold the r esul t (C G D F)

Boer, Sacha wrote:

Hi Tim, I'm not quite sure who posted the original question or whether it's been answered yet, but please bear in mind that up to 5.1 you numeric limit is 30,15 (from the top of my head) and as of 5.2 I believe they stretched it to 60, something.
Anyway, your division of 17,6 by 17,6 would need a theoretical resultfield of 34,12. OS400 will create a "workfield" for you when you code the entire calculation into one statement. 34.12 exceeds the limits so here you'll get the error.

Sacha: You need to read Chapter 21 of the RPG IV Reference. Specifically, the section called "Precision of Intermediate Results". Actually, the precision differs a bit between releases, since the maximum size of a decimal value has changed from 30 (up to v5r1) to 31 (in v5r2) and again to 63 (in v5r3). So the calculation is slightly different in each release.

Consider "(g_g_txamount-linetax)/linetax" where each variable is a (17,6) decimal. First, the subtraction yields an intermediate result with decimal precision (17,6). Next, applying the precision rules for division gives an intermediate with the following precisions:

<=V5R1 -> (30,12)
  V5R2 -> (31,13)
  V5R3 -> (63,45)

In each case, you get 18 digits to the left of the decimal point. If the result of the division has more than 18 digits left of the decimal point, you get an overflow exception. More likely, if you try to assign the value to a target variable with fewer than 19 digits to the left of the decimal (such as a 17,6 decimal), you get an overflow exception.

As someone else suggested, Tim needs to fix up the logic error in his program. Clearly, a computation is yielding a value that's too big for the intended target variable.

Cheers! Hans