Zero X One = 0.001 ?

Al Macintyre  ©¿©
MIS Manager Green Screen Programmer & Computer Janitor of BPCS 405 CD Rel-02 
running on AS/400 mixed mode V4R3 http://www.cen-elec.com Central Industries 
of Indiana--->Quality manufacturer of wire harnesses and electrical 
sub-assemblies

Sharing an SSA bug discovery = ROUNDING ERROR

We have multiple cases of plus or minus 0.001 0.002 0.00001 0.00002 that are 
inexplicable & we just had another hit, where we had sufficient audit trails 
to catch the before & after.  By mistake, billing price was zero & this fact 
was on a listing that we do before invoicing, showing details on all that was 
shipped today, but our Invoicing showed a bad price of 0.001 ... where did 
the 0.001 come from?

Well inside Billing program BIL522 it multiplies 
shipping net price BNET (zero in this case)
in a field 13.5 (00000000.00000) 
times
unit of measure sales conversion WKUMCN (one in this case)
in a field 11.5 (000001.00000)
to get unit price for invoice WNET (should have been zero)
ub a field 13.5 (00000000.00000) 
but to do this math correctly we need 24.10 work area
(00000000000000.0000000000)
except while RPG/400 goes to 30 digits, decimals only go to 9 places.

This is a rounding error, compounded by numbers with absudly high # digits.

When my time permits I plan to look in some costing update programs to see if 
this scenario explains where they are getting these invalid fractions. 
+---
| This is the BPCS Users Mailing List!
| To submit a new message, send your mail to BPCS-L@midrange.com.
| To subscribe to this list send email to BPCS-L-SUB@midrange.com.
| To unsubscribe from this list send email to BPCS-L-UNSUB@midrange.com.
| Questions should be directed to the list owner: dasmussen@aol.com
+---