Re: [Re: RPGILE V4.3 Gotcha]

["Ken Sims at SWS Nevada" <ken.sws@xxxxxxxxx>]

Hi Jim -

>Date: Tue, 05 Oct 1999 14:12:30 -0700
>From: Jim Langston <jlangston@conexfreight.com>
>Subject: Re: [Re: RPGILE V4.3 Gotcha]
>
>Okay, here is something I would probably use it for.
...
>EVAL Percent = (AmountPayable / AmountSales) * 100
>
>In here, we are lucky, we only have 29 total digits.  But, what about the fact
>that Percent was based on 4, 2?  (12.35%)  Or what about next year when
>it overflows again on me, so I kick them up to 15, 2?

Since this was in response to a message I sent, I'll respond.

My first response is ... huh?  What 29 total digits?

You're dividing, not multiplying.  Since I snipped a lot of the message,
I'll remind everyone that AmountPayable and AmountSales are 13.2 in
size.  That means 11 digits to the left of the decimal point, 2 digits
to the right of the decimal point.  Which means for the divide part of
the equation, the maximum number of digits to the left of the decimal
point is 13.  Since it is division, there could be an infinite number
of digits to the right of the decimal point.  Therefore, based on the
manual, the intermediate result for that part will be a total length
of 30, with as many digits to the right of the decimal as possible
without having overflow.  Since we need 13 digits to the left of the
decimal point, that leaves 17 to the right, so the intermediate
result will be treated as 30.17.  No chance of overflow.

You then multiply by 100, which is 3.0.  Ideally you would want the
result field to be 33.17, which is not possible, so it is going to
be treated at 30.14.  Since you are putting it into a field that is
4.2, you are not losing any precision.  Since the intermediate was
never reduced to zero decimal places, you never lost any significant
digits.

So where is the problem???????????????????????????????????????????

Ken
Southern Wine & Spirits of Nevada, Inc.

+---
| This is the RPG/400 Mailing List!
| To submit a new message, send your mail to RPG400-L@midrange.com.
| To subscribe to this list send email to RPG400-L-SUB@midrange.com.
| To unsubscribe from this list send email to RPG400-L-UNSUB@midrange.com.
| Questions should be directed to the list owner/operator: david@midrange.com
+---