RE: Character to Numeric

>Date: Fri, 1 Dec 2000 07:57:32 -0600
>From: "Stone, Brad V (TC)" <bvstone@taylorcorp.com>
>Subject: RE: Character to Numeric
>
>So I just added a half adjust and so far it's worked flawlessly for 3
>years or more.

Brad, as you know, using floating point will invalidate any result with
over 16 significant digits.  It might also fail for some values with
fewer than 16 significant digits.  Are you sure it's worked flawlessly?
Would you know if the occasional result was out by 1 in the least
significant place?

Anyway, if you or anyone else who is using your routine want it to
work for all input within the packed(30,9) range, make these 2 changes.
(Hans and I were talking and he showed me this fix.)

1. Use the (r) extender to keep all decimal places in the result
2. Use %inth on the 10** expression.

C                   eval(hr)  Num = (WrkNum * Sign /
C                                       %inth(10**Decpos))

> as soon as IBM fixes the one that I copied this from.  Better get
> crackin.

I'm not sure what the connection is, but we'll see what we can do to
get that C2N snippet fixed.  I'm not sure we're talking about the
same IBM routine though.  The C2N I know has at least 2 bugs; the (H)
one and also a problem with negative numbers.  Can you please post the
URL to the one you're referring to.

Barbara


+---
| This is the RPG/400 Mailing List!
| To submit a new message, send your mail to RPG400-L@midrange.com.
| To subscribe to this list send email to RPG400-L-SUB@midrange.com.
| To unsubscribe from this list send email to RPG400-L-UNSUB@midrange.com.
| Questions should be directed to the list owner/operator: david@midrange.com
+---