RE: Character to Numeric

Barbara warned:

>> Brad, your subprocedure uses floating point to calculate the result
>> (the result of 10 ** DecPos is a floating point value).  This 
>> means that
>> every once in a while you will return a result that isn't the same
>> as the input value.

Brad Stone replied:

>I haven't had any problems.  This is basically the IBM one.  
>The original IBM one produced wrong results 99% of the 
>time (ie 1.9999999 instead of 2).  So I just added a half 
>adjust and so far it's worked flawlessly for 3 years
>or more.

It fails for large input values - try '12,345,678.12'
This is not outside the realm of realistic numbers being imported from a
spreadsheet program.

Buck Calabro
Aptis; Albany, NY
"Nothing is so firmly believed as
 that which we least know" -- Michel Montaigne
Visit the Midrange archives at http://www.midrange.com
+---
| This is the RPG/400 Mailing List!
| To submit a new message, send your mail to RPG400-L@midrange.com.
| To subscribe to this list send email to RPG400-L-SUB@midrange.com.
| To unsubscribe from this list send email to RPG400-L-UNSUB@midrange.com.
| Questions should be directed to the list owner/operator: david@midrange.com
+---