Re: Convert character to decimal

On 18 Mar 2013 15:51, CRPence wrote:

On 18 Mar 2013 14:30, Jeff Young wrote:

I have a character representation of a numeric field whose value
is '0009671180'. This represents 96.71180

When I attempt to convert it back to a decimal value, I get the
following message:
Message ID . . . . . . : RNX0103 <<SNIP>> An arithmetic operation
resulted in a value which is too large to fit in the target.
<<SNIP>>

The code I am using is:
<snip>
D Work_Rate S 63 15
...
/Free
...
Monitor;
Work_Rate = (%Dec(Oh@AirRate : 10 : 5) / 100000);

The above request should be instead:

Work_Rate = ( %Dec(Oh@AirRate: 10 : 0 ) / 100000 ) ;

Or could request more explicitly for a 10P05-compatible result:

Work_Rate = %Dec( %Dec(Oh@AirRate: 10 : 0 )
/ 100000 ) : 10 : 5 ) ;

$Air_Rate = Work_Rate;
On-Error;
$Air_Rate = *Zero;
EndMon;
</snip>

Since I made my work field more than large enough to hold the
result, why am I receiving this error?

The decimal value 9671180 can not fit into a 10P05 temporary result,
which is what the %Dec('0009671180':10:5 ) is asking to happen, and
what the RNX0103 is diagnosing; i.e. the significant digits, decimal
value 96, would need to be truncated, and that is overflow.

The _decimal digits 96 of_ the _decimal value 9671180 would need to_ be truncated to fit into the 5 positions allowed for, left of the decimal point; i.e. a 10P05 allows 5 digits of significant values, and 5 digits to the right of the decimal point which can be truncated without error.

The request should instead be %Dec('0009671180':10:0) to ensure a
full 10-digit decimal integer number can /fit/ in the temporary
result, and then divide that value by 100000 which makes a value
compatible with storing in a 10P05 [or 63P15] variable. Thus an
unedited string of 10 digits should always fit into anything 10P05 or
larger.

Note that if the character representation is not always all-digits,
then a different approach may be more desirable; well, depending on
how the data gets into the alpha field may also make a different
approach more desirable.

A slight clarification inline above.