Re: %DEC

[Stuart Rowe <rowestu@xxxxxxxxx>]

David

Precision is how many digits TOTAL.  Scale is how many decimals.  So if you
think of numbers in terms of RPG like 31p9, that'd be precision 31, scale 9.

Stu


On Wed, Jan 27, 2010 at 08:52, David FOXWELL <David.FOXWELL@xxxxxxxxx>wrote:

> Hi,
>
> In this expression :
>
>  MyProc ( MyField * MyCoefficient );
>
> MyField is 10S 0, MyCoefficient is 2S 1 and the prototype for MyProc is 31S
> 9.
>
> MyProc will output all decimal places eg 543.748000000. I need to round
> down to two decimal places, eg 543.740000000.
>
> I tried this, which works :
>
>  MyProc ( %DEC ( MyField * MyCoefficient : 10 : 2 ));
>
> My problem is, I don't fully understand what is meant by the precision
> parameter. I just tried 10 when 5 made the program crash.
>
> --
> This is the RPG programming on the IBM i / System i (RPG400-L) mailing list
> To post a message email: RPG400-L@xxxxxxxxxxxx
> To subscribe, unsubscribe, or change list options,
> visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
> or email: RPG400-L-request@xxxxxxxxxxxx
> Before posting, please take a moment to review the archives
> at http://archive.midrange.com/rpg400-l.