|
Isn't there an hspec option to respect decimal positions that will beat that into submission? Rob Berendt -- Group Dekko Services, LLC Dept 01.073 PO Box 2000 Dock 108 6928N 400E Kendallville, IN 46755 http://www.dekko.com |-----------------------------+-------------------------------------------| | Ted J Barry | | | <tjbarry@xxxxxxxxxxxxx> | | | Sent by: | To| | rpg400-l-bounces@midrange.| rpg400-l@midr| | com | ange.com | | | cc| | 07/24/2004 10:32 AM | | | | Subject| | Please respond to | Re: The | | RPG programming on the | target for a | | AS400 / iSeries | numeric | | <rpg400-l@xxxxxxxxxxxx>| operation is | | | too small to | | | hold | | | the re| | | sul t (C G D | | | F) | | | | | | | | | | | | | | | | | | | |-----------------------------+-------------------------------------------| Hi All, I have a similar situation where I have two different database fields. The result field is 10, 4 and the other is 11,4. Obviously too small, but what if you don't want to go around changing all of the database fields? Is there a routine anyone knows of to handle getting the result you want without having to change the db? Thanks, Ted >>> boldt@xxxxxxxxxx 07/20/04 05:39AM >>> Boer, Sacha wrote: > Hi Tim, I'm not quite sure who posted the original question or whether it's been answered yet, but please bear in mind that up to 5.1 you numeric limit is 30,15 (from the top of my head) and as of 5.2 I believe they stretched it to 60, something. > Anyway, your division of 17,6 by 17,6 would need a theoretical resultfield of 34,12. > OS400 will create a "workfield" for you when you code the entire calculation into one statement. 34.12 exceeds the limits so here you'll get the error. > Sacha: You need to read Chapter 21 of the RPG IV Reference. Specifically, the section called "Precision of Intermediate Results". Actually, the precision differs a bit between releases, since the maximum size of a decimal value has changed from 30 (up to v5r1) to 31 (in v5r2) and again to 63 (in v5r3). So the calculation is slightly different in each release. Consider "(g_g_txamount-linetax)/linetax" where each variable is a (17,6) decimal. First, the subtraction yields an intermediate result with decimal precision (17,6). Next, applying the precision rules for division gives an intermediate with the following precisions: <=V5R1 -> (30,12) V5R2 -> (31,13) V5R3 -> (63,45) In each case, you get 18 digits to the left of the decimal point. If the result of the division has more than 18 digits left of the decimal point, you get an overflow exception. More likely, if you try to assign the value to a target variable with fewer than 19 digits to the left of the decimal (such as a 17,6 decimal), you get an overflow exception. As someone else suggested, Tim needs to fix up the logic error in his program. Clearly, a computation is yielding a value that's too big for the intended target variable. Cheers! Hans -- This is the RPG programming on the AS400 / iSeries (RPG400-L) mailing list To post a message email: RPG400-L@xxxxxxxxxxxx To subscribe, unsubscribe, or change list options, visit: http://lists.midrange.com/mailman/listinfo/rpg400-l or email: RPG400-L-request@xxxxxxxxxxxx Before posting, please take a moment to review the archives at http://archive.midrange.com/rpg400-l. -- This is the RPG programming on the AS400 / iSeries (RPG400-L) mailing list To post a message email: RPG400-L@xxxxxxxxxxxx To subscribe, unsubscribe, or change list options, visit: http://lists.midrange.com/mailman/listinfo/rpg400-l or email: RPG400-L-request@xxxxxxxxxxxx Before posting, please take a moment to review the archives at http://archive.midrange.com/rpg400-l.
As an Amazon Associate we earn from qualifying purchases.
This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].
Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.
midrange.com
