× The internal search function is temporarily non-functional. The current search engine is no longer viable and we are researching alternatives.
As a stop gap measure, we are using Google's custom search engine service.
If you know of an easy to use, open source, search engine ... please contact support@midrange.com.



On 2012/7/6 1:07 AM, Roger Harman wrote:
I believe %abs() returns a floating point. You might try wrapping it to
cast to int.

Weight = 10 - %int(%abs(lenpk - lenrd)); // or %inth() depending on your
needs


%int will work, but not because %abs() returns a floating point. (If %abs did return a floating point, it wouldn't fail with an overflow error like Alan is seeing.)

Here's what's happening:

%abs() returns the same type as its operand except that if the operand is an integer, %abs() returns an unsigned integer.

When a numeric literal with no decimal places, like 10, is used with an unsigned integer, the numeric literal is considered to be an unsigned integer literal.

The result of a subtraction between unsigned integers is also an unsigned integer. The result of 10 - %abs() is -3, which gives the overflow error.

You wouldn't need %inth in this case, because if lenpk or lenrd are decimal or float values, the result of %abs would also be decimal or float, and the unsigned-overflow problem wouldn't occur.

The reason for unsigned subtraction (and other unsigned operations) returning unsigned values is that if they returned integer, it would cause expressions with very large unsigned values not to be able to handle values bigger than the maximum value for an integer.

The compiler _could_ have used packed as the result of any operation between integer or unsigned values, but then it would have lost the performance benefit of using integers.

Whether that performance benefit outweighs this kind of annoying and confusing behaviour is something I'm not sure of. But I don't think it would be a good idea to add an option to say that all arithmetic operations should have packed temporary results. An RPG programmer would still have to realize what the problem is before they knew that was a solution, but once the programmer knows what the problem is, there are already lots of ways to get around it.

As an Amazon Associate we earn from qualifying purchases.

This thread ...

Replies:

Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.