Re: Strange Result -- RPG400-L

× The internal search function is temporarily non-functional. The current search engine is no longer viable and we are researching alternatives.
As a stop gap measure, we are using Google's custom search engine service.
If you know of an easy to use, open source, search engine ... please contact support@midrange.com.

Subject: Re: Strange Result
From: Jack Woehr <jwoehr@xxxxxxxxxxxxxxxxxxxxxxxx>
Date: Wed, 21 Oct 2020 09:48:44 -0600
List-archive: <https://archive.midrange.com/rpg400-l/>
List-post: <mailto:rpg400-l@lists.midrange.com>
List-subscribe: <https://lists.midrange.com/mailman/listinfo/rpg400-l>, <mailto:rpg400-l-request@lists.midrange.com?subject=subscribe>
List-unsubscribe: <https://lists.midrange.com/mailman/options/rpg400-l>, <mailto:rpg400-l-request@lists.midrange.com?subject=unsubscribe>

Don't feel bad ...
$ node
Welcome to Node.js v14.2.0.
Type ".help" for more information.

.01 * .01

0.0001

.012 * .011

0.00013199999999999998

On Wed, Oct 21, 2020 at 9:28 AM Scott Klement <rpg400-l@xxxxxxxxxxxxxxxx>
wrote:

John,
The ** (exponentiation) operator outputs data type 8F (8 byte aka 64-bit
aka "double precision" floating point), so unfortunately it will lose
precision just as floating point numbers do.

As an Amazon Associate we earn from qualifying purchases.

Follow-Ups:

Re: Strange Result , Jack Woehr
Re: Strange Result , john erps

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.