× The internal search function is temporarily non-functional. The current search engine is no longer viable and we are researching alternatives.
As a stop gap measure, we are using Google's custom search engine service.
If you know of an easy to use, open source, search engine ... please contact support@midrange.com.



On Thu, Jul 14, 2016 at 12:49 PM, Kevin Adler <kadler@xxxxxxxxxx> wrote:
fieldPrecise will return the precision (total column width) and
fieldScale
(digits after the decimal) will return the scale.

Well, you got that part right. :)

I guess you meant precision = 15, scale = 0 for ITITEM, etc.

Yeah, sorry. The way CLI uses the terms precision and scale is totally
backwards to what I think they should be (and how others use the terms as
well). The worst part is I was consciously trying to get it right, since I
figured someone would "correct" me, but alas I mixed them up anyway...

I'm afraid I am not sure which "CLI" you mean... you did refer to a
CLI API, but I can't stop seeing "command line interface" which
doesn't seem to fit the context.

The terms "precision" and "scale" as used above are definitely the
norm for SQL. I understand the confusion, and I used to get confused
by them myself.

What made me finally grok the terms, used this way, is thinking of
scientific notation and IEEE floating point. When scientists say a
measurement is "precise to 6 digits", they mean it has 6 significant
digits. It doesn't matter whether they are talking on an astronomical
scale or a subatomic one, it's still significant digits. (And see
there, I already used the word "scale".)

So if precision is related to significant digits, then what's left for
scale to be related to is exponent. Astronomers typically deal with
big exponents, particle physicists with small ones (well, negative
ones). It's not that particle physicists need fewer digits (in other
words, it's not that they need less precision). They are just working
at a different scale. The exponent in scientific notation or floating
point essentially tells you where your (fixed number of) significant
digits are situated. And that's what scale does in a database numeric
field.

John Y.

As an Amazon Associate we earn from qualifying purchases.

This thread ...

Follow-Ups:
Replies:

Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.