Re: trimming non-numerics

[Hans Boldt <boldt@xxxxxxxxxx>]

Bob Cozzi wrote:

> Hans,
>  I couldn't agree more, and as you know, I've often said, when asked about
> the performance differences between MOVEL and EVAL... "You've already spent
> more time thinking about it than you'll ever save in actual CPU time, so
> don't worry about it."
> But also, I'm in a situation where I have one client who is processing 10s
> of millions of records, and sometimes 100s of millions in batch. So every
> extra "phempto second" (or how ever you spell that) is important. So if
> _MEMCHR is 3 times faster than %SCAN in this context, and I have to send 46
> million records through the pipe, I'm going to use _MEMCHR.  But more
> importantly, I need to know that _MEMCHR is faster that %scan, and today, I
> have to roll my own to do that.
> Yes there are still systems out there that are not primarily Internet/Web
> based using Interpreted languages, but rather running huge database with
> compiled languages, because they need to speed.
> -Bob

But what's the point in worrying about whether %SCAN or __memchr is 
faster considering that the performance of that application is dominated 
by I/O, which affects the overall performance by a factor of several 
orders of magnitude? Before looking at the issue of %SCAN versus 
__memchr, you look at things like pool sizes, blocking factors, 
persistence, caching frequently accessed data, etc. etc. Right?

I certainly agree that there is a place for compiled languages. At least 
in the world of batch processing of multi-million record files. In the 
case of internet apps, though, the big bottleneck is in network 
latencies, which can easily overwhelm local disk latencies by a couple 
of orders of magnitude. In that realm, overall, the difference in %SCAN 
versus __memchr performance has to be way way down around the 0.0001% range.

(As an aside, I don't want to be seen as advocating that gawd-awful 
interpreted language Perl, but it's been argued that a well written Perl 
program can often perform just as well as a comparable program written 
in the compiled language C++. Why? Well, in that well written program 
coded in Perl, most of the execution time is spent in Perl's run-time 
library, much of which is written in C and compiled with the highest 
level of optimization.)

Cheers! Hans