RE: SQL for performance (overnight batch jobs)

Reeve, I agree. Recommending SQL as one-size-fits-all solution for performance is wrong-headed, IMO. It depends--on the scope of the change, on whether logic is completely rewritten, as Bruce (I think) alluded to in his successful example, on other things.

The parallelism advantage only works if you have multiple processors with SMP installed. So it depends.

The IO paging advantages that IBM mentions will not occur unless access is very sequential (or memory is very large relative to the amount of data to be read in) - Expert Cache (*CALC for paging) determines IO request sizes, and very random access had better not do much more than the 4k or 16k pages. *FIXED will not do larger than 16k IO requests, AFAIK.

My boss, a former IBMer heavily involved in database, has said that you may gain 20% in one area, only to lose 40% in another. You've got to be careful with these blanket recommendations.

These recommendations might be just right for the original posting party. We don't know enough about the situation. But a good PEX profile trace could identify bottlenecks in RPG code and point at ways to improve things without throwing the whole thing over and starting fresh. Or maybe they are recommending a middle ground.

Regards

Vern

At 12:27 PM 1/9/2003 -0500, you wrote:

An important function of batch jobs (large and/or overnight) is
providing batch control totals and other tools accountants (the fact
that "tools" and "accountants" are contiguous in this sentence should
not be construed as a political statement) rely on to demonstrate system
integrity.  In addition, it's a useful form of work management (unless
you like printing 35,000 invoices interactively).

The revenue accounting environment in the transportation/logistics
environment is complex: there are pricing, discounting, revenue split,
revenue allocation, and fuel price fluctuations driving every invoice in
a high-volume (100/day-20,000/day) environment.

Real-time processing is the solution for just about everything but
accounting.  Cutoff dates (period end, etc.) are an integral part of the
"measure and compare" nature of accountancy.

I don't understand how the use of SQL could improve performance in an
otherwise well-designed (proper use of LF's, etc.) batch application.
SQL may offer some flexibility with complex joins (for reporting, of
course) but that's about all I can think of.

-reeve