Re: reading a big file -- MIDRANGE-L

Booth,

In a message dated 97-06-29 00:32:54 EDT, you write:

> I'm not used to big machines and big files.  The box has V3R2 and I think
>  may be an ?F60? (its bigger than their development box, which they keep
>  referring to as a 45).  

Try DSPSYSVAL QMODEL.

>  The file I am concerned with has 5 million records.  I want to go through
>  this file making selections based upon choices made for several fields. 
>  Normally I would use the RPG cycle and bang down through the file (no K),
>  making the selection tests with the most limiting choice first, and as
>  soon as a record fails, goto end of cycle.  Any record that tumbled
>  through that series of tests would be written to a workfile, with a key
>  field made up with the sort-choices as made on the selection screen. Over
>  90% of the time the final work file will hold less that 10,000 records.
>  
>  I've done this a lot on smaller files and smaller machines.  It offers a
>  real easy front-end for the user and is flexible.  
>  
>  But I have never done this on a 5 million record file.  Am I about to make
>  a fool of myself? -- 

Probably not a good idea, although it depends on your selection criteria.  It
is possible to have too many of the latter for ANY method to be efficient.
 The performance of SQL and/or OPNQRYF against 5 million records would be
atrocious on the RISC, nonexistent on the CISC.  IMHO, your best bet is to
build a logical file with ordinal fields that match your RPG logic -- most
important first, and so on.  Perform a SETLL on the logical, and go from
there.

The truth is, until IBM comes out with an AS/400 optimized for Data
Warehousing, there is no really good way to handle selection against a file
of that size.  A logical file is best, but you have the additional DASD
overhead to maintain it.  OPNQRYF and SQL will probably build a temporary
access path of their own, and will be none too speedy about it.  You can
increase performance along the lines of your "No K" RPG program by performing
an OVRDBF on your file and setting a high blocking factor (this can be done
in either a CL program that calls your RPG, or by setting the file to
user-controlled in the F-spec and issuing a call to QCMDEXC with the OVRDBF
prior to opening the file).

JMHO,

Dean Asmussen
Enterprise Systems Consulting, Inc.
Fuquay-Varina, NC  USA
E-Mail:  DAsmussen@AOL.COM

"Man is still the most extraordinary computer of all." -- John F. Kennedy
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* This is the Midrange System Mailing List!  To submit a new message,   *
* send your mail to "MIDRANGE-L@midrange.com".  To unsubscribe from     *
* this list send email to MAJORDOMO@midrange.com and specify            *
* 'unsubscribe MIDRANGE-L' in the body of your message.  Questions      *
* should be directed to the list owner / operator: david@midrange.com   *
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *