RE: processing a file -- MIDRANGE-L


Explanations:

You've got two basic selection methods here.  Test 1, read only the   
sleeted records by keyed access.  Tests 2 ad 3, read the whole file,   
selecting records in RPG.  Test 1 benefits from processing fewer records.   
 Test 2 and 3 benefit from arrival sequence processing, which allows   
double-buffered input.  Test 1 suffers from keyed sequence, which   
probably differs from the arrival sequence, so buffering doesn't help.   
 Test 2 and 3 suffer from having to read all the data in the input file.

All tests suffer from outputting to a keyed file, which requires access   
path maintenance for each output record.  In addition, the output may not   
be buffered.

Suggestions:

1.  Remove the keys from the output PF.  Create an LF that is keyed.   
 Remove the LF member while the PF is being built.  Add the LF member   
after the PF is built.  This defers building of the output file access   
path until after the PF is built.

2.  Specify blocked input and output with OVRDBF.  Read the input file   
using keyed access (as in test 1) and OVRDBF ... NBRRCDS(*N) SEQONLY(*YES   
buffer-size-in-number-of-records).  Output to the non-keyed PF and OVRDBF   
... NBRRCDS(buffer-size-in-number-of-records) SEQONLY(*YES   
same-number-of-records-as-NBRRCDS).  Best buffer size can't be determined   
other than by trial and error.  A good starting point is 4K (number =   
4096/record-length-in-bytes).

 --  
Brian Johnson
Help/Systems, Inc.
brian@helpsystems.com


 ----------
From:  'mail@uucp <boothm@earth.goddard.edu>'
Sent:  Sun, Jul 13, 1997 17:36 PM
Subject:  re: processing a file

Still trying to learn:

ok, I ran the following job 3 ways:
info:  the machine is an F35
      FILE1 is 570,000 records, the PF is not keyed, a LF is.
      FILE2 is the output file, a keyed physical with one 48-character   
key
field
      About 5,800 records are selected for the output file, from the
beginning of FILE1.

test 1:  Ran the job with both files showing the K in the Fspec, and   
using
SETLL.   In the INZSR I set FILE1 to the first selection date, and as   
soon
as the range was exceeded I set on LR and ended the job.

Test 2: Took out the FILE1 "K" and of course then had to read the whole
file, sequentially.

Test 3: Took out the FILE2 "K" too.

Results:

             total time for job     WRKACTJOB %
                                     at 12:24 of the job:
   

Test 1         12:24 min.                80%

Test 2         14:57 min                 62%

Test 3         13:50 min                 28%


Frankly this surprised me.  I did not expect the three choices to be so
close together for time, nor so wide apart on %.

I have figured out that writing out the records selected is what takes   
the
time.  Anyone have any good ideas about speeding up the record writing
part?
...  

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* This is the Midrange System Mailing List!  To submit a new message,   *
* send your mail to "MIDRANGE-L@midrange.com".  To unsubscribe from     *
* this list send email to MAJORDOMO@midrange.com and specify            *
* 'unsubscribe MIDRANGE-L' in the body of your message.  Questions      *
* should be directed to the list owner / operator: david@midrange.com   *
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *