re: processing a file -Reply -Reply

[PaulMmn@xxxxxxxxxxxxx (PaulMmn)]

At 2:19 PM 7/15/97, Scott Cornell wrote:
>Robert Kerr wrote:
>
>>...to gain speed in large file processing you need to
>> change the paradyne... Set up a feeder program to load
>> records into a data queue...now write your process
>> programs to pick records off the queue and do their
>> thing...
>
>Hey, that's pretty slick!  I'm curious though - there is a
>maximum size for data queue objects (I don't know what it is
>off the top of my head) and if you fill up your queue, the
>system pukes (I know 'cause we have a data queue driven
>app that has exactly that problem sometimes).  Doesn't
>your technique run the risk of bombing if your feeder
>program gets too far ahead of of the processing
>programs?  Of course, you can "tune" your performance
>to avoid this by running many instances of the process
>program compared to few (or 1 instance) of the feeder
>program, but I'd be interested to hear your experiences (if
>any) with this problem.


You can do the same thing with a Physical File... there's a 'waitfile'
parameter.  You set this parm (I -think- that's its name) and let an RPG
program (or 2 or 3) read from the file.  When the file has no more records,
the system does not cause an error, it just waits.  Forever.  Or until
another record arrives.

When you wan to kill the task, drop a special record in the file (we use an
*) to signal the program to shut itself down.

Now you're only limited by the max size for a PF.

--Paul E Musselman
PaulMmn@ix.netcom.com


* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* This is the Midrange System Mailing List!  To submit a new message,   *
* send your mail to "MIDRANGE-L@midrange.com".  To unsubscribe from     *
* this list send email to MAJORDOMO@midrange.com and specify            *
* 'unsubscribe MIDRANGE-L' in the body of your message.  Questions      *
* should be directed to the list owner / operator: david@midrange.com   *
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *