|
Date: 7/15/97 5:33 PM RE: re: processing a file -Reply -Reply >Robert Kerr wrote: >>...to gain speed in large file processing you need to >> change the paradyne... Set up a feeder program to load >> records into a data queue...now write your process >> programs to pick records off the queue and do their >> thing... >Hey, that's pretty slick! I'm curious though - there is a >maximum size for data queue objects (I don't know what it is >off the top of my head) and if you fill up your queue, the >system pukes (I know 'cause we have a data queue driven >app that has exactly that problem sometimes). Doesn't >your technique run the risk of bombing if your feeder >program gets too far ahead of of the processing >programs? Of course, you can "tune" your performance >to avoid this by running many instances of the process >program compared to few (or 1 instance) of the feeder >program, but I'd be interested to hear your experiences (if >any) with this problem. > >Scott Cornell, Mercy Information Systems Scott You could condition the call to QSNDDTAQ program with an error handling routine. This routine might call a CL program(or the API) to receive the program messages, You could tell if you just filled the data queue. In that case, you could just DLYJOB for a while to let the processing programs catch up. You still have the record in your driver program buffer, just resend it to the data queue after awhile. This or something like it would handle the filling up of the data queue. I happen to remember John Sears talking about a similar technique about 10 years ago at COMMON on the SYS/38. Without the data queues. A program with the file opened with just Input would read the records, This ensured that they were brought into memory, and then then another program with the file opened for Update would read the records also. The second program would never have a Data Base Page Fault, because the records were always in main memory!. The two programs communicated with each other to tell the first program where the second program was in the file, etc. etc. Roberts example is pretty slick though. It might have some problems if you needed a serial update of the file. One of the programs reading from the data queue might get held up and one of the other ones might update the file out of order. John Carr EdgeTech * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * This is the Midrange System Mailing List! To submit a new message, * * send your mail to "MIDRANGE-L@midrange.com". To unsubscribe from * * this list send email to MAJORDOMO@midrange.com and specify * * 'unsubscribe MIDRANGE-L' in the body of your message. Questions * * should be directed to the list owner / operator: david@midrange.com * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
As an Amazon Associate we earn from qualifying purchases.
This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].
Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.