Re: Multi-threading -- MIDRANGE-L

Venu,

  I heartily endorse Patricks suggestion to use a data queue.  Two
things that us DQ types take for granted that you may not be aware of. 
1) As Patrick states, you may have as many jobs reading from the queue
as you want (even 0 since the entries will wait in the queue), but the
system will send an entry from the queue to exactly one of the readers. 
2) When there is no data in the queue the readers will use exactly *zero
processor resource so if they are needed periodically leave them
waiting.

  Many people don't realize the overhead required to start and stop jobs
on the AS/400. When the amount of real work done by the job is reduced,
it's relative cost per unit of work performed goes up.  If you have 100
records in your input file but each transaction processes quickly you
may find the DQ option saves so much wasted start/stop overhead and
therfore works so effectivly that only 2 or 3 readers can process the
entire 100 records in just a few minutes.

  One of my customers has an application that uses an individual batch
job for each inventory transaction processed. They do over 10,000 per
day and have LOG(4 00 *SECLVL) in the job description (thus creating 10K
joblogs daily).  These processes consume fully 25% of a 640 dual
processor system's resources every day! Try as I might (even told them
I'd do the programming for a percentage of the savings in resources)
they will not switch to a DQ approach, they just keep buying upgrades! 
Don't be like them!!!

  Larry Bolhuis
  Arbor Solutions, Inc
  lbolhui@ibm.net

Patrick Townsend wrote:
> 
> Instead of submitting the jobs to a job queue, why not write the data to
> a data queue. Then you can have a number of jobs wait on the data queue
> and process the records in the queue. You can have as many jobs as you
> like wait on the data queue and they will process simultaneously. The
> QSNDDTAQ and QRCVDTAQ APIs are easy to use in both CL and RPG.
> 
> Patrick
> 
> VENU YAMAJALA wrote:
> >
> > Hi All,
> >
> >  I want to know how to implement multi-threaded jobs on AS400. My 
>requirement
> > is as follows :
> >
> > I have a CL program(CL1). It reads records from a physical file PF1. For 
>each
> > record of PF1, I submit a batch job with different parameters. Now the 
>driver
> > CL (CL1) pgm itself runs in batch. So after submitting CL1 in batch, if I 
>look
> > at the JOBQ, it is forking into several batch jobs (number of records in 
>PF1).
> > But each of this batch jobs are running sequentially i.e, one after the 
>other.
> > I am loosing lot of time here. I want to have a mechanism wherein, the CL1 
>pgm
> > when it is forked to several batch jobs, a specified number of the batch 
>jobs
> > run simultaneously (say 10 at a time). This way I can reduce total 
>run-time. If
> > there are 100 jobs, I can reduce the total time by more than 75%. I know 
>that
> > it is possible but missing lot of points. Can anybody guide me in this? 
>Thanx
> > in advance for any help/suggestion.
> >
> > Regards,
> > Venu
> > +---
> > | This is the Midrange System Mailing List!
> > | To submit a new message, send your mail to MIDRANGE-L@midrange.com.
> > | To subscribe to this list send email to MIDRANGE-L-SUB@midrange.com.
> > | To unsubscribe from this list send email to MIDRANGE-L-UNSUB@midrange.com.
> > | Questions should be directed to the list owner/operator: 
>david@midrange.com
> > +---
> 
> --
> 
> Patrick Townsend                       mailto:townsend@patownsend.com
> Patrick Townsend & Associates, Inc.    http://www.patownsend.com
> +---
> | This is the Midrange System Mailing List!
> | To submit a new message, send your mail to MIDRANGE-L@midrange.com.
> | To subscribe to this list send email to MIDRANGE-L-SUB@midrange.com.
> | To unsubscribe from this list send email to MIDRANGE-L-UNSUB@midrange.com.
> | Questions should be directed to the list owner/operator: david@midrange.com
> +---
+---
| This is the Midrange System Mailing List!
| To submit a new message, send your mail to MIDRANGE-L@midrange.com.
| To subscribe to this list send email to MIDRANGE-L-SUB@midrange.com.
| To unsubscribe from this list send email to MIDRANGE-L-UNSUB@midrange.com.
| Questions should be directed to the list owner/operator: david@midrange.com
+---