Re: Batch Job Queues = multiple

>From Al Macintyre = BPCS 405 CD <- BPCS/36

> Subj:  Re:Batch jobs performance problems 
>  From:        lfry@tokheim.com (Larry Fry)

>  A follow-up question to the "Batch jobs performance problems"
>  
>  Is it documented anywhere that BPCS batch jobs should (or must) be run
>  single-thread, or is it "common knowledge/common sense with BPCS"?  

This was true for BPCS/36.
It is false for BPCS/400.

Before we went with BPCS 405 CD as our Y2K solution, we were looking at ERP 
providers that were in competition with SSA & we had a laundry list of 
grievances with BPCS that had inspired this interest in jumping ship.  JOBQ 
was a minor point on the list, but because it was there, perhaps it led to us 
getting a better explanation, from the BPCS marketing people who wanted to 
keep us as a customer, than is ordinarily available to other BPCS customers.

> We are a new BPCS install and did not have the best guidance 
> during the installation.  We do not this and as far as I am aware
>  the problems we have had that may be related to this type of environment 
>  have are all been corrected by various BMRs.
>  
>  On our old system, the operating system took care of file and record 
locking 
> by specifications within each batch job.  We could also make one or more 
files
>  unavailable to the on-line (interactive) users.  This would allow us to run
>  batch jobs in a controlled environment without impacting users more than 
was
>  necessary.  Not so with AS/400 & BPCS.

Depending on what modifications you want to stick your nose into, you could 
also do this on BPCS/400 - I think the benefits are not worth the trouble.  

BPCS/36 had an enormous volume of logic that said "If program-name-X is 
running right now, do not let program-name-Y run at the same time"  it made 
no difference that the data being processed was not the same data - different 
facility, different warehouse, different all kinds of criteria --- we got 
around it with modifications --- SFC520E released shop orders that were 
exclusive to facility "E" --- SFC520L released shop orders exclusive to 
facility "L".  By this means, we were able to run MRP500 MRP600 CAP500 CAP600 
for all facilities AT THE SAME TIME.  This made it possible for us to run a 
full regeneration every night.  Without it, they had to be saved for the 
weekends, or different facilities different nights.

With BPCS 405 CD, we the users & MIS can collectively create multiple Job 
Queues & have different people running jobs on different queues.  BPCS runs 
fine.  The problem is in AS/400 resources being appropriate to having many 
job queues running stuff at the same time, defeating system purposes of 
having job queues.

>  I have not been able to find anything in the limited documentation we have 
> that indicates this should (or must) be done.
>  
>  Any advice would be helpful.

Single thread applies to user-A needing to run jobs 1-2-3-4-5 in sequence.
User-B can be running a different string of jobs A-B-C-D

When you ask SSA Help Line about such issues, it is important to make 
distinctions between how you accomplish different jobs on different queues at 
same time & which version of BPCS  - Do you have SYS013 on top of your pop-up 
menu?  Does it have an option to give each sign-on session a different jobq?  
Are your virtual sessions named in such a way that 2 or more users will not 
have the same data structure names?  Are the users aware that they should not 
change jobq between dependent jobs - do they know what is meant by the 
terminology "dependent jobs?"  Do people have their work set for JOBQ-hold, 
then move it to a different queue after it is on the default queue, or is 
everything pretty much set to run using the defaults?

> Subj:  Batch jobs performance problems
>  From:        LDEBONDT@am.pnu.com
>  
>       We are live with V6.0.04 C/S since January 4th and during the last 
>       couple of weeks we are having a serious problem with batch jobs 
>       performance due to queuing in the BPCS job queue.

Suggest you check BPCS_L archives on performance issues --- the hardware box 
you are now using had better not be the same hardware box you had on a 
version of BPCS prior to C/S.  There was a big discussion on that thread that 
you ought to review, just in case.
       
>       Since all BPCS jobs should be processed in one, single threaded job 
>       queue (apart from the orderpost en the ceapost job queues), 

Where did you get the notion that all BPCS jobs should be in a single job 
queue?

>  it only takes one large job to block the entire queue.  As a consequence, 
>       people are waiting more than 30 minutes to several hours for a job 
>       that takes only 15 seconds to process, because one or several other 
>       long running jobs are blocking the job queue.

There is a related problem with not easy access to history --- person-A job 
waits 45 minutes for 3 seconds time in the JOBQ --- person-B looks in JOBQ to 
find out what is holding up their job & gets a glimpse of person-A job & 
blames the wrong person.

We seldom have jobs that take more than a few minutes to execute, except for 
those that are deliberately scheduled for restricted access evening time 
periods, such as order purges & cost roll-ups.  Our main problem is that many 
end users have been spoiled - they are accustomed to not having to pay 
attention to the status of what went to job queue - they can ignore job 
queue, with the assurance that anything that went there is done faster than a 
snap of their fingers, except a couple times a month, when some other 
person's job bombs.  

Now the job queue is hung & no body notices.  Various people do their work, 
with the false assumption that stuff they sent to the job queue has in fact 
reached a conclusion.  This leads to other problems & when trouble shooting 
the other problems, someone notices the job queue is hung.  In my opinion, 
this is always a risk.  For some people to say that the solution is to fix 
the program that is bombing right now, is to ignore the reality that 
sometimes stuff bombs & whatever it is today will be something else tomorrow.
  
>       We would like to run the jobs in different job queues, running as 
many 
>       jobs in parallel as possible.  However, this is not without risk and 
>       SSA don't know which jobs can be directed to another queue and still 
>       ensuring that dependency between jobs is preserved as well as data 
>       integrity is preserved.

For our box's resources, the limitation is not SSA but IBM & continuing 
education.  The more job queues running concurrently, the more sluggish could 
be performance for all the on-line interactive users, depending on our 
learning curve of allocating resources.  If person-A can be running SFC520 
release of job-tickets on-line at the same time as person-B is running BIL500 
invoicing on-line, then both person's jobs could be running off of different 
job queues at the same time.

This is a recurring discussion topic at Central.  I am of the opinion that 
people in the same office should have a joint jobq, so if stuff is hung, or 
waiting on other stuff in a queue, and there is the temptation to rearrange 
stuff, or hold stuff, people in accounting should know which of THEIR jobs 
are safe to hold to let other jobs move ahead.  Likewise folks in other 
departments - either they know this stuff or they don't, but it is too much 
to expect that Shipping know about Purchasing jobs or vica versa.

We have been using a handful of job queues without serious sluggishness, and 
are cautiously adding them on a department by department basis - Production 
Control, Shipping, etc. will have their own Jobq.  In theory, dependency is 
preserved, training demands are minimized, when a jobq parade is held up it 
is by someone in your department, who can be asked "Do you really need this 
NOW & not scheduled for the evening?"

>       The only jobs that can be directed to another queue are jobs having a 
>       '1' or '2' in the fourth position of the program name.  But these are 
>       not the kind of jobs that are most heavily used.

Do you have the capability to automatically direct to queue on basis of job 
naming?  We are doing it via SYS013 v.s. location of user work station 
address, and by placing some stuff on the job scheduler.
 
>       Typical jobs that block the job queue are BIL501B, CLD501B, CEA520B, 
>       CST940B, but there are others.
>       
>       Anyone experiencing the same problem ?  Anyone having a clue ?
>       
>       Luc De Bondt
>       Systems coordinator.



+---
| This is the BPCS Users Mailing List!
| To submit a new message, send your mail to BPCS-L@midrange.com.
| To subscribe to this list send email to BPCS-L-SUB@midrange.com.
| To unsubscribe from this list send email to BPCS-L-UNSUB@midrange.com.
| Questions should be directed to the list owner: dasmussen@aol.com
+---