RE: Design for data queue-driven subsystem

We have something similar to what you are looking for.  It is an on-line
transaction processing system.  We have a data area that hold the control
values for min number of server, Thresh hold, max number of server, max time
on queue.  When the server program reads an entry on queue, it checks the
time stamp of when placed on queue.  If the time is exceeded, it submits a
request to a monitoring program, waiting on another queue, to start another
server.  The monitoring program uses system API's to see how many
occurrences are currently active and start another process if the max is not
reached.  If the thresh hold is reached, it can notify us that we are
falling behind.  The monitoring program can be written to handle multiple
data queue base servers.

Truthfully, some is in production and some is in theory or my head for
design but no time to make real.  Currently we do not have the thresh hold
or notification or monitoring program.  We do have a program that monitors
one data queue and all requests go thru it and are passed on to the sever
monitored data queue.  Not the best approach but it works for now.

Chris Bipes

-----Original Message-----
From: Metz, Zak

> I would like to design a relatively generic model for a data
> queue-driven, dynamically scaling subsystem.
> 
> It would seem obvious that the main part of this is the job that
> monitors the queue depth or response time and starts or ends 
> additional
> jobs as needed. But, that's about all I have worked out.
> 
> Has anyone developed a generic design for such a thing. I'm 
> thinking it
> would be controlled by parms such as minimum/maximum number 
> of jobs, min
> delay before additional job started, min depth before additional job
> started, how often to check...but if someone has already gone through
> this exercise perhaps they would consider sharing what a 
> flexible set of
> controlling parameters consists of, and any other gotchas?
> 
> And is there a shorter term for this sort of thing besides "data
> queue-driven, dynamically scaling subsystem?"