RE: Are data queues reliable?

I have used data queues as an integral part of my server designs since 1991. All of the submitted jobs (20 or more) in our payment server are using them to send transactions around to be processed using a centralized "routing table". Each program has a dedicated "incoming" queue, and uses the routing table to know where to send the transaction next based on its origin, status, and transaction type.

They are the SINGLE BEST WAY to have two programs talk to each other in real time. In fact, we also use them with Java (Java toolkit) and PHP (Alan Seiden's Zend PHP toolkit) to interface those programs to RPG. This eliminates the instantiation overhead of CALLing programs, and the complexities of binding them.

One huge benefit is that they provide functions that are scalable. For instance, imagine one incoming and one outgoing queue for an RPG program that does some processing. It gets a record from the "in" queue (CALL QRCVDTAQ), processes, then sends the response to the "out" queue (typically unique to the requesting program that created the work unit.)

We submit the RPG processing job and find that it is not able to keep up. So we simply submit a SECOND, identical, copy of the RPG job. They feed off the queue in round-robin fashion (by default), or the job that is done first will get the next work unit. We can submit as many copies of the RPG as we like to scale the processing.

We do "data queue maintenance" at every startup, where the queues are deleted and re-created, just to keep them fresh.

Highly recommended. And we haven't covered the "sender ID" or keying functions, which amplify the flexibility.

Ira Chandler
Curbstone Corporation

-----Original Message-----
From: MIDRANGE-L [mailto:midrange-l-bounces@xxxxxxxxxxxx] On Behalf Of Rich Loeber
Subject: RE: Are data queues reliable?

We use data queues extensively in many of our products and love them too.
The reliability issue is that a data queue can become damaged during an unscheduled downtime event. Before widespread use of UPS and raid, this was more common than today.
In our products, we always include a delete and rebuild program. That said, I can't remember now when the last time was that we needed to walk a customer through it.