Re: Pre-start Spawning jobs

Hi Chris, I can tell you the outline of what you need to do if that helps. We have many server products that do this.

Yes you will spawn off the child processes before you get a connection. We use a socket pair and sendmsg and recvmsg to communicate between the parent and children. The socket pair are mapped to the child job's file descriptors. We map to 0 and 1 and set this env var - "QIBM_USE_DESCRIPTOR_STDIO=Y". We send connection information to the children via sendmsg using a msghdr structure with the connection socket. You will need to make sure you catch child-has-ended signals so you can reset your counters otherwise you could end up with no child processes if they abend! Make sure you use waitpid with WNOHANG after a child dies or you'll end up with zombie processes. We poll for acks from a child after we pass a connection to ensure we have active and ready children. The ack comes back on the other half of the sock pair that's set to non blocking. If we don't get an ack we verify the # of child jobs and possibly start another one to deal with volume. We do have a child number cap. If there are more than the initial number of child jobs we have an algorithm based on the last started child time that signals a child to end.

Tim


On 3/7/2017 11:34 PM, Chris Bipes wrote:

I have a socket server that is rather critical to be running 24 x 7. Currently it handles up to 200 connections with no problems. (Never really get over 10 at a time.)

But I want to break this up to the listener and child processes.

The way the communications work is:
Each device connects - sends their data and waits for a response. Once they receive the response they disconnect.
The life of a connection is from 1 to 5 seconds. (There is a 30 second Idle time out on the select that closes all the open idle client connections.)

I use two batch job currently, one that waits at a select() and accepts the connections - Reads the socket from the client and sends the data to a data queue for processing.

The second waits at a data queue and sends the queued data to the first program through a persistent connection.

I am thinking if I use a prestart job as the listener and spawn off 5 child processes, I can then hand off each client connection to the child process and change the child to read the socket - send to data queue - wait for keyed response - write response to socket - close. The child would then wait for another connection to be handed off. I want to be able to spawn additional jobs if 4 of the 5 are in use. (Start another two perhaps.) Change the idle routine to end any extra child processes

I have never created a pre-start job with child processes. Does anyone have an example they can share? In looking at the give and take descriptors it looks simple but how do you spawn without a connection made? This is where I get confused. Can you spawn a new process and later hand off a connection?



Chris Bipes
Director of Information Services
CrossCheck, Inc.