Re: Socket Application Question -- MIDRANGE-L

I have a socket application that receives messages from a data queue and
send the message to a remote host via sockets.  Currently the application
uses a single persistent socket connection.  However, the application
needs to be changed so that sends messages to four different hosts for
redundancy.

The easiest solution would be to submit four different jobs, each onetalking to a different host.

Sockets are an API that lets your program control the network layer of theoperating system. Since modern systems are able to speak many differentnetwork protocols, telling us that you're writing a "socket program"doesn't really say much. Is it Unix-domain? TCP? UDP? IPX/SPX?

I'll assume TCP, since TCP is the most common. (Though, I alaways hatemaking assumptions!)

There's a big problem with bridging TCP and data queues. They have atotally different paradigm! Data queues are asynchronus, when you writedata to them, all that gets verified is that the data is placed in thequeue. No verification is made that it ever reaches the destinationprogram -- in fact, in the data queue paradigm, there's no need for theother program to be continually listning. It's possible for a data queueprocessor to be run only once a week, with the inttent of handling allqueued entries for that week.

TCP, on the other hand, is designed for "right here, right now"communications. There has to be a program listening for connections andhandlign them immediately. Nothing is queued for later, if the programcan't receive the data, you get an error. It's more like APPC than dataqueues.

I just don't understand why you'd want to try to bridge them. If yourapplication calls for immediate, real-time communications, use socketsthroughout. If your application calls for asynchronous communications,use data queues throughout. What's the point of bridging them?

I am not sure the best way to switch between sockets. The easiest wayto change the application would be to just switch between sockets buteach time I send a message it would have to wait to receive a replybefore processing a new message. That doesn't seem to be veryefficient.


If you REALLY want to do this...   Here's what you do:

a) Put all of the sockets in non-blocking mode.

b) Create an array of data structures containing a read buffer, writebuffer, and socket descriptor for each socket.c) Loop through the array to build sets of descriptors for the select()API to use.d) For each socket with space in it's read buffer, use FD_SET to turn onthat descriptor in the read set.e) For each socket with data to send in the write buffer, use FD_SET toturn on that descriptor in the write set.

f) Call the select() API with a 0.5 second timeout.

h) Loop through all of the sockets, and read any that are marked readable,write data from teh write buffer to any that are marked writable. Updatethe read/write buffers accordingly.i) Check the data queue(s) with a 0 second timeout. IF any data isavailable to send to the sockets, put it in the appropriate write buffer.j) If there's a full record of data in a socket's read buffer, put it intothe appropriate data queue.

k) go back to the top of the loop (step c)

The 0.5 delay on the select() API will keep the job from consuming toomuch CPU time (unless there's a LOT of traffic on the sockets.)Unfortunately, if there's no traffic on the sockets, it'll also mean thatthe data queues are only checked on 0.5 second intervals, so that mightadd some delay. Of course, you can adjust that timeout to try to get itright.

Alternately, you might use the spawn() API to create a second job so thatone job handles reading from the socket and writing to the data queue, andthe other handles reading from the data queue and writing to the socket.That'll eliminate the 0.5 second delay.

If you're a Java or C programmer, you might consider using threads insteadof separate jobs, but since you only need to start them up once, and sinceyou'll only be submitting (at most) 8 jobs, it probably won't make muchdifference in performance.

It just seems silly to create all this complexity. I'd follow theK.I.S.S. principle and make it all TCP or all data queues..