Re: QRCVDTAQ -- MIDRANGE-L


On 10/07/2006, at 2:20 PM, Steve Richter wrote:

come on.  networking is different story. I have not worked on MQ
applications but I have written a number of dtaq applications. I wrote
a fax system that used a lot of data queues between the PC that had a
fax board and the as400 that had users that wanted to fax stuff. It
started as something that seemed doable, but it got very complicated.

The fundamental problem with data queues is your client code does not
get a definitive return code from the server.

Again, that has nothing to do with queues per se. The queue APIs informyou of successful send or receive and provide error messages orexceptions in the case of failure. Anything over that is an applicationissue. If your sender needs to know the data was successfully receivedthen your design should include that. For example, the receiver couldsend a status entry to another data queue that is periodically checkedby the sender. The sender could maintain a list of sent entries andcheck them off as a completion status is received. If you think aboutit you'll see that the confirmation doesn't have to synchronous so youstill get the asynchronous benefits of queues but don't lose track ofwhat's happening.

3. What you have described repeatedly and well is what I understand
to be the contract model of development - a "user" and a "service"
contract to behave certain ways, such as that the user will validate
data and deliver it to the service in the form the service expects to
see, then the service will process it and return certain data as
agreed upon. I think we do this implicitly, but there is a framework
in which this is more defined, IIRC. Even a development environment
called Eiffel, I think.


that seems fundamentally flawed to me. The server has to validate the
transaction before it is applied to the database. If the client does
validation also that is a duplication of work.

Your thinking is flawed again. The web is a perfect example of whereboth ends probably should do validation. Consider a browser client thatrelies on Javascript to perform field validation. Given that it isreally easy to disable Javascript in browsers, and that the server hasno real way of knowing whether the data it received came from it'sexpected client or anything capable of communicating over HTTP then theserver application has a responsibility to validate the data.

Web applications would typically validate at both ends becausevalidating at the client gives a quicker response to the user, andvalidating at the host ensures accurate data. At least with a webapplication you have a synchronous operation during the POST/RESPONSEcycle so it is easy to have the server return error messages to theclient.

Queues are by their nature asynchronous and thus impose a greaterresponsibility on the sender to provide valid data. However, even hereit is possible to build in asynchronous confirmation if the applicationrequires it.

4. RPC - remote program calls, for some reason, creep into my
thinking here. Or call it asynchronous processing or think about MQ.


Can RPC throw exceptions back to the remote caller? If it can it is
superior to MQ.

Certainly can in the Java implementation. In Unix implementations it'susually done by return codes.

5. As to robust production code that depends on this kind of thing,
it is well known that much of the iSeries code is using user indexes
and user queues (an even lower-level variant of a data queue). I

guess IBM has made a mistake depending on these "unreliable" things.<vbg>


Hah! i5/OS had to run on a 1 CPW B10. Queues and batch processing are
efficient. That is their advantage. The downside is you dare not mess
with the system once you have it working. IBM for example cannot get
the interactive subsystem to support multi threading.

This has nothing to do with queuing. It has everything to do withstorage assumptions. The device management code may use static buffersfor 5250 processing, it may save state between calls, or any one of adozen other things that make retrofitting threads difficult. Notimpossible only difficult. Whether IBM add multi-threading support tointeractive jobs is a question of how much money and time would it takeand who would make use of it. I can count on the fingers of one handthe OS/400 programmers capable of making good use of threads in aninteractive job. Add to that the block mode interface of interactiveterminals and there is no business case to support threading ininteractive jobs.

Of course, if IBM develop a native replacement for 5250 then it wouldmake sense to design that for threads.

The spawn and SBMJOB APIs are also very slow. Is that the nature ofthe beast or is
it that all the queueing and dequeing cannot be circumvented at risk
of breaking the entire mechanism?

It's the nature of the beast as you'd work out if you thought about it.SBMJOB and spawn start JOBS. Jobs are heavy-weight processes. It is thecreation and destruction of a job that makes these slower than startinga thread. Processes on other systems are usually much lighter in systemrequirements than OS/400 jobs but then they don't provide as muchfeedback either. Want to see the search path for a Unix process? Wantto see the call stack for a WinDOS process? Want to see open files?Want to see a running log of the job? Hah! You get what you pay for.

Threads are not a straightforward replacement for spawn--even on othersystems. Using spawn (or even fork and exec) to start a separateprocess is a trivial task. Using threads requires careful thought andeven more careful implementation.


Regards,
Simon Coulter.
--------------------------------------------------------------------
   FlyByNight Software         AS/400 Technical Specialists

   http://www.flybynight.com.au/
   Phone: +61 3 9419 0175   Mobile: +61 0411 091 400        /"\
   Fax:   +61 3 9419 0175                                   \ /
                                                             X
                 ASCII Ribbon campaign against HTML E-Mail  / \
--------------------------------------------------------------------