Re: CGIDEV2 performance versus Net.Data

The generic CGI does run in a named activation group and returns with LR off, so
its overhead is very, very low.

With the sockets server approach, the CGI is still a "traditional CGI program,"
handling all input from and output to the client.  However, it performs no
application logic, leaving that for the sockets server.

The CGI's job is to receive and parse browser input, send each input field and
its value through a socket to the sockets server, receive response information
from the sockets server, and using its externally described HTML, write send
HTML to the browser.

With the first message from the CGI to the the sockets server. the server hands
off to a worker job which takes over communications with the active CGI.  Things
then work synchronously (the connection stays open and the CGI and the sockets
worker job have a conversation) as follows:

- the CGI continues sending inputs until there are no more.  It signals "no
more" by sending a logically empty message.
- the sockets server worker job performs application processing and sends
response messages to the CGI.  The messages contain substitution variables'
names and values, section names, etc.  It also sends logically empty messages as
required to indicate no more variables for the current section, and this is the
last section.
- as the CGI receives all the data needed to write a section, it does so
(WrtSection)
- when the CGI receives the last section, it write it followed by the *fini
section, and returns.
- in the meantime, the sockets worker job closes the socket it was passed and
waits for another request from the server.

There is quite a bit of overlapping, which is very good for performance.

The sockets server can start as many worker jobs as required.  The user can
specify how many worker jobs to prestart and what the maximum number of worker
jobs should be.

The sockets server can be running on the same AS/400 as the HTTP server or it
can be on a different AS/400.

Each worker job stays active until the sockets server is shut down so all the
start-up overhead occurs only once and most of it can be done even before the
first call if the job is prestarted.

The sockets interface is very low level and very fast.

The format of the messages is part of the design, although with this approach we
are using only four:
1. To send an input variable from the CGI to the server.  Contains things like
variable name and value.
2. To send section names and substitution variable names and values from the
server to the CGI.  An "empty" message signals "write the section."
3. To send information about a selection box from the server to the CGI.
4. To send information about a selection box's option from the server to the
CGI.  An empty message says that's all.

An optional feature of the iCTC sockets server is a tool that allows you to use
externally described data structures to define the message layouts.  Simple APIs
then allow you to read and write  (get and put) messages using these formats
with the underlying APIs handling building and decomposing the messages,
converting from internal AS/400 data formats such as packed, binary, etc., to
strings, etc.  The externally described data structures are built once (CRTPF)
and are used by both the sockets client (the CGI) and the sockets server worker
jobs.  It is very slick.

As previously noted, you can read about the iCTC sockets server, including its
documentation, at
http://www-1.ibm.com/servers/eserver/iseries/service/ctc/details/a2_ssrv.htm.
The AS/400 sockets client code is licensed for free.  The sockets server and the
message formatting feature are both licensed for a fee.  If you are interested
in learning more, please contact one of the people listed at our web site:
http://www-1.ibm.com/servers/eserver/iseries/service/ctc/

To summarize some of the advantages of this approach:
1. There is only one CGI program and it is small, efficient, and runs in a named
activation group.  It has to have access to all the externally described HTML,
which it reads only once.
2. The application processing is totally separate from the CGI program and the
CGI protocol.  In addition to knowing their application, programmers have to
know only how to read and write messages to/from the socket, and that is easy to
do.  They don't need to know anything about the CGI protocol.  They do have to
know what input to expect and what output is required and in what order the
output is required (obviously, because standard out is streamed output).
3. Each helper job is started only once and can run for long periods of time (a
day, multiple days, etc.) with files open, internal structures/arrays, etc.,
built once and ready to be reused, etc.
4. The sockets server scales very well by having as many worker jobs as
required.  It can be configured to add worker jobs as required up to a user
specified maximum.
5. The sockets server can run on a separate system, thus taking significant
loads off the system running the HTTP server.
6. The sockets interface, being so close "to the metal" is significantly faster
than data queues, DDM, and remote database, JDBC, etc. interfaces.

I hope this, combined with the information at our web site, answers your
questions.

If not, I'll try to answer them here.

Mel Rothman
CGIDEV2 Author
IBM eServer iSeries Custom Technology Center (iCTC)
http://www-1.ibm.com/servers/eserver/iseries/service/ctc/
Rochester, Minnesota



"Goodbar, Loyd (AFS-Water Valley)" wrote:
>
> These are neat ideas, especially the socket server. But I don't understand
> how a socket server improves performance, I can't picture the
> architecture...
>
> Traditional CGI:
>
> 1. web browser request is sent to
> 2. http server, which calls
> 3. custom CGI program, which generates output back to
> 4. HTTP server, which sends the page to
> 5. web browser
>
> So I can see that each time the CGI program is called, there is a lot of
> activation overhead, unless you're in named activation groups and returning
> with LR off (if using RPG).
>
> Based on your comments, I picture the process as this:
>
> 1. web browser request is sent to
> 2. http server, which calls
> 3. "generic" CGI program, which sends requests to
> 3a. socket server, which evaluates the request and calls a
> 3b. custom CGI program to generate the response ??? back to
> 3c. the socket server, forwards the response to ???
> 3d. the generic CGI, back to ???
> 4. HTTP server, which sends the page to
> 5. web browser
>
> I guess I fail to see where the performance benefits occur. Is 3a a
> neverending socket server job? Is 3c a thread program in the 3a job, or is
> 3c a full program job call by 3a? For the typical pages we process (reading
> realtime from databases), somewhere file opens and initialization takes
> place. I can imagine a single request taking a little longer, due to more
> program calls.
>
> Our AS/400 is not connected to the internet, and probably won't be (at least
> directly). We do have a requirement to develop  inquiries for our suppliers
> to see vendor rating/supplier assessment information, but those requests
> will probably be funneled through a corporate portal. Since we're doing
> traditional CGI, with CGIDEV2 and e-RPG, it would be interesting to know how
> other solutions work and tie together.
>
> Are there any white papers or FAQs for understanding the architectural
> differences between traditional CGI and message/data queue/socket server
> based approaches to web requests?
>
> Loyd
>