Re: CGIDEV2 performance versus Net.Data

"Nathan M. Andelin" wrote:
>
> Mel,
>
> What a pleasant surprise to hear from you.  I sometimes wonder whether
> anyone reads my posts <smiling>.  I respect the work you and Giovanni have
> done on the Easy400 product and Web site.  It's refreshing to hear IBM
> promoting something other than Websphere for Web development.

Thanks, Nathan.  We provide our tools for customers who are not ready for WAS
and Java, for whatever reasons (skills, hardware, etc.).  Our goal is to provide
an alternative way for these customers to exploit their existing skills and keep
their web applications on their iSeries or AS/400 systems.  Some will eventually
adopt WAS and Java; others will stay with CGI.

>
> I apologize in advance for my lengthy response.
>
> From: "Mel Rothman" <mel@rothmanweb.com>
>
> > Nathan, it appears that your CGIDEV2 sample code
> > is returning with LR on.  Based on your notes, it also appears
> > not to be running in a named activation group.  If so, this puts
> > CGIDEV2 at a significant,  unnecessary, and unfair disadvantage.
>
> My Easy400 CGI program does turn on the LR indicator.  And it runs in a new
> activation group.  After consulting with several other CGI programmers, I
> concluded that this was more common.  But maybe we can learn something from
> your experience.

More common doesn't necessarily mean better.  The V4R5 Web Programmer's guide
says in section 1.4.2, CGI Considerations, "There are advantages to running CGI
programs in either a user-named or *CALLER activation group. The performance
overhead associated with activating a CGI every time that is requested can be
drastically reduced."  The operative word is "drastically!"  Any performance
measurements of CGIDEV2 programs should be done in named activation groups after
the first execution.  You may want to look into using named activation groups in
your product as well.

>
> The folks I talked to felt uncomfortable about coding and running CGI
> programs that never freed memory and other resources they used.
>

If the resources are used over and over again, not freeing them is a good deal.
So, for CGI programs that are heavily used, I would definitely use named
activation groups.  In fact, if there is a small group of CGIs that account for
most of a site's activities, I would either give each CGI its own named
activation group or run multiple programs in a single named activation group.
For the latter, I would have each CGI open the same set of HTML files, thus
ensuring that the subprocedures that process the HTML do it only once.  Programs
that are used only rarely could run in a new activation group.


> I recently questioned a Java expert's claim that CGI programs don't scale
> well.  I used the logic that the mechanics were in place to start more
> instances of CGI jobs as the number of concurrent users increased.  But I
> wondered if system resources might become exhausted over a period of time if
> a large enough number of programs ran and never freed the resources they
> used?
>
> My Java friend pointed out that Threads use less system resources than Jobs.
> And that many of the same resources are shared across multiple threads.
> What about Jobs that never free resources?  From your point of view, should
> most CGI programs run in a named activation group, and return without
> freeing resources?  Or should that technique be reserved for special
> circumstances?
>
> BTW, given the multithreaded design of Servlets, would you have any
> references that refute a Java programmer's claim that CGI doesn't scale
> well?


I agree that CGIs probably don't scale as well as servlets, but I don't agree
that they don't scale well.  The iSeries Performance Capabilities Reference
Version 5, Release 1,
http://publib.boulder.ibm.com/pubs/pdfs/as400/V4R5PDF/as4ppcp4.pdf, has a table
on page 99 that shows number of transaction per second per CPW on the Apache
server.  In the non-SSL case, servlets: .46; CGI named activation group .29; CGI
new activation group .06.; Net.Data 0.19.  Note that this is on a model 270-2252
which has 2MB of L2 cache which is critical for good WAS performance.  On less
powerful models without the L2 cache, the gap between servlets and named
activation group CGIs could be quite different.  On many less powerful models,
WAS isn't even an option.

>
> > It can be made even faster by turning off all debugging output
> > by running the SetNoDebug(*on) subprocedure.
>
> I added this call to my Easy400 CGI program and found that it significantly
> improved the performance.  I'll report the results after I get a chance to
> tweak the Net.Data and Relational-Web versions.  The generation of the HTML
> was where most of the CPU time was spent.

Actually, there are three levels of debugging: debugging on, debugging off, and
debugging disabled.  With debugging on (CGIDEBUG data area = '1'), large volumes
of data are written into the CGIDEBUG debugging file.  With debugging off
(CGIDEBUG data area = '0'), the only records written are those for which 'force'
is specified.  In both these cases, the CGIDEBUG data area is checked every time
WrtDebug executes.  With debugging disabled (SetNoDebug(*on)), WrtDebug returns
immediately without checking the data area.
>
> > Perhaps your observation occurred because your program
> > is spending a significant portion of its time in the dbReport
> > subprocedure (or program), thus masking CGIDEV2's
> > performance improvements on the second and subsequent
> > executions.
>
> No.  The same dbReport() procedure is called by each of the versions.  The
> file is opened with each call.  The time it takes to perform it's logic is
> not significant.  All three versions of the application spend most of their
> time in the loop that generates the HTML table.

Okay.  It's clear now that the poor performance is because you are not using a
named activation group.

Although it might not make much of a difference here, when using named
activation groups, it's a good idea to leave files open and avoid the cost of
repeated opens and closes.

>
> > I would guess that CGIDEV2 programs that are properly
> > written for performance would significantly outperform similar
> > Net.Data macros, especially when there is significant procedural
> > logic as opposed to simple set operations.
>
> Got any references to point to?

See reference to iSeries Performance Capabilities Reference, above.
>
> > Also, note, that the current CGIDEV2 version supports multiple
> > HTML files (up to 127) in the IFS.
>
> OK.  I'll make a note of that.
>
> Thanks again,
>
> Nathan M. Andelin
> www.relational-data.com
>
Mel Rothman
CGIDEV2 Author
IBM eServer iSeries Custom Technology Center (iCTC)
Rochester, Minnesota