× The internal search function is temporarily non-functional. The current search engine is no longer viable and we are researching alternatives.
As a stop gap measure, we are using Google's custom search engine service.
If you know of an easy to use, open source, search engine ... please contact support@midrange.com.



On 12 December 2015 at 12:11, Vinay Gavankar <vinaygav@xxxxxxxxx> wrote:

Buck: They do use performance gathering tools (iDoctor etc), and I think it
showed that the process was doing open/close at the rate of about 1000 per
second (300K in a 5 minute period). Hence the panic.

The count of operations is much less important than the accumulated
elapsed time. For instance, if they told you that in the same second
they observed 1000 open/closes, they also observed 50000 MULT
operations, so therefore you must 'do something' about those MULTs.
When the elapsed time for the OPENs is .1 seconds and the elapsed time
of the MULTs is .0001 seconds, and the one, single page fault that is
occurring takes .4 seconds. This is an example, for illustration, but
I have seen altogether too many efforts to optimise for the wrong
issue.

I'm more than a little surprised that the analysts have focussed on
the number of OPENs, because they have 15 files, all of which need to
be updated every time. More interesting to me is the architecture
that has the process calling the same RPG program 15 times in order to
update those 15 files. Yes, that's 15 OPENs but that's not easily
avoided. But it's also 15 *INZSR, 15 internal sets of variables which
are initialised, 15 sets of file control blocks, etc, etc, etc.
Moving all 15 files into one RPG program wouldn't reduce the OPEN
count, but it would reduce all the overhead due to program startup.

Thinking strictly in terms of smallest change that may make a
measurable difference, that would be it. Instead of cycling through a
control file in order to issue OVRDBF and CALL RGPGM for each one, put
all the files in one program and update them all there. The tables
stay the same, the libraries stay the same, there's one different RPG
program and a smallish change to the 'driving' CL program. Run that
for a week and see how the elapsed time (not OPEN counts) changed.
Going back would be easy because it would be only 2 programs.

If that wasn't enough improvement (please tell me they have an actual
elapsed time target... N transactions per second) then the next 'least
intrusive' step is to keep the 15 files open all day. That's another
CLP and some more changes to the RPG that does the updates, in order
to not close the files between calls. There are ramifications though,
as the files will be allocated to this update process all day, so
there needs to be a way to signal the process to shut down gracefully
and release the locks for things like backup.

Personally, I find 'all day locking' to be more intrusive to the
system than Mark's idea of asynchronous updating, but that requires
some new plumbing (another dtaq, another dtaq monitor, etc). Weirdly,
it seems as if the analysts are more concerned with keeping the
programs the same/similar than they are about business processes. ie
N transactions per second vs N OPENs per second.

They are not very keen on on using SQL here, don't know why.

Glenn: To open all the files at once, wouldn't I have to 'define' all the
files (with actual names or dummy names) in the program?

Yes, of course but why would that be a problem for a 'quick fix?'
Typing is cheap compared to thinking, and thinking is cheap compared
to altering an in-production architecture.

Ken: Every update is done to every file defined in the control file. So
keeping a file open is not really an option.

I don't follow this. Put all 15 files in the 'updater' program, make
them USROPN and do a RETURN without a CLOSE or LR. The files will all
remain open. The only thing you need to add is a mechanism to CLOSE
them all for things like backups, IPL, etc.

Jon: Keeping 'all' the files open would imply defining all the files in the
program, right?

It depends on what you mean by 'the program'. I can do this, and all
the files will stay open until end of job:
CL1:
OPNDBF DIVISION1
OPNDBF DIVISION2
CALL RPGDIV1
CALL RPGOTHER
CALL RPGDIV2

RPGDIV1:
FDIVISION1 USROPN
WRITE...
RETURN

RPGDIV2:
FDIVISION2 USROPN
WRITE...
RETURN

RPGOTHER:
FCUSTOMER
READ CUSTREC
SETON LR

RPG typically closes the files on exit, but you can alter that
behaviour with USROPN and RETURN. Files which a program does not
reference are... not affected. If DIVISION1 is open when RPGDIV2 is
called, it will remain exactly as it was when RPGDIV2 ends.

So, the RPG program that is doing the update does not need to
reference all 15 files in order for the job to keep all 15 files open.
But, as Jon notes, something must keep them open. In the above
example, that program is the CLP that kicks off the process.

Mark: It is a very 'elegant' solution, and I am going to shoot for that. I
am not sure that they will accept it as a 'quick fix', but definitely as a
long term solution.

I recommend this idea as a quick fix. But. The solution is a better
architecture. You face what many of us face, although instead of
multiple members, you're dealing with multiple libraries. The true
issue is that the database does not know what division (in my example)
a particular row belongs to, because the division is implied by the
library name. The solution is to add a new column to the tables to
store the division. Everything else - anything else - is a 'hack on
it' quick fix.
--buck

As an Amazon Associate we earn from qualifying purchases.

This thread ...

Follow-Ups:
Replies:

Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.