Re: FNDSTRPDM memory enhancement -- MIDRANGE-L

Hi, Craig:

OS/400 uses a single large virtual address space or "single level" storage.
In this scheme, real main storage is truly just a "cache" of the most
recently
used "pages" of objects recently "touched" (used, read or written to).

So, based on the above, I suspect that what is really happening is,
the first time you run FNDSTRPDM, it "touches" these members,
and their contents are paged into real main storage.  If your system
is not too "busy" and you have lots of main storage, then there is
nothing to force those pages to get paged out ("stolen"), and so,
the next time you run FNDSTRPDM over the same source files
(set of members), you notice that it seems to run much faster.

Unless you have vast amounts of real main storage, I do not think
it would be a "good idea" to submit jobs to read every member,
thus attempting to page them all into real main storage. At some
point, you will exceed the total amount of real main storage and
this will start to force out other pages, which could be pages for
other "production" jobs and users, who may really need that data,
and so, you could have a major negative impact on the overall
performance and throughput of your system...

You could create a separate memory pool, and then run your
RPG program that opens and reads all the members, and run that
batch job in a subsystem that uses only that one memory pool; that
might speed things up... and at least, this way, you limit the total
amount of real main storage that you will use, to just that one
memory pool.  But, the whole time you have this memory pool
allocated to a particular subsystem, no one else can use it.

So, you are probably best off to just let OS/400 take care of
itself... OS/400 also has something called the "Expert Cache"
that you can enable... for a given storage pool... that might
help, too...

Unfortunately, as far as I can tell, SETOBJACC only works
one member at a time, so you would have to issue SETOBJACC
for each member, and there are probably too many members in a
single source file to all fit into one memory pool at a time anyway,
so that would sort of defeat the purpose...

Hope this helps...

Regards,

Mark S. Waterbury

----- Original Message -----
> From: <craigs@xxxxxxxxx>
> To: <midrange-l@xxxxxxxxxxxx>
> Sent: Friday, October 10, 2003 10:34 AM
> Subject: FNDSTRPDM memory enhancement
>
> I have been teetering on whether this should go on the RPG list or the
> MIDRANGE list.  Maybe it is better here.  So, I will make a copy from the
> RPG list.
>
> I have a command I created called FNSTR that searches for a string in any
> source file in any or all libraries in batch.  It has the same options as
> FNDSTRPDM so it is basically a beefed up version of FNDSTRPDM.  As many of
> you have probably noticed, doing a FNDSTRPDM (or 25 then F13) on members
> goes a little slow the first time.  It is slowest when interactive as it
is
> displaying the scan progress.  The second time around on the same command
> on the same set of members goes fast.  My guess it that these members are
> saved in memory since any user doing scans on that set of members goes
> fast.  If you open each member in an RPG program, do a single read, and
> close, I think that places the entire member in memory.  Doing a test on
> this (open, 1 read, close) of each member in a source file in a library
and
> then doing a "25" F13 in that source file showed it being very quick.
Just
> doing an open and close without a read did not speed up the process.
> Every morning a system scan is initiated in batch that basically does a
> DSPFD for all common source physical files to outfiles.  I was thinking of
> taking these resulting outfiles and submitting jobs to batch for each
> library processed for each source file.  Then, whenever someone does any
> type of find, it would be quick.  I think members would open faster too.
> We also have a utility called FNDSRC that uses these outfiles to scan for
> any library containing the input member and source file name.  This
utility
> wouldn't be effected but it comes in very handy.
> Should I do the open, 1 read, close of every source member in batch jobs
(1
> library per job)?  Any system issues that might come up in doing this?
> Am I correct in thinking of this as members stored in memory?  Is this the
> same thing as when a program is replaced while someone is using it and
they
> are running the version still stored in memory until they exit?
> Am I crazy?
>
> Thanks,
> Craig Strong
>
> _______________________________________________
> This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing
list
> To post a message email: MIDRANGE-L@xxxxxxxxxxxx
> To subscribe, unsubscribe, or change list options,
> visit: http://lists.midrange.com/mailman/listinfo/midrange-l
> or email: MIDRANGE-L-request@xxxxxxxxxxxx
> Before posting, please take a moment to review the archives
> at http://archive.midrange.com/midrange-l.
>