Re: File Encapsulation Quandary

Charles,

The caching to avoid record locks uses a cache in a user space. We actually
only use it for a single file where we had lots of record locking issues -
most of the caching is simply for read-only speed of access.

Yeah, we could simply only add the new fields to a new logical and use that
for the programs that need it, but you do end up with more objects (LF's in
this case) on your system. again, we haven't needed to use this for more
than a few files, but it's a useful technique. Although we haven't used it
in anger, there's also the possibility to return more complex data
(possibly from more than one file) depending on the version number
specified. For instance, you could call Get_ItemMaster like this:

rc = Get_ItemMaster( KeyDS : RcdDS : VerNbr )

where VerNbr could be any of the following:

1 - Basic (original) Item Master record
2 - Extended Item Master record (with newer fields added in 2006)
3 - Extended Item Master record (with newer fields added in 2006 and 2008)
4 - Extended Item Master record (all Item Master fields plus Item Master
Extension fields)

I wouldn't exactly call EXTFILE ' somewhat new' - it's been around since
V5R1!

I'm not saying that converting to SQL would necessarily be a good idea,
just that by encapsulating the I/O, it's easier to do with no impact to the
application programs.

Rory

On Wed, Apr 4, 2012 at 2:08 PM, Charles Wilt <charles.wilt@xxxxxxxxx> wrote:

On Wed, Apr 4, 2012 at 4:27 PM, Rory Hewitt <rory.hewitt@xxxxxxxxx> wrote:

FWIW, I've used exactly the sort of file encapsulation that Scott,

Charles

and others seem to hate - essentially creating a service program with
multiple modules, each one containing Get and a Set procedures for each
file. Each procedure passes a data-structure of the entire record format.

Why do I do this?

1. Caching - in some cases, file data is loaded into an array within the
Get procedure and when the Get procedure is called, it first checks the
array, before reading from the file. This provided a
*significant*performance 'boost', compared to simply reading the
record every time. It
also avoids almost all record locks for commonly-accessed records (from
'control' files, for example)

The OS provides SETOBJACC....which would probably provide 75-90% of
the performance boast you see with your custom caching...depending on
the table. Still I could see where custom caching might be useful for
select files. However, I fail to see how caching avoids record
locks....if your just reading, why would you lock the record? If
you're updating, then having the data cached in multiple jobs seems
like a bad idea in the first place. Not to mention you're wasting
space caching the same data in multiple jobs even if it is read
only...

If I needed this, ideally, I'd have a background job I sent I/O
requests to. So the Get_ItemMaster() procedure everybody was calling
would simply put a message on a queue and process the response.

2. Maintainability - although it is rare that file record formats change,
it does happen. Rather as with IBM API's, when we call a Get procedure,

we

pass both the key and also a 'format' identifier (actually, just a

version

number). The Get procedure checks which version number is being passed

and

returns the correct version of the record format data-structure.

I disagree that it's rare....or at least I'd argue that it rarer than
it should be since many i shops try to work around it by re-purposing
old fields, using generic ones that had been added to the file
originally or by using "extension" files.

Funny, the OS already provides a "format" identifier tied to a data
structure...otherwise know as the Format Level Identifier for a
(logical) file's Record Format.

By simply performing all RPG I/O through a logical file with an
explicit file format, you can add fields to the physical whenever you
need to. Given shared access paths, there's no additional overhead to
have multiple logicals with the same key but different fields.

3. Overrides - it's very easy to pass in the name of an overriding file
and/or library and have the procedure do all the file open and close

stuff.

Ok, maybe, given that EXTFILE is somewhat new...

4. Common error-handling - standard error-handling procedures are used
throughout, using a combination of *PSSR, INFSR, MONITOR etc., and

perform

consistent logging.

This one I'll give you. :) But it might be interesting to discuss
the caller's error handling...

5. Extensibility - we can convert our processing to use SQL bit-by-bit,
with no impact to our application programs.

Think I've covered my opinions on this idea.

Charles
--
This is the RPG programming on the IBM i / System i (RPG400-L) mailing list
To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.