Re: File Encapsulation Quandary

On Wed, Apr 4, 2012 at 4:27 PM, Rory Hewitt <rory.hewitt@xxxxxxxxx> wrote:

FWIW, I've used exactly the sort of file encapsulation that Scott, Charles
and others seem to hate - essentially creating a service program with
multiple modules, each one containing Get and a Set procedures for each
file. Each procedure passes a data-structure of the entire record format.

Why do I do this?

1. Caching - in some cases, file data is loaded into an array within the
Get procedure and when the Get procedure is called, it first checks the
array, before reading from the file. This provided a
*significant*performance 'boost', compared to simply reading the
record every time. It
also avoids almost all record locks for commonly-accessed records (from
'control' files, for example)

The OS provides SETOBJACC....which would probably provide 75-90% of
the performance boast you see with your custom caching...depending on
the table. Still I could see where custom caching might be useful for
select files. However, I fail to see how caching avoids record
locks....if your just reading, why would you lock the record? If
you're updating, then having the data cached in multiple jobs seems
like a bad idea in the first place. Not to mention you're wasting
space caching the same data in multiple jobs even if it is read
only...

If I needed this, ideally, I'd have a background job I sent I/O
requests to. So the Get_ItemMaster() procedure everybody was calling
would simply put a message on a queue and process the response.

2. Maintainability - although it is rare that file record formats change,
it does happen. Rather as with IBM API's, when we call a Get procedure, we
pass both the key and also a 'format' identifier (actually, just a version
number). The Get procedure checks which version number is being passed and
returns the correct version of the record format data-structure.

I disagree that it's rare....or at least I'd argue that it rarer than
it should be since many i shops try to work around it by re-purposing
old fields, using generic ones that had been added to the file
originally or by using "extension" files.

Funny, the OS already provides a "format" identifier tied to a data
structure...otherwise know as the Format Level Identifier for a
(logical) file's Record Format.

By simply performing all RPG I/O through a logical file with an
explicit file format, you can add fields to the physical whenever you
need to. Given shared access paths, there's no additional overhead to
have multiple logicals with the same key but different fields.

3. Overrides - it's very easy to pass in the name of an overriding file
and/or library and have the procedure do all the file open and close stuff.

Ok, maybe, given that EXTFILE is somewhat new...

4. Common error-handling - standard error-handling procedures are used
throughout, using a combination of *PSSR, INFSR, MONITOR etc., and perform
consistent logging.

This one I'll give you. :) But it might be interesting to discuss
the caller's error handling...

5. Extensibility - we can convert our processing to use SQL bit-by-bit,
with no impact to our application programs.

Think I've covered my opinions on this idea.

Charles