Re: %lookup on a Data Structure Array

[Rory Hewitt <rory.hewitt@xxxxxxxxx>]

#4. Load the file into an array in a user space. Works alone or with the
cache features described in #2.

On Thu, Nov 10, 2011 at 10:07 AM, Mark S. Waterbury <
mark.s.waterbury@xxxxxxxxxxxxx> wrote:

> Hi, Kurt:
>
> Loading the entire file into one or more arrays will incur the overhead
> of loading the entire file for each job that needs to search the array,
> and this increases the virtual storage used by each and every job that
> needs to access this file.
>
> So, before you decide to load all records of this file into an array and
> search using a binary search, consider the following alternatives:
>
> #1. issue SETOBJACC  to load the entire *FILE into a dedicated memory
> pool; sized based on the size of the file; this will speed up access for
> all jobs using this file, without any coding changes to the programs
> using the *FILE.
>
> #2. create a "server" job that loads the array, and communicates with
> requester jobs via data queues.  Instead of loading every record into
> the array, start out with an empty array, or load just one record. When
> a client program sends a message to the request data queue, requesting a
> "look-up", the server job first looks in the local "cache" (the array),
> and if not found, it then chains out to the file, and adds that record
> to the "cache" array, so any subsequent look-ups for that record will be
> faster, before returning the results via a results data queue (one per
> client).  Typically each client sends the name of its results data queue
> as part of the request message. The server job will need to maintain a
> counter of how many records are in the array, for use on the %lookup BIF.
>
> #3. create a user index, and load the data into the user index, and use
> the MI builtin function _FNDINXEN to do the look-up. This should be
> faster than database I/O, but you will have to incur the overhead to
> initially load the user index.  Note that you can do this just once, and
> use an insert, update and delete trigger attached to the file to
> maintain the *USRIDX whenever the corresponding file is changed.  All
> users or jobs can then share this one *USRIDX object.
>
> _Performance trade-offs_
> On average, how many records or rows of this *FILE would be accessed by
> any given job or interactive user session, in a given day? What
> percentage of the total file?
>
> Hope that helps,
>
> Mark S. Waterbury