Re: Arrays and Performance

[Rory Hewitt <rory.hewitt@xxxxxxxxx>]

Jonathan,

I've used the same technique many times. If it makes sense for you, you
could use a separate cache array - load the first 32767 entries into the
'main' array and then, as entries are accessed by the application, these
entries are copied into the cache array (which is the first thing checked).
In other words:

Pre-start: Load 32767 entries into main array. Clear cache array

1. Check if entry is in cache. If so, go to step 4.
2. Check if entry is in main array. If so, copy to cache array and go to
step 4.
3. Read file and copy data directly into cache array
4. Return cache array entry

In this scenario, you might have entries in the cache array that are not in
the main array, but that's not a problem since the cache array is checked
before the main array. You could add code to step 3 to determine/save which
entries should be loaded into the main array at pre-start in future.

If the cache array is small (e.g. 1000 entries) and ordered, LOOKUP's will
be *very* fast. Of course, this system only works if typically only a subset
of entries are commonly used, otherwise the cache will get full very
quickly. There are many methods to cache management (FIFO, LIFO, MFU) -
which one you use depends on your data.

My experience of user indexes was not as good as others. Mainly this was due
to problems with complex keys in the underlying files.

Rory

On Wed, Aug 4, 2010 at 3:35 AM, Jonathan Mason <
jonathan.mason@xxxxxxxxxxxxxxxx> wrote:

> Hi All
>
> We have a number of programs, some in RPG/400 and others in RPG IV, that
> load reference data from various files into arrays in an attempt to
> reduce the performance impact of multiple chains to database files.
>
> The trouble with this approach is that the arrays are limited in size, a
> maximum 9,999 elements in RPG/400 and 32,767 elements in RPG IV.
>
> One thought was to use the arrays for the first however many records
> would fit and then chain to the database file for the remaining records,
> but there's concern about the performance impact of that approach.
>
> An alternative is to make use of user indexes, which could provide a lot
> more space for storing entries, but we're not sure on the performance
> impact of that approach.
>
> Does anybody have any knowledge of the performance benefits and pitfalls
> of any of these approaches?  What do others do when faced with the same
> problem?
>
> Thanks
>
>
> Jonathan Mason
>
>
> _______________________________________________________
> This message was sent using NOCC v1.14 webmail software
> _______________________________________________________
>
>
>
>
> --
> This is the RPG programming on the IBM i / System i (RPG400-L) mailing list
> To post a message email: RPG400-L@xxxxxxxxxxxx
> To subscribe, unsubscribe, or change list options,
> visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
> or email: RPG400-L-request@xxxxxxxxxxxx
> Before posting, please take a moment to review the archives
> at http://archive.midrange.com/rpg400-l.