Re: Arrays and Performance

Jonathan

You can also use user spaces and pointers to insert data - then use qsort() to sort the data - it's all pointer-based as to where the data is, and you can get the address of a user space. Then use the bsearch() function - binary search - depends on a sorted data set. This is pretty vanilla coding in RPG IV, not possible in RPG III that I know, at least easily. Might be easier to maintain than user indexes - can't speak to that, since I've done this kind of thing and haven't done anything with user indexes.

The Redbook "Who Knew You Could Do That With RPG IV?" has examples of qsort() and bsearch(), IIRC.

User space also has a 16M size limit, so it's in the same realm.

HTH
Vern

On 8/4/2010 7:22 AM, Jonathan Mason wrote:

Hi Dennis

Thanks for that. I was figuring along the same lines as yourself that
there's no guarantee that the entries wanted will be in the array, but to
get around the issue of doing a lookup and then a chain I would do something
like keep a note of the last code loaded and set a flag when the array
overflowed. That way the program could test to see if it needed to
reference the file or the array for the detail information.

I quite like the idea of the user indexes, particularly because for the
program I need to change a 16MB user index would allow 409,000+ entries
which would be more than ample. Actually for the file in question on the
customer's box I've just found out there are only 106 records anyway, so
it's a moot point in this case.

I don't like the idea of having loads of 16MB user indexes lying around, but
we could build in logic to size the index based on the number of records in
the file.

All the best and thanks again

Jonathan



-----Original Message-----
From: rpg400-l-bounces@xxxxxxxxxxxx [mailto:rpg400-l-bounces@xxxxxxxxxxxx]
On Behalf Of Dennis Lovelady
Sent: 04 August 2010 12:39
To: 'RPG programming on the IBM i / System i'
Subject: RE: Arrays and Performance

I suggest a good look at the SETOBJACC command, which will pin specified
objects in memory. For example, the critical index of a table may be locked
in memory via this command, hopefully eliminating most concerns about CHAIN
performance hit.

Like all performance questions, the correct answer to this one depends. I
question the validity of loading "the first x entries" into an array and
leaving the rest to CHAIN, since "the first x entries" will probably not be
the ones you want in any given situation. So for those, you have the CHAIN
impact PLUS the array lookup that theoretically preceded it.

I have used user indexes in the past, and they are wonderfully quick. You
have to decide how well the maintenance and usability of those fits your
shop's needs.

Since there is no index, I expect that the performance of a lookup in a
32K-entry table would struggle to keep up with a similar CHAIN. Not to
mention the impact that such lookup has on the other processes on the
system. For anything of that volume, I wouldn't hesitate to use data files.
I'd avoid user indexes in general for anything that's somewhat fluid just
because of the maintenance aspect.

Dennis Lovelady
http://www.linkedin.com/in/dennislovelady
--
Change is inevitable, except from a vending machine.

We have a number of programs, some in RPG/400 and others in RPG IV,
that
load reference data from various files into arrays in an attempt to
reduce the performance impact of multiple chains to database files.

The trouble with this approach is that the arrays are limited in size,
a
maximum 9,999 elements in RPG/400 and 32,767 elements in RPG IV.

One thought was to use the arrays for the first however many records
would fit and then chain to the database file for the remaining records,
but there's concern about the performance impact of that approach.

An alternative is to make use of user indexes, which could provide a
lot
more space for storing entries, but we're not sure on the performance
impact of that approach.

Does anybody have any knowledge of the performance benefits and
pitfalls
of any of these approaches? What do others do when faced with the same
problem?