Re: Arrays and Performance

[Vern Hamberg <vhamberg@xxxxxxxxxxx>]

Jonathan

You can also use user spaces and pointers to insert data - then use 
qsort() to sort the data - it's all pointer-based as to where the data 
is, and you can get the address of a user space. Then use the bsearch() 
function - binary search - depends on a sorted data set. This is pretty 
vanilla coding in RPG IV, not possible in RPG III that I know, at least 
easily. Might be easier to maintain than user indexes - can't speak to 
that, since I've done this kind of thing and haven't done anything with 
user indexes.

The Redbook "Who Knew You Could Do That With RPG IV?" has examples of 
qsort() and bsearch(), IIRC.

User space also has a 16M size limit, so it's in the same realm.

HTH
Vern

On 8/4/2010 7:22 AM, Jonathan Mason wrote:
> Hi Dennis
>
> Thanks for that.  I was figuring along the same lines as yourself that
> there's no guarantee that the entries wanted will be in the array, but to
> get around the issue of doing a lookup and then a chain I would do something
> like keep a note of the last code loaded and set a flag when the array
> overflowed.  That way the program could test to see if it needed to
> reference the file or the array for the detail information.
>
> I quite like the idea of the user indexes, particularly because for the
> program I need to change a 16MB user index would allow 409,000+ entries
> which would be more than ample.  Actually for the file in question on the
> customer's box I've just found out there are only 106 records anyway, so
> it's a moot point in this case.
>
> I don't like the idea of having loads of 16MB user indexes lying around, but
> we could build in logic to size the index based on the number of records in
> the file.
>
> All the best and thanks again
>
> Jonathan
>
>
>
> -----Original Message-----
> From: rpg400-l-bounces@xxxxxxxxxxxx [mailto:rpg400-l-bounces@xxxxxxxxxxxx]
> On Behalf Of Dennis Lovelady
> Sent: 04 August 2010 12:39
> To: 'RPG programming on the IBM i / System i'
> Subject: RE: Arrays and Performance
>
> I suggest a good look at the SETOBJACC command, which will pin specified
> objects in memory.  For example, the critical index of a table may be locked
> in memory via this command, hopefully eliminating most concerns about CHAIN
> performance hit.
>
> Like all performance questions, the correct answer to this one depends.  I
> question the validity of loading "the first x entries" into an array and
> leaving the rest to CHAIN, since "the first x entries" will probably not be
> the ones you want in any given situation.  So for those, you have the CHAIN
> impact PLUS the array lookup that theoretically preceded it.
>
> I have used user indexes in the past, and they are wonderfully quick.  You
> have to decide how well the maintenance and usability of those fits your
> shop's needs.
>
> Since there is no index, I expect that the performance of a lookup in a
> 32K-entry table would struggle to keep up with a similar CHAIN.  Not to
> mention the impact that such lookup has on the other processes on the
> system.  For anything of that volume, I wouldn't hesitate to use data files.
> I'd avoid user indexes in general for anything that's somewhat fluid just
> because of the maintenance aspect.
>
> Dennis Lovelady
> http://www.linkedin.com/in/dennislovelady
> --
> Change is inevitable, except from a vending machine.
>
>
>    
> > We have a number of programs, some in RPG/400 and others in RPG IV,
> > that
> > load reference data from various files into arrays in an attempt to
> > reduce the performance impact of multiple chains to database files.
> >
> > The trouble with this approach is that the arrays are limited in size,
> > a
> > maximum 9,999 elements in RPG/400 and 32,767 elements in RPG IV.
> >
> > One thought was to use the arrays for the first however many records
> > would fit and then chain to the database file for the remaining records,
> > but there's concern about the performance impact of that approach.
> >
> > An alternative is to make use of user indexes, which could provide a
> > lot
> > more space for storing entries, but we're not sure on the performance
> > impact of that approach.
> >
> > Does anybody have any knowledge of the performance benefits and
> > pitfalls
> > of any of these approaches?  What do others do when faced with the same
> > problem?
> >      
>
>