RE: %lookup on a Data Structure Array -- RPG400-L

I'm aware of SETOBJACC, however I don't want this entire file in memory, only two fields. Also, each of our clients has their own version of the file, so keeping it in memory doesn't seem like a viable option regardless.

I do need to retrieve a value, so I can't simply check for the existence of the record.

I am aware of a potential performance hit of loading the array all at once. These are all batch jobs, and at most there would be 33k records (I'd define the array to be 50k to allow for growth), which is going to load in seconds - so from a batch perspective, extra seconds once is ok.

However this discussion has given me the idea (or maybe someone actually mentioned this and I took it the wrong way, yet lead me to the same conclusion) that I could check the array. If the customer isn't there, then go to the file and get the value I need plus add the customer/value to the array. So in the case of having 33k customers, maybe my job of running 30 million records only uses 15k of those customers, then I've made the array smaller so the lookups would be quicker.

This is all nice, however I think I need to enter a RFE for the RPG team to further enhance data structure array capability.

In regard to User Indexes, I'm not sure where at the point of trade-off occurs. Like - when is it better to use a User Index than an array? I'm sure there's presentations on that out there (and likely even on conference CDs we have).

Thanks,
Kurt

-----Original Message-----
From: rpg400-l-bounces@xxxxxxxxxxxx [mailto:rpg400-l-bounces@xxxxxxxxxxxx] On Behalf Of Raul A. Jager W.
Sent: Thursday, November 10, 2011 12:50 PM
To: RPG programming on the IBM i / System i
Subject: Re: %lookup on a Data Structure Array

In the CHGPF command there isa parameter "KEPPINMEM" I have not yet tried it, but it sounds like it can solve your problem with very litle effort.

Schutte, Michael D wrote:

My question would be are you actually getting data from the file?

If not would

SetLL (...) file;
If %Equal();

Work for you?

It would be equivalent to
Chain (...) file;
If %Found();

You'll save on reading the data from the file into the program.

-----Original Message-----
From: rpg400-l-bounces@xxxxxxxxxxxx
[mailto:rpg400-l-bounces@xxxxxxxxxxxx] On Behalf Of Kurt Anderson
Sent: Thursday, November 10, 2011 1:05 PM
To: RPG programming on the IBM i / System i
Subject: RE: %lookup on a Data Structure Array

A number of you have responded by creating a subfield that is essentially the data structure. Yes, I guess that is a valid option. However I recently moved away from that method with the advent of data structure arrays supporting %lookup. In regard to data structure array limitations, another limitation I encountered was that you MUST have a subfield to search on. So if I have a data structure array with 4 subfields, and the key to the array is all four fields, I need to create another subfield to overlay them instead of simply using the data structure itself. Oh well.

Gary, I suppose I could use the sort and lookup API's. I honestly hadn't thought of that (haven't had to do that sort of thing in quite a while).

Matt, I knew someone would ask. You'll have to take me at my word that I cannot get the data to be in such an order. However, even so, I suppose that type of check may still prove useful (just not as useful if I could guarantee the order of the input).

Dennis, I haven't touched user indexes yet, but I may have to check that out.

Thanks for the responses,
Kurt

-----Original Message-----
From: rpg400-l-bounces@xxxxxxxxxxxx
[mailto:rpg400-l-bounces@xxxxxxxxxxxx] On Behalf Of Helge Bichel
Sent: Thursday, November 10, 2011 11:23 AM
To: 'RPG programming on the IBM i / System i'
Subject: RE: %lookup on a Data Structure Array

d chartds ds qualified
d chartfields dim(62) ascend
d chartorder 20 overlay(chartfields:*next)
d chartx 20 overlay(chartfields:*next)
d charty1 9 0 overlay(chartfields:*next)

-----Original Message-----
From: rpg400-l-bounces@xxxxxxxxxxxx
[mailto:rpg400-l-bounces@xxxxxxxxxxxx] On Behalf Of Kurt Anderson
Sent: Thursday, November 10, 2011 6:09 PM
To: RPG programming on the IBM i / System i (rpg400-l@xxxxxxxxxxxx)
Subject: %lookup on a Data Structure Array

I'm trying to reduce some program I/O for program processing millions of records. I've identified a file that I feel could be loaded into an array instead of having the program chain out for every record. Usually this file is between 100-1000 records.
However one of our clients has 20,000 records in the file. So before making the change, I decided to do a little reading on %lookup, to make sure I'd be using it correctly.
http://www.ibmsystemsmag.com/ibmi/developer/rpg/iSeries-EXTRA--Look-Bef
ore-You--Lookup/

It turns out that I had misunderstood how to trigger the binary search with a %lookup. I thought sorting the array was enough, however to get the binary search one needs to explicitly specify Ascend or Descend on the array.

So I went to add Ascend to a data structure array, and it won't let me:
D ds_Cust DS Qualified Inz Dim( 50000 ) Ascend
D Cust 3p 0
D Cycle 1a
Error: RNF3501E Keyword is not allowed for a data-structure definition; keyword is ignored.

I tried adding the Ascend keyword to the subfield that I use in the lookup, but it doesn't like that either (I wasn't expecting this to work, but at this point I'm grasping at straws).

I suppose I could make these two fields their own arrays. Though I'm not really a fan of that. I love using data structures to group like-data together.

Is there any way to get the new data structure array lookups to use a binary search?

Kurt Anderson
Sr. Programmer/Analyst
CustomCall Data Systems

--
This is the RPG programming on the IBM i / System i (RPG400-L) mailing
list To post a message email: RPG400-L@xxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx Before posting, please take a
moment to review the archives at http://archive.midrange.com/rpg400-l.

--
This is the RPG programming on the IBM i / System i (RPG400-L) mailing
list To post a message email: RPG400-L@xxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx Before posting, please take a
moment to review the archives at http://archive.midrange.com/rpg400-l.

--
This is the RPG programming on the IBM i / System i (RPG400-L) mailing
list To post a message email: RPG400-L@xxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx Before posting, please take a
moment to review the archives at http://archive.midrange.com/rpg400-l.

________________________________

Notice from Bob Evans Farms, Inc: This e-mail message, including any attachments, may contain confidential information that is intended only for the person or entity to which it is addressed. Any unauthorized review, use, disclosure or distribution is strictly prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message and any attachments.

--
This is the RPG programming on the IBM i / System i (RPG400-L) mailing list To post a message email: RPG400-L@xxxxxxxxxxxx To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives at http://archive.midrange.com/rpg400-l.