RE: array handling -- MIDRANGE-L

Richard, then come to our Vermont User Group meeting next time around, at
least.  Let me know your e-mail address and I'll add you to the invite list.
 Plus, ought to be lots of ways to have a dinner some time along.

---------------------------------------------------------
Booth Martin http://www.MartinVT.com
Booth@MartinVT.com
---------------------------------------------------------
-------Original Message-------
From: midrange-l@midrange.com
Date: Sunday, December 02, 2001 04:03:24 PM
To: midrange-l@midrange.com
Subject: RE: array handling
Thanks for the pointers. It's always a bit rough when
the client brings you in for one thing and winds up
asking for something else once you get on site. One
can't really refuse, especially in this economy.
I'll be here in VT for at least 6 months (can't miss
the ski season!) and possibly up to 2 years so we've
got plenty of time to exchange war stories. Let me
know when is good for you.
--- Booth Martin <Booth@MartinVT.com> wrote:
> If you are going to sort a few elements into a large

> array in ascending

> order you can also set a counter and increment it

> with each addition to the

> array as you load it. Then after the array is

> loaded, sort the array, and

> set your counter's value at the (array size -

> counted value). At that point

> you now have a valid initial value for your look-up.

>

> In the example that Joe used, then the look up would

> begin at element (1000

> - 20) or at element 980, so there'd be 979 elements

> ignored in the lookup

> process.

>

> Vermont eh, Rich? great place. If you're around

> this week I'll take you

> out to dinner some night and we can talk AS/400 war

> stories. :)

>

>

---------------------------------------------------------
> Booth Martin http://www.MartinVT.com

> Booth@MartinVT.com

>

---------------------------------------------------------
>

> -------Original Message-------

> From: midrange-l@midrange.com

> Date: Sunday, December 02, 2001 07:16:44 AM

> To: midrange-l@midrange.com

> Subject: RE: array handling

> No, you're right on-base here, Rich. You can

> significantly reduce your

> overhead this way; I use caching a lot for small

> tables. Since the number

> of entries is so small, I don't think we'll be able

> to significantly reduce

> your processing time by the backloading technique,

> but let me explain it and

> see if you're interested in implementing it.

> A LOOKUP operation has to do "N" compare operations.

> The worst case is an

> unsorted array where there is no hit; in this case,

> "N" is the number of

> entries in the array. (Of course, if you're getting

> REALLY pedantic, it's

> actually doing twice that many compares, along with

> a number of increments

> equal to the size of the array, but I think you get

> the picture.)

> As your array gets larger, this can add up to a lot

> of cycles. There are

> two ways to reduce the number of compares. First is

> to order the array -

> that is, make sure the array is in sequence. Knowing

> that, the compiler can

> generate a much more efficient LOOKUP algorithm

> (using a binary scan), where

> the maximum number of compares is actually ((number

> of entries) log 2). For

> a 20 element array, that's five compares, or a

> potential savings of 75%. On

> larger arrays, the savings is even more significant.

> The only downside is

> making sure that all the unused entries at the end

> of your array have "high

> values", such as X'FF'.

> Now you run into the problem of unused elements.

> Let's say you make your

> array 1000 elements long, but you only use 20. In

> that case, most of the

> time is spent chceking unused elements. Even with an

> ascending array, you

> still have to do 10 compares, twice the number

> required to handle 32

> entries. One of the nice things about the LOOKUP

> operation in RPG is that

> you can specify a starting position in the array; it

> will only scan the

> elements from that index forward. So, what you do is

> you load the array

> BACKWARD, starting with the last position in the

> array. Then, whenever you

> do a LOOKUP, you start with the last position that

> you loaded. Depending on

> the number of entries and the size of your array,

> you can probably see that

> this can significantly reduce your processing.

> So what's the best for you? In your case, since the

> number of elements

> doesn't fluctuate very much, you should be able to

> tune your array size to

> be close to the number of entries you need. You may

> occasionally have to

> resize your array, and you MAY want to put a check

> in your maintenance

> program to notify you if someone tries to add more

> records to the file than

> your cache program expects. But the backloading

> technique is probably

> overkill - it's more relevant when you have to

> define a really BIG array

> that may only have a small number of elements in it.

> On the other hand, since you ARE doing this for

> performance reasons, making

> your array ascending may be a smart move. Regardless

> of the size, ascending

> arrays allows the compiler to generate the far more

> efficient binary search

> algorithm, which should reduce processing even in

> your case. Simply add the

> ASCEND keyword to the D-spec defining the array.

> Then, before you load your

> array, do a MOVEA *HIVAL to preload the array with

> X'FF':

> D ARY1 S 2 DIM(20) ASCEND

> (...)

> C MOVEA *HIVAL ARY1

> Hope this helps a little.

> Joe Pluta

> www.plutabrothers.com

> > -----Original Message-----

> > From: midrange-l-admin@midrange.com

> > [mailto:midrange-l-admin@midrange.com]On Behalf Of

> Richard Reeve

> > Sent: Sunday, December 02, 2001 5:33 AM

> > To: midrange-l@midrange.com

> > Subject: RE: array handling

> >

> >

> > Joe,

> >

> > You hit the nail on the head. I am doing this

> > for performance reasons. There are currently less

> > than 20 hold codes (much less actually, currently

> > using 4 so I defined the array(s) as 20. Without

> > doing something along the lines that I described

> then

> > I would have to go out and read the table file for

> > each hold transaction rather than loading the

> array

> > one time and doing a lookup for each hold

> transaction

> > (to get corresponding code). I should always get a

> > hit on the table (after loading).

> >

> > Please let me know if you think that I am missing

> > something.....

> >

> > --- Joe Pluta <joepluta@PlutaBrothers.com> wrote:

> > > Richard, one quick question: are you doing this

> for

> > > performance reasons?

> > > That is, are you caching data in order to avoid

> > > doing reads on the file?

> > >

> > > If not, I'd be interested to know why you are

> doing

> > > this, since it can cause

> > > maintenance headaches down the road - you must

> > > change the program if the

> > > number of entries ever exceeds the size of your

> > > array.

> > >

> > > If you *are* doing this for performance, will

> there

> > > be a significant number

> > > of times when you get a "miss" rather than a

> "hit"?

> > > If the answer is yes,

> > > you might want to "backload" the arrays; this

> can

> > > substantially reduce your

> > > processing time. Let me know and I'll go into

> this

> > > a little further.

> > >

> > > Joe

> > >

> > > > -----Original Message-----

> > > > From: Richard Reeve

> > > >

> > > > Thanks to all who responded. This list is

> > > certainly a

> > > > life saver.

> > > >

> > > > Happy Holidays to all.

> > > >

> > > > Rich

> > >

> _______________________________________________