Re: Dynamic array in interface

On Aug 14, 2017, at 8:02 PM, Chris Pando <chris.pando@xxxxxxxxx> wrote:

I've been doing a lot with dynamic arrays and memory management

Business Problem: Separate addresses into three categories (correctly
formatted with a 5 digit zip code, correctly formatted with a 5+4 zip,
and incorrectly formatted).
Use regular expressions to determine whether or not an address is
correctly formatted. Sort multiple ways, and print (eliminating
duplicates). Number of addresses to
be processed ranges from 10K to 10M.

Technical issues: RPG does not allow for dynamic arrays, and has a
really low limit (16M) for array size. Allowing 128 characters per
address, that limits the total number of elements to
~131K.

Solution: We are going to manage the (data structure) array ourselves,
using an array of pointers and space in the heap. Instead of using
array notation ( array($I) ), we are going to
use basing pointers (just like C).

Note on notation: I tend to start my arrays with an '@'.If a variable
is a pointer, I tend to end it with a '@'. If it is a pointer to a
pointer, I tend to end it with a '@@'. I am not entirely consistent.

@array@ is a pointer to a contiguous space of memory containing
pointers to our heap entries (one heap entry for each array element).
array@@ = @array@ + ($I-1)*16
array@@ -> array@ -> arrayEntry

So to create an array element

Dcl-S array@@ Pointer
Dcl-S array@ Pointer Based(array@@)
Dcl-DS arrayEntry LikeDS(arrayEntry_t) Based(array@)

@array@ = %Alloc(arraySize*16); arraySize is
whatever the intial allocation is to be, I generally use 1024

For each array element (while space remains) to be added:.

array@@ = @array@ + ($I-1)*16
array@ = %Alloc(%Len(arrayEntry))

If the number of elements grows to exceed arraySize

arraySize*= 2
@array@ = %ReAlloc( @array@ : arraySize*16)

I'm working on a more thorough writeup, but, for the moment, a working
example is at https://code.midrange.com/2bf4e99c7d.html
This is two service programs (one for regular expressions, one to
manage memory for an array of an abstract data type, plus binding
source, include source,
and a program utilizing the two service programs. I didn't include any
test data - I figure everybody has a file full of addresses).

This example sorts by:
1. Address
2. Match type and address
3. (left as an exercies) - by state, and then by city, and the by the
5 digit zip code (regardless of whether 5 or 5+4 (e.g. ignore the 4))
then by the alpha part of the street, and
then the numeric part of the street. This would be really tricky with SQL


This started as an example of how to use regular expressions, and is a
pretty good example of how to use qsort().

Chris
---
A useful strategy to use when records are long and swaps are
expensive is to maintain an array of pointers to the records, using
whatever sorting algorithm one chooses.
One can then swap pointers rather than records. Once the pointers to
the records have been arranged into the proper order, the records
themselves can be arranged into
the final sorted order in O(n) time.
Data Structures and Algorithms - Aho, Hopcraft & Ullman,
Addison-Wesley 1983 pg 260

date: Mon, 14 Aug 2017 12:46:23 -0700
from: Jon Paris <jon.paris@xxxxxxxxxxxxxx>
subject: Re: Dynamic array in interface

I am not aware of any plans to support dynamic arrays in RPG (which is what you are effectively asking for) but you are mistaken in your comparison with COBOL.

Yes there is an Occurs Depending On (ODO) option, but the full amount of memory space is always allocated so I don't see how it makes a difference. I have passed structures from RPG with the same layout as a COBOL ODO and it works fine so I'm having trouble understanding your issue.

Also, if you are passing this as a parm from RPG to COBOL only the pointer to the first byte ever gets passed so the actual size of the structure is irrelevant anyway.

You could always use dynamic memory for the array in RPG - If you can give a bit more detail about how the data is passed we might be able to help more with how to implement such an option.

Jon Paris

www.partner400.com
www.SystemiDeveloper.com

--
This is the RPG programming on the IBM i (AS/400 and iSeries) (RPG400-L) mailing list
To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.

Please contact support@xxxxxxxxxxxx for any subscription related questions.

Help support midrange.com by shopping at amazon.com with our affiliate link: http://amzn.to/2dEadiD