RE: Memory allocation strategy

Thanx. I'll look over it how mmap would fit into my current code. Thanx for the pointer to it.

Have a nice weekend all.

Regards

Mihael

-----Original Message-----
From: rpg400-l-bounces@xxxxxxxxxxxx [mailto:rpg400-l-bounces@xxxxxxxxxxxx] On Behalf Of Mark S. Waterbury
Sent: Friday, January 07, 2011 3:18 PM
To: RPG programming on the IBM i / System i
Subject: Re: Memory allocation strategy

Mihael:

See embedded remarks below.

Mark

On 1/7/2011 7:21 AM, Schmidt, Mihael wrote:
Hi,

I am loading some data from a streamfile and I have no clue how big things can get.

The "stat" API will return the size of the streamfile. (See "Unix
type APIs in the InfoCenter.)

Strategy 1: I am allocating some memory (%alloc) and for every line extend the allocated memory (%realloc).
Strategy 2: I get one "big" block of memory and only do a reallocation if the data doesn't fit. (more work on my side)

Doesn't the system do strategy 2 under the hood already? If I allocate 100 bytes the system probably allocates a page (or so) and not only 100 bytes. So a realloc should not trigger another allocation of a whole page.?!

You could easily write some small test programs to empirically determine
the "allocation stragegy" e.g. by reporting the addresses returned.

It is usually better to write your own memory management layer that
allocates large blocks from the OS, and then sub-allocates from within
these large blocks, using whatever custom allocation strategy is deemed
best for the job at hand.

Please enlighten me.

Instead of reading the entire streamfile into memory, use "mmap" (or
"mmap64") API to directly map the streamfile into teraspace. This allows
you to directly address the data in the file as a large array in memory,
and it is paged in on demand.

Thanx in advance.

Mihael Schmidt