RE: Separation of Presentation, BL, and I/O Tiers

Joe,

I put some comments inline:

>>> joepluta@PlutaBrothers.com 04/11/02 05:02PM >>>
> From: David Morris
>
> With a Mediator, you can use one area if you want (I use a user
space)
> that serves many files. The Mediator maintains information about the
> files it is serving and the space where it is loading data.

How do you handle multiple concurrent cursors, then?  I assume you
partition
the user space with your own linked list of pointers.  Once you've
started
doing that, you then have to deal with fragmentation as cursors open
and
close, right?  So this is the real difficult part - managing heap
space
within your user space.  One way would be to have a fixed management
structure in your user space which included a pointer to the actual
(variably sized) data space.  Sort of a Cursor Control Block.

Yes, there is a linked list of pointers. There is connection specific
information, file specific, format, field, etc. They are shared where
possible. We don't open and close cursors, they are open for the
job so fragmentation is not a problem. If it were, the underlying
storage toolkit has a routine to gather up fragmented memory.

> With a Mediator, you can detect the recursive call (you can store a
> stack level by open file) and allocate another piece of memory.

If you're managing multiple cursors as I indicated in the previous
paragraph, this makes perfect sense.  You simply set aside another
chunk of
your user space for another set of records.


> Mediator does not help much here, The best way to do this is to
> register
> an exit and a close routine or use a scope message, either one
imposes
> a new requirement on the caller. Ideally you could call the Mediator
to
> register the cleanup routine but IBMs APIs and scope messages do
> not support a level. The Mediator does simplify the cleanup effort
> because all of the allocation is done in a single module.

It's the same issue with the CCB layout I identified in the first
section.
At some point, you're going to have to free up all those blocks.  The
good
news is that a properly written caller would tell the Mediator to free
up
individual blocks by issuing a "close cursor" command.  The other way
is to
make sure there's an exit procedure identified that does a "close ALL
cursors".

Like I said, we don't close files so storage grows throughout the job.

It hasn't been a problem and when a userspace size is exceeded,
a new one is created and it keeps on going. We hundreds of jobs
running this way processing large files and it scales fine on our
system, which has a CPW of 560. One helpful part is the iSeries
really does better when you leave files open and it doesn't care
where your memory starts and stops. We do push up storage
during the day a few gig, and some of it is qtemp, but it isn't an
issue.

The cleanup happens automatically in this case at job end because
this information is stored in user spaces in qtemp. That was not an
ideal choice looking back because server exits clear qtemp between
calls but do not free activation groups. Those jobs use a custom
qtemp.

Cripes, we're getting closer and closer to my early days of writing
operating system code.  After being hidden from our view for nearly 20
years
by RPG and OS/400, these old concepts are coming back into vogue
<grin>.

Joe