Re: Avoiding disk arms with CLOB

Hi, Dutch:

You might want to look into writing the data into one or more *USRSPC objects. (Each *USRSPC can be up to 16 MB in size.)

Or, you could create an IFS streamfile, and you could use the Unix-type "mmap()" API to map the file into memory. If you start out by creating this streamfile in /tmp, you could take advantage of new features in V7R1 (cannot recall with Technical Reresh level may be needed) that lets the OS keep all /tmp subdirectories and files in memory, so they never get written out to disk (no head and arm movement, etc.). The only issue with this approach is if the system goes down, you lose whatever is in /tmp. So, at some point, if you need to keep this data, you may need to copy it to "elsewhere."

As others have suggested in other replies, we would probably need to know a little more about the structure of the data and what you need to do with it, to give better answers or suggestions.

Hope this helps,

Mark S. Waterbury

> On 9/19/2013 9:21 AM, RPGLIST wrote:

I'm facing the need to process a very large amount of data, which is more
than likely going to come in as a SOAP or XML stream but I need to process
this quickly and without losing time with the disk arms if at all
possible.

I know that a clob will allow me to store a larger set of data but from
what I'm reading, you can't just access and read it like a normal
variable.

I assume, and if I'm wrong feel free to slap with a lead pipe, that like
with regular PF's writes are going to cost me time. I know some
individuals don't care about time or say its not important but I need to
return a response in a matter of seconds like 5-8 or lower if possible and
I'm not going to argue that point. It is what it is.

The data coming in if you put it in a PF would have a record length of
somewhere in the 12k range, so you can imagine it'll be larger with XML
tags and that's a single record, whereas we could receive up to 99 lines
with each line in the 12k+ range.

Now, if someone has a suggestion I'm all ears :)