Re: Dealing with very large blocks of character data

On 11/30/2011 2:07 PM, Mark S. Waterbury wrote:

Does this data contain any embedded null characters (x'00')? If so,
then you may find using the C functions can be somewhat
"problematic" as C strings are "null terminated" -- this means as
soon as a X'00' is encountered, it is considered to be "end of
string." *:-/*

No, it will be simple text.

You might also want to look at the following New MI instructions
available for use as built-in functions in any ILE language:

I'm more familiar with C functions than I am with MI.

Where is this data coming from? Is it in a stream file in the IFS? If
so, you might also want to consider using the _mmap() or _mmap64()
API to map the entire stream file into teraspace, where you can then
easily search it, without the overhead of having to read the file
first.

Hmmm ... the data isn't coming from a stream file, but it's possible that the file could be written to a stream file.

Obviously I'll still need to use the C functions to scan & extract the contents ... but using mmap could eliminate some of the problems with loading the data.

Thanks!

david