Re: Buffered reading of stream files

Sorry for the late reply...
Uh- I think I miss what you are saying. : )

Spanning chunks in the buffering won't be an issue - as you are either going to be reading until you find a delimiter
(i.e. 1 character at a time) or else you are going to use some mechanism like putting the record size in two or four bytes preceeding the record.

In either case, you gain a lot with the fxxx() functions, because the OS is invisibly doing the buffering behind you. In other words, if you do byte by byte reads, you are reading from a buffered stream. If you do two reads per record, size and data, then you are still reading from a buffered stream. Advantage user.

Essentially, if you intend to read in blocks of data and parse through through them, essentially doing your own buffering, all you gain is a small measure of processing time. You do not get a great gain in DASD IO savings, since with the fxxxx() functions,
you are *not* reading the DASD directly.

Hope that makes it a bit clearer.

-Paul

On Feb 5, 2009, at 11:57 AM, Adam Glauser wrote:

Paul Raulerson wrote:

You realize that if you use fread(), all the buffering is handled invisibly for you? You just read however many bytes you want at any one time.

Yes. However, I need to deal with variable length records and
multi-byte record delimiters. Unless I can reliably read the entire
file in to memory (which is probably not the best idea for very large
files), I need to handle the possibility that a record delimiter could
span a break in the 'chunks' I'm reading.

Since I want to create a reusable procedure, I'd rather not make
assumptions about the maximum record size, even though that could
probably improve performance somewhat and simplify the code.
--
This is the RPG programming on the IBM i / System i (RPG400-L) mailing list
To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.