RE: Buffered reading of stream files

Adam;

I use the posix(unix) open(), close(), read()... APIs to process stream file, similar to the C functions. In general I allocate 4M of memory for the file buffer and read the file in 4M blocks, and process the buffer. The tricky part is dealing with "records" that cross the 4M boundary, part is in memory part is still in the file.

Isolating your routine from the parser and making it generic should be fairly easy, pass in the begin and end delimiters and a buffer large enough for a "record" and you should be able to use this procedure almost anywhere.

The big problem I see is performance, just like reading one record at a time from a DB could be a performance hit, reading one byte at a time from a stream file is almost guaranteed to be a poor performer.

I could send you some of my code or Scott Klement has some examples of accessing the IFS on his web site.

Duane Christen

-----Original Message-----
From: rpg400-l-bounces@xxxxxxxxxxxx
[mailto:rpg400-l-bounces@xxxxxxxxxxxx] On Behalf Of Adam Glauser
Sent: Wednesday, February 04, 2009 2:45 PM
To: rpg400-l@xxxxxxxxxxxx
Subject: Buffered reading of stream files

I have a routine which reads a stream file byte by byte
looking for record start and end delimiters. I have
implemented my own buffering.
Now I'm looking at trying to separate the buffered reading
from the actual parsing so that I can handle records without
start delimiters, as well as delimiters of arbitrary length.

It should be relatively easy to modify my current buffering
code to work with different delimiters, but I wonder if I'd
be better off just using the _C_IFS_(fopen,fclose,fread) functions.

What would you choose and why?
--
This is the RPG programming on the IBM i / System i
(RPG400-L) mailing list To post a message email:
RPG400-L@xxxxxxxxxxxx To subscribe, unsubscribe, or change
list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.


______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit
http://www.messagelabs.com/email
______________________________________________________________________

NOTICE: This electronic mail transmission may contain confidential information and is intended only for the person(s) named. Any use, copying, or disclosure by any other person is strictly prohibited. If you have received this transmission in error, please notify the sender via e-mail.