Re: Buffered reading of stream files

Scott Klement wrote:

Do you mean that you want to read your file using fgets()? If so, please be aware that fgets() can only be configured to read lines of text delimited by either LF or CRLF. It can't be used with any arbitrary delimiter.

if you really mean fread(), then I'd suggest setting up some loops and benchmarking the diff. Essentially, fread() and read() work the same -- except that fread buffers the data. Will it perform better than buffering it yourself? Hard to say without benchmarking... and I haven't done any.

Okay, so I got some code working with fread(). Code's still a bit rough, but in case any one is interested, here's what I found.

buffread0: http://code.midrange.com/9def61bedb.html
buffread1: http://code.midrange.com/1e70379662.html

I ran each five times in a row, reading a 5.98 MB file with two repeated records of differing length. Times in microseconds.

buffread0: buffread1:
2373000 5799000
2368000 5831000
2368000 5810000
2367000 5718000
2352000 5737000

So it looks like either I'm doing something really dumb (likely) or fread() is quite a bit heavier than open() with roll-your-own buffering (less likely).

As Simon pointed out, I'm essentially having to double buffer due to not wanting to make assumptions about record length. I suspect the results would be much closer if bufferread1 were to read a larger chunk, and handle the record delimiter finding differently.