Re: scanning user space

[James Rich <james@xxxxxxxxxxx>]

On Tue, 24 Jun 2003, Pete Clifford (@riadne software) wrote:

> a) I guess A and B show the relative overhead of a user space and
> dynamically allocated memory. I'll be doing more ALLOCs in future.
> b) I expected D to be bad and C to be much better, but I didn't expect C to
> be much better than A and B. Why? Have I somehow chosen some magic buffer
> size that corresponds to an internal IFS buffer size or something?

[snip]

> Can anyone explain why C is better than A and B? I wonder whether these
> results would be repeated on an empty machine (mine wasn't - no such
> luxury). I wonder whether A and B are getting paged out while the I-O is
> occurring while C isn't because it's doing more CPU intensive work (multiple
> reads). The CPU usage is similar but the elapsed times are significantly
> different. Any suggestions?

Possible explanation without doing any real tests (apologies to Joe who
suggested we test things more before commenting - an excellent idea):

When you malloc() or use a user space of that size the system needs to
spend time getting the memory together for you to use.  Possibly that
memory may be on disk (paged out).  Regardless, meeting the demands for
that memory may take longer than simply reusing a smaller buffer over and
over.  Too small and you spend all your time going out to disk in an
inefficient way.  Too large and you make inefficient use of memory.  It is
very possible that memory contention is a worse problem than disk
performance in this case.

James Rich