Re: scanning user space -- RPG400-L

I didn't do anything except read the data - no parsing. I was interested in
the different timings of the various methods of reading the contents of the
file. I wasn't proposing to do Rick's job for him. You're right, of course,
that the crux of the matter is the parsing. I suspect that is the area where
the real time savings are to be made.

Pete
----- Original Message ----- 
From: "Scott Klement" <klemscot@xxxxxxxxxxxx>
To: "RPG programming on the AS400 / iSeries" <rpg400-l@xxxxxxxxxxxx>
Sent: Wednesday, June 25, 2003 7:33 PM
Subject: Re: scanning user space


>
> > A.    43.4 seconds elapsed 3.0 seconds CPU time.
> > B.    20.4 seconds elapsed 2.6 seconds CPU time.
> > C.    4 seconds elapsed 2 seconds CPU time.
> > D.    210 seconds elapsed 89 seconds CPU time.
> >
> [SNIP]
> > Can anyone explain why C is better than A and B?
>
> Without seeing your code, it's a little difficult.
>
> Here's what I'm thinking, though, based on assumptions of how the code
> works:
>
> 1) You're reading the entire file in all cases.  As long as the amount
>      of data you're transferring exceeds the minimum amount of disk
>      that's normally transferred to memory, the speed should be about
>      the same for this part of the operation.
>
> 2) A user-space is a disk object.   Sure, if it gets a lot of accesses,
>      the operating system will try to bring it into memory to make it
>      faster, but that's not going to happen instantaneously.   Also,
>      the size being so large probably has an impact on whether it
>      brings it into memory.
>
> 3) ALLOC'ed memory is in RAM.   Sure, it may be paged out to disk if
>      the memory is required by other processes, but since you're doing
>      constant access to it, the OS will try not to page it out if it
>      can help it.   Since it's so large, though, there's a good chance
>      that it will have to page out at least part of it.
>
> 4) The 32k buffer is almost definitely in RAM because it's small, so
>      easy to keep on-hand, and you're using it constantly.   Also,
>      because you're using less memory, you're not slowing down other
>      processes by taking up all of the RAM.
>
> 5) The small-data-at-a-time reads are going to be slow because it has
>      to do more disk transfers to get the same amount of data.
>
> > I wonder whether these results would be repeated on an empty machine
> > (mine wasn't - no such luxury).
>
> The results might have smaller times, but I think you'll get the same
> results.  I'm assuming that all you did was read the file...  I don't see
> why reading it all in to a large block of memory would ever be faster
> than reading it into a small block of memory, and then re-using that
> memory every 32000 bytes.  They should be the same -- or the big
> buffer will be slower if paging is necessary.
>
> > I wonder whether A and B are getting paged out while the I-O is
> > occurring while C isn't because it's doing more CPU intensive work
> > (multiple reads).
>
> Why would that be more CPU intensive?!   Either the read() routine is
> looping loading blocks of data, or your routine is looping.   From the
> CPU's perspective, a loop is an IF and a GOTO.. I don't see why one would
> be more CPU intensive than the other.
>
> Are your benchmarks parsing the file into "lines" (CRLF delimited
> records?)   That would be the real test...
>
>
> _______________________________________________
> This is the RPG programming on the AS400 / iSeries (RPG400-L) mailing list
> To post a message email: RPG400-L@xxxxxxxxxxxx
> To subscribe, unsubscribe, or change list options,
> visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
> or email: RPG400-L-request@xxxxxxxxxxxx
> Before posting, please take a moment to review the archives
> at http://archive.midrange.com/rpg400-l.
>
>
>