On 23-May-2011 07:44 , Boris wrote:
I have a program that copies a DB2 file into an IFS stream file. It
may process millions of records. The problem is that the program
consumes high CPU (more than 90%). I removed all the processing and
only left the read from the DB2 file and write to the IFS file
functions. The program stack shows the write function almost
constantly so I assume the write function is the one to blame. What
is the best way to implement a solution like that?
Please do not post an entire digest; especially when just posting a
new message, as even more of the extra data is unrelated. Just compose
a "new message" to the list as addressee; there should be no reason to
use "reply to" [a digest] just to get that addressee, and if for some
reason that is done, then please trim all of the digest not related to
the message being composed then sent.
High CPU is not necessarily bad. Maximizing CPU, i.e. high CPU, is
generally a good thing, when efficiency is high. However most likely
this scenario is the effect of the underlying write processing
efficiently performing each of the many more than necessary write requests.
The database read will by-default be reading "blocks" of data, though
probably each of the "write" requests in the program is writing only one
record [, or worse just one column of data for one record, or absolutely
worst just one byte of data for one record] of database data to the
stream file for each database record that is "read". And although the
program may "read" just one record from the database, the read is likely
from a block of data, not from the database itself. The resolution in
that case is to perform similar to the database which is reading in
blocks, and that would be to write in blocks. So instead of writing
[what is probably currently] one byte for each write, tens or hundreds
of bytes for each write, the program would do better to write thousands,
and even better to write millions of bytes for each write.