Hi, Boris and Peter.
As the question discussed is absolutely practical for me, I decided to test which is better - my current version with fopen/fputs/fclose or Unix style with open/write/close.
What I did?
I just changed my program from handlers to file descriptions, that is:
fopen->open
fputs->write
fclose->close
All processing - source records reading and output records generation, code page conversion was left "as was".
Source file: 12M records, record size: 117 bytes
Resulting file size 1,6GB
Both programs have been submitted into batch on our test box today morning. Timing retrieved from the system log (DSPLOG).
I made 2 cycles.
First cycle:
My original program:
Started 9:15:01, ended 9:17:55, 98.270 seconds used
Program version that uses Unix style API-s:
Started 9:40:52, ended 9:44:04, 128.497 seconds used
Second cycle:
My original program:
Started 9:54:49, ended 9:57:22, 98.822 seconds used
Program version that uses Unix style API-s:
Started 9:50:58, ended 9:54:10, 129.444 seconds used
CPU was average 32-33% on both jobs, I could not make difference.
From here I can derive, that using handlers is a bit (ca 30%) faster than using file descriptors, however I am not this level specialist to say why it is so...
Additionally, of course, the results apply to our particular machine.
And there was no competition - I've been playing with a program that runs every night in live environment - so these seconds really mean for us.
Jevgeni.
-----Original Message-----
From: c400-l-bounces+jevgeni.astanovski=sampopank.ee@xxxxxxxxxxxx [mailto:c400-l-bounces+jevgeni.astanovski=sampopank.ee@xxxxxxxxxxxx] On Behalf Of Boris
Sent: 26. mai 2011. a. 2:05
To: c400-l@xxxxxxxxxxxx
Subject: Re: [C400-L] Write to IFS file - high CPU
Thanks Peter!
Changing Standard C functions to POSIX did make an improvement and as you
said the program finished much faster. The highest CPU percentage it reached
was 79% as opposed to 91%.
The internal buffer in Standard C functions was a real revelation to me. Any
other hidden gems out of your sleeve :)?
I'm just wondering... The program is doing multiple I/O operations.
Shouldn't an I/O operation release the CPU?
Jevgeni,
It's only a psychological problem until the operator puts the job on hold
because he thinks it's in infinite loop :)
Then it becomes a real problem, because the business unit thinks that other
business jobs will get less CPU.
There seems to be a consensus in this thread that high CPU is pretty much
normal. I'm just having hard time explaining it and I think I will be forced
to put the job in a different class with a lower runtime priority.
Thanks for the code. I'll definitely try it.
midrange.com
