Re: Service Program Overhead

Regarding using CPYTOIMPF to export table data to a CSV stream file which "took about 1 minute 40" while using "a simple Java program to execute an SQL statement and write the results" to a CSV stream file "took about 1 minute 10".... While admittedly there was the caveat noted that "there was no scientific approach to the time calculation, just simple start and end times", it might still be interesting to run the Java program first followed by the CPYTOIMPF. FWiW doing so is still not much of an improved nor a scientific approach to performance testing, but should be a better indication for average of the timings than when omitting entirely the results of a reverse order of the two tests.

Except the v5r2 function and prior for CPYTOIMPF, the relevance of any performance for CPYTOSTMF should be moot with regard to the CPYTOIMPF feature. When using the v5r2 function, whether by data area or actual OS release, the CPYTOSTMF is not used by CPYTOIMPF. Each feature was written entirely separately of the other, and they do not share a common write [nor especially a read\format] algorithm.

Regards, Chuck

James Perkins wrote:

Oops, yes what was supposed to be CPYTOIMPF, sorry about that.

It seemed to work pretty fast for me. I prefer to write a CSV as
you suggest though directly from RPG. I hate using work tables if
I don't have to.

On Fri, Nov 13, 2009 at 11:12, Kurt Anderson wrote:

I don't know if things have changed, but historically I've
always seen significantly slower times when using CPYTOSTMF
than when writing to the IFS directly from RPG via APIs (and
writing directly from RPG to the IFS also has the benefit of
skipping intermediary processing such as writing to a temp file
before passing it along).

I know you said you used CPYFRMIMPF to export the file to CSV,
but I guess that confuses me b/c from the looks of it that
command copies the file from the IFS to your library.

Some info on writing directly to the IFS via RPG: http://www.news400.com/resources/clubtech/tnt400/bo400ng/AS400Q0167.htm

James Perkins wrote:

I had to write a program to write DB data to a stream file in
comma delimited format. No problem, done it many of times.
Then I ran into a problem, running the program seemed to take
hours. Well, I determined that was a bad subprocedure on my
part, so I fixed that and it now takes only about 5 minutes
to run.

While doing this doing this I tried a couple of other
approaches. I wrote the data to a table and used CPYFRMIMPF
to export it to a CSV, this took about 1 minute 40. I also
wrote a simple Java program to execute an SQL statement and
write the results to a stream file, this took about 1 minute 10. ** Note, there was no scientific approach to the time
calculation, just simple start and end times.

<<SNIP>>