Re: Service Program Overhead

[CRPence <CRPbottle@xxxxxxxxx>]

  Regarding using CPYTOIMPF to export table data to a CSV stream 
file which "took about 1 minute 40" while using "a simple Java 
program to execute an SQL statement and write the results" to a CSV 
stream file "took about 1 minute 10"....  While admittedly there was 
the caveat noted that "there was no scientific approach to the time 
calculation, just simple start and end times", it might still be 
interesting to run the Java program first followed by the CPYTOIMPF. 
 FWiW doing so is still not much of an improved nor a scientific 
approach to performance testing, but should be a better indication 
for average of the timings than when omitting entirely the results 
of a reverse order of the two tests.

  Except the v5r2 function and prior for CPYTOIMPF, the relevance 
of any performance for CPYTOSTMF should be moot with regard to the 
CPYTOIMPF feature.  When using the v5r2 function, whether by data 
area or actual OS release, the CPYTOSTMF is not used by CPYTOIMPF. 
Each feature was written entirely separately of the other, and they 
do not share a common write [nor especially a read\format] algorithm.

Regards, Chuck

James Perkins wrote:
> Oops, yes what was supposed to be CPYTOIMPF, sorry about that.
>
> It seemed to work pretty fast for me. I prefer to write a CSV as
> you suggest though directly from RPG. I hate using work tables if
> I don't have to.
>
> On Fri, Nov 13, 2009 at 11:12, Kurt Anderson wrote:
> > I don't know if things have changed, but historically I've
> > always seen significantly slower times when using CPYTOSTMF
> > than when writing to the IFS directly from RPG via APIs (and
> > writing directly from RPG to the IFS also has the benefit of
> > skipping intermediary processing such as writing to a temp file
> > before passing it along).
> >
> > I know you said you used CPYFRMIMPF to export the file to CSV,
> > but I guess that confuses me b/c from the looks of it that
> > command copies the file from the IFS to your library.
> >
> > Some info on writing directly to the IFS via RPG: 
> > http://www.news400.com/resources/clubtech/tnt400/bo400ng/AS400Q0167.htm
> >
> > James Perkins wrote:
> > > I had to write a program to write DB data to a stream file in
> > > comma delimited format. No problem, done it many of times.
> > > Then I ran into a problem, running the program seemed to take
> > > hours. Well, I determined that was a bad subprocedure on my
> > > part, so I fixed that and it now takes only about 5 minutes
> > > to run.
> > >
> > > While doing this doing this I tried a couple of other
> > > approaches. I wrote the data to a table and used CPYFRMIMPF
> > > to export it to a CSV, this took about 1 minute 40. I also
> > > wrote a simple Java program to execute an SQL statement and
> > > write the results to a stream file, this took about 1 minute 
> > > 10. ** Note, there was no scientific approach to the time
> > > calculation, just simple start and end times.
> > >
> > > <<SNIP>>