Re: Any way to improve on CPYTOIMPF's speed?

On Thu, Jan 26, 2017 at 12:45 PM, Dan <dan27649@xxxxxxxxx> wrote:

We have a process that copies 16.7M records from one native table to
another in 13 minutes, then use CPYTOIMPF to convert this to a .CSV file,
which takes 1 hour, 39 minutes.

I thought I had seen a recent thread that claimed that one of Scott
Klement's utilities performed faster than CPYTO???F (can't remember if it
was CPYTOIMPF or CPYTOSTMF), but I came up empty searching for that
thread. Does anyone remember, or was I imagining things?

I don't remember any *recent* thread, or in fact any particular thread
from any time regarding that. However, I will say that the last time I
looked at CPYTOIMPF performance myself (which was years ago), it was
quite inefficient, both in terms of time and memory. Probably disk
too. Basically, it was the bluntest and most brute-force tool
imaginable. Almost anything you could have picked to avoid using it
would have performed better.

I don't know if the situation is still that bad today; I should hope
it's not. But I am confident that in any case, you will be able to do
better by using RPG and the appropriate APIs to work directly with
stream files. Cut out any intermediate steps and just generate the CSV
directly.

John Y.