Re: Any way to improve on CPYTOIMPF's speed?

I'm not selling anything so I don't feel at all guilty about saying this but ...

If you implement my handler it could literally be a matter of minutes to do the job and the _exact_same_ handler will work for every one of the programs you are describing. Same one or two line code change in each. No logic changes required.

Just sayin'


Jon Paris

www.partner400.com
www.SystemiDeveloper.com

On Jan 26, 2017, at 4:00 PM, Dan <dan27649@xxxxxxxxx> wrote:

Jon, Rob, thanks, I will look at those options. It will hinge on whether
our manager deems the current process as too slow. This is just one of 30
such conversions done on a monthly basis, although this one is the largest
by far. The other factor is that our development system is slower than our
new Power 8 box, so some of the speed issues may be mitigated when this
gets implemented there.

- Dan

On Thu, Jan 26, 2017 at 3:47 PM, Jon Paris <jon.paris@xxxxxxxxxxxxxx> wrote:

Two options - one could be really fast to implement.

Fast implementation (one or possibly two line code change):

Use my Open Access handler and plug it in to the program that currently
proceeds the disk file. It will write the IFS file directly with no need
for the CPYTOIMPF. Details and download here:
http://www.ibmsystemsmag.com/ibmi/developer/rpg/Getting-a-
Handle-on-RPG%E2%80%99s-Open-Access/?page=3 the relevant part is page 3
onwards. AN updated version of that article - which describes the template
I use for all OA programs - is here: http://www.ibmsystemsmag.com/
ibmi/developer/rpg/open-access-templates/ and is an enhanced version of
the earlier program.

Slower implementation but fastest operation:

Use the IFS APIs directly. Scott has written on this as have I. In fact if
you study the code in the handler referenced above it actually uses the IFS
APIs to do the job.


Jon Paris

www.partner400.com
www.SystemiDeveloper.com

On Jan 26, 2017, at 12:45 PM, Dan <dan27649@xxxxxxxxx> wrote:

We have a process that copies 16.7M records from one native table to
another in 13 minutes, then use CPYTOIMPF to convert this to a .CSV file,
which takes 1 hour, 39 minutes. The command:
CpyToImpF FromFile(CSVWRK/&TC_IQXnnnH) +
ToStmF( '/csvwrk/' *cat &FileNamNoX *tcat '.csv' ) +
MbrOpt( *Replace ) StmfCCSID( *PCASCII ) +
RcdDlm( *CRLF ) RmvBlank( *Both ) +
OrderBy( IQXSequenc )

FWIW, this is on V7R1 with recent PTFs and TRs. The job shows the input
file is being read in 117-record blocks. Record length is 239 bytes with
29 fields. I also tested the above command without the OrderBy

parameter,

but difference in time to complete was insignificant (2 minutes

difference).

I thought I had seen a recent thread that claimed that one of Scott
Klement's utilities performed faster than CPYTO???F (can't remember if it
was CPYTOIMPF or CPYTOSTMF), but I came up empty searching for that
thread. Does anyone remember, or was I imagining things?

- Dan
--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related

questions.

Help support midrange.com by shopping at amazon.com with our affiliate

link: http://amzn.to/2dEadiD

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: http://amzn.to/2dEadiD

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related questions.

Help support midrange.com by shopping at amazon.com with our affiliate link: http://amzn.to/2dEadiD