Use FTP's module or freeware (not CPY*) for EBCDIC to/from ASCII

[craigs@xxxxxxxxx]

Anyone know of some freeware utilities to replace the inefficient CPY*
commands?
I have found FTP's get/put to perform dramatically better in the EBCDIC
to/from ASCII conversion than CPY, CPYFRMSTMF, CPYTOSTMF, CPYFRMIMPF, or
CPYTOIMPF.  It uses 3% of the CPU instead of all (up to 100%) and takes
only a small fraction of the time of the CPY* commands.  Using FTP seems
like a no-brainer.  Unfortunately, FTP authority in batch is a problem for
us as the generic user profile we use does not have *ALLOBJ authority.  I
think the administrator is worried about one of us programmers getting the
password to gain *ALLOBJ authority (which seems a valid concern).
Now, for an experiment with real stats and a real world example.
Hypothesis: It seems like the CPY* commands could be rewritten by IBM to
use the same EBCDIC to/from ASCII module as FTP.  Or, other utilities could
be used that are more efficient.
Details: The below stats are from a real world example.  The example is
taking a .CSV file in the IFS and copying it to a database file with
fields.  Then, converting that to a flat file as XML for pushing into EDI
software.
For all CPY* examples, 85% CPU is used as the average of free resources.  A
dedicated system would probably take 100% of the CPU.  Tests were repeated
over and over.
1. CPYFRMSTMF with 1640 records about 150 in length from .CSV to flat file
(no DDS with 2000 record length).  half a second.
2. CPYFRMIMPF from flat file (or IFS .CSV file) to external file (with
DDS).  2 seconds.
3. *Just for fun* CPYTOIMPF from external file (with DDS) to .CSV file.  2
seconds.
3b. *Just for fun* Same as 3 except use SQL2CSV instead of CPYTOIMPF.  half
a second and 5% of CPU.
4. Use iconv() in RPG to convert to ASCII from user space and then write()
user space to IFS after XML conversion.  1 second and 3% of CPU.
5. *Just for fun* Same as 4 except write() to flat file instead of IFS
file.  25 seconds.
6. *Just for fun* Same as 4 except do not use iconv() (let write() with
O_TEXTDATA do conversion) to IFS.  1 second and 3% of CPU.
7. CPYFRMSTMF 14,711 records in IFS as XML to flat file.  4 seconds.
8. *Just for fun* CPYFRMSTMF 14,711 recods in IFS as XML to flat file with
no ASCII to EBCDIC.  Almost no time.
9. *Just for fun* Instead of 7, FTP in batch with GET from IFS as XML to
flat file (includes signon).  1 second and 3% of CPU.

So, using steps 1,2,4, and 7 takes about 7.5 seconds with about 6.5 of
those seconds taking all remaining CPU up to 100% of the CPU.  Using FTP
instead of CPYFRMSTMF would cut the time down about 3.5 seconds.  So, only
2 seconds would be used with 100% of the CPU instead of 6.5 seconds.
Notes: A freely available utility was used for #4 above called SQL2XML and
the fun #3b above called SQL2CSV.
http://www.mcpressonline.com/mc/1@xxxxxxxxxxxxxxxxx@.6aed3f76

Conclusion: There are more efficient ways of converting EBCDIC to/from
ASCII than using the CPY* commands.
Again, anyone know of any freeware available that does the same things as
the CPY* commands but is efficient like FTP?  I already see SQL2CSV to
replace CPYTOIMPF.

Thanks!
Craig Strong