Re: Import taking over 24 hours -- RPG400-L

Hi there

Have you considered using the client access upload function ? Going by the performance of the download function compared to other options this may well be worth a try. What I am thinking you may be able to do is:

1. Create a file layout on the iSeries for each of the access tables

2. Run at least one download for each file and save the FDF file when prompted (it''s used by the upload facility to map data I think). You should download the file in the same format as the source data you expect to use when performing the upload. That is, if your source file at the PC end that you are going to upload is a tab delimitted file download a tab delimitted file to get the appropiate FDF file with the target physical file mappings. If you can't download a tab delimitted file then you might want to try downloading a CSV and change your export format at the Access end to match what you downloaded in.

3. Use the upload facility to upload from the Access export file into either your final target file (scary !) or into an interim file that has the same structure as the Access tables then use an RPG to convert the data or other wise process it without having to deal with the conversion aspect.

I have seen this kind of thing a few times and the Upload and Download tools seem to be pretty efficient (and underrated) compared to the other methods I've encountered.

Hope this helps

Regards
Evan Harris

At 01:43 a.m. 10/05/2005, you wrote:

We have been working on an import situation for a couple of months now, and
I would like to get some feedback from some of you on this.  We have a
RS/6000 which is running some older cobol programs, we are not in a position
to replace those programs at this time, what we need to do is import that
data (from standard unix cobol files ACUCOBOL to be exact) into several
physcial files on our 9406-170.

We are using MS Access (which I hate but it serves its purpose here) to dump
the data from the cobol files into tab delimited files which are then ftp'd
over to the 400 and then imported into the physical files.

Everything does work but it is taking around 28 hours for the import process
to complete.

The files can be broken into three groups, customer master data - which
takes about 20 minutes to complete....the flat file is approximately 8.5mb
in size.  We have several line item files which are imported into one single
physical file, the flat files for this group are approximately 400mb.  The
third and final file is a header file which is approx 150mb in size.

We had an outside vendor actually write the code for us which as I said
before does work, but it is taking a very long time to process.

If anyone has any ideas or thoughts on how we can speed this process up I
would love to hear those ideas.  I need to check with the legal department
before posting the code since I am not sure what contractual obligations we
are under as far as the code goes.

I do know that he is splitting the fields up by checking for the tab and
then writing the data to the file. the header file has quite a few fields in
it, and if anyone wants to know the number of fields I can verify that and
re-post.