RE: Import taking over 24 hours

Evan,
Thanks, we had not thought of that and will try it; hopefully at some
point we can automate this process to make it simpler.



-----Original Message-----
From: rpg400-l-bounces@xxxxxxxxxxxx
[mailto:rpg400-l-bounces@xxxxxxxxxxxx] On Behalf Of Evan Harris
Sent: Wednesday, May 11, 2005 3:57 AM
To: RPG programming on the AS400 / iSeries
Subject: Re: Import taking over 24 hours


Hi there

Have you considered using the client access upload function ? Going by
the 
performance of the download function compared to other options this may 
well be worth a try. What I am thinking you may be able to do is:

1. Create a file layout on the iSeries for each of the access tables

2. Run at least one download for each file and save the FDF file when 
prompted (it''s used by the upload facility to map data I think). You 
should download the file in the same format as the source data you
expect 
to use when performing the upload. That is, if your source file at the
PC 
end that you are going to upload is a tab delimitted file download a tab

delimitted file to get the appropiate FDF file with the target physical 
file mappings. If you can't download a tab delimitted file then you
might 
want to try downloading a CSV and change your export format at the
Access 
end to match what you downloaded in.

3. Use the upload facility to upload from the Access export file into 
either your final target file (scary !) or into an interim file that has

the same structure as the Access tables then use an RPG to convert the
data 
or other wise process it without having to deal with the conversion
aspect.

I have seen this kind of thing a few times and the Upload and Download 
tools seem to be pretty efficient (and underrated) compared to the other

methods I've encountered.

Hope this helps

Regards
Evan Harris

At 01:43 a.m. 10/05/2005, you wrote:
>We have been working on an import situation for a couple of months now,

>and I would like to get some feedback from some of you on this.  We 
>have a RS/6000 which is running some older cobol programs, we are not 
>in a position to replace those programs at this time, what we need to 
>do is import that data (from standard unix cobol files ACUCOBOL to be 
>exact) into several physcial files on our 9406-170.
>
>We are using MS Access (which I hate but it serves its purpose here) to

>dump the data from the cobol files into tab delimited files which are 
>then ftp'd over to the 400 and then imported into the physical files.
>
>Everything does work but it is taking around 28 hours for the import 
>process to complete.
>
>The files can be broken into three groups, customer master data - which

>takes about 20 minutes to complete....the flat file is approximately 
>8.5mb in size.  We have several line item files which are imported into

>one single physical file, the flat files for this group are 
>approximately 400mb.  The third and final file is a header file which 
>is approx 150mb in size.
>
>We had an outside vendor actually write the code for us which as I said

>before does work, but it is taking a very long time to process.
>
>If anyone has any ideas or thoughts on how we can speed this process up

>I would love to hear those ideas.  I need to check with the legal 
>department before posting the code since I am not sure what contractual

>obligations we are under as far as the code goes.
>
>I do know that he is splitting the fields up by checking for the tab 
>and then writing the data to the file. the header file has quite a few 
>fields in it, and if anyone wants to know the number of fields I can 
>verify that and re-post.

-- 
This is the RPG programming on the AS400 / iSeries (RPG400-L) mailing
list To post a message email: RPG400-L@xxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.