Re: Import taking over 24 hours

It doesn't appear to be the FTP routine that is taking all the time, it 
seems to be the parsing of the data that is slowing things down.


On Mon, 9 May 2005 11:34:08 -0700, Tony Carolla wrote
> I don't know what takes the most time, but have you tried using 
> iSeries access' "Transfer data to iSeries" function after you have 
> the .csv built? In my experience, this always performs better than 
> CPY...IMPF. And it can be "automated" on a PC, if this is a 
> regularly scheduled thing.
> 
> On 5/9/05, dpalme@xxxxxxxxxxx <dpalme@xxxxxxxxxxx> wrote:
> > 
> > We have been working on an import situation for a couple of months now, 
> > and
> > I would like to get some feedback from some of you on this. We have a
> > RS/6000 which is running some older cobol programs, we are not in a 
> > position
> > to replace those programs at this time, what we need to do is import that
> > data (from standard unix cobol files ACUCOBOL to be exact) into several
> > physcial files on our 9406-170.
> > 
> > We are using MS Access (which I hate but it serves its purpose here) to 
> > dump
> > the data from the cobol files into tab delimited files which are then 
> > ftp'd
> > over to the 400 and then imported into the physical files.
> > 
> > Everything does work but it is taking around 28 hours for the import 
> > process
> > to complete.
> > 
> > The files can be broken into three groups, customer master data - which
> > takes about 20 minutes to complete....the flat file is approximately 
8.5mb
> > in size. We have several line item files which are imported into one 
> > single
> > physical file, the flat files for this group are approximately 400mb. The
> > third and final file is a header file which is approx 150mb in size.
> > 
> > We had an outside vendor actually write the code for us which as I said
> > before does work, but it is taking a very long time to process.
> > 
> > If anyone has any ideas or thoughts on how we can speed this process up I
> > would love to hear those ideas. I need to check with the legal department
> > before posting the code since I am not sure what contractual obligations 
> > we
> > are under as far as the code goes.
> > 
> > I do know that he is splitting the fields up by checking for the tab and
> > then writing the data to the file. the header file has quite a few 
fields 
> > in
> > it, and if anyone wants to know the number of fields I can verify that 
and
> > re-post.
> > --
> > This is the RPG programming on the AS400 / iSeries (RPG400-L) mailing 
list
> > To post a message email: RPG400-L@xxxxxxxxxxxx
> > To subscribe, unsubscribe, or change list options,
> > visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
> > or email: RPG400-L-request@xxxxxxxxxxxx
> > Before posting, please take a moment to review the archives
> > at http://archive.midrange.com/rpg400-l.
> > 
> >
> 
> -- 
> "Enter any 11-digit prime number to continue..."
> "In Hebrew SQL, how do you use right() and left()?..." - Random Thought
> -- 
> This is the RPG programming on the AS400 / iSeries (RPG400-L)
>  mailing list To post a message email: RPG400-L@xxxxxxxxxxxx To 
> subscribe, unsubscribe, or change list options, visit: 
> http://lists.midrange.com/mailman/listinfo/rpg400-l or email: RPG400-
> L-request@xxxxxxxxxxxx Before posting, please take a moment to 
> review the archives at http://archive.midrange.com/rpg400-l.


--
Open WebMail Project (http://openwebmail.org)