Re: Import taking over 24 hours -- RPG400-L

It doesn't appear to be the FTP routine that is taking all the time, it 
seems to be the parsing of the data that is slowing things down.


On Mon, 9 May 2005 11:34:08 -0700, Tony Carolla wrote
> I don't know what takes the most time, but have you tried using 
> iSeries access' "Transfer data to iSeries" function after you have 
> the .csv built? In my experience, this always performs better than 
> CPY...IMPF. And it can be "automated" on a PC, if this is a 
> regularly scheduled thing.
> 
> On 5/9/05, dpalme@xxxxxxxxxxx <dpalme@xxxxxxxxxxx> wrote:
> > 
> > We have been working on an import situation for a couple of months now, 
> > and
> > I would like to get some feedback from some of you on this. We have a
> > RS/6000 which is running some older cobol programs, we are not in a 
> > position
> > to replace those programs at this time, what we need to do is import that
> > data (from standard unix cobol files ACUCOBOL to be exact) into several
> > physcial files on our 9406-170.
> > 
> > We are using MS Access (which I hate but it serves its purpose here) to 
> > dump
> > the data from the cobol files into tab delimited files which are then 
> > ftp'd
> > over to the 400 and then imported into the physical files.
> > 
> > Everything does work but it is taking around 28 hours for the import 
> > process
> > to complete.
> > 
> > The files can be broken into three groups, customer master data - which
> > takes about 20 minutes to complete....the flat file is approximately 
8.5mb
> > in size. We have several line item files which are imported into one 
> > single
> > physical file, the flat files for this group are approximately 400mb. The
> > third and final file is a header file which is approx 150mb in size.
> > 
> > We had an outside vendor actually write the code for us which as I said
> > before does work, but it is taking a very long time to process.
> > 
> > If anyone has any ideas or thoughts on how we can speed this process up I
> > would love to hear those ideas. I need to check with the legal department
> > before posting the code since I am not sure what contractual obligations 
> > we
> > are under as far as the code goes.
> > 
> > I do know that he is splitting the fields up by checking for the tab and
> > then writing the data to the file. the header file has quite a few 
fields 
> > in
> > it, and if anyone wants to know the number of fields I can verify that 
and
> > re-post.
> > --
> > This is the RPG programming on the AS400 / iSeries (RPG400-L) mailing 
list
> > To post a message email: RPG400-L@xxxxxxxxxxxx
> > To subscribe, unsubscribe, or change list options,
> > visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
> > or email: RPG400-L-request@xxxxxxxxxxxx
> > Before posting, please take a moment to review the archives
> > at http://archive.midrange.com/rpg400-l.
> > 
> >
> 
> -- 
> "Enter any 11-digit prime number to continue..."
> "In Hebrew SQL, how do you use right() and left()?..." - Random Thought
> -- 
> This is the RPG programming on the AS400 / iSeries (RPG400-L)
>  mailing list To post a message email: RPG400-L@xxxxxxxxxxxx To 
> subscribe, unsubscribe, or change list options, visit: 
> http://lists.midrange.com/mailman/listinfo/rpg400-l or email: RPG400-
> L-request@xxxxxxxxxxxx Before posting, please take a moment to 
> review the archives at http://archive.midrange.com/rpg400-l.


--
Open WebMail Project (http://openwebmail.org)