RE: Import taking over 24 hours -- RPG400-L

Only one processor at this point.  We will have at least two the next time 
around (2 years)


On Mon, 9 May 2005 11:55:44 -0500, Bob Cozzi wrote
> Of course, now that I said that I recall him mentioning it is a Model
> 170--probably isn't have an n-way system. :)
> 
> -Bob Cozzi
> www.RPGxTools.com
> If everything is under control, you are going too slow.
> - Mario Andretti
> 
> -----Original Message-----
> From: rpg400-l-bounces@xxxxxxxxxxxx [mailto:rpg400-l-bounces@xxxxxxxxxxxx]
> On Behalf Of Bob Cozzi
> Sent: Monday, May 09, 2005 11:50 AM
> To: 'RPG programming on the AS400 / iSeries'
> Subject: RE: Import taking over 24 hours
> 
> Option 2, below is critical if using CPYTO/FRMIMPF. It tends to help 
> speed things up tremendously.
> 
> -Bob Cozzi
> www.RPGxTools.com
> If everything is under control, you are going too slow.
> - Mario Andretti
> 
> -----Original Message-----
> From: rpg400-l-bounces@xxxxxxxxxxxx [mailto:rpg400-l-bounces@xxxxxxxxxxxx]
> On Behalf Of Wilt, Charles
> Sent: Monday, May 09, 2005 11:28 AM
> To: RPG programming on the AS400 / iSeries
> Subject: RE: Import taking over 24 hours
> 
> There's been some good ideas...
> 
> Here's a couple more...
> 
> 1) stop journaling
> 
> 2) Parallelism (particularly if you have more than 1 CPU), if using RPG
> submit X number of jobs that process 1/X of the rows, idea being you 
> want CPU usage to be a close to 100% as possible.  If using 
> CPYFRMIMPF, make sure that DEGREE(*NBRTASKS) is set with CHGQRYA and 
> that the DB2 SMP license program is installed.
> 
> HTH,
> 
> Charles Wilt
> iSeries Systems Administrator / Developer
> Mitsubishi Electric Automotive America
> ph: 513-573-4343
> fax: 513-398-1121
> 
> > -----Original Message-----
> > From: rpg400-l-bounces@xxxxxxxxxxxx
> > [mailto:rpg400-l-bounces@xxxxxxxxxxxx]On Behalf Of dpalme@xxxxxxxxxxx
> > Sent: Monday, May 09, 2005 9:44 AM
> > To: RPG Group
> > Subject: Import taking over 24 hours
> > 
> > 
> > We have been working on an import situation for a couple of 
> > months now, and 
> > I would like to get some feedback from some of you on this.  
> > We have a 
> > RS/6000 which is running some older cobol programs, we are 
> > not in a position 
> > to replace those programs at this time, what we need to do is 
> > import that 
> > data (from standard unix cobol files ACUCOBOL to be exact) 
> > into several 
> > physcial files on our 9406-170.  
> > 
> > We are using MS Access (which I hate but it serves its 
> > purpose here) to dump 
> > the data from the cobol files into tab delimited files which 
> > are then ftp'd 
> > over to the 400 and then imported into the physical files.
> > 
> > Everything does work but it is taking around 28 hours for the 
> > import process 
> > to complete.
> > 
> > The files can be broken into three groups, customer master 
> > data - which 
> > takes about 20 minutes to complete....the flat file is 
> > approximately 8.5mb 
> > in size.  We have several line item files which are imported 
> > into one single 
> > physical file, the flat files for this group are 
> > approximately 400mb.  The 
> > third and final file is a header file which is approx 150mb in size.
> > 
> > We had an outside vendor actually write the code for us which 
> > as I said 
> > before does work, but it is taking a very long time to process.   
> > 
> > If anyone has any ideas or thoughts on how we can speed this 
> > process up I 
> > would love to hear those ideas.  I need to check with the 
> > legal department 
> > before posting the code since I am not sure what contractual 
> > obligations we 
> > are under as far as the code goes.
> > 
> > I do know that he is splitting the fields up by checking for 
> > the tab and 
> > then writing the data to the file. the header file has quite 
> > a few fields in 
> > it, and if anyone wants to know the number of fields I can 
> > verify that and 
> > re-post.
> > -- 
> > This is the RPG programming on the AS400 / iSeries (RPG400-L) 
> > mailing list
> > To post a message email: RPG400-L@xxxxxxxxxxxx
> > To subscribe, unsubscribe, or change list options,
> > visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
> > or email: RPG400-L-request@xxxxxxxxxxxx
> > Before posting, please take a moment to review the archives
> > at http://archive.midrange.com/rpg400-l.
> > 
> >
> 
> -- 
> This is the RPG programming on the AS400 / iSeries (RPG400-L)
>  mailing list To post a message email: RPG400-L@xxxxxxxxxxxx To 
> subscribe, unsubscribe, or change list options, visit: 
> http://lists.midrange.com/mailman/listinfo/rpg400-l or email: RPG400-
> L-request@xxxxxxxxxxxx Before posting, please take a moment to 
> review the archives at http://archive.midrange.com/rpg400-l.
> 
> -- 
> This is the RPG programming on the AS400 / iSeries (RPG400-L)
>  mailing list To post a message email: RPG400-L@xxxxxxxxxxxx To 
> subscribe, unsubscribe, or change list options, visit: 
> http://lists.midrange.com/mailman/listinfo/rpg400-l or email: RPG400-
> L-request@xxxxxxxxxxxx Before posting, please take a moment to 
> review the archives at http://archive.midrange.com/rpg400-l.
> 
> -- 
> This is the RPG programming on the AS400 / iSeries (RPG400-L)
>  mailing list To post a message email: RPG400-L@xxxxxxxxxxxx To 
> subscribe, unsubscribe, or change list options, visit: 
> http://lists.midrange.com/mailman/listinfo/rpg400-l or email: RPG400-
> L-request@xxxxxxxxxxxx Before posting, please take a moment to 
> review the archives at http://archive.midrange.com/rpg400-l.


--
Open WebMail Project (http://openwebmail.org)