RE: Import taking over 24 hours -- RPG400-L

Option 2, below is critical if using CPYTO/FRMIMPF. It tends to help speed
things up tremendously. 

-Bob Cozzi
www.RPGxTools.com
If everything is under control, you are going too slow.
- Mario Andretti


-----Original Message-----
From: rpg400-l-bounces@xxxxxxxxxxxx [mailto:rpg400-l-bounces@xxxxxxxxxxxx]
On Behalf Of Wilt, Charles
Sent: Monday, May 09, 2005 11:28 AM
To: RPG programming on the AS400 / iSeries
Subject: RE: Import taking over 24 hours

There's been some good ideas...

Here's a couple more...

1) stop journaling

2) Parallelism (particularly if you have more than 1 CPU), if using RPG
submit X number of jobs that process 1/X of the rows, idea being you want
CPU usage to be a close to 100% as possible.  If using CPYFRMIMPF, make sure
that DEGREE(*NBRTASKS) is set with CHGQRYA and that the DB2 SMP license
program is installed.


HTH,

Charles Wilt
iSeries Systems Administrator / Developer
Mitsubishi Electric Automotive America
ph: 513-573-4343
fax: 513-398-1121
 

> -----Original Message-----
> From: rpg400-l-bounces@xxxxxxxxxxxx
> [mailto:rpg400-l-bounces@xxxxxxxxxxxx]On Behalf Of dpalme@xxxxxxxxxxx
> Sent: Monday, May 09, 2005 9:44 AM
> To: RPG Group
> Subject: Import taking over 24 hours
> 
> 
> We have been working on an import situation for a couple of 
> months now, and 
> I would like to get some feedback from some of you on this.  
> We have a 
> RS/6000 which is running some older cobol programs, we are 
> not in a position 
> to replace those programs at this time, what we need to do is 
> import that 
> data (from standard unix cobol files ACUCOBOL to be exact) 
> into several 
> physcial files on our 9406-170.  
> 
> We are using MS Access (which I hate but it serves its 
> purpose here) to dump 
> the data from the cobol files into tab delimited files which 
> are then ftp'd 
> over to the 400 and then imported into the physical files.
> 
> Everything does work but it is taking around 28 hours for the 
> import process 
> to complete.
> 
> The files can be broken into three groups, customer master 
> data - which 
> takes about 20 minutes to complete....the flat file is 
> approximately 8.5mb 
> in size.  We have several line item files which are imported 
> into one single 
> physical file, the flat files for this group are 
> approximately 400mb.  The 
> third and final file is a header file which is approx 150mb in size.
> 
> We had an outside vendor actually write the code for us which 
> as I said 
> before does work, but it is taking a very long time to process.   
> 
> If anyone has any ideas or thoughts on how we can speed this 
> process up I 
> would love to hear those ideas.  I need to check with the 
> legal department 
> before posting the code since I am not sure what contractual 
> obligations we 
> are under as far as the code goes.
> 
> I do know that he is splitting the fields up by checking for 
> the tab and 
> then writing the data to the file. the header file has quite 
> a few fields in 
> it, and if anyone wants to know the number of fields I can 
> verify that and 
> re-post.
> -- 
> This is the RPG programming on the AS400 / iSeries (RPG400-L) 
> mailing list
> To post a message email: RPG400-L@xxxxxxxxxxxx
> To subscribe, unsubscribe, or change list options,
> visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
> or email: RPG400-L-request@xxxxxxxxxxxx
> Before posting, please take a moment to review the archives
> at http://archive.midrange.com/rpg400-l.
> 
> 

-- 
This is the RPG programming on the AS400 / iSeries (RPG400-L) mailing list
To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.