Re: Suggestion for improving loop performance

Thanks Chuck/Scott,

I have attached the sample data, after I bring all these records into 1
records.. I have one more operation to do, move this data to DDS file hence
need to have at the least number of fields matching for each record else
would not be able to copy to DDS file.

So all these 16 records will make up a single record in my new file and
then I will do copy to DDS file.

Thanks,
Ashwani Singh
Bank Solutions
Fiserv
Office: +1 – 407 – 357 - 4550
Mobile: +1- 407 – 416 - 3918
Toll Free (Office): +1-811—407 -357-4550
www.fiserv.com

FORTUNE® Magazine’s 2014 World's Most Admired Companies
Facebook: Like Fiserv · Twitter: Follow @Fiserv · LinkedIn: Connect
Fiserv • Careers: Join Fiserv

Please consider the environment before printing this e-mail


-----Original Message-----
From: RPG400-L [mailto:rpg400-l-bounces@xxxxxxxxxxxx] On Behalf Of CRPence
Sent: Tuesday, May 19, 2015 3:41 PM
To: rpg400-l@xxxxxxxxxxxx
Subject: Re: Suggestion for improving loop performance

On 19-May-2015 13:49 -0500, Singh, Ashwani (Lake Mary) wrote:

On 19-May-2015 13:17 -0500, Singh, Ashwani (Lake Mary) wrote:

We are trying to process a 69 million records file which is taking
5 hours plus to load into another file. (We have Block *Yes specified
for the input file (with 69M records) We need to bring the processing
down for sure..

We are trying for the multithread as well..

There are two crucial loops that run in this single program and
causing trouble.

The first loop takes round about .6ms/rec <ed: corrected from
seconds/rec> processed the other one .4ms <ed: corrected from
seconds>.

Our file is a flat file and is | delimited ..

Loop 1:

D ARRAY1 S 5 0 DIM(999)
C '|' SCAN DATA ARRAY1
C Eval X= 1
C Eval A= 0
C Dow ARRAY1(X)> 0
C Eval A+=1
C Eval X+=1
C ENDDO

Loop2:

C DOW A < #Pipesreq
C* <ed: DOW is for the> (Number of pipes actually required) C Eval
Fileds = %subst(fileds:1:%checkr(' ':Fileds)) + '|'
C eval A+=1
C ENDDO

Any suggestion will be highly appreciated..

<<SNIP>>
This is what I am trying to do -

We have one control record running into multiple records of flat file
i.e.

X:01 A|B|C
X:02 D|E|F
X:03 G|H|I

X:01 1|2|3|4|5
X:02 ......


And I know that this record X:01 should contain 5 fields but have 2
missing and rest all are fine.

The output becomes like this

X: A|B|C||D|E|E|G|H|I

Perhaps instead, output should be?:

X: A|B|C|||D|E|F|G|H|I

This whole process is repeated like this...

<<SNIP>>

Suggestion: Do not interrogate how much work is to be done before
actually doing the work; much time can be saved, because the pre-processing
that inherently requires two passes is eliminated, so the data need be
processed just once.

In other words: drop the upfront SCAN, and just start parsing the data.
When the EOR occurs before all elements are accounted, then instead of
tacking on the missing delimiters, know that the missing elements are
representing effectively a NULL value, and process them as such.

FWiW: If the problem scenario is described explicitly as an example
with the setup creating the object(s) and input of some sample data [a
rendition of actual data or facsimile], then the textual descriptions of
the problem such as "this record X:01" are either eliminated as confusing
as they are, or they take on some actual meaning. A reviewer might see an
alternate approach for the problem scenario, rather than concentrating on
the minutiae of two specific loops... which as I allude, may not be worth
keeping at all.

--
Regards, Chuck

--
This is the RPG programming on the IBM i (AS/400 and iSeries) (RPG400-L)
mailing list To post a message email: RPG400-L@xxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives at
http://archive.midrange.com/rpg400-l.


--
This is the RPG programming on the IBM i (AS/400 and iSeries) (RPG400-L)
mailing list
To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.