× The internal search function is temporarily non-functional. The current search engine is no longer viable and we are researching alternatives.
As a stop gap measure, we are using Google's custom search engine service.
If you know of an easy to use, open source, search engine ... please contact support@midrange.com.



I'm only processing a subset of the data. Therefore, positioning by key
is necessary. It's a very large file....

For Example:

Was:
Key             Data

1               aa
1               bb
2               cc
2               dd
3               ee
3               ff

After Change 2 to 4

1               aa
1               bb
3               ee
3               ff
4               cc
4               dd

This is the purpose of the run. For each key, there may be 500,000
records.

Thanks again for the input

Jim Wiant


Once a job is fouled up, anything done to improve it makes it worse. 
Finagle 


-----Original Message-----
From: midrange-l-bounces@xxxxxxxxxxxx
[mailto:midrange-l-bounces@xxxxxxxxxxxx] On Behalf Of Booth Martin
Sent: Thursday, 24 November 2005 11:40
To: Midrange Systems Technical Discussion
Subject: Re: Optimizing File I-O in RPG IV/I-Series - continued


If there is any way you can take the k off the F-spec it will also juice
up
performance.  If you are processing every record anyway, do you care
what
order they are processed in? 

 

---------------------------------

Booth Martin

http://www.martinvt.com

---------------------------------

-------Original Message-------

 

From: Midrange Systems Technical Discussion

Date: 11/23/05 15:46:07

To: Midrange Systems Technical Discussion

Subject: Re: Optimizing File I-O in RPG IV/I-Series - continued

 

If you weren't doing a SETLL to something other than *LOVAL, and you
were

reading every record in the file, I'd suggest trying an Update Primary

file.  RPG automatically blocks these very nicely (and even says so in
an

informational compiler message at the bottom of the compile listing.  I

had some old time trials that suggest that these run like greased

lightning.

 

Rob Berendt

--

Group Dekko Services, LLC

Dept 01.073

PO Box 2000

Dock 108

6928N 400E

Kendallville, IN 46755

http://www.dekko.com

 

 

 

 

 

"Jim Wiant" <Jim.Wiant@xxxxxxxxxxxxxxxx>

Sent by: midrange-l-bounces@xxxxxxxxxxxx

11/23/2005 04:12 PM

Please respond to

Midrange Systems Technical Discussion <midrange-l@xxxxxxxxxxxx>

 

 

To

<midrange-l@xxxxxxxxxxxx>

cc

 

Fax to

 

Subject

Optimizing File I-O in RPG IV/I-Series - continued

 

 

 

 

 

 

Thanks everyone for all the input!

 

To answer a few questions:

                 No, no file triggers in this case.

                 No, no great number of logicals. In most cases with the

files

I'm concerned with, there is only 1 - occasionally 2

                 I will look forward in the future to trying imbedded
SQL

for

this kind of work, because I am indeed changing every occurrence of

DATA1 to DATA2.

                                 there will be no exceptions. Right now,

time prohibits

changing the programs to do that. Perhaps for next time...

 

I tested the READE vs READ idea on a rather large dataset. In this case,

the field being updated is also the primary key. Therefore, I could use

a READ to know when I'd reached the end of the data I wanted to change.

 

Working with 249,271 records, the READE loop changed those records in 70

seconds (processor time).

Working with the same number of records, using READ but leaving BLOCK as

it's default - 72 seconds

Working with the same number of records, using READ and adding

BLOCK(*NO) - 72 seconds

 

I ran these benchmarks on our development machine, which has a fairly

constant load. I was surprised READ didn't show me any advantage, even

though many of you suggested that it would.

 

Therefore, it looks like removing old wood from these files is the best

way to boost the speed of the process, followed by parallel-processing a

few of the updates.

 

Once again, thank you for all the ideas and if I get time to play with

it more I'll try to post another thread.

 

'Best wishes

 

Jim Wiant

 

 

 

 

Once a job is fouled up, anything done to improve it makes it worse.

Finagle

 

 

This message has been sent from Foodstuffs (Auckland) Limited

("Foodstuffs").

 

The information contained in this message and or attachments

is intended only for the person or entity to which it is

addressed and may contain confidential and/or privileged

material. Any review, retransmission, dissemination or other

use of, or taking of any action in reliance upon, this

information by persons or entities other than the intended

recipient is prohibited. If you received this in error,

please contact the sender and delete the material from any

system and destroy any copies.

 

The views and opinions expressed in this message may be those

of the individual and not necessarily those of Foodstuffs,

and are not given or endorsed by it.

 

Please note that this communication does not designate an

information system for the purposes of the Electronic

Transactions Act 2002.

--

This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxx

To subscribe, unsubscribe, or change list options,

visit: http://lists.midrange.com/mailman/listinfo/midrange-l

or email: MIDRANGE-L-request@xxxxxxxxxxxx

Before posting, please take a moment to review the archives

at http://archive.midrange.com/midrange-l.

 

 

--

This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing
list

To post a message email: MIDRANGE-L@xxxxxxxxxxxx

To subscribe, unsubscribe, or change list options,

visit: http://lists.midrange.com/mailman/listinfo/midrange-l

or email: MIDRANGE-L-request@xxxxxxxxxxxx

Before posting, please take a moment to review the archives

at http://archive.midrange.com/midrange-l.

 

.
--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing
list
To post a message email: MIDRANGE-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/midrange-l.

######################################################################
This message was scanned for compliance with Foodstuffs email policies
######################################################################
This message has been sent from Foodstuffs (Auckland) Limited ("Foodstuffs").

The information contained in this message and or attachments 
is intended only for the person or entity to which it is 
addressed and may contain confidential and/or privileged 
material. Any review, retransmission, dissemination or other 
use of, or taking of any action in reliance upon, this 
information by persons or entities other than the intended 
recipient is prohibited. If you received this in error, 
please contact the sender and delete the material from any 
system and destroy any copies.

The views and opinions expressed in this message may be those
of the individual and not necessarily those of Foodstuffs, 
and are not given or endorsed by it.

Please note that this communication does not designate an 
information system for the purposes of the Electronic 
Transactions Act 2002.


As an Amazon Associate we earn from qualifying purchases.

This thread ...


Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.