RE: Keyed file question -- RPG400-L

Let me put this another way. I was using SQL Server to run a query. It was sorted on, let's say, fields A and B. Then I FTP'ed it to an iSeries file. The file I FTP'ed it to had just one field per record. I run a program to parse the data out into individual fields in a second file. This second file is keyed, with key fields C and D. Then I run a second program that updates our database with this data. Since this second program identifies the second file as keyed, I was expecting it to read the file in the "C and D" sorting sequence, even though I was using a READ statement instead of a SETLL and READE (which really shouldn't matter and didn't in this case). And this is exactly what it did.

What it ALSO did was still preserve my "A and B" sequence from the original SQL Server query. This is because within a group of records that had the same "C and D" values, it still sorted in my "A and B" sequence as well. My guess is this is because in a keyed file with key fields with the same values, beyond that records are just sorted in arrival sequence, which in my case happened to be what I wanted.

Hopefully I haven't completely confused everyone. Thanks again for all the quick replies!

Bill

-----Original Message-----
From: rpg400-l-bounces@xxxxxxxxxxxx [mailto:rpg400-l-bounces@xxxxxxxxxxxx] On Behalf Of rpg400-l-request@xxxxxxxxxxxx
Sent: Thursday, August 23, 2012 8:28 AM
To: rpg400-l@xxxxxxxxxxxx
Subject: RPG400-L Digest, Vol 11, Issue 476

Send RPG400-L mailing list submissions to
rpg400-l@xxxxxxxxxxxx

To subscribe or unsubscribe via the World Wide Web, visit
http://lists.midrange.com/mailman/listinfo/rpg400-l
or, via email, send a message with subject or body 'help' to
rpg400-l-request@xxxxxxxxxxxx

You can reach the person managing the list at
rpg400-l-owner@xxxxxxxxxxxx

When replying, please edit your Subject line so it is more specific
than "Re: Contents of RPG400-L digest..."

*** NOTE: When replying to this digest message, PLEASE remove all text unrelated to your reply and change the subject line so it is meaningful.

Today's Topics:

1. RE: Keyed file question (William Howie)
2. Re: Keyed file question (Simmons, Ian)
3. RE: Converting Case with a Varying Field (DeLong, Eric)
4. RE: Keyed file question (KBushard@xxxxxxxxxxxxxxxx)
5. Re: Converting Case with a Varying Field (sjl)
6. Re: Converting Case with a Varying Field (Dennis)

----------------------------------------------------------------------

message: 1
date: Wed, 22 Aug 2012 17:05:14 +0000
from: William Howie <William.Howie@xxxxxxxxxxxxxxxx>
subject: RE: Keyed file question

Thanks for all the replies.........

Sorry I should have been more specific. I am loading the file in a different program than I read it. I think what's happening is that the program is reading it in the keyed sequence defined for the file, and it just so happens that the key fields have values such that the sequence I loaded the file in is staying the same. I was expecting it NOT to stay the same because the sequence I was loading it in was different. Hope this makes sense. Thanks!

Bill

________________________________
Confidentiality Notice: This email message is intended only for the individual or entity to which it is addressed. This email may contain information that is proprietary or privileged, confidential and exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you received this email by accident, please notify the sender immediately and destroy this email and all copies of it.

------------------------------

message: 2
date: Wed, 22 Aug 2012 14:57:01 -0400
from: "Simmons, Ian" <Ian.Simmons@xxxxxxxxxxxxxxxxx>
subject: Re: Keyed file question

This is generally a very simple question - I am surprised it's not been cleared up.

Normally, if you read a keyed file back that's been declared in the F specs with a K in the appropriate position then it will always read it back in key sequence, so long as it isn't already open with an override or is being written to using a buffered (blocked) output by a.n. other program.

Having said that, I still don't quite understand your explanation...

Ian
Ian
Sent from Blackberry

----- Original Message -----
From: William Howie [mailto:William.Howie@xxxxxxxxxxxxxxxx]
Sent: Wednesday, August 22, 2012 01:05 PM
To: rpg400-l@xxxxxxxxxxxx <rpg400-l@xxxxxxxxxxxx>
Subject: RE: Keyed file question

Thanks for all the replies.........

Sorry I should have been more specific. I am loading the file in a different program than I read it. I think what's happening is that the program is reading it in the keyed sequence defined for the file, and it just so happens that the key fields have values such that the sequence I loaded the file in is staying the same. I was expecting it NOT to stay the same because the sequence I was loading it in was different. Hope this makes sense. Thanks!

Bill

________________________________
Confidentiality Notice: This email message is intended only for the individual or entity to which it is addressed. This email may contain information that is proprietary or privileged, confidential and exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you received this email by accident, please notify the sender immediately and destroy this email and all copies of it.
--
This is the RPG programming on the IBM i / System i (RPG400-L) mailing list
To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.

Please consider the environment before printing this email.

E-mail messages may contain viruses, worms, or other malicious code. By reading the message and opening any attachments, the recipient accepts full responsibility for taking protective action against such code. Henry Schein is not liable for any loss or damage arising from this message.

The information in this email is confidential and may be legally privileged. It is intended solely for the addressee(s). Access to this e-mail by anyone else is unauthorized.

------------------------------

message: 3
date: Wed, 22 Aug 2012 20:36:24 +0000
from: "DeLong, Eric" <EDeLong@xxxxxxxxxxxxxxx>
subject: RE: Converting Case with a Varying Field

BZZZZTTTT!!! Sorry! I'm definitely NOT in your camp on this...

I'm always careful to examine my code for efficiency, but NEVER at the expense of needless obfuscation and complexity for the programmer. I might have agreed with your statement about 30 years ago, when IBM midrange sported about 16K of memory, and processed at Khz speeds... Today, processing inefficiencies are much less impactful than before. A scarce assortment of high-performance routines might need to be coded for operational efficiency, but that impacts the api design NOT AT ALL.

I'd hazard that, since you wrap job number in a couple of days, that your applications submit MANY small jobs to jobq, for processing in batch. Each job instance requires significant allocation of resources to start up, initialize, execute, and end... A potential performance improvement to your design might be to use a prestart job to service requests from a data queue. This robust approach allows for several scalability improvements, such as a queue monitor which will start up additional PJs to service request backlog... Slick and performance ready!

-Eric DeLong

-----Original Message-----
From: rpg400-l-bounces@xxxxxxxxxxxx [mailto:rpg400-l-bounces@xxxxxxxxxxxx] On Behalf Of Robert Houts
Sent: Tuesday, August 21, 2012 10:02 PM
To: RPG programming on the IBM i / System i
Subject: RE: Converting Case with a Varying Field

Sorry to disagree with you, as I do greatly respect your opinion, but processing efficiency ALWAYS trumps programmer efficiency. Any extra (read unnecessary) code is less efficient. The problem with most programmers (besides laziness) is that they usually don't look at the big picture, which is the code running in many jobs and possibly being called millions of times in a day. That "possibly negligible" amount of extra processing adds up in a hurry. Our production system does a lot of work (we wrap the job number every two to three days), so we need to remove inefficiency wherever we can.

It seems to me that however "very very fast" the bound calls are, calling the API directly is still faster. And that's the bottom line. Teaching programmers to value programmer efficiency over processing efficiency is a huge mistake.

-----Original Message-----
From: rpg400-l-bounces@xxxxxxxxxxxx [mailto:rpg400-l-bounces@xxxxxxxxxxxx] On Behalf Of Barbara Morris
Sent: Tuesday, August 21, 2012 17:25
To: rpg400-l@xxxxxxxxxxxx
Subject: Re: Converting Case with a Varying Field

On 2012/8/21 7:03 PM, Robert Houts wrote:

... It
isn't rocket science and eliminates all overhead of the service
program. I guess that efficient processing is not a concern to you.

The Integrated Language Environment is optimized to make bound calls very very fast, with the intention that programmers can have many small functions that call each other rather than flattening out the functions to reduce the number of calls. So the cost of calling the two wrapper functions could be negligible compared to the actual upper-casing work.

And I think programmer-efficiency is a factor here. Even if you set up the copy file so all the other parameters can be used without further setup, coding the call to the API is still more complex and error-prone than calling the wrapper.

upper_value = CvtToUpper(some_value);

vs

QlgConvertCase(QlgConvertCase_To_Upper
: some_value
: QlgConvertCase_Temp_Output
: QlgConvertCase_Errcode_Give_Exception
: QlgConvertCase_Job_CCSID);
upper_value = %subst(QlgConvertCase_Temp_Output
: 1 : %len(value));

If I were maintaining this code, I'd much rather see the expressive call to CvtToUpper than the somewhat obfuscated call to the API. Even though I am completely familiar with this API, it's not immediately obvious which is the "real" parameter, some_value.

--
This is the RPG programming on the IBM i / System i (RPG400-L) mailing list
To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.

------------------------------

message: 4
date: Wed, 22 Aug 2012 15:42:49 -0500
from: KBushard@xxxxxxxxxxxxxxxx
subject: RE: Keyed file question

Way back when, I learned that if you put the "K" on the "F" spec, the
program will use the index to read the file, i.e. "sequential by key". If
you don't, it won't.

But I had never run into the "SETLL *START..." <-- good to know!

It's somewhat comforting to me to know that some of those old concepts
still hold true. ;)

Karleen Bushard || IBM i Software Devloper || HOM Furniture, Inc ||
763.767.3777 || VM1777 || kbushard@xxxxxxxxxxxxxxxx

From: "Hiebert, Chris" <chris.hiebert@xxxxxxxxxxxxxx>
To: "RPG programming on the IBM i / System i" <rpg400-l@xxxxxxxxxxxx>,

Date: 08/22/2012 09:15 AM
Subject: RE: Keyed file question
Sent by: rpg400-l-bounces@xxxxxxxxxxxx

In RPG you don't have to define the input file as keyed. When you leave
off the "K" you will read each record in the file once.

I have used this in the case where I am updating the key fields in a
file and only want to process each record once.

FFilename++IPEASF.....L.....A.Device+.Keywords++++
FFILENAME1 UF E DISK
FFILENAME2 UF E K DISK

Filename1 is not keyed.
Filename2 is keyed.

//The first time the file is opened it is positioned at the beginning.
OPEN FILENAME1;

DOU %EOF(FILENAME1);
READ FILENAME1;
IF %EOF(FILENAME1);
LEAVE;
ENDIF;
.. DO STUFF ..
ENDDO;

You can even use this to reposition the file to the begining.
SETLL *START FILENAME1;

Chris Hiebert
Winco Foods, LLC
Applications Development
(208) 672-2047
Chris.Hiebert@xxxxxxxxxxxxxx
-----Original Message-----
From: rpg400-l-bounces@xxxxxxxxxxxx
[mailto:rpg400-l-bounces@xxxxxxxxxxxx] On Behalf Of William Howie
Sent: Wednesday, August 22, 2012 7:20 AM
To: rpg400-l@xxxxxxxxxxxx
Subject: Keyed file question

Hello all...........

This probably seems like a silly question, but oh well, here goes. I
have a file that is defined as a keyed file, both in the DDS and within
the program that I am working on. I am loading data into this file
that won't necessarily be in the same keyed sequence as the file
definition is. Then within the program I just use a READ statement to
loop through it, not any sort of a SETLL and READE scenario. Here's
the question: the file will still get read in the keyed sequence
specified in the file definition, correct? In other words, I should
lose any sort of different sorting order the data might have been loaded
into the file with simply because the file is keyed........right?

I only ask this because it seems to be behaving differently from that.
Thanks in advance!

Bill Howie
Software Developer
AmTrust North America
________________________________
Confidentiality Notice: This email message is intended only for the
individual or entity to which it is addressed. This email may contain
information that is proprietary or privileged, confidential and exempt
from disclosure under applicable law. If you are not the intended
recipient, you are hereby notified that any dissemination, distribution
or copying of this communication is strictly prohibited. If you received
this email by accident, please notify the sender immediately and destroy
this email and all copies of it.
--
This is the RPG programming on the IBM i / System i (RPG400-L) mailing
list To post a message email: RPG400-L@xxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives at
http://archive.midrange.com/rpg400-l.

--
This is the RPG programming on the IBM i / System i (RPG400-L) mailing
list
To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.

------------------------------

message: 5
date: Wed, 22 Aug 2012 17:07:49 -0500
from: "sjl" <sjl_abc@xxxxxxxxxxx>
subject: Re: Converting Case with a Varying Field

+ a gazillion...

;-)))

- sjl

Eric wrote:

BZZZZTTTT!!! Sorry! I'm definitely NOT in your camp on this...

I'm always careful to examine my code for efficiency, but NEVER at the
expense of needless obfuscation and complexity for the programmer. I might
have agreed with your statement about 30 years ago, when IBM midrange
sported about 16K of memory, and processed at Khz speeds... Today,
processing inefficiencies are much less impactful than before. A scarce
assortment of high-performance routines might need to be coded for
operational efficiency, but that impacts the api design NOT AT ALL.

I'd hazard that, since you wrap job number in a couple of days, that your
applications submit MANY small jobs to jobq, for processing in batch. Each
job instance requires significant allocation of resources to start up,
initialize, execute, and end... A potential performance improvement to your
design might be to use a prestart job to service requests from a data queue.
This robust approach allows for several scalability improvements, such as a
queue monitor which will start up additional PJs to service request
backlog... Slick and performance ready!

-Eric DeLong

-----Original Message-----
From: rpg400-l-bounces@xxxxxxxxxxxx [mailto:rpg400-l-bounces@xxxxxxxxxxxx]
On Behalf Of Robert Houts
Sent: Tuesday, August 21, 2012 10:02 PM
To: RPG programming on the IBM i / System i
Subject: RE: Converting Case with a Varying Field

Sorry to disagree with you, as I do greatly respect your opinion, but
processing efficiency ALWAYS trumps programmer efficiency. Any extra (read
unnecessary) code is less efficient. The problem with most programmers
(besides laziness) is that they usually don't look at the big picture, which
is the code running in many jobs and possibly being called millions of times
in a day. That "possibly negligible" amount of extra processing adds up in
a hurry. Our production system does a lot of work (we wrap the job number
every two to three days), so we need to remove inefficiency wherever we can.

It seems to me that however "very very fast" the bound calls are, calling
the API directly is still faster. And that's the bottom line. Teaching
programmers to value programmer efficiency over processing efficiency is a
huge mistake.

-----Original Message-----
From: rpg400-l-bounces@xxxxxxxxxxxx [mailto:rpg400-l-bounces@xxxxxxxxxxxx]
On Behalf Of Barbara Morris
Sent: Tuesday, August 21, 2012 17:25
To: rpg400-l@xxxxxxxxxxxx
Subject: Re: Converting Case with a Varying Field

On 2012/8/21 7:03 PM, Robert Houts wrote:

... It
isn't rocket science and eliminates all overhead of the service
program. I guess that efficient processing is not a concern to you.