Re: the need for speed -- RPG400-L

Eric,

Feisty, picky,  cheeky, in your face with good intentions, inline. =;-}


"Eric N. Wilson" wrote:

> I can agree with John. The first criteria that I use is to write easy to
> read/understand code. Then in testing of batch jobs if performance is bad I
> will go back and optimize the process. Of course I try to code efficiently,
> i.e.. no invariant code in loops etc...

With all due respect, the first objective of any application should be to solve
the problem at hand.  Although you did not state this, I trust that it is also
your "first criteria".

IMHO, accuracy first, usability second, within budget third, speed fourth, 
making
it easy for my successor? Well they can keep up or eat dust <g>

Don't flame me. I'm only half way kidding.  I attempt to write modular, 
reusable,
maintainable code.  Because I'm lazy and I'm the one you will probably be
maintaining it.  I don't want to spend hours trying to figure out what I was
trying to do.  I don't get overly tricky in my code.  It's too hard to remember.
The problem being solved is foremost on my mind. Not -cute- coding techniques.

Easy for the next guy? Not in the formula.  I don't know the skill level of the
next guy and quite frankly could care less.

Oh, oh, I've got it: I'm going to take my 25 years of skill and write at a level
that an intern could understand? NOT!

Just as a side note: I get a kick out of the recurring thread on this list about
"dumbing down" code for the "next" programmer.  If we dumb it down far enough
we'll be doing SAA RPG.  The S/3x and RPG have grown too sophisticated for that.

Glory in your skills and challenge the "next" person to rise to your level of
enlightenment.  You'll be doing them a favor.  Or this profession a favor by
culling the herd.  Just resist the urge to get clever over clear.  But write to
your skill level.



> One thing that would be incredibly helpful is a source level profiler, that
> would give you the amount of time spent executing each line and procedure of
> code. This would greatly help in deciding what sections of a batch program
> could really use the optimization and what you should leave alone.

There was a program that did this about 10 or so years ago (maybe Q38 or what 
was
it DataNetwork?) that took your source code and added an array element (indexed
by source line) incrementor after each line of code.  At the end of the run the
array was dumped to disk and you could see how many times a particular line of
code was executed.

It was a nice exercise in changing your programming technique.  Once your
behavior was changed, the monitor was no longer needed.

It's really an easy thing to write.  Pick an obscure array name for the 
profiler,
insert that into your source member along with an add 1 to array(x) where x is
the source statement number after every line of "C" spec code, at LR write the
array to disk. This program is called within your home grown compile command and
the morphed source member is the one actually compiled.  Your source never gets
touched.

Or output trace to a disk file and cross reference it to a compile list.  Either
way works.

What separates humans from the lower mammals is the ability to create complex
tools. Go forth and create! <g>

> I hate
> relying on knowledge that I have acquired over the years to perform
> optimizations, and with each release of the operating system or change of
> core hardware the things to do could easily become a non issue and in the
> worse case your "optimizations" might cause worse performance.

No tool or profiler will indicate design flaws.  Only performance bottlenecks.
Knowledge is what makes one take a particular approach to solve a problem in the
first place. Don't "hate" knowledge or experience.  They -are- your tools.

> In addition
> it is very difficult to pass this sort of knowledge on because you do not
> remember it until you come across a familiar situation

As with all things, with practice, your ability to teach will improve. Your
memory, well, if it's important to you, you will remember it.  Pop quiz: When's
your mother's birthday? Your Social Security number? Your next day off?

See, I knew you could remember.  You have the ability.  Just choose the subject.

BTW, I've been trying to get the hang of non vocal, you can't see my subtle
nuances of gesture, communication.  One of the first things I got brow beat on
was to use the word "I" to claim, own, and be accountable for an opinion.  A
statement that includes the word "you" is directed to the reader of the post.  
So
when you write "you do not remember" you are claiming that the reader does not
remember, when in fact it is you that does not remember.  Own your view.  Just a
friendly BTW.

> Then the joys of
> running large batch jobs many times for each "optimization" to determine if
> you actually helped anything or not is always one of the great pleasures in
> life. It would be nice to not rely on anecdotal evidence and be able to use
> easily quantifiable data to help in the optimization process.

I've always found that a clock provides easily quantifiable data to help in the
optimization process. <VBG>

Learn on your own, AND learn from others. Anecdotal or scientific testimony has
never convinced me, one way or the other.  My tests are my way of getting a
second opinion.  Now which tests I perform, I admit are dependent upon the 
source
of testimony.

>
>
> All in all I believe that I end up optimizing around 1% of the programs that
> I write, OLTP applications should be written solely with the goal of making
> your code as easy to understand as possible, what user is going to notice
> the difference between one one thousandth of a second and a half a second?

Trust me, the users will.  Especially if they have a customer in their face and
20 more in line.  Seconds add up. Just like in:

> But those sort of time differences make worlds of difference when coding
> batch programs that run over millions and millions of rows/records.

With batch jobs it's easy to see the wasted hours.  And the gains that
performance changes make.  It's only one job.  Take hundreds of users over an 8
hour day and it's easy to "ignore" the accumulated, wasted, hours.

>
>
> I know that in ILE programs you can collect profiling data that the
> optimizer can use to further optimize your code, has there been any thought
> as to making an end user product to profile programs at IBM?

Eric, don't wait for IBM to provide you with the ability to create closed loop
feedback applications.  You can do it now.  The key word is: you.

Or do you want the big brains at Big Blue to subvert your knowledge of
optimization into a Pavlovian response to a "wizard"?

Ah, yes, I glory in the day when a higher power can make my decisions for me and
solve all of the problems I am faced with. =;-}

P.S. Eric, believe me, I'm not picking on you personally.  I don't know you well
enough to make it personal. I've heard this tune before and that recurring echo
is what my response is aimed at.

Besides, your broken soap box was as much box as I needed to climb upon and 
start
spouting. <g>

James W. Kilgore
President
Progressive Data Systems, Inc.


>
>
> OK I think I have broken my soap box :-)
>
> Thanks
> Eric

+---
| This is the RPG/400 Mailing List!
| To submit a new message, send your mail to RPG400-L@midrange.com.
| To subscribe to this list send email to RPG400-L-SUB@midrange.com.
| To unsubscribe from this list send email to RPG400-L-UNSUB@midrange.com.
| Questions should be directed to the list owner/operator: david@midrange.com
+---