Re: Wish list: RPG code refactoring

Hi Buck

thanks for the detailed explication - +1 I think! :)

I finally have a new usable IBM ID, so I'll go vote for this.

In the interim, I find that in order to minimize false positives on searches, I'll add a blank before and after the search string - this means that I'll get ' ON ' and not 'ONLY' or 'FUTON'.

That's not a final answer, but it may help us for awhile anyhow.

Cheers and 3 more days!
Vern

On 2/15/2013 12:57 PM, Buck Calabro wrote:

On 2/14/2013 1:56 PM, Buck Calabro wrote:

I really would like to have the ability
to use Eclipse / RDp to refactor; especially my
legacy code. I hope that other RPG programmers
agree and vote for this idea.

http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=31305

I'm going to take the advice offered me and try to explain what this
refactoring stuff is, and why it's interesting for RPG programmers.

Refactoring is a mathematical term that means 'tinker with a formula so
that it's easier to understand without affecting how the answer comes
out'. Software developers took the idea and ran with it. Refactoring
can mean different things to different developers, but the general idea
at its core remains the same: make the code easier to work with WITHOUT
breaking it.

We're so accustomed to leaving well enough alone (if it ain't broke,
don't fix it!) that the idea of touching code 'just to pretty it up'
seems like a Bad Idea, but the key point of any refactoring technique is
that it's not about looks, it's about better code.

There are many refactoring techniques. Perhaps the simplest is to
rename a variable to something more meaningful. Let's look at this
block of RPG III code:

C N30 ID CHAINPAYROL 54
C 54 ADD 1 NOTON 60
C N54 ADD 1 ON 60

Assume we're in this program because we have other things to do, and we
come across this. Aside from the obvious problem that the left hand
indicator 54 should be conditioned on N30, the counters are named funny.
It increments NOTON when 54 is off, and increments ON when 54 is off!
And at the end of the day, what the heck do NOTON and ON /do/ - what do
they /mean/?

I'd really like to rename NOTON to formerEmployees, and ON to
currentEmployees. How do I do that? Scan & replace is an issue because
scanning for 'ON' will return things like NOTON, ONLY, TONS and my
favourite, SETON. I can't simply change all instances of ON to
currentEmployees because of the false positives. That means if I want
to change the names of those variables, I need to scan and decide which
hits are really ON and which are something else. And then there's the
whole issue of preserving the columns...

It's just enough of a pain in the drain that I just leave those
non-useful names alone, make the simple change I need and go on with
life. Of course, the next programmer in the code is going to need to
mentally decipher the exact same thing I just did, and she too is
probably going to decide that changing the names to something meaningful
isn't worth the amount of effort required to do it.

But what if I could right click on the variable NOTON and choose
Refactor > Rename? What if I got a window popping up asking me for the
new name and whether the scope is local or variable? And RDp then goes
off and renames just that variable? Preserving columns in fixed form
calculations and D specs and shifting columns in free form calculations?
Now the calculus is very different. Because it's easy to rename a
variable, I'm far more likely to change existing code to use more
meaningful names, which pays off for me now, when I'm thinking of the
code and also for every single programmer (including me!) who has to
look at this code in the future.

Whoa!

Let's look at something like Extract Constant. Imagine a legacy program
with sets of arrays. It's one of those sales reports with 4 level
breaks in it. We sell to homes and businesses, so we have 2 element
arrays for cost, sales, profit, margin. One set of each of these arrays
for each of the 5 level breaks (including LR). And now, we've added a
new customer grouping - non-profits. We need to change all of the
places where we do things like 1 DOWLE 2 X and change those 2s to 3s.
Not so bad, you say. Scan and... oh. Yeah. There are places those 2s
are used we don't want to change. Like O specs. Or other, non-array
related calculations. Grumble grumble... Scan and manually replace the
places we need to change, recompile and we're done. Well, except for
the other couple dozen programs like this one. Very tedious, very error
prone.

But what if we could extract out the constant? Highlight the lines we
want to look at, right click and Extract Constant. RDp looks at the
constants and creates a D specification for each one, prompting us for
the name and then replacing 1 DOWLE 2 X with 1 DOWLE MAX_CUST_TYPES X
without mucking up the columns?

Whoa!

But by far, the biggest one on my wishlist is Extract Function, or as I
think of it, Extract Subprocedure. I have an ugly block of code that I
have to get into. It's a business rule that determines the
credit-worthiness of a customer. Of course it has 2 big blocks within
it; one for home and one for business customers. You guessed it: I need
to add another block for non-profits. My biggest concern is testing.
WHen I'm done, in order to test this I need to set up a complete test
database, copy the right records from production master files, create
new test environment transaction files, create new non-profit master
records and imagine all the stuff I'm going to have to look at line by
line to make sure it's working as intended. Yuck.

But what if we could extract out the function? Highlight that block of
code, right click and Extract Subprocedure. RDp prompts for a name,
parameter list and maybe if it's for Export or not. Press OK and RDp
creates a PR, a PI, moves that highlighted code into the PI block and
replaces the mainline code with a CALLP to the new subprocedure, with
the parameter skeleton in place? And maybe even put the export in the
binder source!

It would be so easy to create a new function out of monolithic inline
code, that I'd do a lot more of it. Once I have that function, it's
going to be a LOT easier for me to test it because now I can copy it
into a test jig [1] and instead of hours of database setup and
comparison, I can write a few dozen CALLP lines with every test case I
need to cover. All the edge cases, average cases, test our biggest and
smallest customers - it all goes into the test jig. And the test jig
does the comparisons for me, rather than having to run complex queries
to compare this file to that file, I can directly test the functionality
I'm changing. And ONLY the functionality that I'm changing.

But there's more. Because it's so easy to create a new function call, I
might look at this credit analysis code with its two sub-blocks and
extract out the common code, turning it into a subprocedure that takes a
customerType parameter. Now, adding non-profit has gone from copy a
block, paste a block, change, change, change, change, change, compile,
blah, blah, blah to adding a line to a SELECT block and calling the
non-profit code. No mucking with the already working code for home and
business users.

Whoa!

Why is refactoring interesting to RPG programmers? Because monolithic
code is difficult to change. It's fragile. Unpredictable. By the time
we go into these beasties to add mandatory functionality, we're in it up
to our neck. It's a Big Change. Long testing period and the inevitable
unforeseen side effects afterward. In contrast, refactoring is the
Small Change. I go in, change the variable name, recompile and test.
One small change, one small test. High confidence that there are no
side effects. My code is a tiny bit less fragile because I understand
it better. Then, the next time I'm in there I make another small
change, and test. Later on, another. Still later, I extract out a
function and that requires a different sort of test, but when I'm done,
that code is much easier to deal with. I have local variables I can use
and throw away in those new subprocedures. I have a test jig that I can
run which will prove that the changes I'm making now haven't broken
previously working code.

Refactoring is interesting to us because it gives us the confidence that
making these small changes won't break external functionality. That's
huge. Programming isn't about typing. Programming is about thinking.
Any help the IDE can provide me with the ability to think clearer is a
giant step in the right direction.

Thanks for your patience with me.
--buck

[1] Or put the subprocedure in a service program and use RPGUnit to test it!

Whoa!