9.1 Code Coverage first impressions

I like it.

As a first cut, it's impressive. There are two requirements that I know
of so far:
1) the program needs to run via SBMJOB
2) the program needs to be compiled DBGVIEW(*SOURCE) *LIST doesn't work
yet.

I have this in house utility I wrote to compare two files on a field by
field basis. It uses the QUSLFLD API to generate a list of fields and
allows one to specify the key for JOINing the test and production files,
as well as fields to ignore (like timestamps). I recently modified it
to allow me to output the differences to tables instead of printer files
(selected by the command) and I found that testing the code wasn't very
straightforward. It wasn't until I ran the code coverage on it that I
realised the parts I'd forgotten to test. Here is my experience with
running code coverage over this.

My utility has a command, a CPP written in CL and an ILE RPG program
that does most of the work. In addition, it calls out to several
utility subprocedures in my utility service program. The point of this
is that I can't simply do a SBMJOB on the ILE RPG program in the middle
and hope to see it all work. This isn't a limitation!

Because my program is in the middle of several steps, I can't use the
simpler approach of auto-configuring a code coverage configuration.
(Right click the *PGM object). I need to make my own configuration,
which turns out to be pretty easy. If you use Eclipse to do Java or C++
work, this looks very similar to setting up a debug configuration.

Run > Compiled Code Configurations... opens up the list of configs.
There aren't any yet, so double click the IBM i: Code Coverage Batch
Application icon and RDi will create one with the appropriate template.
In the What To Analyze tab:
1) Choose a name (I picked CMPF to jog my memory)
2) Choose a connection
3) Add a program to be analysed

My library list is OK, so I didn't need to change anything in the Source
tab. Don't know what the Common tab does yet; left it alone. The Code
Coverage tab has the option to create a new analysis or append to an
existing one. Left that alone too. Now back to the How To Start tab.
It only allows a SBMJOB. Here's my command:
SBMJOB CMD(CMPF PRDF(prod/masterfile) TSTF(test/masterfile) KEYS(LOCAL
MBRID) OMTFLD(RAED GOVED2 REDIST LOCSECRET) output(*print) )
INQMSGRPY(*DFT) This command will invoke the CPP which will call CMPFR1
- the program I want to run CC analysis on.

Click Apply, then Coverage and off it goes. It submits a new job and
keeps track of the lines that the test job hit. When it's done, it
opens up 2 views: at the bottom, a Compiled Code Coverage Launch History
view and at the top, a Code Coverage Report view. Expand the tree to
see the subprocedures which are called. I didn't see coverage of
subroutines; there may be a switch for that that I haven't come across yet.

Two of my procedures had 100% coverage. The rest have varying amounts.
Double click on one and an editor window opens up showing the code in
browse mode. But this editor window has red / green (not covered /
covered) indicators on the left so you can see what has and hasn't been
tested yet. The red/green colours are adjustable in Window >
Preferences > General > Appearance > Colors and Fonts.

Comments do not have coverage indicators on them, so you'll see a
stretch of green down the left broken up by comments and blank lines.

IF statements are interesting. This particular test evaluated to false.

g if prdMbr <> '*FIRST';
r create_alias += '(' + prdMbr + ')';
g endif;

Though only the middle line is red, it stood out enough that it wasn't a
problem for me.

This SELECT followed the *ONLY path

g select;
g when rcdFmt = '*ONLY';
g rcdFmt = inrcdFmt;
g when rcdFmt <> '*ONLY' and
rcdFmt <> inrcdFmt;
r rcdFmt = '*MULTI';
r other;
r rcdFmt = '*ERROR';
r endsl;

The AND line had no indicator and the ENDSL line was red. I didn't
really care about the SELECT or ENDSL lines so the fact that they're
indicated differently wasn't an issue for me.

I'll post a short procedure so you can get a sense for how it gets
indicated. This shows 86% covered in the Report view.

pgenSQLoutput...
g p b

// generate the SQL OUTPUT() keywords
dgenSQLoutput pi 75 varying
d inOutput 10 const varying
d inOutFil 10 const varying
d inOutLib 10 const varying
d inOutMbr 10 const varying

d rtnvar s 75 varying inz

// default
g rtnvar = 'output(*print)';

// generate outfile keywords
g if inOutput = '*OUTFILE';
r rtnvar = 'output(*outfile) outfile(' +
inOutLib + '/' + inOutFil +
') outmbr(' + inOutMbr + ')';
g endif;

g return rtnvar;

pgenSQLoutput...
g p e

You may remember that I said recently modified this to use
OUTPUT(*OUTFILE). By creating another config - this time with
OUTPUT(*PRINT) - and changing the Code Coverage tab to Append to a
specific result, I can run the analysis again and the lines that were
touched with OUTPUT(*OUTFILE) are added to the lines that were touched
with OUTPUT(*PRINT). How cool is that? If I create enough separate
coverage configs I can hit the error cases as well as all the various
options.

This has already helped me think about ways to write my code so that it
is easier to write TDD style tests for, with code coverage as a way to
verify that the final assembly has been exercised.

I did find one glitch; I'm not sure how pervasive it is yet. My program
doesn't have a *ENTRY PLIST, it has a PR/PI pair for the mainline:

// *entry plist
dcmpfr1 pi extpgm('CMPFR1')
d inPrdFil 10
d inPrdLib 10
d inPrdMbr 10
d inTstFil 10
d inTstLib 10
d inTstMbr 10
d inSrcFil 10
d inSrcLib 10
d inSrcMbr 10
d inKeyStruc like(keyStruc)
d inRcdFmt 10
d inCase 10
d inAdlFldsStruc like(AdlFldsStruc)
d inOmtFldsStruc like(OmtFldsStruc)
d inOutput 10
d inOutFil 10
d inOutLib 10
d inOutMbr 10
d inJobType 1

The Code Coverage Report view indicates that 0% of CMPFR1() was covered.
This might be technically true: there is no CALLP to this procedure.
But all of those lines do actually execute, so it seems like this is a
problem interpreting which subprocedure the processed lines should be
numbered among.

By the looks of the CC Config dialogue, I think I could put all of the
programs and service programs that make up this application in one
config and see a report that deals with them as a group.

There are some things which I didn't play with because this app is so
simple: Code Coverage Report filters, exporting a report, the HTML and
PDF reports (instead of the Workbench report). I have to admit that the
colour coding of the HTML report seems better to these eyes than the
Workbench report, but that's a very personal preference.

This might be the first truly new RPG-related thing since the Code/400
days. I'm very happy.
--buck