How much testing required after CVTRPGSRC?

(The following originally appeared as a response in a thread on the WDSC
(RDi) list. Someone advised off-list that I should post it here as well.)

I was searching the archives on CVTRPGSRC to see what impact / issues one
could expect to see by converting RPG-III to RPG-IV.

We're using an old ERP package written in RPG-III. I have always been of
the mindset to convert any RPG-III source to RPG-IV before I modify it.
Especially now with RDi, where the outline view doesn't work with RPG-III
and hoop-jumping necessary to debug OPM programs. Unfortunately, I'm new
here and I'm getting pushback when I need to modify an RPG-III program.
The "standard" here is to convert only when it's a major rewrite or
something that needs to be done can't be done in RPG-III. The argument is
that any such application must be completely retested, so, while the effort
to convert is minimal, the big cost is in the effort to QA the app.

It's been at least 15 years since I did any significant conversions to
RPG-IV, but I never recall anything that "broke" as a result of simply
converting an app. NOTE that this means no attempts to convert stuff like:
MMDDYY MULT 10000.01 YYMMDD
to
Eval YYMMDD = MMDDYY * 10000.01
as an example. >>> For the purpose of this discussion, I am strictly
talking about taking an RPG-III source, running CVTRPGSRC on it, checking
the conversion report, and compiling the RPG-IV program. The result should
be a functionally-identical program, 100% of the time. <<< True? False?

When I was searching a few weeks ago to refute the idea that a converted
program needed to be *completely* retested, I found a few threads on the
RPG list from 2006 where some were asking about any “gotchas” with using
CVTRPGSRC. Comments from people whose judgment I trust on this list were
unanimous in that a converted app *must* be completely retested. If this
is still the case, I will have tremendous difficulty selling the idea of
converting every RPG-III program to RPG-IV anytime we do any kind of
modification, because the QA resources are extremely tight in this shop.
It seems to me that, with ~20 years of experience with CVTRPGSRC, surely
IBM would have worked out any kinks in the conversion and/or the report
identifying potential issues.

So, my question to the group is this: Has anyone *ever* found any
functional differences or issues after doing a straight CVTRPGSRC and
getting a clean report from the conversion report?

- Dan