Re: Test environments (was: upgrade)

If I may be permitted one small (!) reply, as I don't want to be accused of 
turning this into a sales thread.

I concur  that macro record and playback can go part of the way toward testing 
- but then you have the issue of maintaining the
macro when screens or fields change - an issue that most of the generic record 
and playback tools offer - are the users competent
scripting language programmers?

Also, and this applies to the parallel running situation, too; there is the 
need to verify the test results, and this is quite often
where there is an disproportionate amount of time spent looking to find perhaps 
only a few, if any, errors, and then documenting
them.

We have published a white paper about our view to approaching testing and I am 
more than happy to send a copy to anyone who would
like one - no sales follow up from me (promise!)

Regards

Jamie Coles
Original Software

----- Original Message -----
From: "Tom Liotta" <qsrvbas@netscape.net>
To: <midrange-l@midrange.com>
Sent: Wednesday, August 28, 2002 2:16 AM
Subject: Re: Test environments (was: upgrade)


A couple generic comments in-line...


midrange-l-request@midrange.com wrote:

>From: "Jamie Coles"
>
>  So the old tests have been recorded and can be played back whilst you record 
>new
>ones, and then these get played back when you record the next set.

For many basic tests, something as simple as a recorded ClientAccess macro 
could be used to automate testing. Not for everything,
but for testing that each menu option actually invoked a program, that the 
logged-on user didn't run into authority errors, etc.

Simple macro recording and playback isn't used nearly to full advantage for 
testing. With modification, the macros could be made
very intelligent. Much of this only needs doing once.


>One of the main problems with testing or running parallel with the old and the 
>new system is the extra amount of time needed to
>check the test results.

On contract, I rewrote a Local Improvement District billing application for a 
City a number of years ago. The IS manager resigned to
take a better position towards the end of the project, just before full system 
testing began. The new manager insisted on full
parallel testing until everything checked out in detail (which had much merit 
in concept) as the basis for "Acceptance".

Because a reconciliation between old and new took major time for City staff and 
because LID billing was on a monthly cycle, parallel
testing was a major problem. Surprise! The old and new didn't match by 
significant margins at many points. A few items were found
easily by me and corrected in the new system, but major items remained that 
took a lot of detail checking over years of history.
They started seeing that the big problems were mostly because the old system 
had always had major flaws that were never detected.

So, the month would pass while reconciliation went on. A new month-end would 
arrive and a new parallel also. And corrections for old
errors would allow new errors to be uncovered. After the first parallel, no 
significant errors were ever discovered in the new
version; all were errors in the old. In one major case, an LID was found that 
had _NEVER_ been billed to the property owners for a
number of years. You can imagine the political difficulties for the City in 
getting things current there.

But the parallels continued to be mandated. Meanwhile, my contract payment is 
delayed for months awaiting "Acceptance".

Ever since, I've been extremely leery of the value of parallel testing without 
clear guidelines.


>Once an error has been spotted the next difficulty is providing the developer 
>with the actual scenario that led to the error being
>created so that it can be replicated and then fixed - not always an easy task 
>- because us users never really know if we pressed
>"enter" or "space" !!

A very good point to bring up. Worth keeping in mind. We've all had users say 
"No, I'm _POSITIVE_ I hit <F16>, not <F4>." Automated
testing bypasses this.


Tom Liotta

--
Tom Liotta
The PowerTech Group, Inc.
19426 68th Avenue South
Kent, WA 98032
Phone  253-872-7788
Fax    253-872-7904
http://www.powertechgroup.com



__________________________________________________________________
Your favorite stores, helpful shopping tools and great gift ideas. Experience 
the convenience of buying online with Shop@Netscape!
http://shopnow.netscape.com/

Get your own FREE, personal Netscape Mail account today at 
http://webmail.netscape.com/

_______________________________________________
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@midrange.com
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/cgi-bin/listinfo/midrange-l
or email: MIDRANGE-L-request@midrange.com
Before posting, please take a moment to review the archives
at http://archive.midrange.com/midrange-l.