RE: Project Estimation

Phil,

First up, I represent a company selling Automated Testing Solutions,  -
the product is called TestBench on the AS/400 (iSeries)  - but hopefully
that will qualify my comments as they are based on customer experience
rather than sales literature.

The time taken to run the tests and analyse the results is a "how long
is a piece of string" question dependant on various factors including
knowledge of the system, how complex it is to set up test data, how many
test scripts you need to run and how often these need to be repeated.

Initially when creating test data it is considered better to extract
data that is relevant to the program that is being tested from the live
database and then protect it so that it can be used time after time.
This can save a great deal of time to create a test database only to
have it "corrupted" when the first test is run and subsequent tests
start with different data so the results are different- but are they
caused by a error in the program or a different set of test data?  A
tool that will automatically analyse the database and then extract the
relevant data is good.

When running the test scripts - an automated tool will allow you to
record them once and run them many times, whilst you are doing other
jobs - or even playing golf!!   And there is no need to laboriously
check results for errors - the tool will highlight these for you - and
even print a report of the errors with the screen shots and expected and
actual results all there for you - automatically.

One thing to watch if you do invest in an automated tool - and that is
to make sure the test scripts are easy to maintain and change.  The
users will know their work, but may not be very good at using scripting
languages to change the scripts.  

You mention you wanted other suggestions for things to look out for -
one area that is vitally important to the tests run on AS/400's (which
are heavily batch oriented applications) is the ability to test the
database as well as the user front end. 

Without slipping into "sales" mode (too much)  - our product will show
you the effects on the database of every keyed input, track data queues,
track parameters that are passed between programs etc.    

For instance if there is an error found in the front end isn't it better
to show the programmer every step that took place to create the error
rather than say "when I think I hit that button" this appeared on my
screen" 

We have a number of white papers discussing testing - again based on
experience not sales literature as well as discussions about linking in
with the major Change Management tools and Web and GUI testing.

Please feel free to visit our website or email me off-list.

Jamie Coles

www.origsoft.com 

Sorry for the long reply - hope it is of help.