Home
– Project Management
– Software Testing –
Software Quality
Assurance – About Me
WinRunner vs. QuickTest Pro
Quick Comparison
Table of Contents
WinRunner
Compared to QuickTest Pro
Today Mercury has two functional testing tools – WinRunner (WR) and QuickTest Professional (QTP). WinRunner has been around since about 1995 while QuickTest Pro has been available since about 2002. We have lots of companies asking us “What’s the difference between WinRunner and QuickTest Pro?” and “If they are both functional testing tools from the same company, then which one of the tools should we use?” This document was developed as a short simple reference in helping to understand the basics about the tools in order to answer these two questions.
Mercury’s position on the two tools is this:
The strategic direction for Mercury and its customers for Functional Testing is product integration. The new releases of QTP and WR improve already existing integration where customers can leverage complete WinRunner assets (scripts/functions) from QuickTest Pro without any modification to these assets.
QuickTest Pro 8 has recently been released and WinRunner version 8.0 is planned for November 2004. There are no plans to phase out WinRunner.
QuickTest Pro remains Mercury’s prime product for best customer satisfaction and experience. For customers who need both products, Mercury continues to offer Mercury Functional Testing as a combined package of both product offerings. The Business Process Testing system will be extended to incorporate components designed in WinRunner for the WinRunner 8.0 release in November.
Environment Coverage Comparison:
Common environments shared by both WinRunner and QuickTest Pro:
Web-Related Environments |
IE, Netscape, AOL |
|
JDK, Java Foundation Classes, AWT |
|
Symantec Visual Café |
|
ActiveX Controls |
ERP/CRM |
Oracle: Jinitiator, 11i, NCA |
Custom Client Server |
Windows |
|
C++/C |
|
Visual Basic |
Operating Systems |
Windows 98, 2000, NT, ME, XP |
Legacy |
3270, 5250 Emulators |
|
VT100 |
WinRunner Only Environments:
Custom Client/Server |
PowerBuilder |
|
Forte |
|
|
|
Centura |
|
Stingray |
|
SmallTalk |
ERP/CRM |
Baan |
|
PeopleSoft Windows |
|
Siebel 5, 6 GUI Clients |
|
Oracle GUI Forms |
QuickTest Pro Only Environments:
ERP/CRM |
SAP |
|
Siebel 7.x |
|
PeopleSoft 8.x |
.Net |
WinForms |
|
WebForms |
|
.Net controls |
Web Services |
XML, HTTP |
|
WSDL, SOAP |
|
J2EE, .Net |
Multimedia |
RealAudio/Video |
|
Flash |
Feature Comparison:
Common features found in both WinRunner and QuickTest Pro:
Record/Replay |
ODBC & Excel Connectivity |
Code Editor & Debugger |
Recovery Manager |
Shared Object Repository |
Rapid Object Import |
Numerous Checkpoints |
Analog |
Script & Function Libraries |
|
WinRunner Only Environments:
Function Generator |
Database Integration |
Run Wizard |
TSL |
MDI |
|
QuickTest Pro Only Environments:
ActiveScreen |
TestGuard |
Tree View |
ScriptFusion |
Data Table |
VBScript |
Function Generator* (coming in v7.0) |
Run Wizard* (coming in v7.0) |
QuickTest Pro is our functional testing tool of choice!
Most customers are in either of two situations:
Existing WinRunner customers asking “Should we switch to QuickTest Pro?”
New customers asking “Which should we get, WinRunner or QuickTest Pro?”
We have been implementing Mercury’s products since 1992 and have senior level expertise in WinRunner. We have been implementing WinRunner since the very first version of it. So like many existing WinRunner customers we have a huge knowledge investment in WinRunner ourselves. However we have worked on the Mercury development team on the QuickTest Pro 6.0 and 6.5 releases, and have solid real-world experience in implementing QuickTest Pro since the 6.0 release. We have found it to be a great tool to use and recommend it to all customers! Overall, we recommend using QuickTest Pro unless for some reason you have to use WinRunner due to an unsupported environment not existing in QuickTest Pro.
Overall, QuickTest Pro is easier to use and implement for both technical and non-technical testers in comparison to WinRunner. QTP offers many features that are found in WinRunner, but are easier to use. QTP also offers many features not found in WinRunner that make test script creation, enhancement and maintenance easier.
Let’s quickly discuss some key issues:
The WinRunner interface forces the user to look directly at TSL
code. TSL is the WinRunner programming
language developed by Mercury. It is
based on the “C” programming language and therefore looks very similar. For testers who do not have a technical
background, they are not always comfortable with having to always look at
code. However QuickTest Pro offers a
“Tree View” which is an icon-based view of the script. This is very easy to get used to and
non-technical people adapt to it quicker and feel more comfortable working with
it. For the technical user, they can
always switch over to the “Expert View” in QuickTest and look directly at code,
and program away using VBScript.
WinRunner uses TSL which is a proprietary language of Mercury. These types of languages can be very
restrictive and you are limited on available resources. QuickTest Pro use the Microsoft programming
language VBScript which is very powerful, has lots of capabilities and there
are lots of resources available.
We feel that there are many more “point and click” features in QuickTest
Pro than WinRunner. This makes its use
easier.
All the same features found in WinRunner are found in QuickTest Pro plus
more.
Data Table integration is much easier and simpler with QuickTest Pro.
Data Driven Testing is easier with more options in QuickTest Pro.
Script enhancements are typically easier with QuickTest Pro because it
has the Active Screen where the windows and objects in your application are
captured for later use. Using a “point and click” capability you can easily
interface with objects, their definitions and create checkpoints after having
recorded a script – without having to navigate back to that location in your
application like you have to with WinRunner.
This greatly speeds up script development.
QuickTest Pro currently has built in integration with WinRunner in order
to be able to call existing scripts, which is great for customers who already
have a large test script investment with WinRunner. Likewise WinRunner 8.0 is to have the ability to also call
QuickTest Pro scripts. This eases the
transition from WinRunner to QuickTest Pro for existing customers.
Parameterization is much easier in QuickTest Pro and basically anything
in QuickTest Pro can be parameterized (statements, checkpoints and the Object
Repository).
Capturing various output values is easier and simpler with QuickTest
Pro. Using this capability with
parameterization enables you to easily develop scripts that can do more
in regards to testing.
We have been implementing QuickTest Pro in real-world environments and have found it much easier to use, advance features are easier to implement and the script development is quicker. We really enjoy using QuickTest Pro and highly recommend it as the functional testing tool to use.
QuickTest Pro is our functional testing tool of choice!
This product is a mature tool that has been around since approximately 1995. It interfaces with most of the leading development toolkits using the WindowsAPI and toolkit DLLs to interface with the “Application Under Test”.
WinRunner offers a recording feature that will watch the individual tester and generate a test script to simulate the same actions just performed. The script is displayed as a program which can be enhanced with checkpoints, logic and special coding/programming.
WinRunner also has integration with Excel spreadsheets for data driven testing and the ability to write data out in Excel format or in simple text files.
Here is the description from the Mercury “Features and Benefits” section of the WinRunner web page:
Significantly
increase power and flexibility of tests without any programming: The Function Generator presents a quick and error-free
way to design tests and enhance scripts without any programming knowledge.
Testers can simply point at a GUI object, and WinRunner will examine it,
determine its class and suggest an appropriate function to be used.
Use
multiple verification types to ensure sound functionality: WinRunner provides checkpoints for text, GUI, bitmaps,
URL links and the database, allowing testers to compare expected and actual
outcomes and identify potential problems with numerous GUI objects and their functionality.
Verify
data integrity in your back-end database: Built-in Database Verification confirms values stored in the database
and ensures transaction accuracy and the data integrity of records that have
been updated, deleted and added.
View,
store and verify at a glance every attribute of tested objects: WinRunner's GUI Spy automatically identifies, records
and displays the properties of standard GUI objects, ActiveX controls, as well
as Java objects and methods. This ensures that every object in the user
interface is recognized by the script and can be tested.
Maintain
tests and build reusable scripts: The
GUI map provides a centralized object repository, allowing testers to verify
and modify any tested object. These changes are then automatically propagated
to all appropriate scripts, eliminating the need to build new scripts each time
the application is modified.
Test
multiple environments with a single application: WinRunner supports more than 30 environments,
including Web, Java, Visual Basic, etc. In addition, it provides targeted
solutions for such leading ERP/CRM applications as SAP, Siebel, PeopleSoft and
a number of others.
Simplify
creation of test scripts: WinRunner's
DataDriver Wizard greatly simplifies the process of preparing test data and
scripts. This allows for optimal use of QA resources and results in more
thorough testing.
Automatically
identify discrepancies in data:
WinRunner examines and compares expected and actual results using multiple
verifications for text, GUI, bitmaps, URLs, and databases. This ensures stable
functionality and execution of business transactions when the application is
released into production.
Validate
applications across browsers:
WinRunner enables the same test to be used to validate applications in Internet
Explorer, Netscape, and AOL. This saves testing time and reduces the number of
scripts that must be developed and maintained.
Automatically
recover tested applications from a crash: Unexpected events, errors, and application crashes during a test run
can disrupt the testing process and distort results. WinRunner's Recovery
Manager enables unattended recovery and provides a wizard that guides the
process of defining a recovery scenario.
Leverage
investments in other testing products:
WinRunner fully integrates with our other testing solutions, including
LoadRunner for load testing and TestDirector for global test management.
Moreover, organizations can reuse WinRunner test scripts with QuickTest
Professional.
- WinRunner “Features and Benefits” webpage from Mercury:
http://www.mercury.com/us/products/quality-center/functional-testing/winrunner/features.html
QuickTest Professional provides an interactive, visual environment for test development.
Here is the description from the Mercury Interactive “How it Works” section of the QuickTest Pro web page:
Mercury QuickTest Professional™ allows even novice testers to
be productive in minutes. You can create a test script by simply pressing a
Record button and using an application to perform a typical business process.
Each step in the business process is automated documented with a plain-English
sentence and screen shot. Users can easily modify, remove, or rearrange test
steps in the Keyword View.
QuickTest Professional can automatically introduce checkpoints to verify
application properties and functionality, for example to validate output or
check link validity. For each step in the Keyword View, there is an
ActiveScreen showing exactly how the application under test looked at that
step. You can also add several types of checkpoints for any object to verify
that components behave as expected, simply by clicking on that object in the
ActiveScreen.
You can then enter test data into the Data Table, an integrated spreadsheet
with the full functionality of Excel, to manipulate data sets and create
multiple test iterations, without programming, to expand test case coverage.
Data can be typed in or imported from databases, spreadsheets, or text files.
Advanced testers can view and edit their test scripts in the Expert View, which
reveals the underlying industry-standard VBScript that QuickTest Professional
automatically generates. Any changes made in the Expert View are automatically
synchronized with the Keyword View.
Once a tester has run a script, a TestFusion report displays all aspects of the
test run: a high-level results overview, an expandable Tree View of the test
script specifying exactly where application failures occurred, the test data
used, application screen shots for every step that highlight any discrepancies,
and detailed explanations of each checkpoint pass and failure. By combining
TestFusion reports with Mercury Quality Management, you can share
reports across an entire QA and development team.
QuickTest Professional also facilitates the update process. As an application
under test changes, such as when a “Login” button is renamed “Sign In,” you can
make one update to the Shared Object Repository, and the update will propagate
to all scripts that reference this object. You can publish test scripts to
Mercury Quality Management, enabling other QA team members to reuse your test
scripts, eliminating duplicative work.
QuickTest Professional supports functional testing of all popular environments,
including Windows, Web, .Net, Visual Basic, ActiveX, Java, SAP, Siebel, Oracle,
PeopleSoft, terminal emulators, and Web services.
- QuickTest Pro “How it Works”
webpage from Mercury:
http://www.mercury.com/us/products/quality-center/functional-testing/quicktest-professional/works.html
We like QuickTest Pro and now prefer implementing it over WinRunner. When you get into advance testing scenarios, QuickTest Pro has more options and they are easier to implement compared to WinRunner in our opinion.
Do to the similarities in concept and features, an experienced WinRunner user can easily convert to QuickTest Pro and quickly become an efficient Test Automation Engineer!
We recommend that existing customers begin all new development with QuickTest Pro and use the built-in feature of calling WinRunner scripts from QuickTest Pro for all existing WinRunner scripts that they already have. As older scripts require updates and time permits, we recommend replacing them with QuickTest Pro scripts. Eventually you will be able to convert your test script library with all QuickTest Pro scripts.