Assignment-9

Aim:- Study of different Automation testing tools.

Automation
Automation is the use of strategies, tools and artifacts that reduce the need of manual or
human involvement or interaction in unskilled, repetitive or redundant tasks.

Test Automation is the use software to execute tests rather than executing them manually.
Test Automation sets up the preconditions executes the test compares the actual outcomes to
expected outcomes and reports the results without human interaction. Test automation is
often used to automate regression tests (tests that are run on each iteration of a software
build) to ensure basic functionalities remain stable.

The following types of testing can be automated:-

 Functional - testing that operations perform as expected.

 Regression - testing that the behavior of the system has not changed.
 Exception or Negative - forcing error conditions in the system.
 Stress - determining the absolute capacities of the application and operational
infrastructure.
 Performance - providing assurance that the performance of the system will be adequate
for both batch runs and online transactions in relation to business projections and
requirements. (Batch test is a group of scripts.. you can run multiple script with automation tool
when you run multiple script with automation tool that is called batch testing. QTP an inrunner
both are support batch test)
 Load - determining the points at which the capacity and performance of the system
become degraded to the situation that hardware or software upgrades would be required.

The Benefits of Automation and Tools :-

Automated Software Testing Saves Time and Money-Software tests have to be

repeated often during development cycles to ensure quality. Every time source code is
modified software tests should be repeated. For each release of the software it may be tested
on all supported operating systems and hardware configurations. Manually repeating these
tests is costly and time consuming. Once created, automated tests can be run over and over
again at no additional cost and they are much faster than manual tests. Automated software
testing can reduce the time to run repetitive tests from days to hours. A time savings that
translates directly into cost savings.

Automated Software Testing Improves Accuracy-Even the most conscientious

tester will make mistakes during monotonous manual testing. Automated tests perform the
same steps precisely every time they are executed and never forget to record detailed results.
Automated Software Testing Increases Test Coverage-Automated software testing
can increase the depth and scope of tests to help improve software quality. Lengthy tests that
are often avoided during manual testing can be run unattended. They can even be run on
multiple computers with different configurations. Automated software testing can look inside
an application and see memory contents, data tables, file contents, and internal program
states to determine if the product is behaving as expected. Automated software tests can
easily execute thousands of different complex test cases during every test run providing
coverage that is impossible with manual tests. Testers freed from repetitive manual tests have
more time to create new automated software tests and deal with complex features.

Automated Software Testing Does What Manual Testing Cannot-Even the

largest software departments cannot perform a controlled web application test with thousands
of users. Automated testing can simulate tens, hundreds or thousands of virtual users
interacting with network or web software and applications.

Automated Software Testing Helps Developers and Testers-Shared automated

tests can be used by developers to catch problems quickly before sending to QA. Tests can
run automatically whenever source code changes are checked in and notify the team or the
developer if they fail. Features like these save developers time and increase their confidence.

Automated Software Testing Improves Team Morale-This is hard to measure but

we’ve experienced it first hand, automated software testing can improve team morale.
Automating repetitive tasks with automated software testing gives your team time to spend
on more challenging and rewarding projects. Team members improve their skill sets and
confidence and, in turn, pass those gains on to their organization.

The following areas must be automated first:-

1. Highly redundant tasks or scenarios

2. Repetitive tasks that are boring or tend to cause human error
3. Well-developed and well-understood use cases or scenarios first
4. Relatively stable areas of the application over volatile ones must be automated.

Automated testers must follow the following guidelines to get the benefits of automation:-

 Concise: As simple as possible and no simpler.

 Self-Checking: Test reports its own results; needs no human interpretation.
 Repeatable: Test can be run many times in a row without human intervention.
 Robust: Test produces same result now and forever. Tests are not affected by changes in the
external environment.
 Sufficient: Tests verify all the requirements of the software being tested.
 Necessary: Everything in each test contributes to the specification of desired behavior.
 Clear: Every statement is easy to understand.
 Efficient: Tests run in a reasonable amount of time.
 Specific: Each test failure points to a specific piece of broken functionality; unit test failures
provide "defect triangulation".
 Independent: Each test can be run by itself or in a suite with an arbitrary set of other tests in
any order.
 Maintainable: Tests should be easy to understand and modify and extend.
 Traceable: To and from the code it tests and to and from the requirements.

QTP Testing tool

QTP stands for QuickTest Professional, a product of Hewlett Packard (HP). HP QTP is an
automated functional Testing tool that helps testers to execute automated tests in order to
identify any errors, defects or gaps in contrary to the expected results of the application
under test.

HP QTP uses Visual Basic Scripting (VBScript) for automating the applications. The
Scripting Engine need not be installed exclusively, as it is available as a part of the Windows
OS. The Current version of VBScript is 5.8, which is available as a part of Win 7. VBScript
is NOT an object-oriented language but an object-based language.

Advantages of QTP

1. It supports record and playback

2. It uses active screen to record scripts and helps tester in referring the screen object
properties
3. It has excellent object identification process or mechanism
4. It supports different add-ins like Oracle, Java, SAP, NET, Web Forms, People soft,
etc..
5. It allows you to enhance the existing tests even without the AUT through an active
screen
6. It supports popular automation frameworks- keyword driven testing approach,
modular testing approach, data driven testing approach, etc..
7. It comes with an inbuilt IDE
8. It can be integrated with Test management tools like Quality Center, Test Director,
and Winrunner
9. Different types of suites like Smoke, Regression, Sanity can be easily maintained
10. It supports XML
11. Test reporting is possible through QTP for analysis purpose
12. Easy to maintain

Disadvantages of QTP

1. Unlike Selenium, QTP works in Windows operating system only.

 2. Not all versions of Browsers are supported and the testers need to wait for the patch to
be released for each one of the major versions.
3. Having said, that it is a commercial tool, the licensing cost is very high.
4. Even though scripting time is less, the execution time is relatively higher as it puts
load on the CPU & RAM.

Rational Robot
Rational Robot is a complete set of components for automating the testing of Microsoft
Windows client/server and Internet applications. (Rational Robot user guide)

Rational Robot is an automated functional regression testing tool.

Automated Functional Regression Testing

• Functional Test: Functional Tests are designed to make sure that the application performs
as it was intended.

• Regression Test: A regression test is a test where an application is subjected to a suite of

functional tests at each build to ensure that everything that worked continues to work.

Components Of Rational Robot

 Rational Administrator - Create and manage Rational projects to store your testing
information.
 Rational TestManager - Review and analyze test results.
 Object Properties, Text, Grid, and Image Comparators - View and analyze the results
of verification point playback.
 Rational SiteCheck - Manage Internet and intranet Web sites.

Bugzilla Tool

Bugzilla is an open-source issue/bug tracking system that allows developers effectively to

keep track of outstanding problems with their product. It is written in Perl and uses MYSQL
database.

Bugzilla is a Defect tracking tool, however it can be used as a test management tool as such
it can be easily linked with other Test Case management tools like Quality Center, Testlink
etc.

This open bug-tracker enables users to stay connected with their clients or employees, to
communicate about problems effectively throughout the data-management chain.
Key features of Bugzilla includes

 Advanced search capabilities

 E-mail Notifications
 Modify/file Bugs by e-mail
 Time tracking
 Strong security
 Customization
 Localization

Creating a Bug-report in Bugzilla

Step 1) To create a new bug in Bugzilla, visit the home-page of Bugzilla and click
on NEW tab from the main menu

Step 2) In the next window

1. Enter Product
2. Enter Component
3. Give Component description
4. Select version,
5. Select severity
6. Select Hardware
7. Select OS
8. Enter Summary
 9. Enter Description
10. Attach Attachment
11. Submit

NOTE: The above fields will vary as per your customization of Bugzilla

NOTE: The mandatory fields are marked with *.

In our case field's

 Summary
 Description

Are mandatory

If you do not fill them you will get a screen like below
Step 4) Bug is created ID# 26320 is assigned to our Bug. You can also add additional
information to the assigned bug like URL, keywords, whiteboard, tags, etc. This extra-
information is helpful to give more detail about the Bug you have created.

1. Large text box

2. URL
3. Whiteboard
4. Keywords
5. Tags
6. Depends on
7. Blocks
8. Attachments
Step 5) In the same window if you scroll down further. You can select deadline date and also
status of the bug. Deadline in Bugzilla usually gives the time-limit to resolve the bug in
given time frame.

The Advantages of Bugzilla are:

1. it is an open source widely used bug tracker;
2. it is easy in usage and its user interface is understandable for people without technical
knowledge;
3. it easily integrates with test management instruments;
4. it integrates with an e-mailing system;
5. it automates documentation.

Loadrunner tool
HP LoadRunner is an automated performance and test automation product for application
load testing: examining system behaviour and performance, while generating actual load. A
software testing tool, HP LoadRunner works by creating virtual users who take the place of
real users' operating client software, such as Internet Explorer, sending requests using the
HTTP protocol to IIS or Apache web servers. HP LoadRunner can simulate thousands of
concurrent users to put the application through the rigors of real-life user loads, while
collecting information from key infrastructure components (Web servers, database servers
etc.) The results can then be analyzed in detail to explore the reasons for particular behavior.

Broadly, LoadRunner supports RIA (Rich Internet Applications), Web 2.0 (HTTP/HTML,
Ajax, Flex and Silverlight etc.), Mobile, SAP, Oracle, MS SQL Server, Citrix, RTE, Mail
and above all, Windows Socket. There is no competitor tool in the market which could offer
such wide variety of protocols vested in single tool.

What is more convincing to pick LoadRunner for performance testing is the credibility of
this tool. LoadRunner has long established reputation as often you will find clients cross
verifying your performance benchmarks using LoadRunner. You'll find relief if you're
already using LoadRunner for your performance testing needs.

LoadRunner is tightly integrated with other HP Tools like Unified Functional Test (QTP) &
ALM (Application Lifecycle Management) with empowers you to perform your end to end
Testing Processes.

LoadRunner works on a principal of simulating Virtual Users on the subject application.

These Virtual Users, also termed as VUsers, replicate client's requests and expect
corresponding response to pass a transaction.

LoadRunner contains the following components:

 VuGen: Captures end-user business processes and creates an automated performance

testing script, also known as a Vuser script.
 Controller: Organizes, drives, manages, and monitors the load test.
 Analysis: Helps you view, dissect, and compare the results of the load tests.

LoadRunner Architecture

Broadly speaking, the architecture of LoadRunner is complex, yet easy to understand.

Suppose you are assigned to check performance of Amazon.com for 5000 users

In a real life situation, these all these 5000 users will not be at homepage but in different
section of the websites. How can we simulate different

VUGen:

VUGen or Virtual User Generator is an IDE (Integrated Development Environment) or a rich

coding editor. VUGen is used to replicate System Under Load (SUL) behaviour. VUGen
provides a "recording" feature which records communication to and from client and Server in
form of a coded script - also called VUser script.

So considering the above example, VUGen can record to simulate following business
processes:

1. Surfing the Products Page of Amazon.com

2. Checkout
3. Payment Processing
4. Checking MyAccount Page

Controller

Once a VUser script is finalized, Controller is the main component which controls the Load
simulation by managing, for example:

 How many VUsers to simulate against each business process or VUser Group
 Behaviour of VUsers (ramp up, ramp down, simultaneous or concurrent nature etc.)
 Nature of Load scenario e.g. Real Life or Goal Oriented or verifying SLA
 Which injectors to use, how many VUsers against each injector
 Collate results periodically
 IP Spoofing
 Error reporting
 Transaction reporting etc.

Taking analogy from our example controller will add following parameter to the VUGen
Script

1) 3500 Users are Surfing the Products Page of Amazon.com

2) 750 Users are in Checkout

3) 500 Users are performing Payment Processing

4) 250 Users are Checking MyAccount Page ONLY after 500 users have done Payment
Processing
Even more complex scenarios are possible

1. Initiate 5 VUsers every 2 seconds till a load of 3500 VUsers (surfing Amzon product
page) is achieved.
2. Iterate for 30 minutes
3. Suspend iteration for 25 VUsers
4. Re-start 20 VUSers
5. Initiate 2 users (in Checkout, Payment Processing , MyAccounts Page) every second.
6. 2500 VUsers will be generated at Machine A
7. 2500 VUsers will be generated at Machine B

Agents Machine/Load Generators/Injectors

LoadRunner Controller is responsible to simulate thousands of VUsers - these VUsers

consume hardware resources for example processor and memory - hence putting a limit on
the machine which is simulating them. Besides, Controller simulates these VUsers from the
same machine (where Controller resides) & hence the results may not be precise. To address
this concern, all VUsers are spread across various machines, called Load Generators or
Load Injectors.

As a general practice, Controller resides on a different machine and load is simulated from
other machines. Depending upon the protocol of VUser scripts and machine specifications, a
number of Load Injectors may be required for full simulation. For example, VUsers for an
HTTP script will require 2-4MB per VUser for simulation, hence 4 machines with 4 GB
RAM each will be required to simulate a load of 10,000 VUsers.

Taking Analogy from our Amazon Example, the output of this component will be

Analysis:

Once Load scenarios have been executed, the role of "Analysis" component comes in.

During the execution, Controller creates a dump of results in raw form & contains
information like, which version of LoadRunner created this results dump and what were
configurations.

All the errors and exceptions are logged in a Microsoft access database, named, output.mdb.
The "Analysis" component reads this database file to perform various types of analysis and
generates graphs.

These graphs show various trends to understand the reasoning behind errors and failure
under load; thus help figuring whether optimization is required in SUL, Server (e.g. JBoss,
Oracle) or infrastructure.
Below is an example where bandwidth could be creating bottleneck. Let's say Web server
has 1GBps capacity whereas the data traffic exceeds this capacity causing subsequent users
to suffer. To determine system caters to such needs, Performance Engineer needs to analyse
application behaviour with abnormal load. Below is a graph LoadRunner generates to elicit
bandwidth.

Advantages of loadrunner:
1. No need to install it on the server under test. It uses native monitors. For Ex: perfmon for
windows or rstatd daemon for Unix
2. Uses ANSI C as the default programming language1 and other languages like Java and
VB.
3. Excellent monitoring and analysis interface where you can see reports in easy to
understand colored charts and graphics.
4. Supports most of the protocols2.
5. Makes correlation3 much easier. We will dig into correlation through a series of posts
later.
6. Nice GUI generated script through a one click recording, of course you would need to
modify the script according to your needs.
7. Excellent tutorials, exhaustive documentation and active tool support from HP.

Disadvantages of loadrunner:
1. Programming/Scripting language is used to represent the captured protocol data and
manipulate the data for play-back.
2. Protocol is simply a language that your client uses to communicate with the system.
3. Correlation is a way to substitute values in dynamic data to enable successful playback.
 Silk Test
Silk Test is a tool specifically designed for doing REGRESSION AND
FUNCTIONALITY testing. It is developed by Segue Software Inc.

SilkTest is powerful software testing automation tool for running automated test cases on the
front end. It is a tool for testing Web based applications across different browsers. Silk Test
is the industry’s leading functional testing product for e-business applications, whether
Window based, Web, Java, or traditional client/server-based. Silk Test also offers test
planning, management, direct database access and validation, the flexible and robust 4Test
scripting language, a built in recovery system for unattended testing, and the ability to test
across multiple platforms, browsers and technologies.

SilkTest maximizes your productivity by providing a completely integrated recovery system,

which makes it possible for you to run your tests unattended. When the application you are
testing fails, or even crashes, the recovery system restores the application to its base state so
that the rest of your tests can continue to run.

Mostly anything you can do in a Web browser. You can enter text in text fields, click
buttons, check boxes, get files, and clear your cache with SilkTest.

Silk Test Working-

During SilkTest testing, it interacts with the GUI to submit operations to the application
automatically. Thus SilkTest can simulate the actions of a person who is exercising all the
capabilities of an application and verifying the results of each operation. The simulated user
(SilkTest) is said to be driving the application. The application under test reacts to the
simulated user exactly as it would react to a human user.

SilkTest consists of two software components that execute in separate processes: They are

1. The SilkTest host software.

2. The 4Test Agent software.

SilkTest Host Software: This host software is the SilkTest component you use to develop,
edit, compile, run and debug your test scripts and test plans. The machine that runs this
component is often referred to as the host machine.

4Test Agent Software: This SilkTest Agent is the component of SilkTest that interacts with
the GUI of your application. This Agent translates the commands in your 4Test scripts into
GUI specific commands, driving and monitoring the application you are testing. The Agent
can run locally on the same machine on which the Host is running or, in a networked
environment, any number of Agents can run on remote machines. In a networked
environment, the machine that runs the Agent is often referred to as the remote machine

Segue Testing Methodology

SilkTest compliments the Segue Testing Methodology. This methodology consists of six
phases:

1.Plan.
2.Capture.
3.Create.
4.Run.
5.Report.
6. Track.

1. Plan: You begin the automated testing process by creating a test plan. A test plan is a
document that helps you to plan and manage the testing of your application. You develop test
plans in SilkTest’s test plan editor.

2. Capture: In this capture phase, you introduce SilkTest to the AUT by building a
framework for your testing. First you record a test frame, which is a central repository for
information about your application.

3. Create: After you have defined your test requirements and created a framework for testing
your application, you can begin creating test cases.

4. Run: This run phase consists of selecting one or more test cases and executing them
against your application. You have the option of running individual test scripts, a collection
of test scripts called a suite, or selecting specific test cases (using attributes) in a test plan, or
the entire test plan.

5. Report: During reporting phase, you review statistical information about the running of
each test case. SilkTest automatically records the statistics in a results file when the test case
is executing. From this results file, you can generate pass/fail reports.

6. Track: This is the final phase to identify defects in the application you are testing. If you
want to implement a defect-tracking system, to monitor your application as it progresses
through development and testing to ensure that the application is performing as expected.

SilkTest Features

SilkTest offers a complete solution for the automated testing of Web and client/server
applications.

1. SilkTest’s organization of resources within a project.

 2. With SilkTest you can perform functional, operability and regression testing, test
planning and management using the test plan editor, and database validation using
DB Tester.
3. The Basic Workflow guides you through the process of creating a test case, while the
Data Driven Workflow assists you in creating a data driven test case.
4. It has automatic completion within the Editor.
5. Easy recording of tests.
6. It has true object recognition.
7. Unattended testing with built-in recovery system.
8. Fully integrated test plan creation and maintenance.
9. Browser and platform independence.
10. It has multiple testing options using browser extensions.

Selenium automation testing tool

Selenium is a free (open source) automated testing suite for web applications across different
browsers and platforms. It is quite similar to HP Quick Test Pro (QTP now UFT) only that
Selenium focuses on automating web-based applications. Testing done using Selenium tool
is usually referred as Selenium Testing.

Selenium is not just a single tool but a suite of software's, each catering to different testing
needs of an organization.

It has four components.

1. Selenium Integrated Development Environment (IDE)

2. Selenium Remote Control (RC)
3. WebDriver
4. Selenium Grid

At the moment, Selenium RC and WebDriver are merged into a single framework to
form Selenium 2. Selenium 1, by the way, refers to Selenium RC.
  Selenium IDE
Selenium Integrated Development Environment (IDE) is the simplest framework in the
Selenium suite and is the easiest one to learn. It is a Firefox plugin that you can install as
easily as you can with other plugins. However, because of its simplicity, Selenium IDE
should only be used as a prototyping tool. If you want to create more advanced test cases,
you will need to use either Selenium RC or WebDriver.

 Selenium Remote Control (Selenium RC)

Selenium RC was the flagship testing framework of the whole Selenium project for a long
time. This is the first automated web testing tool that allowed users to use a programming
language they prefer. As of version 2.25.0, RC can support the following programming
languages:

 Java
 C#
 PHP
 Python
 Perl
 Ruby

 WebDriver
The WebDriver proves itself to be better than both Selenium IDE and Selenium RC in
many aspects. It implements a more modern and stable approach in automating the browser's
actions. WebDriver, unlike Selenium RC, does not rely on JavaScript for Automation. It
controls the browser by directly communicating with it.
The supported languages are the same as those in Selenium RC.

 Java
 C#
 PHP
 Python
 Perl
 Ruby

 Selenium Grid
Selenium Grid is a tool used together with Selenium RC to run parallel tests across
different machines and different browsers all at the same time. Parallel execution means
running multiple tests at once.

Features:

 Enables simultaneous running of tests in multiple browsers and environments.

 Saves time enormously.
 Utilizes the hub-and-nodes concept. The hub acts as a central source of Selenium
commands to each node connected to it.

Advantages of Selenium

1. Selenium is a suite of tools that helps in automating only web applications.

You should have a basic understanding of Java or any other object-oriented
 programming language for using Selenium.It has capabilities to operate
across different browsers and operating systems.

2. Selenium IDE -a Firefox plugin that lets testers to record their actions as
they follow the workflow that they need to test.

3. Selenium RC-was the flagship testing framework that allowed more than
simple browser actions and linear execution. It makes use of the full power
of programming languages such as Java, C#, PHP, Python, Ruby and PERL
to create more complex tests.

4. Selenium WebDriver-is the successor to Selenium RC which sends

commands directly to the browser and retrieves results.

5. Selenium Grid-is a tool used to run parallel tests across different machines
and different browsers simultaneously which results in minimized execution
time.

Disadvantages of Selenium

1. Supports only web based applications.

2. No IDE, so the script development won't be fast

3. Cannot access controls within the browser.

4. No default test report generation.