OLAT

SELENIUM TESTING

Selenium based OLAT UI Testing Lavinia Dumitrescu (Informatikdienste, MELS)

OLAT
SELENIUM TESTING

Objectives:
Overview testing

What is Selenium Testing Framework? OLAT Selenium Testing Framework. What is this, why do we need it? Testing Workflow in OLAT

Run an existing JUnit test class in OLAT testing setup, create

JUnit test class.

your first Selenium

Quo Vadis OLAT Testing?

OLAT
SELENIUM TESTING

1. Testing in OLAT

White box testing - JUnit

the well known JUnit test classes from olat3/webapp/WEB-INF/test/java, triggered by the nightly process

Black box load Testing JMeter (e.g. Enroll, Download, Start Courses, etc.)

automated via cronjob

Black box functional testing UI Testing with the Selenium Framework

triggered by the nightly process

OLAT
SELENIUM TESTING

1.1 Load Testing Overview

OLAT
SELENIUM TESTING

2. Selenium Testing Framework

Selenium is a suite of tools to automate web app testing across many platforms. Selenium Concepts:
A command is what tells Selenium what to do. Each command is one line in the test table of the form: command, target, value. Selenium commands come in three 'flavors': Actions, Accessors and Assertions. Actions are commands that generally manipulate the state of the application. They do things like "click this link" and "select that option". If an Action fails, or has an error, the execution of the current test is stopped. Assertions are like Accessors, but they verify that the state of the application conforms to what is expected. Examples include "make sure the page title is X" and "verify that this checkbox is checked". Element Locators tell Selenium which HTML element a command refers to. The strategies for locating elements are: identifier,id, name,dom, xpath, link (textPattern). Rollup rule: Logic that describes how one or more Selenium commands can be grouped into a single command, and how that single command may be expanded into its component Selenium commands. The single command is referred to simply as a "rollup". UI-Element is an extension for the Selenium IDE and Remote Control that makes it possible to define a mapping between semantically meaningful names of elements on webpages, and the elements themselves. The mapping is stored in a file using JavaScript Object Notation. The file may be shared by the IDE and driven tests running via the Selenium server. It also offers a single point of update (user-extensions.js) should the user interface of the application under test change. com.thoughtworks.selenium.Selenium - Defines an object that runs Selenium commands.

(http://seleniumhq.org/projects/core/reference.html ,chrome://selenium-ide/content/ui-doc.html)

OLAT
SELENIUM TESTING

2. Selenium Testing Framework (continues)

How does a selenium java JUnit test look like ?
public class SeleniumTestingDemoTest3 extends SeleneseTestCase { public void setUp() throws Exception { setUp("http://selenium.olat.uzh.ch/olat/dmz/", "*chrome");} public void testNew() throws Exception { selenium.open("/olat/dmz/"); selenium.click("ui=dmz::username()"); selenium.type("ui=dmz::username()", "administrator"); selenium.click("ui=dmz::password()"); selenium.type("ui=dmz::password()", "nightly1"); selenium.click("ui=dmz::login()"); selenium.waitForPageToLoad("30000"); selenium.click("ui=tabs::learningResources()"); selenium.waitForPageToLoad("30000"); assertTrue(selenium.isTextPresent(Learning")); selenium.click("ui=learningResources::menu_courses()"); selenium.waitForPageToLoad("30000"); selenium.click("ui=learningResources::content_showContentOfLearningResource(nameOfLearningResource=Demo Course)"); //... }}

OLAT
SELENIUM TESTING

2. Selenium Testing Framework

Overview:

Firefox

Selenium IDE records test cases

javascript

Selenium Java JUnit Test Class controls the browsers via the RC Server

OLAT
SELENIUM TESTING

2. Selenium Testing Framework

(continues)

Selenium Remote Control (RC) is a test tool that allows you to write automated web application UI tests in any programming language against any HTTP website using any mainstream JavaScript-enabled browser. Selenium RC comes in two parts: 1. A server which automatically launches and kills browsers, and acts as a HTTP proxy for web requests from them. That is, the Selenium Server is acting as a client-configured proxy for the browser that you automatically started with "getNewBrowserSession". Specifically, it configures your browser to use the Selenium Server as a proxy in its browser preferences. 2.
(http://seleniumhq.org/projects/remote-control/)

Client libraries for your favorite computer language.

The RC server also bundles Selenium Core, and automatically loads it into the browser. By default, Selenium looks for a file called "user-extensions.js", and loads the javascript code found in that file. This file provides a convenient location for adding extensions to Selenium, like new actions, checks and locator-strategies, without needing to modify the core Selenium sources.

OLAT
SELENIUM TESTING

2. Selenium Testing Framework .

Selenium Remote Control (RC) Overview:

user-extensions.js javascript automatically launches/kills browsers HTTP client-configured proxy selenium-server.jar

selenium-client-driver.jar Selenium Java JUnit Tests

(extends com.thoughtworks.selenium.SeleneseTestCase)

OLAT
SELENIUM TESTING

2. Selenium Testing Framework

Overview:

Firefox

Selenium IDE

javascript

Records test cases

Selenium Java JUnit Test Class controls the browsers via the RC Server

OLAT
SELENIUM TESTING

2. Selenium Testing Framework (continues)

Selenium IDE (Firefox-Add-on ) is a complete Integrated Development Environment for Selenium tests. Scripts may be automatically recorded and edited manually providing autocompletion support. The moment you start Selenium IDE, it starts recording commands.
Start by clicking on a link; youll see a clickAndWait appear in the table panel of your Selenium window. Selenium records everything from clicking links to entering data in input fields and submitting forms. To stop recording, hit the red circle in the top-right. You can then add in commands within the table - I suggest starting with verifyTextPresent to check if a snippet of text is found within the page (expressions allowed). Features: 1. Record (in Firefox) and playback (in a JavaScript-enabled browser) 2. 3. 4. 5. 6. Intelligent field selection will use IDs, names, or XPath as needed Autocomplete for all common Selenium commands Save tests as HTML, Java(JUnit) classes, or other formats Support for Selenium user-extensions.js file Option to automatically assert the title of every page - Video: showing how to record and play back a google search. http://wiki.openqa.org/download/attachments/400/Selenium+IDE.swf?version=1

Links: - http://seleniumhq.org/projects/ide/

OLAT
SELENIUM TESTING

2. Selenium Testing Framework

(continues) How to configure the Selenium IDE? (go to Selenium IDE/Options/Options...) Apply rollups
user-extensions.js

Enable UI-Element

OLAT
SELENIUM TESTING

2. Selenium Testing Framework

test

(continues) Stop recording

Selenium IDE screenshot (HTML Format) Run Apply rollups

selenium command call: command, target, value

OLAT
SELENIUM TESTING

3. OLAT Selenium Testing Framework overview

javascript selenium-client-driver.jar com.thoughtworks.selenium.Selenium

OLAT Context
(context.xml)

OLAT App. Layer

Java objects - Selenium macros

ui-map xml files XPaths as UI Locators ant gen_olat-ui-map OLAT Testing Framework Selenium independent

Selenium JUnit Test Classes

user-extensions.js

OLAT
SELENIUM TESTING

3. OLAT Selenium Testing Framework

The Abstraction layer around Selenium contains:

The Context class - hides the complexity and differences of running a selenium test locally (customcontext.xml) on a developer machine, or in the nightly (defaultcontext.xml) context. The Context provides the Selenium instances, a default user for each role (administrator, author, student) and encapsulates the SetupType info (SINGLE_VM, TWO_NODE_CLUSTER, etc.). The ui-map xml files - contains the XPath element locators used for OLAT testing. The user-extension.js is then dynamically generated via the gen_olat-ui-map ant target. (see olat3ext/tests/seleniumtests/ui-map). During the test recording in Selenium IDE, the Application abstraction layer - this is an abstraction around OLAT functionality from the perspective of an OLAT user, it provides Selenium based implementation for the OLAT workflows.

OLAT
SELENIUM TESTING

3. OLAT Selenium Testing Framework

Application abstraction layer (continues)

(continues)

Is has the role to decouple the test cases from the selenium (com.thoughtworks.selenium Selenium) implementation. It is designed as an hierarchical object model with a OLATWorkflowHelper instance as root, acquired via the Context class. From the test developer point of view an OLATWorkflowHelper instance incapsulates the services offered by a just logged in OLAT. Motivation: increasing the code reuse and maintainability of tests classes. Related reading: Webdrivers Page Objects
http://code.google.com/p/webdriver/wiki/PageObjects

Open Problems Language dependent UI locators. The UI locators are language dependent which is very inconvenient. It would have been better if the UI locators could use IDs or names but this is not possible in OLAT. We must find a solution to make the UI locators language independent.

OLAT
SELENIUM TESTING

4. Testing workflow in OLAT

Create new test class and test it locally. If OK, CVS commit.
The local test setup must be defined into the customcontext.xml (this overwrites the defaultcontext.xml)

Automatically deploy and run triggered by the nightly process

The newly commited test class gets deployed and run together with all selenium test classes. This is triggered by the nightly process.

Test results evaluation each tests run generates a JUnit report as an html doc. The
test result report contains the list with the errors or failures, if any. The System.out from JUnit report contains all the info for debugging an error/failure. An error/failure in a new commited class must be fixed by the author. If an error/failure occurs in an old test class than it is Lavinias responsability to fix the problem.

OLAT
SELENIUM TESTING

4. Testing workflow in OLAT Overview

(continues)

Test development (locally)

Automatically Deploy & Run (seleniumnightly)

Results evaluation

OLAT
SELENIUM TESTING

4.1 Develop new JUnit test class

Old style: (e.g. org.olat.test.demo.SeleniumDemo1Test) Record the test case with the Selenium IDE The Selenium IDE well use the already defined XPath elements locators (see ui-map dir). If it cannot find one, it uses a link or an id locator. (XPath element locator sample:
<xpath-ui-element name="username" xpath="//input[@id='lf_login']" description="username input field" testcase="dmz/type(dmz::username,foo)"/>)

export it in java and use as superclass the

org.olat.test.util.selenium.BaseSeleneseTestCase

use the org.olat.test.util.setup.context.Context class for the test setup use the selenium (com.thoughtworks.selenium.Selenium) instances, got via the Context, directly. Based on OLAT application abstraction layer (e.g. org.olat.test.demo.SeleniumDemo1Test) get an org.olat.test.util.selenium.olatapi.OLATWorkflowHelper via the Context class. develop the test case based on the olatapi package.

OLAT
SELENIUM TESTING

4.2 Automatically deploy and run selenium tests (nightly process)

nightly.sh/donightly.sh checkout, build, tagging, start nightly.olat.uzh.ch run junit tests build docu tagging last_stable rebuild & restart

cron

selenium.olat.uzh.ch (recording instance)

selenium-nightly.sh rebuild & restart

seleniumnightly-singlevm (test run instance)

HTTP

(olat3ext...)run-test.sh ant alltests

seleniumnightly-cluster (test run instance)

HTTP

test results (html)

- rebuilds & restarts several times - runs the selenium tests javascript

Firefox Selenium RC Server Firefox Firefox

OLAT
SELENIUM TESTING

4.3 Test results report (http://nightly.olat.uzh.ch/testresults/selenium/00latest/html/index.html)

OLAT
SELENIUM TESTING

4.4 Test results evaluation

Frequent error patterns:

ERROR: Element ui course::menu_link(link=Forum) not found

Possible reasons: - A test bug (e.g. The html element is not visible for a certain role) - A timing issue (e.g. The element is not yet visible, the test needs a delay) - A setup issue (e.g. This html element is not there in the nightly setup but visible locally) - The properties strings were changed in xxx.properties files but not updated in ui-map - A previous RedScreen

Translation Issue encountered. Well, title says it all. Our seleniums test fails if the string
'translation::::' occurs anywhere.

ERROR: Element ui learningResources::content_clickLearningResource(nameOfLearningResourc e=resourceName) not found. This means that your test expects a course to be there but it can't find it.

OLAT
SELENIUM TESTING

5. Tools
Firebug as Firefox add-on, for support with the XPath detection.
(https://addons.mozilla.org/de/firefox/addon/1843)

OLAT

SELENIUM TESTING

OLAT Selenium Mitarbeit Melden bei "Kristina Isacson fr Organisation <kristina.isacson@id.uzh.ch> OLAT JUnit Testcases schreiben Bug Reports mit Selenium Testcase versehen

OLAT
SELENIUM TESTING

6. Getting started with selenium testing in OLAT

Steps: 1. 2. 3. CVS checkout olat3ext/tests/seleniumtests project in eclipse, and configure project. Run ant target: gen_olat-ui-map (olat3ext/tests/seleniumtests/build.xml) it generates the userextensions.js Configure a SeleniumServer Run in eclipse: add new Java Application Run in eclipse, with the following settings: 4. 5. 6. Project: seleniumtests Main class: org.openqa.selenium.server.SeleniumServer Program arguments: -userExtensions C:\eclipse\workspace\seleniumtests\build\classes\user-extensions.js Classpath: seleniumtests (add the selenium-server.jar to the project classpath)

Run the SeleniumServer. So we have a running selenium RC server! Install the Selenium IDE, in Firefox (download at http://seleniumhq.org/projects/ide/). Go to 'Tools'>'Selenium IDE' to launch the tool in Firefox after install. Add the user-extensions.js to the Selenium IDE, by going to Selenium IDE/Options/Options.../General/Selenium Core extensions/ browse and add the previously generated file (...user-extensions.js). AND select 'Enable UI-Element' and then restart Selenium IDE.

OLAT
SELENIUM TESTING

6. Getting started with selenium testing in OLAT (continues)

7. configure a customcontext.xml for running the tests locally against say the selenium.olat.uzh.ch, use the coustomcontext.xml.example as template and replace the map element with this:
<map> <entry key="seleniumHost" value="localhost"/> <entry key="instanceId" value="olat"/> <entry key="jmsBrokerUrl" value="tcp://localhost:61616"/> <entry key="adminUsername" value="administrator"/> <entry key="adminPassword" value="nightly1"/> <entry key="authorUsername" value="author"/> <entry key="authorPassword" value="test"/> <entry key="studentUsername" value="test"/> <entry key="studentPassword" value="test"/> <entry key="guestUsername" value="guest"/> <entry key="guestPassword" value="guest"/> <entry key="singleVmOlatUrl" value="http://selenium.olat.uzh.ch/olat/dmz/"/> <entry key="multiVmOlatUrl1" value="http://selenium.olat.uzh.ch/olat/dmz/"/> <entry key="multiVmOlatUrl2" value="http://selenium.olat.uzh.ch/olat/dmz/"/> <!-this must only be outcommented if the firefox.exe is not in default path

<entry key="browserId" value="*firefox [absolute path to firefox.exe]"/> --> </map>

OLAT
SELENIUM TESTING

6. Getting started with selenium testing in OLAT (continues)

8. Run an existing test class. (olat3ext/tests/seleniumtests) Lets take the org.olat.test.demo.SeleniumDemo1Test.java class and run it as JUnit Test. 9. Create your own test class Go to the recording olat instance: http://selenium.olat.uzh.ch/olat/dmz/ in Firefox Start Selenium IDE, add as Base URL: http://selenium.olat.uzh.ch/olat/dmz/ Record test case (e.g. login, select Learning resources, select Courses, open Demo Course, Show content, Course editor, insert a course element as Roots first child, Publish, Select all, Logout) Go to the Selenium IDE, click stop recording, and click Apply rollup rules. See picture (next slide). Save test in html format, if you want to play it later again. Click Save Test Case As... this saves the test case as html. (XxxTest.html). Save test only if you want to play it later again. Play test case to check if it is OK: Run test case, click Play current test case. Export test case in Java format: Export Test Case As.../Java Selenium RC. Change package name and java class name to correspond to the filename. Change class to extend BaseSeleneseTestCase, delete setUp() call, and add:
Context context = Context.setupContext(getFullName(), SetupType.SINGLE_VM); selenium = context.createSelenium();

Run test from eclipse, as JUnit. Your class should look like: org.olat.test.demo.SeleniumDemo1Test.java.