Lesson 4: Ranorex Test Suite

As already mentioned in Lesson 2 it is recommended to seperate a test case into reusable and parametrizable test automation modules. Within the Ranorex test suite you can not only easily create new test cases by combining already existing automation modules but also specify global parameters and data connectors to set up data driven test cases. Within the following lesson you learn about:

Test Suite Editor

After the creation of a new test suite project a Ranorex Studio project automatically containes a <ProjectName>.rxtst file. By default the test suite file is open automatically.

: Ranorex test suite file

Looking on the project created during the lessons 1 - 3 you can now create a new test case based on the existing modules. To open up the test suite you can double click the *.rxtst file or use the 'View Test Suite' button shown in the toolbar panel.

: Ranorex Studio Toolbar

Add a New Test Case

You can add a new test case by clicking the 'NEW' button. Depending on the current item selection of the test suite you are allowed to add different types of test suite elements. A test suite can be made up of the following items:

Test Cases
Record Module
Code Module
Folder

Select the 'Test Case' item to create a new test case at the current position.

: Add new test case

Note: Within a test suite test cases can be added to different layers. A test case can also be added as a child to an existing test case or to a folder item.

The second test case should check the connection state of the VIP Database application. First select and then click on the test case to rename it to 'CheckConnection'.

: Rename the test case

Reuse of Existing Modules

Now, you are ready to reuse two existing record modules from the Ranorex Module Browser View. You can simply drag and drop items from the module browser into the test case or you can add new or existing items using the context menu as follows:

: Add an existing recording module

: Select 'StartSystemUnderTest' module from the list

Press 'OK' to add the record module to the test case. Repeat the same to add the record item 'CloseApplication' to the test case.

Insert a New Recording Module

Now the test case only covers only starting and closing of the application under test using two exisiting modules. In order to check the connection state of the VIP Database application after starting, you can create a new recording for the test case. Use the context menu again but this time to insert a new recording into the test case.

: Insert a new record module

Before you start with a new recording to check the connection state of the VIP Database application, you can run the executable manually or simply open and run the 'StartSystemUnderTest' recording. The new recording module should cover the following steps:

Disconnect Application - Selection of menu item 'File' | 'Disconnect'
Connect Application - Selection of 'File' | 'Connect'
- Confirm Dialog 'Establish DB Connection'
Verify 'Online' Status Text

Keep in mind that you need to select the 'Global Recording' option because the system under test will be started by another recording module.

Note: In the event that the first action is carried out within a .NET based application is a click performed on a menu item, the click is not really executed. To avoid this situation you need to set the current focus first by clicking the title bar for example.

: New created recording scenario

Now the test case 'CheckConnection' is ready to get executed. Use the context menu item 'Play Selected Test Case' to run it and see whether it works or not.

: Play selected test case

General Structure of a Test Suite and its Test Cases

During the last section of lesson 4 you learned how to add new test cases to the test suite by combining existing modules with a newly created recording. To see the different ways of organizing a more complex test suite project open the sample test suite project 'VIPTestSuite' from the Ranorex Studio start page:

: Open sample test suite project 'VIPTestSuite'

The sample includes the different types of elements within a test suite and covers all possible combinations.

: Ranorex test suite structure

#1 Folder	Used to group multiple test cases
#2 Test Case	Represents a test case which can contain modules, a setup or teardown region or other test cases
#3 Setup Region	Groups modules used to prepare a test case (e.g. start system under test, initialize a database, etc.)
#4 Tear Down Region	Groups modules used to clean up a test case (e.g. deleting files generated during test execution, closing application, etc.)
#5 Recording Module	Automation module generated by recording
#6 Code Module	Automation module written in code

Setup and Teardown Regions

The Setup and Tear Down regions are used to prepare and clean up a single test case execution.

The setup region will be executed before any other module held by the test case and should hold any modules needed to bring the system under test in exactly the state your test can start from. A typical application of this section is to start the application under test and log in with a user.

The teardown region will be executed when the execution of the test case has been finished, which means after all modules have been executed, or an error has aborted the test case. The teardown region should hold any modules needed to clean up the system under test and bring it to the original state. A typical application of this section is to delete all added data and close the application under test.

The setup region will automatically be placed at the beginning and the teardown region will automatically be placed at the end of a test case.

Use the context menu to specify which modules of a test case should be part of the setup or teardown process.

: Add module to setup region

: Nested test case for a general setup and teardown region

Note: If you want to define one setup and one teardown region for a set of test cases simply nest your test cases as shown in the picture on the right.

In order to quickly deactivate a certain module instead of deleting it from the test case use the context menu item 'Disable'.

Searching For Elements
Use the 'Search' text box to find elements in the Test Suite.

: Search result for text 'AddVIP'

Running a Test Suite

To run a test suite click the 'RUN' button shown within the test suite editor's toolbar. Use the check boxes to specify whether single test cases or group of test cases should be part of a test suite run. Create different test suite run configurations using the drop down box from the toolbar.

These test suite run configurations define if a specific test case will be executed or not.

For example one test suite run configuration can be defined where the data driven tests stored in your test suite will be executed and one test suite run configuration can be defined where they won’t.

: Activate different test suite run configurations

: Add or remove test suite run configurations

Running Tests without Ranorex Studio

As you already learned within Lesson 1 Ranorex Studio creates an executable file from your test suite project. In order to execute the test suite in a runtime environment, you need to have the generated executable (*.EXE) with the test suite file (*.RXTST) in the same directory. You can execute the test suite outside Ranorex Studio using

the Ranorex Test Suite Runner
via command line

Ranorex Test Suite Runner

Simply double-click the *.RXTST file from the project's output folder to open the Ranorex Test Suite Runner.

: External Ranorex Test Suite Runner

You can use the Ranorex Test Suite Runner to run the test suite, execute certain test cases or just play a specific automation module. Additionally one can create new run configurations the same way as within a Ranorex Studio project.

Running Test via Command Line

You have the following options to execute the test suite from the command line:

<GeneratedTestSuite>.exe /<argument>

Allowed Arguments:

help|?

Prints this help text.

testcase|tc:<name of test case>

Runs this test case only. (e.g. <GeneratedTestSuite>.exe /testcase:TestCase1)

testsuite|ts:<path to testsuite file>

Runs the test cases defined by the test suite (*.rxtst) file. By default

the *.rxtst file with the same name as the <testsuiteexe> is used</testsuiteexe>
<testsuiteexe>- or the first *.rxtst file in the same folder as <testsuiteexe>.</testsuiteexe></testsuiteexe>

runconfig|rc:<configuration name>

Runs the test cases of the specified configuration defined by the *.rxtst file. Configurations can be edited using Ranorex Studio or TestSuiteRunner. By default, the currently selected run config is used. (e.g. GeneratedTestSuite.exe /runconfig:TestRun)

reportfile|rf:<report file path>

Sets the name (and path) of the report file. By default, the filename specified in the *.rxtst file is used.

reportlevel|rl:Debug|Info|Warn|Error|Success|Failure|<any integer>

Sets the minimum report level that log messages need to have in order
to be included in the log file.
These levels correspond to the following integer values:
Debug=10, Info=20, Warn=30, Error=40, Success=110, Failure=120

module|mo:<module name or guid>

Runs the module with the specified name or guid.
Assemblies loaded by <TestSuiteExe> and assemblies referenced in the *.rxtst file are searched.
(e.g. GeneratedTestSuite.exe /module:Recording1)

param|pa:<global parameter name>=<value>

Creates or overrides values for global parameters specified in the test suite.
(e.g. GeneratedTestSuite.exe /param:Parameter1="New Value")

runlabel|rul:<custom value>

Sets a custom runlabel for the test run.

listconfigparams|lcp:<custom value>

Lists all settable configuration parameters and their values.

listglobalparams|lp:<custom value>

Lists all global parameters and their values.

Test Suite Settings

Open the test suite's settings dialog by selecting the context menu item 'Properties' on the test suite root node.

: General settings of a test suite

: Global parameters table

General Test Suite Settings

Name	Specifies the name of the test suite (same as shown within the test suite editor)
Description	Description of the test suite (same as shown within the description column of the test suite editor)
Show progress dialog	Specifies whether a progress dialog should be shown during test suite execution or not
Report Level	Specifies the level of report shown with the report file

Additional Report Settings

Report File Directory	Specifies the directory for the generated report files
Reporting Filename	Specifies the filename generated by the report; move the mouse over the 'Description' label to see the variable values
Custom XSL Filename	Specifies a customized style sheet file used instead of the Ranorex style sheet to present the reports
Auto Save Interval	Specifies how often the report file is saved during an execution

Global Parameters

Globally specified parameters can be accessed and used by every test case within the test suite. The VIPTestSuite example project uses a global parameter to specifiy the directory path for the application under test. You can easily connect global parameters with variables as shown in the recording module 'StartSystemUnderTest'. In addition you can use the global parameter to transfer values or states between test cases.

Test Case Settings

The 'General' tab of the test case's properties dialog is mainly used to setup how a broken test case impacts other test cases within the test suite.

Name	Name of the test case
Description	Description of the test case (same as shown within the description column of the test suite editor)
Report Level	Specifies the level of report shown with the report file; by default the level of report is set to 'Inherit' which means that the test case inherits the setting from its parent (test case or test suite)
Error Behaviour	Specifies the behaviour of the test case and the test suite in case of on error Set to 'Continue with iteration' if the case should not break in case of an error caused by a particular data set Set to 'Continue with sibling' if the case should not break and continue with the next test case Set to 'Stop' in order to break the test suite run in case of an error

: Test case properties dialog

: Specifiy error behavior for multiple test cases

Use the context menu to change the report level or the error behavior for multiple test cases.

: Specifying error behavior for multiple test cases