Introduction

Test Suite Execution screen enables Manual Testing and scheduling Automation Testing by QA teams. Test suites are executed on different platforms and the run results are marked on this screen. There are two views available for test execution screen - Default View and Detail View.

You can select the view as per your preference. This page refers to the Default View of the test execution screen.

1 Introduction
2 Executing Test Suites
3 Attachments
4 Comments
5 Execute Individual Test Cases
6 Execution Status of Test Cases with Parameterized Values
7 View Statistics of Execution
8 Bulk Operations
9 Assign Build for Test Executions
10 Create New Build
11 Link Latest Test Case Version at Run-time
12 Test Result Log
13 Change Log
14 Export Test Execution
15 Share Test Execution Runs with Other Users

To read about the Detail View of the test execution screen, refer to Test Execution Screen Detail View.

Executing Test Suites

Open the test suite detail page and select the Test Execution tab.

Execution Time and Remaining Time

You can view the Execution Time and the Remaining Time calculated for Test Execution.

Calculation of Execution Time and Remaining Time for the test suite is useful for the QA Manager in Test Planning. It gives accurate estimates for the test suite. These test case level estimates refine each time a test suite is executed. Testers can make recommendations to have the test estimates revised based on the last few executions in different environments.

Two columns, Execution Time and Remaining Time, are added to the grid at the following places in the application:

Test Executions tab in Test Suite detail page.
Test Executions tab in Test Case detail page.
Execution runs in the Issue detail page.

Calculating Execution Time of a Test Suite

Execution Time of a test suite is calculated based on Estimate Time of all the test cases in that test suite.

Calculating the Remaining Time of a Test Suite

The Remaining Time is reduced for test cases with “Pass” and “Not Applicable” status only.

The test cases with “Not Run”, "Failed", "Blocked" or "any other custom" execution status will be considered as remaining, as the work still remains pending for these execution statuses.

Example

The following example shows the calculation of both Execution Time and Remaining Time.

	Execution time	Status	Remaining time

	Execution time	Status	Remaining time
TC1	2	Pass	-
TC2	3	Not Applicable	-
TC3	4	Failed	4
TC4	5	Blocked	5
	14		9

Execution Time = 14

Remaining Time = 9

Users can carry out manual testing for individual test suites by clicking on the Execute icon

under the Actions column.

Test Execution Screen Default View

The Test Run screen displays the list of test cases covered under the test suite.

The Default View of the Test Execution screen allows users to view, sort, and filter test cases using various parameters. This feature helps organize and streamline the test execution process.

Filtering Test Cases on Execution Screen

You can filter the test executions with basic and advanced filters. Refer Filters to know more.

Notes

You can filter the linked test cases in the test suite by the folder path.
You can also view the linked test cases of specific folders or the subfolder hierarchy of the folder by clicking on the include entities from subfolders checkbox. By default, this option is selected.

Sorting Test Cases on the Execution Screen

You can easily sort test cases on the Test Execution screen by clicking on the respective column headers. Sorting icons (up or down arrows) appear on the header to indicate the selected sort order.

Organizing Columns

The Execution screen has system-defined fields, pre-selected by default. You can further customize their view by selecting additional fields, including custom-defined fields, using the Arrange Column icon.

For instance, you can add the Test Case Folder Path as a column to display the complete folder hierarchy of each test case. While all user-defined fields, except Large Text fields, can be displayed for test cases, all types of user-defined fields are supported for test steps.

Fields can be shown or hidden as needed, and columns can be rearranged by dragging and dropping. Refer Managing Grid Columns to know more.

Components of the Default Execution Screen

A. The main row on the screen displays test cases.

B. Expanded test case to view test steps within it. You can expand or collapse individual steps as well as all steps.

Click on the information icon to view test suite details like:

The platform against which the test is to be run.
Run Attributes: Run Attributes of the platform. You can also tag Platform Attributes at run time. Read more about Tagging Attributes at Runtime in Managing Platforms.
Release: Release as selected for the test execution.
Cycle: Cycle as selected for the test execution.
Build: The Build on which the test is being executed.

This screen also enables the users to update the following parameters:

A. Assign tester to the test case: Open the drop-down and select the user to whom the test case is to be assigned.

B. Log issues/defects to the test case. Read more about Adding/Linking Issues to Test Case and Test Step.

C. Add comments to the test case.

D. Add attachments to the test case.

E. Execution Type: The execution type for the test suite can be set as - automated or manual. By default, this column remains hidden. You can make this column visible from the column list. Also, by default, the execution type is "Manual". Users can change the value by opening the drop-down menu and selecting "Automated".

Use Case: Testers need to run particular test cases as Manually and Automated way. Sometimes testers have to manually mark the status of test cases on the Execution screen. Therefore, testers need an ability to mark the executions as Automated or Manual on the execution screen.

Users can also set the Execution Type for bulk test cases at once. Refer to the Bulk Operations section on this page.

F. Planned Date: Users can set the planned date for test suite execution from this screen manually. Like other execution fields, Planned Date column could be rearranged, shown or hidden, filterable and exported to excel/pdf/word for test execution exports.

Steps:

Go to the Test Execution Screen and open the Default View.
Click on the planned date field for the test case you wish to update.
Select the desired date from the calendar.
Click the checkmark to confirm and set the date.

Modify Execution Time Stamp

Users can modify the test case execution date and time for each test case under the Executed At column.

The following admin-level settings are configured under Administration > General Settings to enable the functionality.

Go to the Edit Execution Time section.
Set the Allow manual edit of last executed date/time flag to On.

If enabled, then only allow editing of the last executed date field in Test Execution

The Executed At field is editable only if this flag is turned On.

The Date and Time can be changed at two levels: individual test cases and bulk test cases through Bulk Operation.

Setup vs. Attended Time

Users can enter Actual Setup Time and Actual Attended Time for each test case on the execution screen. Refer to Tracking Actual Execution Time with Stopwatch Timer document to know about timer settings.

Attachments

Users can add attachments to individual test cases as well as test steps. Refer to Managing Attachments for more details.

Test Case and Test Step Attachments

Attachment Level: The column indicates the level at which the attachments are added.

Test Case: The attachments are added from the Test Case module. Attachments added at the Test Case level can be downloaded. You cannot delete the attachments added from the Test Case module.
Execution: The attachments are added from the Execution screen. Attachments added at the Execution level can be downloaded and deleted.

Comments

On the test execution screen, you can add multiple comments for each test case with the purpose of efficiently collaborating on the testing and keeping track of the conversation.

Steps:

1. Open the test execution screen.

2. Click on the Comment icon for the test case.

The Comment pop-up opens. You can add multiple comments on the screen.

A. An existing comment

B. Add a new comment

To add a new comment, the wide text area opens.

3. Save the comment after adding.

You can edit and delete an existing comment.

A. Edit a Comment: Click on the Edit icon for the comment to edit its content.

B. Delete a Comment: Click on the Delete icon for the comment to delete the comment.

Tagging Users in a Comment

You can tag QMetry users in the Comments section, which improves collaboration among the team. An email notification will be triggered to the user who is tagged on adding/updating the comment.

QMetry users can be tagged by their usernames in the Comments section. For example, @<username>

Execute Individual Test Cases

The test case execution status derives from test case step status and vice versa. Assign Execution Status to individual test steps/test cases.

The derivation of Execution Status of test case and test step depends on the Derive Test Case Execution status from Test Case step status settings done in the Project module. The settings remain “On” by default, which derives test case execution status from test case step status and vice versa.

An option is provided to disable auto-deriving test case execution status from step statuses and vice-versa is provided under Project Settings. Refer to Project Management for more details.

When the Derive Test Case Execution Status from Test Case Step Status settings are turned “On” in the Projects module: The test case execution status derives from test case step status and vice-a-versa as per the set priority of Execution Status as described in Execution Status Management.

When the Derive Test Case Execution Status from Test Case Step Status settings are turned “Off” in the Projects module, When the flag is turned Off, users need to manually change the status of the test case, as it will not get changed as per the status assigned to test steps underneath.

Execution Status of Test Cases with Parameterized Values

The execution screen displays the execution status at the parameter group combination level when executing parameterized test cases.

Use case:

As a tester analyzing a failed test case execution with multiple parameter group combinations, you can quickly identify both the failed parameter combinations or unexecuted cases with a "Not Run" status.

Notes

The execution status for parameter groups will only be displayed for projects in which the “Derive Test Case Execution Status from Test Step Status” setting is enabled.
Only the consolidated execution status for each parameter group combination is displayed based on the test step execution performed for that combination. This execution status is not editable.
The execution status of each parameter combination follows the similar status prioritization logic used to calculate the test case execution status. Refer Execution Status Management to know more.

View Statistics of Execution

Click the Statistics button at the top of the screen.

This displays a status bar regarding the execution status of test cases. The status bar is generated based on the status of each test case execution (i.e., Not Run/Passed/Failed/Blocked/Not Applicable/any other user-defined status) of each test case.

Bulk Operations

Execution Status

You can also assign execution status to multiple test cases at a time.

Use Case: A new build is received from the development team, and the QA Manager has scheduled a Regression Test. The scope covers multiple modules and functionalities thereof. Testers are well aware of the test cases and test steps associated with each module. So they jump-start on executing the test cases one after another without going into details. They keep running continuous execution without updating the run status of each test case. At the end or a certain point they just update the status of bulk test cases with a few clicks instead of assigning status one by one on each test case execution.

To update the Execution Status, perform the following steps:

Open the Bulk Operations drop-down and select Execution Status.
Select test cases to apply the status.
Select the status on the Execution Status drop-down.
Add a comment to explain the status change(optional).
Click Update to save the changes.

Execution Type

You can assign Execution Type in bulk to test cases whose execution type is similar.

1. Open the Bulk Operations drop-down and select the Execution Type.

2. Select test cases on the grid.

3. Open the Execution Type drop-down and select the type that is to be assigned to the selected test cases.

Assignee

Users with ANY of the following permissions can Assign, Modify, Unassign test execution testers:

Test Suite Modify Permissions
Test Suite Execute Permissions.

You can assign test cases in bulk to a tester to execute. It saves QA Lead and Manager's time to assign multiple test cases to the same tester.

Open the Bulk Operations drop-down and select the Assignee option on it.
Select test cases on the grid.
Open the Assignee drop-down and select the tester to whom all the selected test cases are to be assigned for execution.

Planned Date

Users with ANY of the following permissions can modify the Planned On Date:

Test Suite Modify Permissions
Test Suite Execute Permissions.

You can set a single planned date for multiple test cases in bulk.

To bulk set planned date, perform the following steps:

Open the Bulk Operations drop-down and select Planned Date.
Select the test cases from the grid.
Open the Planned Date calendar and select the date for the selected test cases.
If no planned date is available, click on Set as Unplanned.

The planned date for all the selected test cases gets updated in bulk.

Issues

Users can link QMetry issues or Jira defects to multiple test cases at a time.

User Case: A testing team has some test suites containing many test cases. They want to link a Jira issue with multiple test cases. Linking an issue to each execution can be very time consuming. Thus, they want the ability to link a JIRA issue to multiple executions at once.

1. Open the Bulk Operations drop-down and select Link Issues.

2. Select test cases on the grid.

3. Click Assign Issues.

Then follow the same mentioned process of linking existing issues. Read more about Adding/Linking Issues to Test Case and Test Step.

When opening the issue in Jira, the test case links in Jira are displayed in accordance with the defects linked with test cases in QMetry.

Execution Comment

You need Test Suite Execution Permissions to add execution comment.

You can add execution comments for multiple test case runs in bulk. This comes handy when you need to add the same comment for multiple test case execution runs in a test suite.

To set execution comment in bulk, perform the following steps:

On the Execution Screen, open the Bulk Operations menu, and select Execution Comment.
Select test case executions using check boxes and filters.
In the text box, enter the comment, and select the check mark to save.

You can update up to 1,000 records in a single operation.
The Select All option is not available for test case dependent projects.
To use rich text in comments, enable the Rich Text Editor in Project settings.
Rich Text Editor supports formatted text but excludes image insertion, image copy-paste, and user tagging.
You can add comments at the Test Case Execution Level and not at the Test Step Execution Level.
The maximum comment length is 65,535 characters.

Execution Time

Users can modify the execution date and time for multiple test cases at a time.

To enable the editing of the Execution date and time, go to Customization > General Settings & Audit and turn the Allow manual edit of last executed date/time flag On.

1. Open the Bulk Operations drop-down and select the Execution Time option on it.

2. Select test cases on the grid.

3. Select date and time of execution using calendar and click Apply.

The date and time is changed under the Executed At column for the selected test cases.

Assign Build for Test Executions

QMetry defines a Build as a specific iteration of the product/domain to be tested. The feature allows the tester to assign a "build" (aka patch, drop, or increment) to test suites that indicate the version of the product being tested. It helps to track multiple builds of software more dynamically without having to add separate test suites with every new build.

Refer to Assign Build for Test Executions for more details.

Create New Build

Users can create builds from the Execution screen.

Click on the Set Your Build button, select the test cases to which you want to apply the build and select the build from the drop-down.
Manage Build: Clicking on the Manage Build button opens Builds section (Projects > Builds) in another tab. Refer to Managing Builds to know more about adding a new build.

Link Latest Test Case Version at Run-time

Users can associate latest version of the test case to the test suite from the Execution screen by syncing test case version.

In case the test case updated with a new version, which is not associated with the execution, then the Version will show the Refresh icon in blue color.

Sync a Single Test Case to Latest Version

Clicking on the Refresh icon, the test case version will be updated to the latest one. Any changes related to test case parameters and test steps will be updated. The execution status of the updated test case will be reset to Not Run.

Bulk Sync Test Cases to Latest Version

You can sync multiple test cases to their latest version at once.

On the Test Execution Screen, open Bulk Operations drop-down menu and select Sync with Latest Version.
Select the test cases to sync with their latest version.
Click Sync Test Cases.

Test Result Log

The Test Result Log displays detailed information about each test. As each test completes, the status column updates with the test execution result. It generates a full-detailed log of all actions it performs during the test run. The test log provides the test run summary, indicates the results, and contains detailed information for each test operation.

Open the Test Suite Execution screen.

Click on the Settings icon at top right corner and click on the Test Result Log option.

The test log contains all relevant information about the test run, such as Test Case Summary, Test Case Version, Test Case Step, Status, Executed By, and Executed Date and time and so on.

You can apply filter view required logs.

Change Log

Open the Test Suite Execution screen.

Click on the Settings icon at top right corner and click on the Change Log option.

The Change log displays logs of the following activities carried out on the execution screen for a particular Test Execution.

Assign Tester
Add/Edit Comment
Set Build
Change Execution At (time)
From Bulk Operation: Assign Tester, Change Execution At (time)

Change Log helps you to track the changes done at the execution level.

The details include Actor, Action, Test Case Summary, Field, New Value, Old Value, New Value and Date. You have the option to filter the records.

If any changes are done at test step level, then the step details are also displayed in the Action column. It helps users to identify which test step was modified and track the activities in the steps.

Export Test Execution

You can export test executions in Excel and PDF.

Export Test Execution to Excel

Steps

1. On the execution screen.

2. Click on the cog icon at the top right corner of the application header.

3. Select Export.

The next screen opens with the following options. Select options to export test execution.

Test Execution Export Options

Export Test Execution with steps in multiple rows: It will export test cases with steps arranged in multiple rows of excel sheet.
Export Test Execution with steps in single row: It will export test cases with steps included in a single cell.
Export Test Execution without Steps: It will export test cases without steps.

Field Export Options

Visible Fields: Select the option to export only visible fields on the screen. The export will exclude the columns which are hidden.
All Fields: Select the option to export all the fields irrespective of whether the columns are visible or hidden.

Select Export Format

XLSX
CSV

4. Click Export.

Export Test Execution to PDF

Refer to Export from Execution Screen.

Share Test Execution Runs with Other Users

Share Test Execution Runs

QMetry allows easy sharing of test executions with other users. Users can share the asset summary and link for easy access to the latest version.

Steps

Go to the Test Suite module and select Test Executions.
Click the Share icon at the top.
Enter one or more email addresses.
Write a message (optional but recommended) describing the test case or required actions.
Click on the Share button.

Copy Test Asset Link

Users can copy the test case link and share it with other users to provide direct access to a specific test case. Authorized members can access the test case directly via the shared link.

Steps to Copy Test Execution Runs

Open the test case details page.
Locate and click on the Copy Link icon ( ) on the breadcrumb navigation.
Users can share this link via any preferred communication channel.