/
Test Suites Module

Back to QMetry All Products Help Page

Test Suites Module

Introduction

A test suite in QMetry is a collection of test cases organized to test a complete scenario, such as BVT, smoke, or regression tests. Test suites are part of specific cycles but can be copied to new cycles for future testing. They are executed with environmental or configuration details to provide context on the testing setup.

image-20240808-081102.png

 

Test Suites Module - UI

The Test Suite module UI in QMetry enables users to:

  • Create and organize test suites for efficient test management.

  • Define and manage test case execution plans within each test suite.

  • Monitor the status and results of test cases and suites.

  • Configure test suite settings and parameters to tailor them to specific testing needs

Icons on Test Suite List View:

image-20240808-083257.png

Column Icon

Column Name

Column Icon

Column Name

Linked Platform(s) Count

Linked Test Case(s) Count

Total Test Case Executions

Total Unassigned Test Case Executions

Total Remaining Execution Time (mins)

Total Execution Time (mins)

Manage Grid Columns

Users can personalize their view through the Arrange Columns option, and these customizations are saved for future sessions.

image-20240808-083452.png

 

Customization Options:

  • Show More Columns: Add additional columns to the view.

  • Hide Columns: Remove less important columns from the view.

  • Re-arrange: Change column order using drag and drop.

Users can find specific columns using the search box within the Arrange Columns menu. Additionally, they have the option to select all columns, clear selections, or restore default settings.

  • Resize Columns: Users can adjust the width of the columns.

Filter Test Suites

QMetry simplifies test suite filtering, allowing users to sort by summary, release, cycle, status, owner, and more.

image-20240808-084610.png

  • Basic Filters: Basic filters like summary, release, cycle, and platform are provided for easy filtering.

  • Advanced Filters: For more precise filtering, users can also add custom fields to the filter criteria by clicking on the '+' sign.

image-20240808-085350.png

  • Clear Filter: Click the Clear Filters button above the module grid or in the left pane to reset the view to its default state.

  • Reset Filters: Select the Reset Filters option to revert all advanced filter fields to their default settings, removing any custom selections.

  • Clear Sorting: Click the Clear Sorting button above the module grid to remove any applied sorting preferences.

image-20240808-090749.png

Managing Test Suite Folders

Users can organize test suites into folders for more efficient management. Each folder groups test suites with similar attributes, enabling bulk operations such as cut, paste, copy, edit, and delete.

Folders can be created directly under the root directory or as subfolders within existing folders. Test suites within a folder inherit the default attributes of that folder. The number of test assets, including both active and archived test suites, is displayed next to each folder name.

image-20240808-091544.png

Adding a Test Suite Folder

The tree/folder panel is expandable, allowing users to widen the area for longer folder names. Users can also hover over a folder to view its full name.

image-20240808-091717.png

  • On the Test Suite tree, click on the New Folder button to add a new folder.

  • Enter Folder Name and click Create.

  • The new folder is reflected on the tree.

Ensure that users have the required permissions for the operation by checking Administration > Roles for individual roles.

Editing a Test Suite Folder

image-20240808-092211.png

To edit a test suite folder, these are the steps:

  1. Select the folder in the tree view and click the ‘Edit’ icon on the toolbar.

  2. The folder detail screen will open in editable mode.

Deleting a Test Suite Folder

To delete a test suite folder, users need to archive the test suite first. Here are the steps:

  1. Select the folder on tree and click on the Delete icon on the toolbar.

  2. On deleting a folder, all the sub folders and entities within the folder will be deleted.

Root folder cannot be deleted.

Include/exclude Test Suites from Sub Folders at Parent Folder

Users can choose to include or exclude test assets from sub-folders in the parent folder view. By default, assets from sub-folders are shown. To hide them, open the Options drop-down and uncheck "Include Entities from Sub-Folder."

Test Suite Detail Page

To view the Test Suite detail page, click the test suite entity Key in the grid view, which will display detailed information in the current window. For easier comparison, users can also open the page in a new tab.

The main screen of the Test Suite module displays test suites within the Project based on the selected folder.

image-20240808-093801.png

Organize Tabs

The detail page of QMetry assets features various tabs. Users can customize the tab order, and this arrangement is saved for each user.

Navigation

Once on the test suite details page, users can return to the main test suite screen by clicking on the breadcrumbs.

Tabs

  • Individual Section Tabs - The Test Suites detail page includes tabs for Test Executions, Test Cases, Platforms, Details, Release & Cycles, Attachments, Comments, and Change Log. Each tab shows and allows editing of relevant details.

  • Combined View - The tab shows the previous view of the details page. All sections are visible on a single page, but the information is read-only and cannot be edited.

Managing Test Suites

Managing test suites in QMetry allows users to efficiently add, organize, filter, and view detailed information for test suites within a project, streamlining test management and comparison.

Adding a Test Suite

To add a test suite, follow these steps:

  1. Open the Test Suite module.

  2. Click on the New button on the toolbar.

  1. Create Test Suite screen opens.

image-20240808-110501.png

  1. Enter the following fields to add a test suite:

  • Summary - Enter the name of the test suite.

  • Owner - Assign test suite owner.

  • Status - Select suitable status from the drop down.

  • Associated Releases: While adding a test suite, select the release and cycle for which the test suite is being added. The list of releases appears according to the releases added on the project details page (Projects > Project / Release / Cycle).

  • Associated Cycles: Cycle list is prompted for the selected release.

  • Description: Enter the description for the test suite. This can be added as either Plain Text or Rich Text.

    • Rich Text: The Rich Text Editor allows users to format text, insert images, hyperlinks, and tables, reducing the chances of missing important details.

    • Image Upload: Users can add images up to 1 MB to any Large Rich Text Editor field (System or Custom) by:

      • Browsing and uploading the image.

      • Inserting the image via URL.

      • Copying and pasting the image directly into the editor.

  • To use rich text editor, it must be enabled for the Project from - Projects > Project / Release / Cycle. Refer Managing Projects, Releases and CyclesManaging Projectsfor more details.

  • Multiple images can be copied and pasted simultaneously. Images can be previewed by clicking on them.

Notes:

  • The supported file formats to upload/copy-paste the image are jpg, jpeg, png, and gif.

  • The maximum file size should be 1 MB (for images uploaded through browsing).

  • Users can paste the html/xml content into the Source editor of the "Large Text" field type and it auto-renders the code to show the text outputs on the screen.

  • Platforms: It is mandatory to link the platform for the execution of the test suite. Read more details in the section below.

  • Link Test Cases: It is mandatory to link at least one test case to create a test suite. Read more details in the section below.

  • User Defined Fields: If UDFs are added for the test case module, then they are displayed on the Create Test Suite screen and Test Suite Details screen.

Automation Testing

For Automation Testing, refer to QMetry Automation Agent.

Linking Platforms with Test Suite

Linking a platform is mandatory for executing a test suite. Users must associate platforms with test suites to conduct test runs. A default platform, “No Platform,” is provided for new test suites. Users can add additional platforms via Projects > Platforms.

  1. Go to the Platforms section on the Test Suite creation page.

  2. Click on the Link button.

image-20240808-115032.png

  1. The Link Platforms pop-up opens with the list of Platforms on it. "No Platform" is available by default. 

  2. Filter Platform: To manage a long list of platforms, users can filter by Platform Name, Creation Date, or Created By.

image-20240807-115958.png

  1. Users can also create a new platform from this screen by clicking on the ‘Create New Platform’.

  2. Select single or multiple platforms from the list.

  3. Click the "Link Selected Platforms" button.

View Linked Platforms

After creating the test suite, users can view all linked platforms from the Platform tab. This tab allows users to add, delete, and view both active and archived linked platforms.

image-20240808-114833.png

Linking Test Cases with Test Suite

At least one test case should be linked with the test suite to complete the test suite creation.

There are three ways to link test cases with the Test suite. Linking can be done while adding or editing a test suite. 

  • Link Test Cases from Requirement

  • Link Test Cases 

  • Link By Entity Key

image-20240808-115309.png

Link Test Cases from Requirement

Linking a requirement to a test suite automatically includes all test cases associated with that requirement, eliminating the need for individual selection. This method ensures comprehensive requirement coverage and traceability, enabling the QA team to monitor requirements from creation through execution. Additionally, test cases from requirements in different projects can also be linked.

image-20240808-115648.png

Linking test cases from a requirement minimizes manual effort and saves time by automatically including all relevant test cases. This approach also reduces the risk of overlooking any test cases planned for the requirement.

To link test cases from requirement, follow these steps:

  1. Click the Link Test Cases from Requirement button.

  2. A pop-up shows only requirements with associated test cases.

  3. Users can filter requirements based on Summary, Version ID, Priority, Labels, Releases, Cycles, Created By, or Folder Path.

  4. Select requirements to be linked.

  5. Click the Link Selected Requirement button.

Test cases linked to the selected requirements will be added to the test suite.

Link Test Cases 

Users can link test cases individually by clicking the Link button. They can filter test cases, select multiple test cases, and then link them to the test suite simultaneously.

image-20240808-120138.png

Users can apply Basic or Advanced Filters to quickly find test cases. Additionally, they can view test cases based on the following criteria:

  • Show Test Cases Linked to Releases and Cycles: Displays test cases associated with the release and cycle linked to the current test suite.

  • Show Test Cases Not Linked to Any Test Suite: Identifies test cases not linked to any test suite for the selected release and cycle, highlighting uncovered test cases.

  • Hide Test Cases Already Linked to This Test Suite: Hides test cases already linked to the test suite to avoid duplicate linkage.

Users can link test cases from different projects by selecting the project from the Project drop-down. The screen will then display test cases for the chosen project.

  • Select Test Cases: Choose the test cases you want to link to the test suite. Filters will adjust according to the selected project, with additional filters for Labels and Folder paths available.

  • Link Specific Versions: If a test case has multiple versions, expand it to view all versions. Select the specific version to link with the test suite.

  • Linking Across Projects: To link test cases from multiple projects, select one project at a time. Search and link test cases for the current project before switching to another project to avoid losing previous selections.

  • Finalize Linking: After selecting test cases, click the Link Selected Test Cases button.

Link by Entity Key 

Users can link the latest version of a test case or a requirement to a test suite using comma-separated test case keys. A maximum of 500 entities can be linked at once.

image-20240808-121305.png

Notes:

  • Test case keys will be ignored if the test case is not found, archived, or "Not Approved" (if eSignature is enabled).

  • If a test case key is listed multiple times, it will be linked only once.

  • The latest version of a test case will not replace an older version if the older version is already linked. Only unarchived versions will be linked.

  • Skip Duplicate Linkage:

    • On: The latest version will be ignored if an older version is already linked.

    • Off: The latest version will be linked regardless of existing versions.

  • Linking test cases from a requirement automatically adds all associated test case versions to the test suite. If eSignature is enabled, only "Approved" versions will be linked.

To link entity key, follow the below steps:

  1. Click on the Link by Entity Key button.

  2. Two options will appear - From Requirement and From Test Case.

  3. Select either of the options and a pop-up appears.

  4. Enter Requirement entity keys or Test case entity keys.

View Linked Test Cases

Users can view linked test cases on the Test Cases Tab of the Test Suite Detail Page. From this tab, users can also link or unlink test cases, reorder them, and export all linked test cases.

image-20240808-124044.png

 

  • Link: Users can link individual test cases or all test cases associated with a requirement using the following options:

    • Test Case

    • Test Case from Requirement

    • By Entity Key From Requirement

    • By Entity Key From Test Case

  • Unlink Selected: Users can unlink selected test cases from this screen.

Unlinking test cases after execution will also remove all associated executions from the test suite, including those across all releases, cycles, and platforms.

  • Export All Test Cases: Users can export all test cases linked with a test suite using this button.

  • Change Order: Test cases linked to the test suite will appear in the same order on the test execution screen. However, users can adjust this sequence to match the desired execution flow. Use the Cog icon and select one of the following options:

    • Move Up

    • Move Down

    • Shift to First

    • Shift to Bottom

Export Test Cases Linked to Test Suite

Users can export test cases linked to a test suite, allowing QA Managers to review selected fields in a PDF or Word format.

Required Permissions:

  • Integration app must be enabled in the QTM subscription.

  • Export permission for the Test Suite module is required. Role-based access control is managed under Customization > Roles.

Options:

  • Export all test cases

  • Export selected test cases

Steps:

  1. Navigate to Test Suite > Test Cases tab.

  2. To export all test cases, click the Export All Test Cases button at the top of the test case list.

image-20240807-125238.png

  1. To export desired test cases, select the test cases to export and click on the Export Selected Test Case button.

image-20240807-125358.png

  1. Select the format PDF or Word to export the test cases.

  2. Select the page orientation Portrait or Landscape.

  3. Open the Test Case Fields drop-down, and open the Test Cases/Test Steps tab.

  4. Select the fields you want to export the details of the required fields of the test cases and test steps.

  5. Click Save.

  1. Select from the Additional Options:

  • Test Suite Details

  • Test Case Approval Details (This option is available only for the eSingature-enabled projects.)

  • Issues Linked to Test Cases

  • Requirements Linked to Test Case

  1. Click on the Export button.

Users can view the progress in the Scheduled Task section.

Editing a Test Suite

An in-place editor is available on detail screens for all test assets. Hover over a field and click the edit icon to open it in editable mode. Users can then enter a new value and save the changes. To modify a value on the test suite details page, simply hover over the field and click the edit icon.

image-20240807-130043.png

Associate Releases & Cycles

Releases and cycles are initially associated with a test suite during its creation. To update these associations, users can do so via the test suite details page or the edit page.

To associate additional releases and cycles with an existing test suite:

  1. Click the Add New button to enable the drop-down menus for selecting releases and cycles.

  2. Choose the desired releases and cycles from the drop-down lists.

  3. Click Save to apply the new associations. The updated releases and cycles will be displayed in the grid.

To remove the release and cycle association for the test suite, click on the Unassociate button for the combination of release and cycle.

image-20240807-130455.png

View Archived Release & Cycles

Users can apply filters to view the required releases and cycles. To view archived releases and cycles, select the Show Archived checkbox in the filter options.

Adding Attachments

The tab allows users to add attachments to the test suite. Read more Managing Attachments.

Test Execution

On the Test Execution tab, records are shown by platform after associating a release, cycle, and platform with a test suite. The number of test execution records corresponds to the various combinations of platforms, releases, and cycles linked to the test suite.

The Test Executions tab in the Test Suites module displays only those test suites that have at least one Release, Cycle, and Platform linked with them.

image-20240808-132812.png

The screen displays details like Test Suite Summary, Release, Cycle, Platform, Status, and Execution Time.

Customize the Test Execution tab view

image-20240807-131111.png

Users can adjust the Test Execution tab view in the test suite to meet their needs:

  • Show/Hide Columns: Choose which columns to display or hide.

  • Arrange Columns: Reorder columns to fit your preferred layout.

  • Resize Columns: Adjust column widths for better visibility.

  • Sort Executions: Organize test suite executions by different columns to easily find related information.

Notes:

  • The Execute button, Automation Execution button, and Summary could not be arranged or kept hidden on the screen.

  • The preferences applied to the Test Execution tab view are preserved.

The Test Execution view configured in the Test Suite > Test Execution tab will also apply to:

  • Bulk Operations: For eSignature-enabled projects, during bulk approval processes for statuses like Approved, Request Changes, and Closed.

  • Traceability Report: When tracing by test executions.

From the Test Execution Screen, users can Assign Testers or Bulk Execute Test Cases.

A. Assign Tester:

To assign a tester to execute a test suite on a specific platform:

  1. Click on the Assign Tester tab. The Select User drop-down will become available.

  2. Select the test suite(s) by checking the corresponding boxes.

  3. Choose the user from the drop-down menu.

The selected tester will be assigned to execute the chosen test suite on the designated platform.

B. Bulk Execute:

Users can perform bulk execution of test cases from this screen. The process for bulk execution is detailed below.

Archive Test Execution/Test Run

The Test Execution Archive feature allows users to lock executions to prevent any further changes.

image-20240808-134353.png

Test executions can be archived in the following scenarios:

  • When the entire Test Suite is archived.

  • When the associated Release or Cycle is archived.

  • When the associated Platform is archived.

  • When the Test Execution itself is archived.

  • When the Project is archived.

Archived test executions will not appear on the Test Executions tab of the test suite.

Archive Test Runs Manually

  1. Open the test suite detail page and go to the Test Execution tab. Click on the Archive button for the test run to archive.

The confirmation message pops up. Click Yes to proceed.

View Archived Test Runs

To view archived test runs, select the Show Archived Records check box in the filter on the Test Execution tab. This will display test runs archived due to platforms, releases, cycles, or the test run itself.

For imported data, any changes in the status of archived test case executions will be ignored.

Impact of Archiving a Test Execution/Test Run

  • Any new test case linked to the test suite will not be added to the archived execution.

  • Existing linked test cases would not be allowed to unlink if they are linked to an archived execution.

  • When new version of test case is created, it cannot be force-synced or manually sycned in the archived execution.

  • In case of test case steps update in the existing version, any change in test case step data in that test case version will reflect in test cases under archived execution.

  • Bugs, attachments which are part of archived executions cannot be unlinked.

  • Test Case and Step Comments, Execution Status, Actual, Expected outcome added during execution cannot be changed for archived executions.

  • When "archived" execution is unarchived, any test cases that are linked to the test suite, are auto-linked with the execution after unarchive.

  • If an execution's release, cycle or platform are archived, it will not be allowed to unarchive.

  • Deleting any executed test case or test case version will remove the executions from the archived executions.

  • In a cloned project (with data), the archived test run also get copied.

  • For eSignature enabled project, State (In-Review/Approve/Closed) of the test Run will not be allowed to change.

  • Closed test execution can be archived but on unarchive test execution nothing will happen.

  • Test case data grid related modification will not be reflected in archived execution.

Unarchived Test Runs/Test Executions

  • Missing test cases will be linked on unarchive test execution with associated test case version and latest data grid values.

  • Existing linked Test case version will not get modified on unarchive test execution.

  • Manually archived test execution will not get unarchived if any associated archived Platform/Release/Cycle is unarchived.

  • If Test Case-Test suite Data Grid Mapping is modified after archive test execution, then older mapping (steps) will be removed on unarchive test execution.

  • In case of parameterized test case, edited value of test case will not affect test cases under the archived test run. Once the test run is unarchived, the values will be updated in the respective test cases.

When a test suite with manually archived test runs is copied within the project, the archived test runs will be copied as archived in the test suite same like release/cycle/platform. Options while copying the test suite like Test Case version Sync, Reset to Not Run and Reset Assignee also impact the archived test runs in copied test suite.

Bulk Execute 

This feature enables bulk assignment of Execution Status, Execution Type, and Assignee for test suites with multiple test cases, platforms, releases, and cycles. Once triggered, the assignments are processed in the background.

Steps

1. Go to the Test Suite module.

2. Open a test suite detail page and select the Test Execution tab on it. The screen displays associated test cases.

3. Select the test cases you want to execute and click on the Bulk Execute button.

Maximum 20 test execution runs can be selected at a time.

4. The next screen displays the combination of test cases with their associated Platforms, Releases, and Cycles.

On this screen, users can:

  • Assign an assignee to the test cases

  • Assign an execution status to the test cases

  • Assign an execution type to the test cases

The image below denotes selection range of respective check boxes on the screen.

 

A. Platforms

B. Releases & Cycles

C. Platform Attribute: To view Platform attributes, click on the icon adjacent to platform name.

D. Select a particular Release & cycle for all test cases on the current page

E. Select a Platform with all test cases under all releases & cycles

F. Select individual test cases

Back to QMetry All Products Help Page