Test Data Parameterization
Introduction
The Test Data Parameterization feature enables testers to run the same test case multiple times with different data inputs without duplicating test cases. It replaces hard-coded values with parameters, allowing QA teams to store and share reusable test data across QMetry projects. This feature helps maintain consistency and manageability and facilitates the quick creation of automated data-driven tests.
Example
The following values need to be validated against Employee IDs in an organization.
Employee ID | Designation | User Type |
|---|---|---|
1011 | QA Engineer | Regular |
1012 | Developer | Admin |
1013 | DevOps | Super Admin |
1014 | QA Engineer | Read Only |
1015 | Developer | Regular |
Here, Employee ID, Department, and User Type are parameters with specific values. These parameter values create a data grid. First, create the parameters with their values, and then create the data grid.
Step 1: Add Parameters
Navigate to:
- Test Cases > Data Parameter.
Click the New button to create a new parameter.
Enter the following details to create a parameter:
Parameter Name: Parameter names should be unique across projects. Characters, number, and some special characters -_&.' are allowed.
Project: The drop-down displays projects with "Modify" rights of Test Data Parameterization module. Select project(s) to associate the parameter with. Users can select multiple projects to link the parameter with. Maximum 10 projects can be associated at a time.
Description: Enter a short description for the parameter. (This is optional)
Values: Enter parameter values. For example, for "User Type" parameter, the values can be "Read Only", "Regular", "Super Admin" and "Admin".
This way parameters for Employee ID and Departments can also be added.
Edit Parameters
Navigate to:
- Test Cases > Data Parameter.
There are two ways to open the edit screen, follow either of them:
Click on the specific parameter to edit.
Click on the Cog icon of the parameter and select Edit.
The Parameter details page opens in editable mode.
Associated project cannot be removed if the parameter is used in test case of that project.
If a parameter is updated with new values, then the existing test execution will not get affected.
Archive Parameters
Parameters used in Data Grid cannot be archived.
To archive a single parameter, click on the Cog icon for that parameter and select Archive.
To archive multiple parameters, click the "Cog" icon and select "Archive."
On the next screen, select the parameters to archive and click the "Archive" button.
Unarchive Parameters
To unarchive parameters, click the "Cog" icon and select "Unarchive."
The screen will show all archived parameters. Select the parameters to unarchive and click the "Unarchive" button.
Delete Parameters
Parameters used in the test case (any version of it) cannot be deleted.
For deleting a parameter, it needs to be archived first.
Select the Show Archived check box to view archived parameters along with other parameters on the grid.
These archived Parameters can be deleted individually. Click the Cog icon for the parameter to delete and select Delete.
Step 2: Add Data Grid
Data Grid is kind of a table that has rows and columns within it.
Steps to Create Data Grid
Once users have added parameters with values, they can create different combinations of these values—known as a Data Grid—to run the test.
Navigate to:
- Test Cases > Data Grid.
Click the + icon to create a new Data Grid.
Enter the DataGrid Name, Project, Description and Values. (The parameters are described above in the Add Parameters section)
Maximum 100 rows and 20 columns can be added in a Data Grid.
This way users can create multiple Data Grid for multiple data sets that will be executed.
If users have already entered Data Grid manually, then the use of Autofill Values option will reset those values.
Data Grid List View
Once Data Grids are created, they will be displayed in a list view. Users can apply filters to the records, including: Projects (for which users have "View" rights), Name, Created Date, Updated Date, and Archived status.
Create Data Grid directly from Parameters
Users can create a data grid directly from the parameters.
Click on the Cog icon and select Create DataGrid.
Select Records
The Create Datagrid screen displays all available parameters. Select the parameters to include in the Data Grid by checking the corresponding boxes and click "Next" to proceed.
Step 3: Create a Data Grid
Enter a DataGrid Name, select the associated Project(s), and provide a Description.
Values: Parameters from Step 1 will be displayed. Users can add more parameters by clicking the "Column" button.
Additional options like "Row" and "Autofill Values" can be used as described in the Create Data Grid section.
Step 4: Click Create.
Creating Bulk Data for Performance Testing
To create a Data Grid in bulk, such as for Performance Testing, use the Autofill Values option to speed up the process.
Go to Test Cases > Data Grid.
Enter the DataGrid Name.
Select the Project(s) to associate with the Data Grid. Multiple projects can be selected.
Set the columns as needed.
Click the "Autofill Values" button.
It provides two options:
Unique Values: Creates a Data Grid with unique combinations of values in sequential order.
Randomize Data: Creates a Data Grid with repeated values, regardless of how many times they are used.
For Randomize Data, specify the number of rows to generate. Up to 100 rows can be created at a time.
Click Yes to proceed.
The rows are created with random values as shown below.
Edit Data Grid
Navigate to:
- Test Cases > Data Grid.
There are two ways to open the edit screen:
Click the Data Grid to edit.
Click the "Cog" icon for the Data Grid and select "Edit."
The Data Grid details page will open in editable mode.
Associated projects cannot be removed if the Data Grid is used in test cases for those projects. Updating the Data Grid will not affect existing test executions.
Archive Data Grid
To remove a Data Grid from the list, archive it.
Data Grids mapped to test cases cannot be archived.
Go to Test Cases > Data Grid.
Click the "Cog" icon and select "Archive."
To archive multiple Data Grid records at a time, select records using check boxes and the click on the Archive button.
Unarchive Data Grid
Navigate to:
- Test Cases > Data Grid.
Click on the cog icon and select Unarchive.
To unarchive multiple Data Grid records at a time, select records using check boxes and click on the Unarchive button.
Delete Data Grid
To delete a data grid, users have to archive the Data Grid first.
Click the "Cog" icon and select "Unarchive" to view all archived Data Grids.
To delete a Data Grid, click the "Cog" icon for the Data Grid and select "Delete."
Step 3: Use Parameters in Test Cases
Once the user creates the test data sets with Parameters and Data Grids, these parameters can be used in test steps as mentioned below:
Go to the Test Cases Module.
Open a test case detail page and navigate to the Test Steps section.
Insert a parameter by pressing the "@" key on the keyboard.
A list of parameters will appear. Parameters can be used in Step Description, Input Data, and Expected Outcome. Press "@" to see the available parameters. The test step field will look like this:
Step 4: Link Test Cases to Test Suite
Link both parameterized and manual test cases to the test suite. Parameterized test cases are tagged as "Parameterized."
A color-coded indicator on the execution screen reveals if parameter values are missing, helping the test execution planner quickly identify test cases with unassigned parameters.
Use Case: A test execution planner, who is responsible to create and plan the test executions and assign them to the testers for actual executions, needs an indicator to identify parameterized test cases that contain a parameter that does not have any data values assigned for executions.
Color of @ sign | Indication |
@ in Grey | It indicates non-parametrized test cases. |
@ in Blue | It indicates parameterized test cases that do not have data values (for either all or any of the parameters) assigned for executions. |
@ in Green | It indicates parameterized test cases having values assigned for all parameters. |
→ When either all or any one of the parameters do not have values, the @ sign appears in Blue.
→ Once values are assigned/mapped to all the parameters, the @ sign appears in Green.
Mapping Data Grid
On the test suite detail page, go to the Test Cases section. If parameterized test cases are linked, their information will be displayed on the screen as shown below.
Click on the @ sign to add values to the parameters. The @ sign for parameterization is highlighted and has a hyperlink on it.
Users can select one of the following Data Sources:
Select DataGrid: Choose this to use a Data Grid from the Data Parameterization module. The drop-down will show all available Data Grids, and data will populate accordingly.
Fill Data Manually: Alternatively, enter parameter values manually and map them to the parameters.
Click "Save" to save the Data Grid mapping.
Step 5: Execute Parameterized Test Cases
Open test suite detail page and open the Test Execution tab on it. Click on the play icon to navigate to the Test Execution Screen to run the test and record the status.
On the Test Execution screen, clicking "Expand All" will expand test cases with parameterization to show Parameter Groups. The step count at the test case level reflects the number of test steps multiplied by the number of data sets.
Expand each Parameter Combination to view the test steps.
Execute the test steps and test cases on the screen.
If the Data Grid Mapping is not done
When a parameter is added but values are not yet mapped, the test step on the Test Execution screen will show an "@" mark, indicating that the values have not been synced in the Data Grid.
A tip on the screen provides a link to map the parameter with its value. Clicking this link opens the test suite detail page.
Go to the Test Cases section and map the Data Grid for the test case.
After mapping, refresh the Test Execution screen.
Expand the test case to see the parameter values and view the steps with the assigned values.
If a new column is added to the Data Grid:
Go to the test suite detail page and open the Test Cases section.
Click the "@" sign to access the Data Grid Mapping screen.
Click the "Sync Columns" button to update the Data Grid with the new column.
The newly added parameter will be added as a column. Now map the values and update the Data Grid.
Edit In | Impact on Test Execution Screen |
|---|---|
Edit Data Grid Mapping:
| When the Data Grid is updated, changes are reflected on the Test Execution screen. The test case will be updated, and the step count will change accordingly. Editing the Data Grid from the Test Suite (for linked parameterized test cases) will only make temporary changes that do not sync with test parameter data. |
Edit Test Case: Any changes to test steps in a test case will be reflected on the Test Execution screen. Add a step to the test case and save it as a new version. | On the Test Execution screen, syncing the test case will update it to the new version, increasing the step count. Each combination will now include the newly added step. |
Import Parameters
Users can import parameters from an Excel file into QMetry, allowing them to use bulk parameters in the system.
Import Templates - Import Parameter Sample File.xlsx
Click on the Cog icon and select Import.
On the Import screen:
Select File: Browse and choose the Excel file with parameters.
Project: Choose the project(s) where the parameters will be used. Data parameters are visible across projects but must be associated with the specific project to be used.
Import Options:
Overwrite Duplicate Parameters: Replace existing parameters with new values from the file.
Append Parameter Values: Add new values to existing parameters without removing old ones.
Skip Duplicate Parameters: Ignore existing parameters during import, preserving their current values.
Import Limitation
100 parameters at a time.
1000 parameter values per parameter at a time.
Click Import.
View Notification Window on the application header.
View Scheduled Task
Import Data Grid
Users can import Data Grids from an Excel file into QMetry to use them in bulk.
Import Templates - Import Datagrid Sample File.xlsx
Click on the Cog icon and select Import.
The Import screen opens.
Choose File: Browse the excel file that contains Parameters.
Project: Select the project. Multiple Projects can be selected. Data Parameters are visible across projects irrespective of its linkage with the project. To use the Data Parameter within project, it should be associated with that particular project.
Click Import.
View (A) Notification Window and (B) Scheduled Task on the application header.
Audit Log Report
Audit logs for Create, Update, Delete, Archived, and Unarchived are added for Data Grid and Parameter in Audit Log Report (Audit Compliance > Audit Log Report).