Track test execution

Applies to CucumberStudio Cloud, and to Enterprise ver. 3.3 - 3.5.6.0. Last updated on November 17, 2023

Definition

Test execution is managed with test runs. A test run is a snapshot of your project that includes all the tests or just a subset of them. A test run creates a kind of branch of the project where current definitions of tests are stored. It means that you can continue working on your scenarios and modify them without impacting the test run and the execution. Of course your can synchronize your test run at any time with the new definitions of scenarios.

For each test of a test run, you can add one or more test results to keep track of your test execution progress.

CucumberStudio test runs

Click the image to enlarge it.

Create a test run

When creating a new test run, you can choose between 4 options:

  • include all the tests,

  • select the same tests as a previous test run,

  • select a subset of tests (you can filter by folders and scenarios),

  • create an empty test run that will be used to integrate external test results.

Then click Create button at the bottom of the page.

Create test run

Click the image to enlarge it.

Once your test run is created you can see the list of tests and metrics.

Test run overview

Click the image to enlarge it.

From this page, you can:

  • navigate between test runs,

  • select a test for execution,

  • select multiple tests for bulk status update,

  • select or unselect all tests in the test run for bulk status update,

  • select or unselect all tests in a folder for bulk status update,

  • show or hide tags of tests and folders,

  • synchronize all the tests with the latest definitions of scenarios,

  • add scenarios to the current test run,

  • export the data into an excel file,

  • archive the test run,

  • automate the test run,

  • reset test results,

  • delete the test run.

If the test run contains many execution cycles, you can see the history by clicking on the Executions history tab:

CucumberStudio execution history

Click the image to enlarge it.

Assign a test to a user

You can assign a test to a user in the Assignees section. This user should be a project member.

CucumberStudio test assignees

Click the image to enlarge it.

You can search for assigned tests in a test run using the Search tests input by typing the assignee name.

Assignee search

Click the image to enlarge it.

Execute a test manually

When selecting a test, you can add one or more execution results. An execution result contains a description, a date, the author of the execution and the status. The author is automatically updated. You can add a description and attach files to an execution result (screenshot…).

You can also track execution step by step. The first click marks the step as passed, the second one as failed and last one as undefined.

CucumberStudio test execution

Click the image to enlarge it.

You can use the Passed and next action to mark the test as Passed and pass to the next one:

CucumberStudio test execution: passed and next

Click the image to enlarge it.

Below the execution results, you have access to the execution results history:

CucumberStudio execution results history

Click the image to enlarge it.

From the test page you can as well:

  • add tags to your test

  • assign the test to one or many project members

Bulk status update

You can select multiple tests with the check boxes and update their status.

Bulk status update

Click the image to enlarge it.

Automated execution

To see how to automate your tests and link CucumberStudio to your continuous integration server see the Automate your tests section.

Test run metrics

At any time you can click on the test run name (left column) to view the metrics. You can also export test runs into an excel file and then customize your metrics.

Test run metrics

Click the image to enlarge it.

You can go back to your test runs list by clicking on the test run menu. From this list, you can export the content of all your test runs into a single excel file (one test run by tab), customize your metrics, and other actions on the existing test runs.

CucumberStudio test runs list

Click the image to enlarge it.

Add a new scenario to a test run

You can add a new scenario to a test run directly from the scenario page by selecting Add to test run from the menu.

Add scenario to test run

Click the image to enlarge it.

Or you can add many scenarios from the folder page. Just select your scenarios and choose Add to test run.

CucumberStudio test run add scenarios from folder

Click the image to enlarge it.

Archive test runs

Because your test runs list can grow very quickly, you can hide some of your test runs thanks to the Archive option.

Archive test run

Click the image to enlarge it.

You can find archived test runs in your test runs list by selecting the dedicated filter.

CucumberStudio archived test runs

Click the image to enlarge it.

An archived test run will not appear in the test results list of a scenario, and you will not be able to add a scenario in it or watch its metrics on the project dashboard. To unarchive a test run, select Unarchive from the menu.

Highlight search results