Skip to content

Running Tests & Test Results

Overview

Running a test in Test Explorer creates a test run — a collection of tests that execute together and report their outcome as a group. This page covers how test runs are executed, where to find and read their results in the Test Results view, how failures are decorated in the Navigator and workflow editor, how the debug features work with testing, and how to run and analyze tests using the Designer agent in the AI Chat view.

What You Will Learn

  • What a test run is and how many tests run concurrently
  • Where to find the Test Results view and how it's organized
  • How to read a test's result and logs on the Results and Logs tabs
  • How failing unit tests are decorated in the Navigator and workflow editor
  • How workflow unit tests integrate with the workflow debugger, including breakpoints and generated Debug configurations
  • How to run and troubleshoot tests conversationally using the Designer agent in the AI Chat view

Test Runs

As described in Testing in Martini Designer, you can start a test run from Test Explorer by running all tests, every test in a directory, or a single test. A directory or single-test action groups the selected tests into one test run and waits for every test in it to finish before considering the run done. Run All is the exception; it creates the two test runs described below.

Keep these behaviors in mind when starting or reviewing a run:

  • Concurrency: All test runs share a single FIFO worker pool that executes at most 5 tests concurrently across the whole workspace, not per run. For example, since running all tests creates two test runs — one for Unit Tests and one for HTTP Request Tests — both runs draw from that same shared pool rather than each getting their own 5 workers. Additional tests wait their turn in order until a worker becomes free.
  • Numbering: Test runs are numbered starting at 0 and incrementing with every new run you start, so you can tell runs apart even if their contents overlap.
  • Skipped tests: A run still includes any skipped tests — for example, workflow tests named with the Skip suffix described in Workflow Tests, or requests using the SkipTest.request naming described in HTTP Request Integration Tests. Skipped tests aren't executed, but they're still listed in the run and marked as skipped.
  • Running all tests: Clicking Run All in the Test Explorer toolbar runs every test in the workspace, but this creates two separate test runs — one for the Unit Tests root node and one for the HTTP Request Tests root node — rather than a single combined run.

The Test Results View

Test run results appear in the Test Results view, located by default in the bottom panel alongside the Console and Output views. You can show or hide it with the Toggle Test Results action in the Test Explorer toolbar.

The right side of the view lists your test runs, each labeled with its run number — for example, "Test run #0". Expanding a test run reveals the individual tests that were part of it, each shown with an icon indicating whether it failed, passed, or was skipped.

Click any test — from either the Test Results view or the Test Explorer tree — to display its outcome on the left side of the Test Results view, split across a Results tab and a Logs tab.

Reading Results

The Results tab shows what happened when the selected test ran, and the details differ slightly by test type.

For workflow tests (Unit Tests):

  • Passed: Displays "Test passed successfully in <ms>ms.", showing the actual elapsed time for that test.
  • Failed, with an assertion error: Displays the assertion error message and stack trace. Entries in the stack trace are clickable and navigate you directly to the failing node in the workflow.
  • Failed, with no assertion error reported: Displays "Test failed without errors, please check the test workflow has a 'success' output property set to true when the test is successful." This points you back to the success output contract described in Workflow Tests.
  • Skipped: Displays "Test was skipped and did not execute."

For HTTP request integration tests (HTTP Request Tests), the tab shows the results of the request's test script, which typically contains several individual martini.test() cases using Chai assertions. Each test case in the script is listed with its own passing, failing, or skipped status. If the overall test passed, the tab also displays "Test passed successfully in <ms>ms." at the top. If any test case within the script fails, the HTTP request test as a whole is marked failed.

Reading Logs

The Logs tab shows what was logged while the test ran, and what it contains also depends on the test type:

  • Unit Tests: Shows the logs produced by the workflow during its run. If the test didn't produce any output, the tab displays "The test case did not report any output."
  • HTTP Request Tests: Shows only request-level logs — the request's start time, end time, and response status — rather than workflow logs, since no workflow is involved.

Failures from the latest test run are decorated directly in the Navigator and on workflow nodes, so you can spot a regression without opening the Test Results view. This decoration currently applies to unit tests only; it isn't yet supported for HTTP request tests.

  • The Navigator shows an overlay icon on any workflow test file that failed in the latest run.
  • In the workflow editor, failing nodes are decorated with an icon. Hover over it to see the failure message as a tooltip, without needing to switch to the Test Results view.
  • Decoration always reflects only the latest test run: if a previously failing test isn't included in a later run, it is no longer decorated, even though it failed before.

Debug Integration for Workflow Unit Tests

Workflow unit tests are built on top of the regular workflow debugger rather than a separate test runner: when a workflow test runs, Martini Designer executes it exactly as it would any other workflow debug session, generating a debug configuration for it automatically. This integration means you can use the same debugging tools you already use on your workflows to diagnose why a test is failing:

  • Add breakpoints to the test workflow itself, or to any workflow it calls, and step through execution the same way you would when debugging a regular workflow run.
  • Inspect variables and node output at each breakpoint to pinpoint where the test's behavior diverges from what the success output contract (see Workflow Tests) expects.

Because every test run generates a debug configuration, running your test suite repeatedly can produce a large number of them over time. To keep the Debug configuration picker focused on your regular workflows, it filters out test debug configurations by default. Click the beaker icon in the picker to toggle that filter on or off and reveal the test configurations when you need them.

Running and Analyzing Tests with the Designer Agent

Beyond Test Explorer, you can ask the Designer agent to run and analyze your tests directly from the AI Chat view. The Designer agent has specialized skills and tools for testing, so you can describe what you want in plain language instead of navigating the Test Explorer tree yourself:

  • Ask it to run all tests, the tests in a specific directory, or a single named test.
  • Ask it to troubleshoot a failing test — the Designer agent retrieves the failure details (such as the assertion error and stack trace described in Reading Results) and explains the likely cause in its response.

This gives you a conversational alternative to manually running a test in Test Explorer and inspecting its outcome in the Test Results view, particularly useful when you want a quick explanation of why a test failed instead of tracing the stack trace yourself.

Want to run this same test suite automatically whenever code changes? See Running Tests in CI/CD for the official Docker image that runs every test in your project and produces a JUnit test report in any CI/CD pipeline.

Helpful Resources