unittest Integration

Report results from Python's built-in unittest framework to TMS. The plugin provides a custom TMSTestRunner that replaces the default runner and sends results to TMS as tests complete.

Not yet published: do not pip install tms-plugin

The tms-plugin Python package this page describes is not published on PyPI or on plugins.provalab.io yet. The name is unclaimed on PyPI, so installing it by name would fetch whatever a third party uploads under it. In the meantime, run your unittest suites with pytest (it collects unittest.TestCase classes unchanged) and the published pytest plugin. Check the plugin catalogue for the official install path when it ships.

Install

Until tms-plugin ships, use the pytest plugin, which is published as a wheel:

pip install --upgrade https://plugins.provalab.io/packages/pytest_tms_plugin-1.0.13-py3-none-any.whl
pytest tests/        # collects unittest.TestCase classes and reports them to TMS

Requirements (for tms-plugin, once published): Python >= 3.9

Configure

1. Create your test file with TMS case IDs

There are two ways to mark tests with TMS case IDs.

Option A: Use the @tms_case decorator (recommended)

tests/test_users.py
import unittest
from tms_plugin.integrations.unittest_runner import tms_case

class TestUserAPI(unittest.TestCase):

    @tms_case(101)
    def test_create_user(self):
        """Create a new user via the API."""
        user = create_user(name="Alice", email="[email protected]")
        self.assertIsNotNone(user.id)
        self.assertEqual(user.name, "Alice")

    @tms_case(102)
    def test_get_user_by_id(self):
        """Retrieve a user by ID."""
        user = get_user(1)
        self.assertEqual(user.name, "Alice")

    @tms_case(103)
    def test_delete_user(self):
        """Delete a user."""
        result = delete_user(1)
        self.assertTrue(result.success)

Option B: Use docstring annotation

tests/test_users.py
import unittest

class TestUserAPI(unittest.TestCase):

    def test_create_user(self):
        """TMS_CASE_ID: 101
        Create a new user via the API."""
        user = create_user(name="Alice", email="[email protected]")
        self.assertIsNotNone(user.id)

2. Set up the TMSTestRunner

run_tests.py
import unittest
from tms_plugin.client import TMSClient
from tms_plugin.integrations.unittest_runner import TMSTestRunner

# Initialize the TMS client
client = TMSClient(
    api_key="your-api-key",
    api_url="https://tms.yourcompany.com",
)

# Load your test suite
suite = unittest.TestLoader().loadTestsFromTestCase(TestUserAPI)

# Or load all tests from a directory
# suite = unittest.TestLoader().discover('tests')

# Run with TMS reporting
runner = TMSTestRunner(
    client=client,
    organization_id=1,
    test_plan_id=10,          # Creates a new test run automatically
    environment="staging",
    verbosity=2,
)
result = runner.run(suite)

Using an existing test run:

runner = TMSTestRunner(
    client=client,
    organization_id=1,
    test_run_id=123,          # Report to an existing test run
    environment="staging",
    verbosity=2,
)

Run

python run_tests.py

Output:

test_create_user (tests.test_users.TestUserAPI) ... ok
test_get_user_by_id (tests.test_users.TestUserAPI) ... ok
test_delete_user (tests.test_users.TestUserAPI) ... FAIL

======================================================================
TMS Test Results
======================================================================
Test Run ID: 853
Total: 3, Passed: 2, Failed: 1, Blocked: 0, Skipped: 0
Pass Rate: 66.67%
======================================================================

View Results in TMS

Open TMS and go to your project
Click Test Runs in the sidebar
Find the run named "Unittest Run - 2026-02-20 14:30:00"
Click into it to see individual test results with error details

Advanced Configuration

Using environment variables for secrets

Keep secrets out of your code:

run_tests.py
import os
from tms_plugin.client import TMSClient

client = TMSClient(
    api_key=os.environ["TMS_API_KEY"],
    api_url=os.environ.get("TMS_API_URL", "https://tms.yourcompany.com"),
)

export TMS_API_KEY=your-api-key
python run_tests.py

Using as a context manager

from tms_plugin.reporter import TestReporter

client = TMSClient(api_key="your-api-key", api_url="https://tms.yourcompany.com")

with TestReporter(client=client, organization_id=1, test_run_id=123) as reporter:
    reporter.report_result(
        test_case_id=101,
        status=TestStatus.PASSED,
        execution_time=5,
        actual_result="User created successfully",
    )

Integration with `unittest.main()`

tests/test_users.py
import unittest
from tms_plugin.client import TMSClient
from tms_plugin.integrations.unittest_runner import TMSTestRunner, tms_case

class TestUserAPI(unittest.TestCase):
    @tms_case(101)
    def test_create_user(self):
        self.assertTrue(True)

if __name__ == '__main__':
    client = TMSClient(api_key="your-key", api_url="https://tms.yourcompany.com")
    runner = TMSTestRunner(client=client, organization_id=1, test_plan_id=10)
    unittest.main(testRunner=runner)

Verify It Works

Install the plugin (not yet published, see the notice at the top of this page; the steps below describe the tms-plugin API)
Create a test and runner:

test_tms_verify.py
import unittest
from tms_plugin.client import TMSClient
from tms_plugin.integrations.unittest_runner import TMSTestRunner, tms_case

class TestVerify(unittest.TestCase):
    @tms_case(1)
    def test_verify_tms_integration(self):
        """Verify TMS integration is working."""
        self.assertEqual(1 + 1, 2)

if __name__ == '__main__':
    client = TMSClient(api_key="your-api-key", api_url="https://tms.yourcompany.com")
    runner = TMSTestRunner(client=client, organization_id=1, test_run_id=1)
    unittest.main(testRunner=runner)

Run it:

python test_tms_verify.py

Check the terminal for the TMS summary
Open TMS and confirm the test result appears

Common issues

"Either test_run_id or test_plan_id must be provided": Pass one of these to TMSTestRunner
Tests run but nothing appears in TMS: Make sure tests use @tms_case() decorator or TMS_CASE_ID: in the docstring
Import error: Make sure tms-plugin is installed in your current Python environment

Install​

Configure​

1. Create your test file with TMS case IDs​

2. Set up the TMSTestRunner​

Run​

View Results in TMS​

Advanced Configuration​

Using environment variables for secrets​

Using as a context manager​

Integration with unittest.main()​

Verify It Works​