unittest Integration

Report results from Python's built-in unittest framework to ProvaLab.io. The plugin provides a custom TMSTestRunner that replaces the default runner and sends results to ProvaLab.io as tests complete.

Install

pip install tms-plugin

Requirements: Python >= 3.9

Configure

1. Create your test file with ProvaLab.io case IDs

There are two ways to mark tests with ProvaLab.io case IDs.

Option A: Use the @tms_case decorator (recommended)

tests/test_users.py

import unittest
from tms_plugin.integrations.unittest_runner import tms_case

class TestUserAPI(unittest.TestCase):

    @tms_case(101)
    def test_create_user(self):
        """Create a new user via the API."""
        user = create_user(name="Alice", email="[email protected]")
        self.assertIsNotNone(user.id)
        self.assertEqual(user.name, "Alice")

    @tms_case(102)
    def test_get_user_by_id(self):
        """Retrieve a user by ID."""
        user = get_user(1)
        self.assertEqual(user.name, "Alice")

    @tms_case(103)
    def test_delete_user(self):
        """Delete a user."""
        result = delete_user(1)
        self.assertTrue(result.success)

Option B: Use docstring annotation

tests/test_users.py

import unittest

class TestUserAPI(unittest.TestCase):

    def test_create_user(self):
        """TMS_CASE_ID: 101
        Create a new user via the API."""
        user = create_user(name="Alice", email="[email protected]")
        self.assertIsNotNone(user.id)

2. Set up the TMSTestRunner

run_tests.py

import unittest
from tms_plugin.client import TMSClient
from tms_plugin.integrations.unittest_runner import TMSTestRunner

# Initialize the ProvaLab.io client
client = TMSClient(
    api_key="your-api-key",
    api_url="https://tms.yourcompany.com",
)

# Load your test suite
suite = unittest.TestLoader().loadTestsFromTestCase(TestUserAPI)

# Or load all tests from a directory
# suite = unittest.TestLoader().discover('tests')

# Run with ProvaLab.io reporting
runner = TMSTestRunner(
    client=client,
    organization_id=1,
    test_plan_id=10,          # Creates a new test run automatically
    environment="staging",
    verbosity=2,
)
result = runner.run(suite)

Using an existing test run:

runner = TMSTestRunner(
    client=client,
    organization_id=1,
    test_run_id=123,          # Report to an existing test run
    environment="staging",
    verbosity=2,
)

Run

python run_tests.py

Output:

test_create_user (tests.test_users.TestUserAPI) ... ok
test_get_user_by_id (tests.test_users.TestUserAPI) ... ok
test_delete_user (tests.test_users.TestUserAPI) ... FAIL

======================================================================
ProvaLab.io Test Results
======================================================================
Test Run ID: 853
Total: 3, Passed: 2, Failed: 1, Blocked: 0, Skipped: 0
Pass Rate: 66.67%
======================================================================

View Results in ProvaLab.io

1. Open ProvaLab.io and go to your project
2. Click Test Runs in the sidebar
3. Find the run named "Unittest Run - 2026-02-20 14:30:00"
4. Click into it to see individual test results with error details

Advanced Configuration

Using environment variables for secrets

Keep secrets out of your code:

run_tests.py

import os
from tms_plugin.client import TMSClient

client = TMSClient(
    api_key=os.environ["TMS_API_KEY"],
    api_url=os.environ.get("TMS_API_URL", "https://tms.yourcompany.com"),
)

export TMS_API_KEY=your-api-key
python run_tests.py

Using as a context manager

from tms_plugin.reporter import TestReporter

client = TMSClient(api_key="your-api-key", api_url="https://tms.yourcompany.com")

with TestReporter(client=client, organization_id=1, test_run_id=123) as reporter:
    reporter.report_result(
        test_case_id=101,
        status=TestStatus.PASSED,
        execution_time=5,
        actual_result="User created successfully",
    )

Integration with unittest.main()

tests/test_users.py

import unittest
from tms_plugin.client import TMSClient
from tms_plugin.integrations.unittest_runner import TMSTestRunner, tms_case

class TestUserAPI(unittest.TestCase):
    @tms_case(101)
    def test_create_user(self):
        self.assertTrue(True)

if __name__ == '__main__':
    client = TMSClient(api_key="your-key", api_url="https://tms.yourcompany.com")
    runner = TMSTestRunner(client=client, organization_id=1, test_plan_id=10)
    unittest.main(testRunner=runner)

Verify It Works

1. Install the plugin:

pip install tms-plugin

2. Create a test and runner:

test_tms_verify.py

import unittest
from tms_plugin.client import TMSClient
from tms_plugin.integrations.unittest_runner import TMSTestRunner, tms_case

class TestVerify(unittest.TestCase):
    @tms_case(1)
    def test_verify_tms_integration(self):
        """Verify ProvaLab.io integration is working."""
        self.assertEqual(1 + 1, 2)

if __name__ == '__main__':
    client = TMSClient(api_key="your-api-key", api_url="https://tms.yourcompany.com")
    runner = TMSTestRunner(client=client, organization_id=1, test_run_id=1)
    unittest.main(testRunner=runner)

3. Run it:

python test_tms_verify.py

4. Check the terminal for the ProvaLab.io summary
5. Open ProvaLab.io and confirm the test result appears

Common issues

• "Either test_run_id or test_plan_id must be provided": Pass one of these to TMSTestRunner
• Tests run but nothing appears in ProvaLab.io: Make sure tests use @tms_case() decorator or TMS_CASE_ID: in the docstring
• Import error: Make sure tms-plugin is installed in your current Python environment