Skip to main content

Robot Framework Integration

Report Robot Framework test results to ProvaLab.io using a listener. The ProvaLab.io listener hooks into Robot's execution lifecycle and sends results as tests complete.

Install​

pip install tms-plugin

Requirements: Python >= 3.9, Robot Framework >= 5.0

Configure​

1. Tag your test cases with ProvaLab.io IDs​

Use the tms:<id> tag format to link Robot test cases to ProvaLab.io:

tests/login.robot
*** Settings ***
Library    SeleniumLibrary

*** Test Cases ***
Login With Valid Credentials
    [Tags]    tms:101    smoke    critical
    Open Browser    https://app.yourcompany.com/login    chrome
    Input Text    id=email    [email protected]
    Input Text    id=password    password123
    Click Button    id=login-btn
    Page Should Contain    Dashboard
    [Teardown]    Close Browser

Login With Invalid Password
    [Tags]    tms:102    regression
    Open Browser    https://app.yourcompany.com/login    chrome
    Input Text    id=email    [email protected]
    Input Text    id=password    wrong-password
    Click Button    id=login-btn
    Page Should Contain    Invalid credentials
    [Teardown]    Close Browser

Login With Expired Account
    [Tags]    tms:103    regression
    Open Browser    https://app.yourcompany.com/login    chrome
    Input Text    id=email    [email protected]
    Input Text    id=password    password123
    Click Button    id=login-btn
    Page Should Contain    Account expired
    [Teardown]    Close Browser
Tag format

You can use either tms:123 or tms_case_id:123 as the tag format. Both are recognized by the listener.

2. Run with the ProvaLab.io listener​

Option A: Pass configuration via Robot variables

robot --listener tms_plugin.integrations.robot_listener.TMSListener \
      --variable TMS_URL:https://tms.yourcompany.com \
      --variable TMS_API_KEY:your-api-key \
      --variable TMS_ORGANIZATION_ID:1 \
      --variable TMS_TEST_RUN_ID:123 \
      tests/

Option B: Use environment variables

export TMS_URL=https://tms.yourcompany.com
export TMS_API_KEY=your-api-key
export TMS_ORGANIZATION_ID=1
export TMS_TEST_RUN_ID=123

robot --listener tms_plugin.integrations.robot_listener.TMSListener tests/

Option C: Auto-create test run from test plan

robot --listener tms_plugin.integrations.robot_listener.TMSListener \
      --variable TMS_URL:https://tms.yourcompany.com \
      --variable TMS_API_KEY:your-api-key \
      --variable TMS_ORGANIZATION_ID:1 \
      --variable TMS_TEST_PLAN_ID:10 \
      tests/

Run​

robot --listener tms_plugin.integrations.robot_listener.TMSListener tests/

The listener reports results as each test finishes. After the suite completes, ProvaLab.io metadata is added to Robot's output:

==============================================================================
Login Tests
==============================================================================
Login With Valid Credentials                                          | PASS |
Login With Invalid Password                                           | PASS |
Login With Expired Account                                            | FAIL |
Account expired message not found
==============================================================================
Login Tests                                                           | 2 passed, 1 failed
==============================================================================

ProvaLab.io results appear in Robot's output.xml metadata:

ProvaLab.io Test Run ID: 854
ProvaLab.io Total Tests: 3
ProvaLab.io Passed: 2
ProvaLab.io Failed: 1
ProvaLab.io Pass Rate: 66.67%

View Results in ProvaLab.io​

  1. Open ProvaLab.io and go to your project
  2. Click Test Runs in the sidebar
  3. Find the run (by ID or name "Robot Run - Login Tests - 2026-02-20 14:30:00")
  4. Click into it to see individual test results with error messages

Advanced Configuration​

Constructor arguments​

You can pass configuration directly to the listener constructor:

robot --listener tms_plugin.integrations.robot_listener.TMSListener:base_url=https://tms.yourcompany.com:api_key=your-key:organization_id=1:test_run_id=123 tests/

Environment selection​

robot --listener tms_plugin.integrations.robot_listener.TMSListener \
      --variable TMS_ENVIRONMENT:staging \
      tests/

Screenshot capture​

The listener looks for screenshots in the screenshots/ directory that match the test name. If you're using SeleniumLibrary, configure it to save screenshots there:

*** Settings ***
Library    SeleniumLibrary    screenshot_root_directory=screenshots

Configuration precedence​

The listener resolves configuration in this order (first wins):

  1. Constructor arguments
  2. Robot variables (--variable TMS_API_KEY:...)
  3. Environment variables (TMS_API_KEY=...)

Verify It Works​

  1. Install the plugin:
pip install tms-plugin
  1. Create a simple test:
tests/tms_verify.robot
*** Test Cases ***
Verify ProvaLab.io Integration
    [Tags]    tms:1
    Log    ProvaLab.io integration is working
    Should Be True    ${True}
  1. Run it:
robot --listener tms_plugin.integrations.robot_listener.TMSListener \
      --variable TMS_URL:https://tms.yourcompany.com \
      --variable TMS_API_KEY:your-api-key \
      --variable TMS_ORGANIZATION_ID:1 \
      --variable TMS_TEST_RUN_ID:1 \
      tests/tms_verify.robot
  1. Check the output for ProvaLab.io metadata
  2. Open ProvaLab.io and confirm the test result appears under your test run
Common issues
  • "ProvaLab.io API key not provided": Make sure TMS_API_KEY is set as a Robot variable or environment variable
  • Tests run but nothing appears in ProvaLab.io: Make sure tests have the tms:<id> tag
  • "Either test_run_id or test_plan_id must be provided": Pass one of TMS_TEST_RUN_ID or TMS_TEST_PLAN_ID