CLI Tool
The TMS CLI lets you manage your entire testing workflow from the terminal. Create projects, run API and load tests, analyze failures with AI, and pipe results into your CI/CD pipeline -- all without opening a browser.
The TMS CLI is not yet available as a package: it is not on PyPI (do not run pip install tms-cli, the name is unclaimed there and would resolve to whatever a third party uploads) and it is not listed on plugins.provalab.io. It currently lives only in ProvaLab's private source repository. This page documents the command reference so you can plan automation around it; check the plugin catalogue for the official install path when it ships. In the meantime, the same operations are available through the REST API and the MCP tools.
Installation
Not yet available. Once published, the CLI will be installable from plugins.provalab.io and this section will carry the exact command.
Verify the installation:
tms --version
# TMS CLI, version 1.0.0
The CLI requires Python 3.8 or later and a running TMS instance to connect to.
Authentication
Before you can run commands, you need to log in so the CLI knows who you are and which TMS instance to talk to.
Log in with email and password
tms auth login --email [email protected] --password yourpass
If you omit --email or --password, the CLI will prompt you interactively (the password is hidden as you type).
Connect to a specific TMS instance
By default the CLI connects to http://localhost:8000. To point it at your company's TMS server, pass the --api-url flag:
tms auth login --email [email protected] --password yourpass \
--api-url https://tms.yourcompany.com
Use an API key instead
If you prefer API key authentication (recommended for CI/CD), create a key under API Keys at auth.provalab.io/api-keys (keys start with tms_) and set it directly:
tms auth set-api-key --key tms_...
Verify your session
tms auth whoami
Example output:
Current User:
Email: [email protected]
Name: Jane Smith
Organization: Acme Corp
Role: admin
Log out
tms auth logout
This removes the saved token from ~/.tms/config.json.
Quick command reference
| Command | What it does |
|---|---|
tms auth login | Log in and save your session |
tms auth whoami | Show current user info |
tms auth logout | Clear saved credentials |
tms projects list | List all projects |
tms projects create | Create a new project |
tms projects get <id> | View project details |
tms test-plan list | List test plans |
tms test-plan execute <id> | Execute a test plan |
tms api collection list | List API test collections |
tms api run start | Start an API test run |
tms load scenario list | List load test scenarios |
tms load run start | Start a load test |
tms ai chat | Chat with the AI assistant |
tms ai generate test-cases | Generate test cases with AI |
tms ai analyze failures | Analyze test failures with AI |
tms ai analyze flaky-tests | Detect flaky tests |
Configuration file
The CLI stores your connection details in ~/.tms/config.json:
{
"api_url": "https://tms.yourcompany.com",
"api_key": "tms_...",
"default_project_id": 1,
"default_organization_id": 1
}
You can also configure the CLI through environment variables:
| Variable | Description | Default |
|---|---|---|
TMS_API_URL | TMS API base URL | http://localhost:8000 |
TMS_API_KEY | TMS API key | None |
TMS_CONFIG_DIR | Config directory | ~/.tms |
TMS_TIMEOUT | Request timeout (seconds) | 30 |
Environment variables take precedence over values in config.json.
Output formats
By default the CLI renders beautifully formatted tables in your terminal. You can switch to machine-readable formats for scripting:
# JSON output
tms projects list --format json > projects.json
# CSV output (opens in Excel)
tms api run results 123 --format csv > results.csv
Combine JSON output with jq for powerful filtering:
tms api run results 42 --format json | jq '.items[] | select(.status == "failed")'
Troubleshooting
"API key not found"
You need to log in first:
tms auth login --email [email protected] --password yourpass
"Connection refused"
Make sure the TMS backend is reachable. Check your API URL with:
tms auth whoami
"Command not found: tms"
Make sure the CLI is installed and on your PATH (see the notice at the top of this page: it is not yet published, so there is no pip install for it). With a user-level install, pip puts the tms script in ~/.local/bin:
export PATH=$PATH:~/.local/bin
Next steps
- Managing Projects -- Create and organize your projects from the CLI
- Running Tests -- Execute API tests, load tests, and test plans
- AI from the CLI -- Generate tests, analyze failures, and chat with the AI agent