AI API

The ProvaLab.io AI endpoints let you generate test cases from requirements, analyze test failures for root causes, chat with an intelligent QA agent, and detect flaky tests -- all programmatically.

---

Generate test cases

Have the AI create test cases from a natural language requirement or user story.

POST /api/v1/ai/organizations/{organization_id}/generate-test-cases

Request body (text input)

Field	Type	Required	Description
requirement	string	Yes	The requirement or user story to generate tests from
module	string	No	Module name for context
priority	string	No	Default priority for generated tests
context	string	No	Additional context (e.g., tech stack, constraints)

curl -X POST https://your-tms-instance.com/api/v1/ai/organizations/1/generate-test-cases \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "requirement": "As a user, I want to reset my password via email so that I can regain access to my account if I forget my password",
    "module": "Authentication",
    "priority": "high",
    "context": "Email service is configured with SendGrid"
  }'

Response 200 OK

{
  "generated_test_cases": [
    {
      "title": "Verify password reset email is sent for valid email",
      "description": "Test that a password reset email is sent when the user requests a reset with a valid registered email",
      "priority": "high",
      "type": "functional",
      "preconditions": "User account exists with verified email",
      "steps": [
        {
          "step_number": 1,
          "action": "Navigate to password reset page",
          "expected_result": "Reset page is displayed with email input field"
        },
        {
          "step_number": 2,
          "action": "Enter registered email address",
          "expected_result": "Email is accepted"
        },
        {
          "step_number": 3,
          "action": "Click 'Send Reset Link' button",
          "expected_result": "Success message is displayed"
        }
      ],
      "expected_result": "Password reset email received within 5 minutes",
      "tags": ["password-reset", "email", "authentication"]
    },
    {
      "title": "Verify password reset fails for unregistered email",
      "description": "Test that no email is sent and a generic message is shown when requesting a reset with an unregistered email",
      "priority": "high",
      "type": "functional",
      "preconditions": "No account exists for the test email",
      "steps": [
        {
          "step_number": 1,
          "action": "Navigate to password reset page",
          "expected_result": "Reset page is displayed"
        },
        {
          "step_number": 2,
          "action": "Enter unregistered email address",
          "expected_result": "Email is accepted (no error shown)"
        },
        {
          "step_number": 3,
          "action": "Click 'Send Reset Link' button",
          "expected_result": "Generic success message is displayed (no email enumeration)"
        }
      ],
      "expected_result": "No password reset email is sent",
      "tags": ["password-reset", "security", "authentication"]
    }
  ],
  "metadata": {
    "total_generated": 2,
    "model_used": "claude-sonnet-4-20250514",
    "confidence_score": 0.92
  }
}

File upload

You can also upload a requirements document (PDF, DOCX, or TXT) using multipart/form-data instead of providing the requirement as text. Include the file field with your document and optionally set document_type to requirements or user_story.

---

Analyze a test failure

Get AI-powered root cause analysis for a failed test.

POST /api/v1/ai/organizations/{organization_id}/analyze-failure

Request body

Field	Type	Required	Description
test_result_id	integer	Yes	The ID of the failed test result
include_logs	boolean	No	Include execution logs in analysis (default: false)
include_screenshots	boolean	No	Include screenshots in analysis (default: false)

curl -X POST https://your-tms-instance.com/api/v1/ai/organizations/1/analyze-failure \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "test_result_id": 123,
    "include_logs": true,
    "include_screenshots": true
  }'

Response 200 OK

{
  "analysis": {
    "root_cause": "Element selector timeout -- the login button's CSS class was changed from .btn-login to .btn-primary in the latest deployment",
    "confidence": 0.85,
    "suggestions": [
      "Update the element selector from '.btn-login' to '.btn-primary'",
      "Use a data-testid attribute for more stable selectors",
      "Add a retry mechanism for element lookups with a 10-second timeout"
    ],
    "similar_failures": [
      {
        "test_case_id": 45,
        "similarity_score": 0.92,
        "resolution": "Updated element selector to use data-testid"
      },
      {
        "test_case_id": 78,
        "similarity_score": 0.84,
        "resolution": "Added explicit wait before element interaction"
      }
    ]
  }
}

The similar_failures field shows previous failures with matching patterns and how they were resolved, helping you fix the issue faster.

---

Chat with the AI agent

ProvaLab.io provides a conversational AI agent that understands your test data. You can ask questions, request analyses, and get recommendations through a chat interface.

Create a conversation

POST /api/v1/ai/organizations/{organization_id}/chat/conversations

Field	Type	Required	Description
title	string	No	Conversation title

curl -X POST https://your-tms-instance.com/api/v1/ai/organizations/1/chat/conversations \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "title": "Sprint 12 Test Analysis"
  }'

Response 201 Created

{
  "id": 1,
  "title": "Sprint 12 Test Analysis",
  "created_at": "2025-11-15T10:00:00Z"
}

Send a message

POST /api/v1/ai/organizations/{organization_id}/chat/conversations/{conversation_id}/messages

Field	Type	Required	Description
message	string	Yes	Your question or instruction
context	object	No	Additional context (project_id, date range, etc.)

curl -X POST https://your-tms-instance.com/api/v1/ai/organizations/1/chat/conversations/1/messages \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "What are the top 5 failing test cases this week?",
    "context": {
      "project_id": 1,
      "date_range": "last_7_days"
    }
  }'

Response 200 OK

{
  "message_id": 1,
  "user_message": "What are the top 5 failing test cases this week?",
  "ai_response": "Based on the test execution data from the last 7 days, here are the top 5 failing test cases:\n\n1. **User Login with Invalid Password** - Failed 12 times (80% failure rate)\n2. **Shopping Cart Checkout** - Failed 8 times (53% failure rate)\n3. **Payment Processing Timeout** - Failed 6 times (40% failure rate)\n4. **Search Results Pagination** - Failed 5 times (33% failure rate)\n5. **User Profile Image Upload** - Failed 4 times (27% failure rate)\n\nThe login test failures correlate with a deployment on Wednesday that changed the authentication flow.",
  "actions_taken": [
    {
      "type": "query_database",
      "query": "Queried test results for failures in last 7 days"
    }
  ],
  "created_at": "2025-11-15T10:00:00Z"
}

The actions_taken field shows what the AI did behind the scenes to answer your question (e.g., database queries, API calls).

Example questions you can ask

• "What is the pass rate trend for the last month?"
• "Which test suites have the most failures?"
• "Show me all flaky tests in the authentication module"
• "Compare the results of test run 42 and test run 45"
• "What areas have no test coverage?"

List conversations

GET /api/v1/ai/organizations/{organization_id}/chat/conversations

curl https://your-tms-instance.com/api/v1/ai/organizations/1/chat/conversations \
  -H "Authorization: Bearer YOUR_TOKEN"

Response 200 OK

{
  "items": [
    {
      "id": 1,
      "title": "Sprint 12 Test Analysis",
      "message_count": 5,
      "created_at": "2025-11-15T10:00:00Z",
      "last_message_at": "2025-11-15T10:15:00Z"
    },
    {
      "id": 2,
      "title": "Flaky Test Investigation",
      "message_count": 3,
      "created_at": "2025-11-14T14:00:00Z",
      "last_message_at": "2025-11-14T14:20:00Z"
    }
  ]
}

---

Detect flaky tests

Use machine learning to identify tests that pass and fail intermittently.

POST /api/v1/ai/organizations/{organization_id}/detect-flaky-tests

Request body

Field	Type	Required	Description
project_id	integer	Yes	Project to analyze
min_executions	integer	No	Minimum executions to consider (default: 10)
threshold	number	No	Flakiness threshold 0-1 (default: 0.7)

curl -X POST https://your-tms-instance.com/api/v1/ai/organizations/1/detect-flaky-tests \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "project_id": 1,
    "min_executions": 10,
    "threshold": 0.5
  }'

Response 200 OK

{
  "flaky_tests": [
    {
      "test_case_id": 42,
      "test_case_title": "User profile update",
      "flakiness_score": 0.85,
      "total_executions": 50,
      "pass_count": 35,
      "fail_count": 15,
      "pattern": "Intermittent timeout on API call"
    },
    {
      "test_case_id": 67,
      "test_case_title": "Shopping cart total calculation",
      "flakiness_score": 0.62,
      "total_executions": 40,
      "pass_count": 33,
      "fail_count": 7,
      "pattern": "Race condition between cart update and price recalculation"
    }
  ],
  "summary": {
    "total_analyzed": 150,
    "flaky_count": 2,
    "flaky_percentage": 1.3
  }
}

The flakiness_score ranges from 0 (completely stable) to 1 (extremely flaky). The pattern field describes the AI's assessment of why the test is flaky.

When to investigate

Tests with a flakiness score above 0.5 are worth investigating. They waste CI/CD time, erode team trust in the test suite, and can mask real bugs.

---

Using AI endpoints in your workflow

Here is an example of a complete AI-powered workflow: generate tests from a requirement, run them, and analyze any failures.

# 1. Generate test cases from a user story
curl -X POST https://your-tms-instance.com/api/v1/ai/organizations/1/generate-test-cases \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "requirement": "Users should be able to export their data as CSV from the settings page",
    "module": "Settings"
  }'

# 2. After running the tests, analyze any failures
curl -X POST https://your-tms-instance.com/api/v1/ai/organizations/1/analyze-failure \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "test_result_id": 456,
    "include_logs": true
  }'

# 3. Ask the AI agent for recommendations
curl -X POST https://your-tms-instance.com/api/v1/ai/organizations/1/chat/conversations/1/messages \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Based on the recent failures in the Settings module, what should we prioritize fixing?"
  }'

---

Next steps

• Test Management API -- Core CRUD endpoints for projects, tests, and runs
• Integrations API -- Set up webhooks and connect your tools
• CLI Tool -- Access all AI features from the command line