Start testing your voice agent

1. Install the CLI

The Vexa CLI wraps the REST API and makes it easy to run suites from a terminal or a CI step. You can also call the API directly withcurl or any HTTP client.

npm install --save-dev vexa-cli

2. Get an API key

Sign in to your Vexa dashboard and navigate to Settings → API keys. Create a new key and copy it. Keys start with vexa_sk_. Set it as an environment variable so it is available to the CLI and your scripts:

export VEXA_API_KEY=vexa_sk_live_xxxxxxxxxxxxxxxxxxxx

In CI, store the key as a secret (see the CI section below). Never commit it to source control.

3. Register your agent

Register your voice agent by telling Vexa where to reach it and which protocol it speaks. Vexa connects to your agent the same way a real telephony platform does: it dials in over WebSocket or SIP and speaks as a simulated caller.

curl -X POST https://api.vexavoice.tech/v1/agents \
  -H "Authorization: Bearer vexa_sk_live_xxxx" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Card Bot v2",
    "endpoint": "wss://bot.acme.com/voice",
    "protocol": "twilio-twiml"
  }'

The response includes an id you will use in every run. Copy it.

{
  "id": "agent_01J9KXPR7WQVHB2N5M8ZD3CA4T",
  "name": "Card Bot v2",
  "endpoint": "wss://bot.acme.com/voice",
  "protocol": "twilio-twiml",
  "createdAt": "2025-03-14T09:41:22Z"
}

Supported protocols: twilio-twiml, vapi, retell, raw-sip.

4. Run your first suite

A suite is a set of simulated call scenarios. Vexa ships with built-in suites for common voice agent types. Use suite_card_replacement for this walkthrough.

npx vexa@latest run \
  --agent agent_01J9KXPR7WQVHB2N5M8ZD3CA4T \
  --suite suite_card_replacement

The run starts immediately. The CLI streams progress. Each scenario in the suite is dialed concurrently (default concurrency: 8).

5. Read the scores

When the run completes, fetch it by its ID. The top-level fields give you the aggregate verdict; the turns array gives you every exchange.

curl https://api.vexavoice.tech/v1/runs/run_01J9KZ7YH5CQNM3FX8WBR4DV2E \
  -H "Authorization: Bearer vexa_sk_live_xxxx"

{
  "id": "run_01J9KZ7YH5CQNM3FX8WBR4DV2E",
  "agentId": "agent_01J9KXPR7WQVHB2N5M8ZD3CA4T",
  "suiteId": "suite_card_replacement",
  "status": "complete",
  "verdict": "pass",
  "overall": 0.91,
  "taskSuccess": 0.89,
  "werPct": 4.2,
  "avgLatencyMs": 340,
  "bargeIns": 2,
  "turns": [
    {
      "speaker": "agent",
      "text": "Hi, this is Maya. How can I help you today?",
      "latencyMs": 320,
      "bargeIn": false,
      "success": true,
      "tone": "warm",
      "flag": null,
      "note": null
    },
    {
      "speaker": "sim",
      "text": "I need to replace my damaged card.",
      "latencyMs": 0,
      "bargeIn": false,
      "success": true,
      "tone": "neutral",
      "flag": null,
      "note": null
    },
    {
      "speaker": "agent",
      "text": "Of course. Can I confirm the last four digits of the card?",
      "latencyMs": 295,
      "bargeIn": false,
      "success": true,
      "tone": "professional",
      "flag": null,
      "note": null
    }
  ],
  "failingTurn": null,
  "createdAt": "2025-03-14T09:41:22Z",
  "completedAt": "2025-03-14T09:49:37Z"
}

A verdict of pass means task success across all scenarios met the default threshold of 0.80. You can override thresholds per run or per suite.

Gate CI on a regression

Add a Vexa step to your GitHub Actions workflow. The CLI exits with a non-zero code if any threshold is breached, which fails the build.

# .github/workflows/voice-regression.yml
name: Voice regression

on:
  push:
    branches: [main]
  pull_request:

jobs:
  vexa:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run Vexa regression suite
        run: |
          npx vexa@latest run \
            --agent ${{ secrets.VEXA_AGENT_ID }} \
            --suite suite_card_replacement \
            --fail-below taskSuccess=0.85 \
            --fail-below werPct=8.0
        env:
          VEXA_API_KEY: ${{ secrets.VEXA_API_KEY }}

The --fail-below flag accepts any scored field: taskSuccess, overall, or werPct (where lower is worse means you use --fail-above). See the Guides for more CI patterns.

Every run returns these aggregate fields. The API reference lists every field and its type.

What a failing run looks like

When verdict is fail, the failingTurn object pinpoints the first turn that did not pass, with a flag explaining why and a note written by the scoring model.

{
  "id": "run_01J9KZ7YH5CQNM3FX8WBR4DV2E",
  "status": "complete",
  "verdict": "fail",
  "overall": 0.62,
  "taskSuccess": 0.48,
  "werPct": 12.1,
  "avgLatencyMs": 890,
  "bargeIns": 7,
  "turns": [ ... ],
  "failingTurn": {
    "index": 5,
    "speaker": "agent",
    "text": "Your replacement card has been ordered.",
    "latencyMs": 1240,
    "bargeIn": false,
    "success": false,
    "tone": "flat",
    "flag": "task_incomplete",
    "note": "Agent confirmed dispatch without verifying mailing address"
  }
}

Turn flags include: task_incomplete, wrong_info, barge_in_unhandled, latency_spike, tone_mismatch, off_script. See the API reference for the full enum.