CLI Report

The CLI report is the default output format for agenticassure run. It renders test results directly in the terminal as a formatted table using Rich , with color-coded status indicators and a summary line.

Generating a CLI Report

The CLI report is produced by default when you run scenarios:


agenticassure run scenarios/ --adapter my_agent.MyAgent

You can also request it explicitly:


agenticassure run scenarios/ --adapter my_agent.MyAgent --output cli

Report Structure

A CLI report consists of three sections: a header, a results table, and a summary line.

Header

The header shows the suite name, a unique run ID, and a timestamp:


Run Results: search-agent-tests
Run ID: a1b2c3d4-e5f6-7890-abcd-ef1234567890
Timestamp: 2026-03-10T14:30:00+00:00

Results Table

The main table contains one row per scenario:


┌─────────────────┬────────┬───────┬──────────┬──────────────────────────────────────┐
│ Scenario        │ Status │ Score │ Duration │ Details                              │
├─────────────────┼────────┼───────┼──────────┼──────────────────────────────────────┤
│ weather_query   │ PASS   │  1.00 │   245ms  │ Output contains expected text;       │
│                 │        │       │          │ Tool 'get_weather' called correctly   │
│ greeting        │ PASS   │  1.00 │   129ms  │ Output contains expected text        │
│ bad_input       │ FAIL   │  0.00 │   312ms  │ Expected tool 'search' was not called│
│ timeout_test    │ ERROR  │  0.00 │ 30001ms  │ Scenario timed out after 30.0s       │
└─────────────────┴────────┴───────┴──────────┴──────────────────────────────────────┘

Columns

Column	Description
Scenario	The name of the scenario as defined in the YAML file
Status	`PASS` (green), `FAIL` (red), or `ERROR` (red). ERROR indicates an exception occurred during execution rather than a scoring failure.
Score	The average score across all scorers for this scenario, from `0.00` to `1.00`
Duration	Wall-clock time for the scenario execution in milliseconds
Details	Scorer explanations (semicolon-separated if multiple scorers) or the error message if an error occurred

Summary Line

Below the table, a summary line shows aggregate statistics:


Summary: 75% passed | Score: 0.75 | Duration: 30687ms

The pass rate is color-coded:

Pass Rate	Color
80% or above	Green
50% to 79%	Yellow
Below 50%	Red

Multi-Suite Runs

When multiple suites are loaded (e.g., from a directory with several YAML files), each suite gets its own table and summary. After all suites, an overall summary is printed:


Overall: 18/20 scenarios passed across 4 suite(s)

When to Use the CLI Report

The CLI report is best suited for:

Local development — Quick feedback while iterating on scenarios or adapters.
Interactive debugging — The details column shows exactly why each scenario passed or failed.
Quick checks — Fastest way to verify that everything works before generating a shareable report.

For sharing results with others or archiving, consider the HTML report or JSON report.

What’s Next

HTML Report — Standalone HTML reports for sharing and archival.
JSON Report — Structured JSON output for programmatic consumption.
agenticassure run — Full reference for the run command.