Skip to Content
CLI Referencelist

agenticassure list

The list command displays all scenarios found in one or more YAML files, along with their metadata. It is a quick way to inspect what scenarios exist, which scorers they use, and how they are tagged — without running anything.

agenticassure list [OPTIONS] [PATH]

Arguments

ArgumentDefaultDescription
PATH. (current directory)A file or directory containing YAML scenario files. Must exist.

Options

OptionShortTypeDefaultDescription
--tag-tString (repeatable)NoneFilter scenarios by tag. Can be specified multiple times.
--json-outputFlagfalseOutput the scenario list as a JSON array
--helpFlagShow help and exit

What It Shows

For each scenario, the list command displays:

  • Suite — The name of the suite the scenario belongs to.
  • Name — The scenario name.
  • Input — The input text that will be sent to the agent.
  • Scorers — The scorer(s) assigned to the scenario (comma-separated).
  • Tags — The tags assigned to the scenario (comma-separated), or - if none.

Table Output (default)

agenticassure list scenarios/
                        Scenarios
┌──────────────────┬───────────────┬─────────────────────────┬──────────┬───────────────┐
│ Suite            │ Name          │ Input                   │ Scorers  │ Tags          │
├──────────────────┼───────────────┼─────────────────────────┼──────────┼───────────────┤
│ search-tests     │ weather_query │ What is the weather in  │ passfail │ tools,weather │
│                  │               │ San Francisco?          │          │               │
│ search-tests     │ greeting      │ Hello, how are you?     │ passfail │ basic         │
│ regression-tests │ tool_chain    │ Book a flight to NYC    │ passfail │ tools,booking │
└──────────────────┴───────────────┴─────────────────────────┴──────────┴───────────────┘

3 scenario(s) found

The table is rendered using Rich for clean formatting in the terminal.

JSON Output

Use --json-output to get the scenario list as a JSON array, suitable for piping to other tools or scripts:

agenticassure list scenarios/ --json-output
[
  {
    "suite": "search-tests",
    "name": "weather_query",
    "input": "What is the weather in San Francisco?",
    "scorers": ["passfail"],
    "tags": ["tools", "weather"]
  },
  {
    "suite": "search-tests",
    "name": "greeting",
    "input": "Hello, how are you?",
    "scorers": ["passfail"],
    "tags": ["basic"]
  }
]

The JSON output is written to stdout, making it easy to integrate with jq, scripts, or other tooling:

# Count scenarios per suite
agenticassure list scenarios/ --json-output | jq 'group_by(.suite) | map({suite: .[0].suite, count: length})'
 
# Get all unique tags
agenticassure list scenarios/ --json-output | jq '[.[].tags[]] | unique'

Filtering by Tags

Use --tag to display only scenarios that have a matching tag. Multiple --tag flags use OR logic — a scenario is included if it matches any of the specified tags.

# Show only scenarios tagged "tools"
agenticassure list scenarios/ --tag tools
 
# Show scenarios tagged "tools" or "basic"
agenticassure list scenarios/ --tag tools --tag basic

Tag filtering works with both table and JSON output modes:

agenticassure list scenarios/ --tag smoke --json-output

Path Resolution

Path handling is identical to other commands:

  • Single file: Lists scenarios from that file only.
  • Directory: Recursively scans for all .yml and .yaml files.
# List from a single file
agenticassure list scenarios/search_tests.yaml
 
# List from all files in a directory
agenticassure list scenarios/
 
# List from the current directory
agenticassure list

Examples

List all scenarios in a directory:

agenticassure list scenarios/

List scenarios from a single file with tag filter:

agenticassure list scenarios/search_tests.yaml --tag tools

Export scenario metadata as JSON for scripting:

agenticassure list scenarios/ --json-output > scenario_manifest.json

Combine with validate for a quick audit:

agenticassure validate scenarios/ && agenticassure list scenarios/

What’s Next

Last updated on
validateinit