agenticassure init

The init command scaffolds a new AgenticAssure project by creating a scenarios/ directory with example scenario files. It is the fastest way to get started with a working project structure.

agenticassure init [DIRECTORY]

Arguments

Options

What It Creates

Running agenticassure init produces the following structure inside the target directory:

<directory>/
  scenarios/
    example_scenarios.yaml

scenarios/example_scenarios.yaml

The generated file contains a complete, valid suite with two example scenarios:

suite:
  name: example-agent-tests
  description: Example test scenarios for your AI agent
  config:
    default_timeout: 30
    retries: 1
 
scenarios:
  - name: basic_greeting
    input: "Hello, how are you?"
    expected_output: "hello"
    scorers:
      - passfail
    tags:
      - basic
 
  - name: tool_usage
    input: "What is the weather in San Francisco?"
    expected_tools:
      - get_weather
    expected_tool_args:
      get_weather:
        location: "San Francisco"
    scorers:
      - passfail
    tags:
      - tools

The first scenario demonstrates a basic output-matching test. The second demonstrates tool call verification, including expected tool names and arguments.

Usage

Initialize in the current directory:

agenticassure init

Initialize in a specific directory:

agenticassure init my-agent-project

If the target directory does not exist, it will be created (including any intermediate directories).

If the scenarios/ subdirectory or example_scenarios.yaml file already exists, it will be overwritten.

Output

After running, init prints a confirmation and next steps:

Initialized AgenticAssure project in /home/user/my-agent-project
  Created: /home/user/my-agent-project/scenarios/example_scenarios.yaml

Next steps:
  1. Edit scenarios in the 'scenarios/' directory
  2. Create an adapter for your agent
  3. Run: agenticassure run scenarios/

What to Do After Init

1. Edit the Example Scenarios

Open scenarios/example_scenarios.yaml and replace the example scenarios with tests relevant to your agent. Update the suite name, inputs, expected outputs, expected tools, scorers, and tags to match your use case.

You can also create additional .yaml files in the scenarios/ directory. AgenticAssure loads all YAML files recursively when you point it at a directory.

2. Write an Adapter

Create a Python module that wraps your agent and implements the AgentAdapter protocol:

# my_agent.py
from agenticassure.adapters.base import AgentAdapter
from agenticassure.results import AgentResult
 
class MyAgent:
    def run(self, input: str, context=None) -> AgentResult:
        # Call your agent and return the result
        response = your_agent.invoke(input)
        return AgentResult(output=response)

3. Validate Your Scenarios

Before running, verify that your scenario files are correctly structured:

agenticassure validate scenarios/

4. Run Tests

Execute your scenarios against your agent:

agenticassure run scenarios/ --adapter my_agent.MyAgent

5. (Optional) Create a Config File

To avoid passing --adapter every time, create an agenticassure.yaml in your project root:

adapter: my_agent.MyAgent

Then you can simply run:

agenticassure run scenarios/

What’s Next

• Quickstart — Full walkthrough from install to first test run.
• agenticassure run — Running test scenarios.
• Scenarios & Suites — Complete YAML format reference.
• Adapters — Writing an adapter for your agent.