Adapters

An adapter is the bridge between AgenticAssure and your AI agent. It translates the test runner’s requests into whatever API or interface your agent uses, and translates the agent’s response back into a structured format that scorers can evaluate.

Why Adapters Exist

AI agents come in many shapes: raw OpenAI API calls, LangChain AgentExecutors, custom orchestration frameworks, proprietary APIs, and more. AgenticAssure cannot know how to call your agent directly. Instead, you implement a thin adapter class that satisfies a simple protocol, and AgenticAssure handles everything else — loading scenarios, running scorers, retrying, reporting.

This design keeps AgenticAssure decoupled from any specific agent framework and lets you test any agent that can accept a text input and produce a text output.

The AgentAdapter Protocol

The AgentAdapter protocol defines the contract your adapter must fulfill:


from typing import Any, Protocol, runtime_checkable
from agenticassure.results import AgentResult
 
@runtime_checkable
class AgentAdapter(Protocol):
    def run(self, input: str, context: dict[str, Any] | None = None) -> AgentResult:
        """Execute the agent with the given input and return structured results."""
        ...

Your adapter class does not need to inherit from AgentAdapter. It only needs to implement a run method with the correct signature. Python’s structural subtyping (the Protocol and runtime_checkable decorators) handles the rest.

Parameters

input — The scenario’s input string. This is the prompt or message your agent should process.
context — An optional dictionary of additional context passed through from the runner. This can carry session state, user metadata, or any other information your agent needs beyond the input string.

Return Value

The run method must return an AgentResult instance. See the Results documentation for full details on AgentResult.

Implementing an Adapter

Minimal Example

The simplest possible adapter wraps a function that takes a string and returns a string:


from agenticassure.results import AgentResult
 
class MyAgent:
    def run(self, input, context=None):
        # Call your agent however you need to
        response = my_agent_function(input)
        return AgentResult(output=response)

That is all AgenticAssure requires. The adapter receives the scenario input, calls your agent, and returns an AgentResult with at minimum the output field populated.

Returning Tool Calls

If your agent uses tools (function calling), capture them in the tool_calls field so that scorers like passfail can verify the correct tools were invoked:


from agenticassure.results import AgentResult, ToolCall
 
class MyToolAgent:
    def run(self, input, context=None):
        result = call_my_agent(input)
 
        tool_calls = []
        for call in result.tool_invocations:
            tool_calls.append(ToolCall(
                name=call.function_name,
                arguments=call.args,
                result=call.return_value,  # optional
            ))
 
        return AgentResult(
            output=result.final_answer,
            tool_calls=tool_calls,
        )

Returning Token Usage

Track token consumption for cost monitoring and benchmarking:


from agenticassure.results import AgentResult, TokenUsage
 
class MyTrackedAgent:
    def run(self, input, context=None):
        result = call_my_agent(input)
 
        return AgentResult(
            output=result.text,
            token_usage=TokenUsage(
                prompt_tokens=result.usage.input_tokens,
                completion_tokens=result.usage.output_tokens,
            ),
        )

The TokenUsage model also exposes a total_tokens property that sums prompt and completion tokens.

Returning a Full AgentResult

Here is an adapter that populates every available field:


import time
from agenticassure.results import AgentResult, TokenUsage, ToolCall
 
class FullAdapter:
    def run(self, input, context=None):
        start = time.perf_counter()
        result = call_my_agent(input, session=context)
        latency = (time.perf_counter() - start) * 1000
 
        return AgentResult(
            output=result.answer,
            tool_calls=[
                ToolCall(
                    name=tc.name,
                    arguments=tc.args,
                    result=tc.output,
                )
                for tc in result.tool_calls
            ],
            reasoning_trace=result.chain_of_thought,  # list of strings
            latency_ms=latency,
            token_usage=TokenUsage(
                prompt_tokens=result.prompt_tokens,
                completion_tokens=result.completion_tokens,
            ),
            raw_response=result.raw,  # store the raw API response for debugging
        )

How the Adapter Is Resolved

When you run agenticassure run, the CLI resolves the adapter in the following order:

1. The `--adapter` Flag (Highest Priority)

Pass a Python dotted path to your adapter class:


agenticassure run scenarios/ --adapter mypackage.agents.MyAgent

The path is split at the last . — everything before it is the module path, and the last component is the class name. AgenticAssure imports the module, instantiates the class with no arguments, and verifies it satisfies the AgentAdapter protocol.

2. Configuration File

If no --adapter flag is provided, AgenticAssure looks for a configuration file in the current working directory:

agenticassure.yaml — checked first:
```
adapter: mypackage.agents.MyAgent
```
agenticassure.toml — checked second:
```
adapter = "mypackage.agents.MyAgent"
```

The adapter value follows the same dotted-path format as the CLI flag.

3. No Adapter (Dry Run)

If no adapter is found through either method, AgenticAssure validates and lists the scenarios without executing them. It prints a summary table and a message explaining how to supply an adapter. This is useful for verifying your scenario YAML files are well-formed before wiring up an agent.

Built-In Adapters

AgenticAssure ships with adapters for common frameworks. These are convenience wrappers — you can always write your own.

OpenAIAdapter

Wraps the OpenAI chat completions API with function calling support:


from agenticassure.adapters.openai import OpenAIAdapter
 
adapter = OpenAIAdapter(
    model="gpt-4",
    system_prompt="You are a helpful assistant.",
    tools=[...],  # OpenAI function definitions
    api_key="sk-...",  # optional, defaults to OPENAI_API_KEY env var
)

This adapter automatically captures tool calls, token usage, latency, and the raw API response.

Install the optional dependency: pip install agenticassure[openai]

LangChainAdapter

Wraps a LangChain AgentExecutor:


from langchain.agents import AgentExecutor
from agenticassure.adapters.langchain import LangChainAdapter
 
executor = AgentExecutor(agent=my_agent, tools=my_tools)
adapter = LangChainAdapter(agent_executor=executor)

This adapter extracts tool calls and reasoning traces from LangChain’s intermediate steps.

Install the optional dependency: pip install agenticassure[langchain]

Tips for Writing Adapters

Keep adapters thin. The adapter should do as little as possible beyond calling your agent and mapping the response into AgentResult. Business logic, retries, and error handling belong in the runner or the agent itself, not in the adapter.

Handle exceptions intentionally. If your agent raises an exception, you can either let it propagate (the runner will catch it and record an error result) or catch it in the adapter and return an AgentResult with an empty output. Letting exceptions propagate is usually better because the runner’s error reporting will capture the exception type and message.

Use context for shared state. If your scenarios need session context, authentication tokens, or other ambient state, pass them through the context dictionary rather than hardcoding them in the adapter.

Measure latency in the adapter. Wrap your agent call with timing code and populate latency_ms. This gives you per-scenario latency data in your reports, which is valuable for performance benchmarking.

Store the raw response. The raw_response field on AgentResult is a good place to stash the unprocessed API response for post-hoc debugging. It accepts any type and is not used by any built-in scorer.

Test your adapter independently. Before plugging it into AgenticAssure, verify that your adapter returns a valid AgentResult for a few representative inputs. This saves time debugging when scenarios start failing.

Adapter instantiation must be zero-argument. When resolved via the --adapter CLI flag or config file, the adapter class is instantiated with no arguments (adapter_cls()). If your adapter needs constructor parameters, either use default values, read from environment variables, or wrap the parameterized adapter in a zero-argument factory class.