Drivers

A driver is how Firedrill sends each step of a drill to your agent and reads its answer. Pick the one that matches how real users or callers reach your agent. You set it in two places: driver.profile in the descriptor and the driver block in the local plan.

Profile	Use it when your agent is	Your agent must
`http.v1`	An HTTP API or chat endpoint	Accept a POST and return the reply
`jsonl-process.v1`	A command-line program	Read JSON lines on stdin and write JSON lines on stdout
`function-wrapper.v1`	A function in your code	Be called from a small wrapper script
`playwright.v1`	A web app used in a browser	Be driven by a Playwright flow file
`signed-http-event.v1`	A webhook receiver	Accept signed HTTP events

What your agent receives

Each drill is a series of steps. A step is a user message, a task, or an event. Your agent only receives the step itself. It never sees the persona’s goals or the evaluation criteria. Every step carries a session key that identifies the drill’s conversation. Use it to keep conversation state per session. Your agent’s answer becomes the reply in the drill. A non-empty string is used as it is. Any other JSON value is stored as JSON. An empty answer means no reply. If a step may have reached your agent but no full answer came back, Firedrill marks it as unknown and does not retry it.

http.v1

Firedrill sends each step as an HTTP request to your service on your machine.

Plan driver block

{
  "profile": "http.v1",
  "service_alias": "agent",
  "path": "/chat",
  "method": "POST",
  "stream": "none",
  "session_header": null,
  "request_template": "message-text",
  "response_pointer": "/reply"
}

Field	Description
`path`, `method`	Where to send steps. `method` is `POST` or `PUT`.
`request_template`	`message-text` sends `{"message": "<text>", "session": "<session key>"}`. `task-json` sends the task input as the body.
`response_pointer`	A JSON pointer to the reply in your response, such as `/reply`. Use `null` to take the whole body.
`stream`	`none`, `sse` or `ndjson`. With a pointer, text pieces from each streamed event are joined.
`session_header`	A header name that also receives the session key, or `null`.

Your service must answer with a 2xx status. Redirects are not followed. The service needs http readiness in the plan. With the example above, your agent receives:

POST /chat
content-type: application/json

{"message": "Hi, my order arrived broken.", "session": "drill-7f3a"}

and answers:

{"reply": "Sorry to hear that. Can you share the order number?"}

jsonl-process.v1

Firedrill starts your program for each drill and talks to it over stdin and stdout.

Plan driver block

{ "profile": "jsonl-process.v1", "service_alias": "agent" }

Each line Firedrill writes to stdin:

{"id": "step-1.1", "session": "drill-7f3a", "kind": "message", "input": "Hi, my order arrived broken."}

Your program answers on stdout with the same id and either output or error:

{"id": "step-1.1", "output": "Sorry to hear that. Can you share the order number?"}

Rules:

Set stdout readiness in the plan, for example {"kind": "stdout", "pattern": "READY"}, and print that line when your program is ready.
After the ready line, every stdout line must be a JSON object. Write logs to stderr.
Firedrill sends one step at a time.
The service must use mode: launch.

function-wrapper.v1

Use this when your agent is a function. You write a short wrapper that imports it and speaks the same JSON lines protocol as jsonl-process.v1.

Plan driver block

{
  "profile": "function-wrapper.v1",
  "root_alias": "app",
  "wrapper": "firedrill_wrapper.py",
  "runtime": { "executable": "python3", "args": [] }
}

firedrill_wrapper.py

import json
import sys

from mypkg import handle

print("firedrill-wrapper-ready", flush=True)
for line in sys.stdin:
    request = json.loads(line)
    try:
        result = handle(request["input"], request["session"])
        print(json.dumps({"id": request["id"], "output": result}), flush=True)
    except Exception as error:
        print(json.dumps({"id": request["id"], "error": str(error)}), flush=True)

The wrapper must print a line containing firedrill-wrapper-ready before the first step. Set the descriptor’s adapter_sha256 to the SHA-256 of the wrapper file. If the file changes, run stops and asks you to approve again.

playwright.v1

Use this for an agent people use through a web page. Firedrill opens a fresh headless Chromium context for each session and runs your flow file.

Plan driver block

{
  "profile": "playwright.v1",
  "root_alias": "app",
  "flow": "firedrill.flow.mjs",
  "service_alias": "web",
  "start_path": "/"
}

firedrill.flow.mjs

export async function login(page, identity) {}

export async function send(page, step) {
  await page.fill("#message", step.text);
  await page.click("#send");
  return await page.locator(".reply").last().innerText();
}

Export	Required	Description
`login(page, identity)`	Yes	Signs in. `identity` is `null` for anonymous agents.
`send(page, step)`	Yes	Performs one step. Its return value is the reply.
`reset(page)`	No	Clears state between steps if needed.

The step object has step_id, sequence, kind, session_key, text and input. Install Playwright in your project first. Firedrill does not install it. Set adapter_sha256 to the SHA-256 of the flow file. To capture masked screenshots, add "screenshots": {"mask_selectors": ["#card-number"]} to the driver block.

signed-http-event.v1

Use this for an agent that reacts to webhooks. Each scenario sends one signed event to your service.

Plan driver block

{
  "profile": "signed-http-event.v1",
  "service_alias": "receiver",
  "path_prefix": "/webhooks",
  "signing_secret_env": "WEBHOOK_SECRET"
}

Firedrill sends the event’s body with these headers:

Header	Value
`firedrill-delivery-id`	A unique ID for the delivery
`x-firedrill-signature`	`sha256=` and the hex HMAC-SHA256 of the raw body, keyed with your secret

The signing secret is read from WEBHOOK_SECRET in your environment. It never leaves your machine. Any 2xx response counts as delivered. The drill ends with delivery_acknowledged, because Firedrill sees that you received the event, not the work it started. The scenario for this driver needs a target_input of kind event. See Scenarios.

Signed-in test users

Set authentication to synthetic-session in the descriptor when your agent needs a signed-in user. Firedrill passes a test session to your agent: in a request header for HTTP, in a headers object for JSON lines, and through login(page, identity) for Playwright. Credentials are never put in environment variables. MFA, passkeys and single sign-on flows are not supported.

Connect synthetic tools

Route your agent’s API and MCP calls.

Run drills

Plan, apply and run.

Get started

Coding agents

Connect your agent

Build a world

Run drills

Read results

Account

Reference

What your agent receives

http.v1

jsonl-process.v1

function-wrapper.v1

playwright.v1

signed-http-event.v1

Signed-in test users

Connect synthetic tools

Run drills