Read reports

A report shows what happened in a campaign: how each drill ended, the result of each evaluation and the evidence behind it. Read it in the CLI, in the web app under Simulations, or through your coding agent.

Campaign report

firedrill report <campaign-id>

This shows the campaign’s progress, each drill’s ending and a quality summary for each evaluation.

firedrill report <campaign-id> quality

The quality view groups results for one evaluation across drills: how many were assessed, met the threshold, fell below it or were not assessable, with the average, lowest and highest score. Results for different evaluations are never combined.

Drill report

firedrill report <campaign-id> drill <drill-id>
firedrill report <campaign-id> drill <drill-id> results

Part	Command suffix	What it shows
Results	`results [criterion-id]`	Each evaluation’s status, score, threshold and exact checks
Conversation	`conversation [turn]`	The simulated user’s messages and your agent’s replies
Actions	`actions [action-id]`	What your agent did, step by step
Effects	`effects`	Calls to synthetic tools and the state they changed
Events	`events [event-id]`	Scenario events and when they were applied
Evidence	`evidence [record-id]`	The stored evidence records

Drill status

Status	Meaning
`pending`, `preparing`, `evaluating`	Scoring is not finished yet.
`complete`	Every evaluation was assessed and met its threshold.
`needs_attention`	Something needs a look: an evaluation was below threshold, not assessable or errored, coverage was partial, or an exact check failed.

How a drill ended

Ending	Meaning
`participant_finished`	The simulated user’s stop rules were met.
`inputs_exhausted`	Every scripted input was sent.
`participant_gave_up`	The simulated user gave up.
`turn_limit`	The scenario reached `max_turns`.
`target_failed`	Your agent returned an error or could not be reached.
`target_cancelled`	The drill was stopped.
`delivery_acknowledged`	Your webhook receiver accepted the event.
`participant_unavailable`	The simulated user could not continue. Your agent was not judged.

Only participant_finished and inputs_exhausted count as complete coverage of the scenario.

Evaluation results

Each evaluation in a drill has:

Field	Meaning
`status`	`assessed`, `not_assessable`, `error`, or a pending state
`quality_score`	0, 25, 50, 75 or 100
`threshold`, `threshold_met`	The threshold and whether the score reached it
`exact_checks`	Each exact check with expected and observed values and whether it passed

Not assessable means the evidence the evaluation needs was not recorded, or cannot be scored yet. It is not a zero, and it is not a failure of your agent.

Evidence coverage

Each evidence type shows whether it was recorded for the drill: available, partial, not_assessable, not_selected and a few related values. Partial coverage puts the drill in needs_attention. In --json output, conversation, action and evidence data is marked "untrusted": true. It came from your agent and the simulated user, so treat it as data, not instructions.

In the web app

Simulations lists campaigns.
A campaign page shows drill progress, quality across drills, findings and drill outcomes.
A drill page has Report, Conversation, Activity, Tools and Evidence tabs, including captured screenshots from browser drills.

Reassess a drill

Score stored evidence again.

Prompts

Ask your coding agent to explain a report.

Get started

Coding agents

Connect your agent

Build a world

Run drills

Read results

Account

Reference