> ## Documentation Index > Fetch the complete documentation index at: https://docs.firedrill.run/llms.txt > Use this file to discover all available pages before exploring further. # Run your first drill > Connect an existing agent to an isolated world, run a behavioral test, and inspect the resulting evidence. This guide explains Firedrill's repository contract. The files belong beside the agent code, can be reviewed in pull requests, and work without an account. The npm packages are still an unpublished release candidate. After an authorized publication, the normal project-local installation will be `npm install --save-dev @firedrill/cli`, followed by `npx firedrill init`. Until then, use a reviewed packed release artifact or an authorized framework source checkout. The exact available flags are documented in the [CLI reference](/cli/reference). For your existing agent, start with [local tools](/guides/start-local-tools): ```sh theme={null} firedrill init firedrill serve ``` Choose or create tools, inspect their data and behavior, then connect the existing agent. No scenario or test is required to get a usable backend. To learn the complete **testing** loop through an explicit example instead: ```sh theme={null} firedrill init --path template firedrill validate firedrill firedrill inspect ``` That creates repository-owned source, validates it, runs one passing drill in an isolated world, prints the self-contained report path, and opens the local inspector. No account or network service is involved. For saved results, open `.firedrill/reports/index.html`. This central page lets you search and filter recorded executions, then open an individual report's task, checks, tool calls and data changes. It refreshes when the CLI or SDK saves reports; no inspector server is required to browse it. A custom `--report-dir` has its own `index.html`. The CLI prints this path once as `All reports`, and JSON/SDK results expose `reportIndex`. Individual run folders remain portable evidence bundles. The central index is just navigation over verified bundles; unavailable reports are listed without unsafe links, and bounded-history limits are stated on the page. Copy the index and its run folders together if you intend to share a browsable collection. Keep generated output out of Git. ## 1. Pick a verified starting path In an interactive terminal, `firedrill init` guides tool selection, optional authoring, and local startup. A catalog package requires installation consent if missing; custom tools need no key or account. The wizard does not require you to invent a test task before starting the backend. In JSON, CI, or piped use, bare `init` remains read-only. Use `--tool`, `--custom`, or a legacy `--path` to make an explicit setup selection; `--start` opts into foreground execution. ```sh theme={null} firedrill init firedrill init --path firedrill-agent firedrill init --custom my-tool --json firedrill init --tool @scope/firedrill-tool firedrill init --path coding-agent # or: firedrill init --path template # or: firedrill init --path manual ``` The explicit `--path firedrill-agent` and `--path coding-agent` forms install the same skill and repository brief under `.agents/`. Firedrill Agent uses the separately installed `@firedrill/agent` and your `ANTHROPIC_API_KEY`; its default task is preparing tools, not inventing a test suite. The template path remains an explicit complete runnable example. The legacy manual path creates a world shell and probe tool; prefer `--custom ` for an editable stateful backend. Existing files are never silently replaced. Setup ensures `.firedrill/` is Git-ignored. Alternatively, copy `examples/quickstart` into a temporary directory or inspect it in place. It contains one intentionally plain agent and one Tool so the framework concepts stay visible. ```text theme={null} firedrill.json firedrill/ world.yaml tools/workspace/ workspace.tool.yaml behavior.js scenarios/empty.scenario.yaml targets/local-agent.target.yaml drills/set-record.drill.yaml suites/workspace-conformance.suite.yaml agent.mjs ``` `agent.mjs` is the agent being tested; the `firedrill/` folder describes its controlled surroundings and tests. The Tool declaration owns record/input/output schemas; its behavior module owns the fake consequences. `world.yaml` owns shared starting conditions and permissions, scenarios provide starting data or variations, and each drill owns its task and assertions. The example agent is a deterministic HTTP client, not an LLM. The generated template follows the same folders, with its demonstration agent kept separately in `firedrill-example/agent.mjs`. Both are teaching conventions, not required project layouts. Resource files are discovered recursively under `sourceRoot`; descriptive kebab-case filenames with `.tool`, `.scenario`, `.target`, `.drill` or `.suite` suffixes make their purpose obvious. YAML, YML and JSON are supported. References use stable in-file IDs, so changing a display title need not move a file. Existing flat projects remain valid and need no migration. Do not rerun `init` just to rearrange an existing world. If you choose to move source, preserve IDs, update relative Tool module paths and any configured world path, then validate. Set `sourceRoot` and `world` in `firedrill.json` if you prefer another source folder or world filename; neither `tools/` nor any other example subfolder is mandatory. Run `firedrill validate`, inspect the discovered work with `firedrill plan`, then run `firedrill`. A fresh SQLite world is created, the agent is invoked with only its declared binding, assertions inspect the resulting consequences, and the terminal prints a local HTML report path. ### What belongs in Git | Commit | Keep local and ignored | | --------------------------------------------------------------- | -------------------------------------------------- | | `firedrill.json` | `.firedrill/builds/` | | `firedrill/**/*.yaml`, JSON, and behavior modules | `.firedrill/worlds/` and `.firedrill/runs/` | | Agent/test integration code | `.firedrill/reports/` and `.firedrill/tool-tests/` | | The installed coding-agent skill, when the team wants it shared | `.firedrill/contributions/` | The ignored side is reproducible runtime output and may contain synthetic data, model output, and evidence. Provider keys belong in the agent's normal ignored environment files or secret manager—never in either Firedrill source or reports. Repository data is the reproducible starting definition, not a live database dump. Each trial, retry, and concurrent run materializes its own SQLite file from the pinned build plus scenario. Runtime Tool calls mutate that isolated file and never write records back into YAML or JSON. Editing repository data creates the starting state for later builds and runs; retained older worlds and evidence remain unchanged. All Tools selected for one trial share that trial's world database, which is what makes cross-Tool consequences, the clock, pending events, and the ordered journal atomic. A project may therefore retain many SQLite worlds without assigning one database per vendor or Tool. ## 2. Replace the fixture with the agent's real boundaries Work from the interfaces the agent already uses: * Model each required action surface as a Tool operation with typed input, output, and deterministic behavior. * Put baseline records, actors, permissions, time, and initial events in the world. * Put each meaningful starting condition or provider failure in a scenario. * Choose one target matching how the agent already runs: module, command, local HTTP, or an SDK callback. * Give the target only the direct, HTTP, MCP, or CLI bindings it needs. * Write drills around observable consequences and safety invariants, not phrasing in the model response. Tools are not limited to REST APIs. They describe capabilities and consequences; direct, HTTP, MCP, and CLI are transport adapters over the same Tool behavior and SQLite world. The generic HTTP adapter does not automatically reproduce a vendor's URL and payload conventions. Keep that translation at the agent's normal client seam or in a small repository adapter instead of adding vendor branches to Firedrill core. ### Reuse an installed Tool package when one fits A reusable Tool is an ordinary package dependency, not a framework feature switch. Install it with the project's package manager, then select it once in `firedrill.json`: ```sh theme={null} pnpm add -D @scope/firedrill-tool ``` ```json theme={null} { "schemaVersion": 1, "sourceRoot": "firedrill", "world": "world.yaml", "toolPackages": ["@scope/firedrill-tool"] } ``` Install the selected Tool package itself; do not add its Firedrill implementation dependencies to the application. The compiler embeds the approved Tool behavior runtime into the locked artifact. The generated [Tool catalog](/guides/tool-discovery) shows operation-level fidelity for known packs. The selected package's own documentation gives its Tool id, operations, state contract, and setup. `firedrill validate` reads and locks the selected declaration without executing behavior. Use `firedrill tool inspect ` to see exactly what was selected, then `firedrill tool validate ` or run a drill to execute it locally. Your repository still owns its initial data, actors, scenarios, targets, and drills. Firedrill never edits the installed package. Browse the reviewed community catalog with `firedrill tool list`, or find one integration with `firedrill tool search `. The printed install source is exact and can be passed to `firedrill tool add --install` after review. ## 3. Keep the agent integration at one seam For a command target, Firedrill sends a JSON invocation on stdin and supplies the declared binding variables, such as `FIREDRILL_HTTP_URL` and `FIREDRILL_HTTP_TOKEN`, their MCP equivalents, or `FIREDRILL_CLI_URL` and `FIREDRILL_CLI_TOKEN`. For an external target, `runDrills()` supplies the same values in `binding.environment`. A direct binding is available only to a module or callback target that declares it. Firedrill does not become the model-provider credential store. An external callback uses the provider configuration already available to its owning process. A command target can map only the host variables it needs: ```yaml theme={null} environmentFromHost: ANTHROPIC_API_KEY: ANTHROPIC_API_KEY ``` Unlisted host variables are not inherited by the command. The target's `timeoutMs` covers the complete agent interaction, including all model turns and Tool calls, so choose it for the slowest expected end-to-end loop rather than one request. Repoint the agent's existing test configuration, or use a separate test-only adapter around its ordinary entry point. Do not modify production agent logic, duplicate every action, or scatter test-mode branches through business logic. When an ordinary test needs different starting data or a temporary Tool behavior, use the repository-level SDK rather than editing source and restoring it: ```ts theme={null} const result = await runDrills({ root: process.cwd(), drill: "one-explicit-drill", setup: { scenario: { state: [ { action: "upsert", packageId: "record-store", namespace: "records", rowId: "primary", value: { status: "ready" }, }, ], }, bindings: { environment: { RECORDS_BASE_URL: "FIREDRILL_HTTP_URL" } }, }, agent: ({ task, binding, signal }) => runExistingAgent({ task, environment: binding.environment, signal }), }); ``` The `setup` object is serialized, normalized, hashed, and compiled into a derived immutable build. It may add or replace starting state rows and actors, set virtual time, activate declared faults, append initial events, select installed Tool packages, point one declared Tool at a repository-owned behavior module, and map temporary bindings onto names the agent already understands. It never writes those choices back to YAML, JSON, or SQLite behind the report. Command targets receive binding aliases automatically; caller-owned targets pass `binding.environment` through the agent's existing configuration seam. Unsupported protocol mappings fail before agent execution. Full details are in the [`@firedrill/sdk` guide](/sdk/local-typescript#per-test-synthetic-data-and-tools). HTTP-bound agents can discover loaded operations at `GET $FIREDRILL_HTTP_URL/v1/tools` and call one at `POST /v1/operations/{packageId}/{operationId}` with bearer authentication. Discovery is not an access grant: the kernel checks actor permissions on invocation. MCP-bound agents use the supplied Streamable HTTP URL and token; discovered names are `{packageId}.{operationId}`. CLI-bound agents call `firedrill world tools --json` and `firedrill world call --input '{...}' --json`. The protocol READMEs define the exact envelopes. ## 4. Run and diagnose ```sh theme={null} firedrill validate firedrill plan firedrill firedrill run --trials 3 --seed 42 firedrill run --json firedrill run --suite --concurrency 4 firedrill run --tag safety --shard 1/2 firedrill run --watch firedrill report verify .firedrill/reports/ firedrill tool inspect firedrill tool validate firedrill tool test ``` Exit code `0` means every selected drill passed. Exit code `1` means source, execution, or assertions failed. Exit code `2` means the CLI invocation itself was invalid. Human and JSON modes carry the same diagnostics and report locations. Every trial retains its exact world and writes terminal, JSON, JSONL, JUnit, and self-contained HTML evidence under the current project's `.firedrill/` directory. That directory is generated and Git-ignored by default. Use `firedrill report verify ` to check the exact file set, hashes, schemas, identities, evidence ordering, and generated projections without an account or network. This proves bundle integrity, not authorship. The report's reproduction command loads its content-addressed build with `--build-hash` and reruns the recorded seed. Keep that project-local build directory while reproducing; if it was removed, restore the source revision and setup that produced the displayed hash and compile it again. A drill timeline can span hours of virtual time while running locally in minutes. It declares actors, ordered interactions, a horizon, invariant checkpoints, and Tool-call and event budgets; it is still a drill and uses the same runner and evidence. Watch mode queues edits and reruns without overlapping. To inspect change, compare two verified report directories: ```sh theme={null} firedrill compare .firedrill/reports/ .firedrill/reports/ ``` Read the compatibility grade before interpreting deltas. A build or Tool-lock change is descriptive evidence, not proof that the agent regressed. Tool conformance uses ordinary drills rather than a second test language. Name the suite `-conformance` (or pass `--suite`), cover every declared operation/error/event/fault/subscription, and run `firedrill tool test `. Firedrill executes it twice against one immutable build and checks same-seed state and trajectory hashes. After it passes, a human who owns the source may prepare a non-uploading review bundle: ```sh theme={null} firedrill tool contribute --accept-apache-2.0 ``` This copies only the Tool declaration and its exact local behavior dependency closure, blocks common secret patterns, writes checksums and a portable conformance summary, and never overwrites, uploads, or opens a pull request. ## 5. Let an authoring agent iterate to green A coding agent should begin with `.agents/firedrill/BRIEF.md` and `.agents/skills/firedrill/SKILL.md`, inspect the real agent's tool clients, existing mocks, fixtures, and failure tests, create or select Tool packages, then run `firedrill validate --json` repeatedly until diagnostics are empty. It should run a small passing and intentionally failing drill before adding breadth. It must not invent unsupported fidelity or change production behavior merely to satisfy a fixture. The optional local Firedrill Agent follows that same skill rather than a private format: ```sh theme={null} pnpm add -D @firedrill/agent export ANTHROPIC_API_KEY=your_key firedrill agent ``` It uses the Claude Agent SDK with the developer's key. It may send selected repository content to Anthropic under Anthropic's applicable terms, but sends nothing to Firedrill Cloud. Repository discovery/search skips known secret and generated paths; file edits stay inside the selected repository; shell, generic web, Git publication, and subagents are disabled. A run defaults to 40 turns, a \$2 spend ceiling, and a 15-minute deadline. These controls reduce accidental exposure but do not sandbox ordinary repository code that the Agent authors and later executes. Review its diff as you would any coding-agent change. The compiler and drill runner—not the Agent's narrative—remain the authority. Published JSON Schemas are available from `@firedrill/compiler/schema/*` and `@firedrill/contracts/schema/*`. Source diagnostics include stable codes, file locations, paths, and corrective suggestions for machine use.