Skip to main content
Firedrill Cloud can run a selected drill suite for a pull request and put the result where the team already reviews code. The repository still owns the agent command. Firedrill provides the isolated world, evaluates its recorded consequences, and retains the evidence.
Pull requests are one trigger. The same suite can run after a push, on a schedule, or through a manual workflow dispatch.

​
What a reviewer receives

A completed pull-request suite publishes:
  • a Firedrill check on the exact candidate commit;
  • a pull-request summary with passed, failed, and inconclusive drills;
  • a base-versus-head comparison when both revisions were evaluated;
  • a private link to assertions, Tool calls, changed state, logs, and admitted captures; and
  • a terminal success, failure, cancellation, or infrastructure outcome.
Firedrill does not turn an incomplete run into a pass. A target crash, expired interaction, world failure, or unevaluated assertion remains distinguishable from an agent behavior failure.

​
Configure the workflow

1

Connect the repository

Install the Firedrill GitHub App for the repository and connect it to the current project. The installation grants repository identity and permission to publish the final check and marker-owned pull-request summary.
2

Choose the environment and suite

Select the hosted environment and the focused suite that should run for code review. Keep longer or higher-trial simulations on push, schedule, or manual triggers when they would slow normal review.
3

Declare the agent command

Use the same non-interactive command that runs the agent locally. JavaScript, TypeScript, and Python runners use the same process boundary.
4

Commit the generated workflow

Add the generated workflow under .github/workflows/. It checks out the exact revision and exchanges GitHub OIDC identity for short-lived Firedrill access. No long-lived Firedrill API key is stored in the repository.
5

Decide whether it blocks merging

GitHub branch protection decides whether the check is required. Firedrill can be informational while the suite is being introduced.

​
Agent command contract

The Action sends one invocation JSON value to the configured command on standard input. The command writes one result JSON value to standard output and sends human-readable logs to standard error.
The agent process receives only the active world binding and host values that the workflow explicitly admits. It does not receive the GitHub token, OIDC token, Firedrill control credentials, or another tenant’s state.

​
Base and head comparison

For a pull request, Firedrill binds each run to an immutable Git revision. The base and candidate results are compared only after both are sealed. The comparison shows factual changes in assertions, evidence, and world state; it does not infer that every difference is an improvement or regression. Examples of useful merge-gating assertions include:
  • no duplicate external action after a lost response;
  • no write outside the active actor’s permissions;
  • protected state remains unchanged after a failed operation; and
  • required follow-up work occurs before the agent reports completion.
Use repeated trials for stochastic quality questions. Do not turn a small-sample pass rate into a deterministic merge gate.

​
Evidence and captures

Logs, screenshots, recordings, and files follow the suite’s capture policy: off, always, or retain-on-failure. The runner must provide those artifacts or register a capture driver; Firedrill does not silently record the developer’s machine. Evidence links remain private. A signed-in project member can inspect the result, and an unauthorized recipient can request access to that exact suite without gaining project access.

Run drills in any CI provider

Generate a local GitHub workflow or use the same exit-code and report contract in another CI system.