The Firedrill GitHub App and hosted CI are in private preview.
Repository connection
- Begin a GitHub App installation from your organization.
- Claim the verified installation after authorization.
- Choose one visible repository and branch.
- Discover
firedrill.json files at the exact branch head.
- Connect one configuration to a Firedrill project.
Each build attempt records the repository, configuration path, requested revision,
resolved commit, and diagnostics. Reconnect or update explicitly when the selected
source changes.
CI authentication
GitHub Actions exchanges its OIDC token for a short-lived Firedrill credential
bound to the verified workflow invocation and exact revision. You do not need to
store a long-lived Firedrill credential in the repository.
The exchange validates repository and workflow identity before admitting a suite.
The returned credential is limited to that CI work; it is not a browser session or
general project token.
Drill suites
A CI suite pins immutable revisions and expands the selected drills into attempts.
Your workflow claims the next interaction, runs the agent from the checked-out
commit, connects it to the supplied world binding, and submits the real result.
Firedrill retains each case’s assertions and evidence, then reports the suite
status and private evidence URL. A check should block only on configured behavioral
invariants; model quality metrics may require multiple trials and an explicit
threshold.
Browser checks
Hosted browser tests can be saved separately and selected for a pull request. A
browser CI configuration pins each saved test version and verifies the preview
application’s reported revision before and after execution. Change-aware selection
falls back safely when the changed-file set cannot be mapped.
Browser evidence and world-drill evidence remain distinct. Passing visible-page
assertions does not prove Tool state unless the drill also checks that state.
A successful repository build proves compilation of an exact revision. A
successful agent drill requires the customer-owned runner to execute the agent
and Firedrill to seal its actual world evidence.
