Skip to main content
Open the report link returned by the CLI or SDK, or select your project/environment in the portal’s Runs view. Batch and run reports read the same saved records as CLI and SDK inspection. You do not need a separate portal save step. The portal is for inspecting executions and evidence. Creating a simulation or executing your local application remains a coding-agent, CLI, SDK, or CI workflow.

​
Read four separate facts

Completed execution does not prove task success. No checks or rubric is a valid observation-only run, not an automatic quality score. An unavailable evaluator does not mean the agent failed.

​
Start with a batch

Set BATCH_ID to the ID returned by simulate, submit, or the batch list. Keep its original project/environment selected:
Status shows retained progress. Summary shows population counts, dimensional distributions, and available measurements. The run list gives the exact IDs for deeper inspection. Queued or claimed work is not proof that a target executed. For coding agents, compact inspection preserves counts and read handles without embedding large payloads:
Follow returned cursors with unchanged filters. A returned page is not the entire population.

​
Inspect an individual run

Set RUN_ID to a run ID from the batch’s run list:
The readable report is a summary. Use JSON for exact typed fields, including fractional AI scores. The operator view supplies bounded evidence references. A value read returns a JSON fragment with digest and continuation metadata; join all returned fragments before parsing a large value. To continue activity, use the report’s returned cursor. To narrow activity, use exact event kinds or producer prefixes from the report:
An empty filtered page does not establish that no other activity occurred.

​
Understand scores and constraints

Semantic rubrics assess meaning against authored anchors. Jev’s fractional 0–4 score is retained separately from ordinal levels and is not rounded into a CI verdict. Scores are not probabilities of correctness. Exact checks show satisfied, violated, or unknown. A hard constraint violation remains visible even when an AI score is high. Required missing assessments cannot be assumed to pass. Read assessment history without requesting new evaluation:
Reassessment is an explicit separate action on frozen evidence. It preserves the target’s execution and is not another trial.

​
Inspect evidence attribution

A report can include actual input/output, runner stdout/stderr, observations, Tool operations, state changes, scheduled work, clock actions, and available artifacts. Tool-free runs have no invented Tool activity. Distinguish runner-reported observations from service-attested effects. A model’s statement that it changed a record is not proof the change occurred. A screenshot observes pixels; it does not independently establish an external database mutation. Producer ordering and timestamps help interpret activity, but should not be treated as a universal causal clock across independent sources. Capture gaps and unknown provenance remain unknown.

​
Compare trials and configurations

Use the batch summary to see assessed, missing, unassessable, interrupted, and completed counts. Fractional score summaries remain within matching rubric/evaluator cohorts and separate from ordinal bins. Use saved-batch comparisons for paired scenario/configuration/repetition slots. Inspect mismatched definitions, evaluator changes, missing pairs, and divergent evidence. More trials and descriptive averages do not guarantee coverage or future reliability. Timing, token usage, and target cost are measurements, not task scores. Missing usage is not zero. Customer-reported target inference remains separate from evaluator usage.

​
Scope, availability, and recovery

A report belongs to an exact project/environment and assessment context. Broad account access does not make an ID from another selected scope valid. Links preserve their recorded scope; authenticate with an authorized identity rather than editing IDs until a report opens. A Tool’s expiry or reset does not erase already-retained history. Payload retention/removal can leave metadata without original input, output, or media. Inspect lifecycle state instead of treating unavailable content as an empty successful run:
Reads never execute your target or request assessment. Preserve private recovery journals after interruption. Resume the original operation using its returned recovery direction; a fresh simulate command creates new work. Cancellation, confirmed stopping, evidence availability, and cleanup remain distinct.