# Trace Diagnosis With CLI `provon diagnose` runs Provon's built-in diagnostic Rules against a trace source. It is designed for coding agents, cloud jobs, CI artifacts, exported production traces, and persisted agent transcripts. ```text trace source -> deterministic signals -> optional Agent adjudication -> Findings ``` The command does not start or configure a model runtime. The calling Agent supplies semantic judgment through a hash-bound structured response. The CLI can run on a developer machine, in CI, in a container, or as a cloud job. `source` describes the evidence input available to that process; it does not describe where diagnosis executes. ## Execution Boundary Source-based diagnosis: - does not require a Provon API key; - does not upload the source; - does not persist a Finding; - does not create a GitHub issue; - does not modify the repository or source files. It returns evidence-backed Finding objects to stdout. Server-side Rule Runs and the Findings inbox remain a separate remote workflow. ## Supported Inputs ```bash provon diagnose ``` `source` can be: | Source | Accepted evidence | | ------------------------- | ------------------------------------------------------- | | Transcript file | Claude Code, Codex, or Pi persisted session | | Transcript directory | Recursively discovered supported Agent sessions | | OTLP JSON file | Standard `resourceSpans` or trace request envelope | | OTLP protobuf file | Binary trace export request | | Provon span JSON or JSONL | Canonical normalized `OTelSpan` records | | Provon trace bundle | Directory containing `spans.jsonl` from `traces export` | | `-` | Standard input | Input formats: ```text auto transcript otlp-json otlp-protobuf provon-spans ``` Automatic detection is the default: ```bash provon diagnose ./trace-evidence provon diagnose ~/.claude/projects/example/session.jsonl provon diagnose ./otlp-traces.json provon diagnose ./traces.pb ``` Override detection when stdin or an ambiguous extension is involved: ```bash cat ./spans.jsonl | provon diagnose - --input-format provon-spans provon diagnose ./capture.bin --input-format otlp-protobuf ``` The same trace/span identity appearing more than once is deduplicated before diagnosis. ## Rule Selection By default, all five built-in Rules run: ```text runtime-reliability tool-correctness task-fulfillment answer-grounding conversation-health ``` Select one or more Rules by repeating `--rule`: ```bash provon diagnose ./trace-evidence \ --rule runtime-reliability \ --rule tool-correctness ``` Override publication thresholds for this execution: ```bash provon diagnose ./trace-evidence \ --min-score 0.5 \ --min-confidence 0.7 ``` Both values must be between 0 and 1. ## Deterministic-Only Mode ```bash provon diagnose ./trace-evidence --deterministic-only ``` This mode: - returns only Findings supported without semantic adjudication; - emits no adjudication request; - makes no model call; - is predictable for CI and high-volume screening; - intentionally has lower recall for grounding, fulfillment, and causal questions. Do not treat `--deterministic-only` as equivalent to complete diagnosis. ## Signal Adjudication Without `--deterministic-only`, the first pass can return: ```json { "ok": true, "data": { "status": "needs_adjudication", "findings": [], "adjudicationRequests": [ { "schemaVersion": "provon-signal-adjudication-v1", "diagnosisInputHash": "...", "instructions": "...", "evidenceBrief": "...", "signals": [], "evidence": [], "responseContract": {} } ] }, "warnings": [], "nextActions": [ { "label": "Finalize with current Agent adjudication", "command": "provon diagnose ./trace-evidence --adjudication-response " } ] } ``` The request contains the only candidates the Agent may confirm. The Agent must judge whether each candidate is: - a genuine fault; - the earliest uncorrected cause; - not corrected or compensated for by a later step. It must not invent a new candidate or use trace IDs, span IDs, conversation IDs, or timestamps as the `signalSignature`. ### Response Shape Create one response per `diagnosisInputHash`. The response can be a single object or a `responses` envelope: ```json { "responses": [ { "schemaVersion": "provon-signal-adjudication-v1", "diagnosisInputHash": "copy-from-the-request", "outcome": "failed", "confidence": 0.92, "signals": [ { "failureModeKind": "goal_not_completed", "signalSignature": "copy-a-candidate-signature", "severity": "high", "confidence": 0.9, "title": "Required verification was skipped", "summary": "The agent claimed completion without running the required verification.", "rootCauseSummary": "The first uncorrected fault is the omitted verification step.", "suggestedFix": "Run and report the required verification before claiming completion.", "failedActor": "implementation-agent", "causeStep": "Claimed completion before verification", "causeStepIndex": 8, "genuineFault": true, "earliestUncorrected": true, "notCorrected": true, "causalCheckReason": "No later step performed or compensated for the verification." } ] } ] } ``` Use values from the request's supplied signals. A response with an unknown failure mode or non-matching signal signature cannot confirm that candidate. Run the same diagnosis again with the response: ```bash provon diagnose ./trace-evidence \ --adjudication-response ./adjudication-response.json ``` The command recomputes the diagnosis input and requires the response hash to match. A changed trace, Rule, threshold, or evidence projection invalidates stale adjudication. ## Output Every successful CLI command returns an `{ ok, data, warnings, nextActions }` envelope. `data` contains: | Field | Meaning | | ---------------------- | -------------------------------------------------------------- | | `status` | `completed` or `needs_adjudication` | | `source` | Resolved path, detected format, and loaded files | | `summary` | Span, trace, conversation, Rule, Finding, and rejection counts | | `traceIds` | Distinct trace IDs in the input | | `findings` | Promoted Findings returned by this execution | | `adjudicationRequests` | Pending hash-bound requests | Parse and rejected-record messages appear in the envelope's `warnings`. When adjudication is required, `nextActions` includes the follow-up command. Use `--include-candidates` to inspect promoted, suppressed, pending, and adjudication-rejected candidates: ```bash provon diagnose ./trace-evidence --include-candidates ``` This is useful for tuning thresholds and understanding why a signal did not become a Finding. ## CI Pattern ```bash provon diagnose ./artifacts/agent-trace.jsonl \ --input-format provon-spans \ --deterministic-only \ --min-score 0.5 \ --min-confidence 0.7 ``` CI should consume the structured JSON result rather than scrape human text. Decide explicitly which severities or Finding counts fail the job; Provon does not impose a repository-specific gate. ## Evidence Requirements Diagnosis needs more than valid OTLP: - a stable conversation boundary; - the user's goal and constraints; - ordered model, tool, retrieval, and participant activity; - tool arguments, results, failures, and retries; - a terminal answer or explicit non-completion; - trace and span IDs for evidence references. When the input contains spans but no diagnosable trajectory, the command fails with `diagnose.no_trajectories`. ## Related Docs - [Tracing quickstart](./tracing-quickstart.md) - [Tracing CLI](./tracing-cli.md) - [Agent transcript sync](./agent-transcripts.md) - [Diagnosis-ready tracing](./tracing-best-practices.md) - [Findings](./findings.md)