Skip to content

Gateway Evidence And Trace Capture

Gateway Trace Capture turns a model call into the evidence used by Provon traces, conversations, diagnostic Rules, and Findings. It is separate from request success: inference can

View as Markdown Open the plain-text version of this page.
On this page Browse sections
text
Gateway request -> root operation span -> provider attempt spans
                                +-> usage, cost, errors, guardrail events
                                +-> conversation and agent context

What A Gateway Trace Contains#

The root span can include:

  • Gateway route, status, duration, and time to first chunk;
  • selected provider, Provider Key, credential fingerprint, and model mapping;
  • requested and response models;
  • input, output, cache, and reasoning tokens;
  • normalized input, output, and total cost;
  • OpenTelemetry GenAI request and response fields;
  • conversation, user, session, agent, and workflow context;
  • normalized provider error classification;
  • guardrail mode, decision, and evidence;
  • captured textual request and response bodies.

Each upstream attempt is a child span with:

  • provider and upstream URL;
  • Gateway and upstream model IDs;
  • target and credential identity;
  • fallback index and retry count;
  • status, duration, and normalized failure classification.

Provider secrets and raw credential values are never stored as trace attributes.

Add Diagnostic Context#

Send context as Gateway request headers:

bash
-H "x-otel-gen-ai-conversation-id: conversation-123" \
-H "x-otel-user-id: user-456" \
-H "x-otel-session-id: session-789" \
-H "x-otel-gen-ai-agent-id: support-agent" \
-H "x-otel-gen-ai-agent-name: Support Agent" \
-H "x-otel-gen-ai-agent-version: 2026-08-01"

The mapping is defined in Gateway API.

Use one stable conversation ID across all turns that belong to the same user goal. Provon uses that identity to reconstruct ordered conversation trajectories. Without it, each Gateway request may be diagnosed as an isolated trace.

Use pseudonymous user IDs unless the Workbench genuinely needs direct identity. Header values are telemetry and follow the project's retention boundary.

Capture Modes#

Configure Providers -> Request traces:

Mode Behavior
All Persist every Gateway request
Filtered Persist requests matching at least one configured filter
Off Do not persist Gateway traces

Default mode is All.

Filtered mode supports:

Filter Match
Failed responses Non-successful final response
Blocked requests Guardrail or Gateway-policy block recorded during execution
Recovered requests A later attempt recovered the request
Slow requests Duration at or above a threshold
High cost Calculated request cost at or above a USD threshold
High tokens Provider-reported token count at or above a threshold
Specific status codes Exact status codes or ranges
Provider error category Normalized categories such as rate limit or upstream timeout

When Filtered mode has no explicit filters, Provon defaults to failed, blocked, and recovered requests.

Choose A Capture Strategy#

Use All when:

  • validating a new integration;
  • reconstructing multi-turn conversations;
  • running diagnostic Rules over representative production behavior;
  • comparing healthy and unhealthy trajectories;
  • investigating intermittent routing or model changes.

Use Filtered when:

  • telemetry volume is more important than complete conversation reconstruction;
  • only failures, recoveries, slow calls, or high-cost calls need evidence;
  • the selected filters still preserve the context required by downstream Rules.

Use Off only when Gateway routing is needed without persisted request evidence.

Filtering individual calls can fragment a conversation. A diagnostic Rule cannot reason about turns that were never captured. For projects that rely on conversation Findings, prefer All and use project retention to control storage duration.

Request And Response Bodies#

For a trace selected by capture policy:

  • textual request bodies are attached to the root span;
  • textual response bodies are attached when capture completes within the runtime limit;
  • oversized or truncated response bodies are not attached as full payloads;
  • normalized GenAI fields may still be available independently of the attachment.

The default response capture limit is 2 MiB and can be changed by the deployment runtime.

Treat prompts, responses, tool arguments, and context headers as potentially sensitive. Use Gateway guardrails to monitor, redact, or block supported PII and secret patterns before they become provider traffic and trace evidence. Apply project telemetry retention to the resulting data.

Trace Semantics#

Gateway traces use OpenTelemetry-aligned GenAI attributes where a semantic convention exists:

Evidence Attributes
Operation gen_ai.operation.name
Provider gen_ai.provider.name
Models gen_ai.request.model, gen_ai.response.model
Usage gen_ai.usage.*
Cost gen_ai.cost.*.usd
Conversation gen_ai.conversation.id
Agent gen_ai.agent.id, gen_ai.agent.name, gen_ai.agent.version
HTTP http.request.method, http.response.status_code, http.route

Gateway-specific routing, target, credential, guardrail, and error facts use gateway.* and provon.guard.* attributes. They remain objective execution evidence; diagnostic labels are derived later by Rules and Findings.

Verify The Evidence Path#

After a request:

  1. Record the x-provon-request-id response header.
  2. Open Traces and locate the root span with the same trace ID.
  3. Confirm the conversation, user, and agent context.
  4. Expand provider attempt spans.
  5. Compare requested, Gateway, and upstream model IDs.
  6. Inspect usage, cost, error, retry, fallback, and guardrail attributes.
  7. Confirm the trace appears in the expected conversation.

If the request succeeded but no trace appears:

  • confirm Trace Capture is not Off;
  • in Filtered mode, confirm at least one filter matched;
  • confirm the request used the same project API key as the Workbench project;
  • confirm the deployment has a Gateway telemetry sink or trace store;
  • check the jobs or telemetry runtime if capture is asynchronous.

From Trace To Finding#

Gateway capture does not label every unsuccessful request as a Finding. Provon first:

  1. reconstructs the conversation trajectory;
  2. extracts objective facts from model and tool spans;
  3. runs deterministic diagnostic signals;
  4. applies model adjudication only where configured and needed;
  5. publishes a Finding only when Rule score and confidence thresholds pass.

This separation keeps Gateway execution facts distinct from diagnostic conclusions.

See Findings for the Rule, Run, Finding lifecycle.