Skip to content

Tracing Quickstart

Send one diagnosis-ready agent trace to Provon over standard OTLP/HTTP, inspect it, and run the built-in diagnostic Rules with the CLI.

View as Markdown Open the plain-text version of this page.
On this page Browse sections

This guide uses a direct OTLP JSON request so the transport and evidence are visible. For application SDK setup, continue with OpenTelemetry setup. To capture model calls through Provon instead, use the Gateway quickstart.

Prerequisites#

  • A running Provon deployment.
  • A project API key with telemetry:ingest and telemetry:read.
  • curl, openssl, and a POSIX-compatible shell.
  • The provon CLI for the inspect and diagnose steps.

Configure the deployment and project:

bash
export PROVON_BASE_URL="http://127.0.0.1:3000"
export PROVON_API_KEY="your_project_api_key"
export PROVON_PROJECT_ID="your_project_id"

1. Send A Trace#

The example creates one bounded agent run with a child tool call. It uses stable operation names, explicit participant identity, readable input and output, and a conversation ID.

bash
TRACE_ID="$(openssl rand -hex 16)"
CONVERSATION_ID="quickstart-$TRACE_ID"
ROOT_SPAN_ID="$(openssl rand -hex 8)"
TOOL_SPAN_ID="$(openssl rand -hex 8)"
START_NS="$(($(date +%s) * 1000000000))"

curl "$PROVON_BASE_URL/v1/traces" \
  -H "Authorization: Bearer $PROVON_API_KEY" \
  -H "Content-Type: application/json" \
  --data-binary @- <<JSON
{
  "resourceSpans": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": { "stringValue": "quickstart-support-agent" }
          },
          {
            "key": "deployment.environment.name",
            "value": { "stringValue": "development" }
          }
        ]
      },
      "scopeSpans": [
        {
          "scope": { "name": "provon.quickstart" },
          "spans": [
            {
              "traceId": "$TRACE_ID",
              "spanId": "$ROOT_SPAN_ID",
              "name": "invoke-support-agent",
              "kind": 1,
              "startTimeUnixNano": "$START_NS",
              "endTimeUnixNano": "$((START_NS + 3000000))",
              "status": { "code": 1 },
              "attributes": [
                {
                  "key": "gen_ai.operation.name",
                  "value": { "stringValue": "invoke_agent" }
                },
                {
                  "key": "gen_ai.conversation.id",
                  "value": { "stringValue": "$CONVERSATION_ID" }
                },
                {
                  "key": "gen_ai.agent.id",
                  "value": { "stringValue": "support-agent" }
                },
                {
                  "key": "gen_ai.input.messages",
                  "value": {
                    "stringValue": "[{\"role\":\"user\",\"content\":\"Check whether order 123 can be refunded.\"}]"
                  }
                },
                {
                  "key": "gen_ai.output.messages",
                  "value": {
                    "stringValue": "[{\"role\":\"assistant\",\"content\":\"Order 123 is eligible for refund review.\"}]"
                  }
                }
              ]
            },
            {
              "traceId": "$TRACE_ID",
              "spanId": "$TOOL_SPAN_ID",
              "parentSpanId": "$ROOT_SPAN_ID",
              "name": "lookup-refund-policy",
              "kind": 1,
              "startTimeUnixNano": "$((START_NS + 1000000))",
              "endTimeUnixNano": "$((START_NS + 2000000))",
              "status": { "code": 1 },
              "attributes": [
                {
                  "key": "gen_ai.operation.name",
                  "value": { "stringValue": "execute_tool" }
                },
                {
                  "key": "gen_ai.conversation.id",
                  "value": { "stringValue": "$CONVERSATION_ID" }
                },
                {
                  "key": "gen_ai.agent.id",
                  "value": { "stringValue": "support-agent" }
                },
                {
                  "key": "gen_ai.tool.name",
                  "value": { "stringValue": "lookup_order" }
                },
                {
                  "key": "gen_ai.tool.call.id",
                  "value": { "stringValue": "call-quickstart-1" }
                },
                {
                  "key": "gen_ai.tool.call.arguments",
                  "value": { "stringValue": "{\"order_id\":\"123\"}" }
                },
                {
                  "key": "gen_ai.tool.call.result",
                  "value": {
                    "stringValue": "{\"status\":\"eligible_for_review\"}"
                  }
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}
JSON

printf 'trace id: %s\n' "$TRACE_ID"
printf 'conversation id: %s\n' "$CONVERSATION_ID"

A successful request returns asynchronous acceptance:

json
{
  "partialSuccess": {},
  "queued": true,
  "jobId": "ing_..."
}

This confirms staging and queue admission, not query readiness. See the OTLP/HTTP API for the complete delivery contract.

2. Verify The Evidence#

Open Traces and search for the printed trace ID. The trace should contain:

text
invoke-support-agent
`-- lookup-refund-policy

Verify:

  • service.name is quickstart-support-agent;
  • the tool span is a child of the agent span;
  • both spans use the printed conversation ID;
  • the root input and output are readable;
  • the tool arguments and result are retained.

If the trace is not visible yet, wait for background ingest and summary materialization. Use Tracing troubleshooting when an accepted payload never appears.

3. Inspect It From The CLI#

Configure CLI authentication:

bash
provon auth token set \
  --api-key "$PROVON_API_KEY" \
  --project "$PROVON_PROJECT_ID"

Search and summarize the trace:

bash
provon traces search "$TRACE_ID" --search-type trace-id --range 1h
provon traces summarize "$TRACE_ID"

traces summarize reports telemetry facts such as duration, spans, errors, tokens, cost, and slow operations. It does not run diagnostic Rules or create Findings.

4. Diagnose The Same Evidence With The CLI#

Export a trace bundle, then diagnose it without uploading or persisting a Finding:

bash
provon traces export "$TRACE_ID" --out ./quickstart-trace
provon diagnose ./quickstart-trace --deterministic-only

The healthy example should normally produce no Finding. That is a successful result: traces are evidence, while Findings are supported problems.

Remove --deterministic-only when the calling Agent can perform the hash-bound adjudication round trip. See Trace diagnosis with CLI.

Next Steps#