Skip to content

Provon AI Gateway

Provon AI Gateway is an OpenAI-compatible model-call path that produces diagnostic evidence as it routes. One request can apply project policy, reach a cloud or self-hosted model,

View as Markdown Open the plain-text version of this page.
On this page Browse sections
text
agent -> Provon AI Gateway -> model provider
              |
              +-> trace evidence -> Rules -> Findings -> GitHub repair Issue

The Gateway is not required to use Provon. Choose it when Provon should sit on the model boundary. Keep calling providers directly and use OpenTelemetry when the application must own that boundary.

Start With One Request#

The first integration has two success conditions:

  1. The application receives a provider-compatible model response.
  2. The same request appears in Traces with its provider attempt and conversation context.

Follow the Gateway quickstart to run that complete check. Do not add logical model names, Auto Router, fallback, or enforcing guardrails until a provider-qualified request passes both conditions.

Choose The Right Capture Path#

Requirement Use
Provon should route the model request and enforce policy AI Gateway
The application must keep calling its provider directly OpenTelemetry ingest
The run already exists as a Claude Code, Codex, or Pi session Agent transcript sync
The agent needs tool, queue, handoff, or application spans too Gateway for model calls plus application OpenTelemetry

Gateway capture observes the model boundary. It cannot infer application operations that never cross that boundary. Combine it with application spans when a diagnosis needs the full agent trajectory.

What Changes In The Application#

Most OpenAI-compatible clients change three values:

text
Base URL: http://127.0.0.1:3000/gateway/v1
API key:  a Provon project API key
Model:    openai/gpt-5-mini

The application no longer receives the upstream provider credential. Provon stores that credential inside a project-scoped Provider Key. Optional x-otel-* headers add stable conversation, user, session, agent, and workflow identity to the resulting trace.

See Migrate to the Gateway for a staged application change and SDK integrations for client examples.

Core Objects#

Object Used by Responsibility
Project API key Agent application Authenticates one project and grants gateway:invoke
Provider Key Provon runtime Defines an upstream target, credential, priority, timeout, and model access
Model mapping Gateway router Maps a public Gateway model ID to an upstream model and its capabilities
Model Policy Gateway router Resolves a project-owned plain model name to ordered targets
Trace Capture Evidence path Decides which Gateway requests become persisted OpenTelemetry trace evidence

Keep these responsibilities separate. In particular, a pricing catalog entry does not make a model routable, and an upstream provider key is not a Gateway client credential.

Request Lifecycle#

A request passes through these stages:

  1. Resolve the project and verify gateway:invoke.
  2. Reserve request, token, and cost limits attached to the project API key.
  3. Evaluate request guardrails.
  4. Resolve the model against Provider Keys, mappings, policy, and required capabilities.
  5. Execute the eligible target with configured retry or fallback behavior.
  6. Evaluate response guardrails and return the provider-compatible response.
  7. Settle usage and persist the trace when selected by Trace Capture.

The final response shows what won. The trace shows every attempted target and the objective facts that led to the result.

Model Values#

Model value Routing intent
openai/gpt-5-mini Pin provider resolution to openai
support-model Apply a matching project Model Policy, then registry inference
provon/auto Select an eligible mapped target through Auto Router

Provon filters candidates by endpoint and requested features before scoring or recovery. Routing cannot make an unsupported feature available.

Documentation By Task#

Start And Migrate#

  • Gateway quickstart: run Provon, connect one target, send a request, and verify its trace.
  • Migrate to the Gateway: inventory compatibility, change credentials and base URL, add evidence context, canary, and roll back.
  • SDK integrations: configure HTTP, OpenAI Python, OpenAI JavaScript, native Messages, and compatible frameworks.

Configure And Control#

  • Model providers: choose managed, BYOK, custom, or self-hosted targets and configure model mappings.
  • Routing and reliability: use provider-qualified models, Model Policies, Auto Router, retries, fallback, circuit breaking, and affinity.
  • Gateway governance: attach request, token, and cost limits and apply request/response guardrails.

Diagnose And Operate#

  • Gateway evidence: preserve conversation context, choose Trace Capture, and understand span semantics.
  • Production guide: set rollout gates, security, privacy, and rollback.
  • Gateway troubleshooting: isolate authentication, model resolution, provider, policy, streaming, and missing-trace failures.

Reference#

  • Gateway API: base URLs, auth, context headers, discovery, endpoints, limits, errors, and request IDs.
  • Keep Request traces set to All during integration.
  • Use a stable x-otel-gen-ai-conversation-id for every multi-turn user goal.
  • Start with a provider-qualified model.
  • Query discovery instead of assuming provider or endpoint parity.
  • Add API-key limits before exposing a key to an uncontrolled workload.
  • Run new guardrails in Monitor before enforcement.