Integrate SDKs With The Gateway

Complete the Gateway quickstart first so the provider, project key, and trace path are known to work.

Choose An Integration#

Provon does not require a Provon-specific inference SDK. The stable contract is the Gateway base URL, Bearer authentication, request model, and optional evidence-context headers.

Shared Configuration#

export PROVON_GATEWAY_URL="http://127.0.0.1:3000/gateway/v1"
export PROVON_API_KEY="your_project_api_key"

PROVON_API_KEY is a Provon project API key with gateway:invoke. Do not put an upstream provider key in the application.

Clients that read the standard OpenAI environment variables can often be redirected without code changes:

export OPENAI_BASE_URL="$PROVON_GATEWAY_URL"
export OPENAI_API_KEY="$PROVON_API_KEY"

Use a provider-qualified model such as openai/gpt-5-mini during migration. Introduce plain model names or provon/auto only after routing policy has been tested.

Direct HTTP#

Any HTTP client can call the Gateway:

const response = await fetch(`${process.env.PROVON_GATEWAY_URL}/chat/completions`, {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.PROVON_API_KEY}`,
    'Content-Type': 'application/json',
    'x-otel-gen-ai-conversation-id': 'conversation-123',
    'x-otel-gen-ai-agent-id': 'support-agent',
  },
  body: JSON.stringify({
    model: 'openai/gpt-5-mini',
    messages: [{ role: 'user', content: 'Summarize this ticket.' }],
  }),
});

if (!response.ok) throw new Error(await response.text());
console.log(await response.json());

Log the x-provon-request-id response header with the application request:

console.log(response.headers.get('x-provon-request-id'));

OpenAI Python SDK#

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["PROVON_API_KEY"],
    base_url=os.environ["PROVON_GATEWAY_URL"],
    default_headers={
        "x-otel-gen-ai-agent-id": "support-agent",
        "x-otel-gen-ai-agent-name": "Support Agent",
    },
)

response = client.chat.completions.create(
    model="openai/gpt-5-mini",
    messages=[{"role": "user", "content": "Summarize this ticket."}],
    extra_headers={
        "x-otel-gen-ai-conversation-id": "conversation-123",
        "x-otel-user-id": "user-456",
    },
)

print(response.choices[0].message.content)

The same client can use the Responses API when the selected target supports it:

response = client.responses.create(
    model="openai/gpt-5-mini",
    input="Summarize this ticket.",
    extra_headers={
        "x-otel-gen-ai-conversation-id": "conversation-123",
        "x-otel-user-id": "user-456",
    },
)

print(response.output_text)

OpenAI JavaScript SDK#

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.PROVON_API_KEY,
  baseURL: process.env.PROVON_GATEWAY_URL,
  defaultHeaders: {
    'x-otel-gen-ai-agent-id': 'support-agent',
    'x-otel-gen-ai-agent-name': 'Support Agent',
  },
});

const response = await client.chat.completions.create(
  {
    model: 'openai/gpt-5-mini',
    messages: [{ role: 'user', content: 'Summarize this ticket.' }],
  },
  {
    headers: {
      'x-otel-gen-ai-conversation-id': 'conversation-123',
      'x-otel-user-id': 'user-456',
    },
  },
);

console.log(response.choices[0]?.message.content);

Streaming uses the SDK's normal stream option. Provon preserves the provider-compatible stream and records final usage and attempt evidence when the provider exposes it.

Frameworks And OpenAI-Compatible Clients#

For LangChain, LlamaIndex, Vercel AI SDK, or another framework:

1. Select its OpenAI or OpenAI-compatible provider adapter.
2. Set the base URL to PROVON_GATEWAY_URL.
3. Set the API key to PROVON_API_KEY.
4. Use a model ID available in the Provon capability matrix.
5. Add the context headers through the framework's default-header or per-request option.

Framework configuration names change across releases. The invariant contract is:

Base URL:      https://<gateway-origin>/gateway/v1
Authorization: Bearer <Provon project API key>
Model:         <provider>/<model>, plain policy model, or provon/auto

Do not add a second /v1 when the configured Gateway URL already ends in /gateway/v1. The client must be able to send Authorization: Bearer <Provon project API key>. A client that hard-codes provider-specific authentication without a header override is not drop-in compatible.

Native Messages Requests#

Provon also exposes /gateway/v1/messages for provider-native Messages payloads. Use Bearer authentication even when the upstream provider normally uses x-api-key:

curl "$PROVON_GATEWAY_URL/messages" \
  -H "Authorization: Bearer $PROVON_API_KEY" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -H "x-otel-gen-ai-conversation-id: conversation-123" \
  -d '{
    "model": "anthropic/claude-haiku-4.5",
    "max_tokens": 256,
    "messages": [{"role": "user", "content": "Summarize this ticket."}]
  }'

The selected target must declare the messages endpoint and native passthrough capability. Query the capability matrix instead of assuming parity.

Diagnostic Context#

Stable context makes individual model calls useful to conversation diagnostics:

Use the same conversation ID for every turn in one user goal. Use non-secret, pseudonymous values; these headers become retained trace attributes.

Set request-scoped values such as conversation, session, and user IDs on each request. Do not place one user's identifiers in process-wide default headers. Agent name and version can be client-wide when one client instance represents one deployed agent.

Migration Checklist#

1. Connect one Provider Key and keep Request traces set to All.
2. Verify the application endpoint ends in /gateway/v1.
3. Replace the provider key with a Provon project key.
4. Start with a provider-qualified model.
5. Send stable conversation and agent identifiers.
6. Test streaming, tools, structured output, and multimodal inputs separately.
7. Verify the root and provider-attempt spans in Traces.
8. Add limits, guardrails, and recovery policy before increasing traffic.

For rollout and rollback gates, continue with Run The Gateway In Production.

Related Docs#

• Gateway quickstart
• Migration guide
• Gateway API
• Model providers
• Routing and reliability
• Gateway evidence
• Gateway troubleshooting