Skip to content

AI Models

Provon can import, run, fine-tune, and deploy open-weight models behind the same AI Gateway contract used for cloud providers.

View as Markdown Open the plain-text version of this page.

AI Models are available in the Node deployment. The Cloudflare runtime can route to remote providers but does not run local inference or training workloads.

Choose A Model Path#

Goal Start with
Use a managed or BYOK cloud model Model providers
Import and serve a local or private model Self-hosted model runtime
Train a checkpoint from a reviewed Dataset Model fine-tuning
Install and run a model directly from the command line Models CLI
Route traffic to a running model Gateway routing

Runtime Lifecycle#

flowchart LR
  catalog["Model catalog"] --> import["Import model"]
  import --> install["Install profile and engine"]
  install --> serve["Start inference"]
  serve --> gateway["Gateway self/ target"]
  dataset["Reviewed Dataset"] --> tune["Fine-tuning job"]
  tune --> checkpoint["Checkpoint"]
  checkpoint --> serve

The Node runtime owns model records, artifacts, engine installation, and lifecycle coordination. Inference runs in services/python-inference; training runs in services/python-fine-tuning.

Base Models And Checkpoints#

A base model is imported from a catalog source such as Hugging Face Hub. A runtime profile describes the artifact format and compatible inference engine.

A fine-tuning job references a base model and an immutable Dataset manifest. Successful jobs produce checkpoints with lineage back to both inputs. Deploying a checkpoint registers an inference profile and exposes it as a Gateway target.

Deployment Boundary#

Running an AI model is different from self-hosting the Provon platform:

  • Self-hosting Provon determines where the Workbench, APIs, telemetry, and jobs run.
  • Self-hosting a model provides one private upstream target for the AI Gateway.

The two can run on different hosts when network and credential boundaries permit it.

  1. Confirm host and engine support.
  2. Import one exact model and install one compatible profile.
  3. Serve it locally and send a direct provider-qualified request.
  4. Verify the resulting trace before adding routing policy.
  5. Fine-tune only from a reviewed Dataset.
  6. Deploy a checkpoint behind a stable logical model name.
  7. Compare new traces with the behavior that motivated the change.