Skip to content

Model CLI

Use provon model to search the model catalog, install artifacts and inference engines, and run a model from the local runtime.

View as Markdown Open the plain-text version of this page.

These commands operate on local runtime state. They do not start the Provon Node server.

Inspect The Runtime#

bash
provon model status
provon model activity

Use PROVON_BLOBS_DIR or XDG_DATA_HOME to control where runtime state and model weights are stored. The model runtime state file is fixed at <data-dir>/data/model-runtime.json.

Find And Import A Model#

Search the installable catalog:

bash
provon model search --query gemma --format mlx

Import an exact Hugging Face repository into the local catalog:

bash
provon model import mlx-community/gemma-4-e2b-it-4bit

List installed models or the full catalog:

bash
provon model list
provon model list --all
provon model list --category text --format gguf

Inspect a model:

bash
provon model get <model-id>
provon model get <model-id> --profile <profile-id>

Install A Model#

bash
provon model install <model-id> --profile <profile-id>

The default --engine auto policy installs the compatible inference engine when needed. Use --engine none when the engine is managed outside the CLI.

Supported engines include llama.cpp, sglang, and vllm. Supported engines and profiles depend on the host and model format.

Serve A Model#

bash
provon model serve <model-id> --profile <profile-id> --foreground

Model serving currently stays attached to the CLI process. Stop the process to stop the service. The command reports the self/ Gateway model value when the service is ready.

Remove Artifacts#

bash
provon model remove <model-id> --profile <profile-id>

Environment Variables#

text
PROVON_BLOBS_DIR                  Root data directory; also resolves models/ and data/ defaults
PROVON_MODELS_DIR                 Override the directory for cached model weights
XDG_DATA_HOME                     Fallback data root when PROVON_BLOBS_DIR is unset
PROVON_PYTHON_INFERENCE_URL       Base URL of the python-inference service
PROVON_PYTHON_INFERENCE_API_KEY   API key for the python-inference service
PROVON_TRANSFORMERS_DEVICE          Transformers device override
PROVON_TRANSFORMERS_DTYPE          Transformers dtype override
PROVON_TRANSFORMERS_TRUST_REMOTE_CODE=1  Allow remote code in Transformers models