Model improvement

The mistake it made once.

Recurring failures become reviewed training examples, a fine-tuned checkpoint, and a deployed target — then new traces prove whether the behavior actually changed.

Explore the improvement loop Fine-tuning guide
tool-corrections-v3 checkpoint serving

Dataset

128

Manifest

locked

Fine-tune

done

Serving

self/v3

Malformed tool calls

Base model 76%
After fine-tune 18%

Measured on new traces, not on the training set.

A reproducible loop

From evidence to a measured checkpoint.

  1. Collect

    01

    Trace-backed examples enter a dataset with one objective.

    $ provon datasets create "tool-corrections"

  2. Review

    02

    Keep coherent examples; follow each back to its trace.

    $ provon datasets examples list <dataset-id>

  3. Train

    03

    A frozen manifest feeds an SFT or DPO job.

    $ curl -X POST $API/fine-tuning/jobs

  4. Deploy

    04

    The checkpoint serves as a self/ Gateway target.

    $ curl -X POST $API/checkpoints/<id>/deploy

Collection objectives

Decide what the next model should learn.

01

Preserve successful behavior

02

Correct failed conversations

03

Learn from user corrections

04

Teach project knowledge

Operational safeguards

Automation with boundaries.

Manifest locked

Later dataset edits cannot change a queued job.

Full lineage

Base model, job, and checkpoint stay traceable.

Threshold gated

Training queues only after minimum count and cooldown.

Stays inside

Training and serving run on infrastructure you control.

Build the loop from real evidence.

Create a focused dataset and measure the next checkpoint.

Read the guide