# Dataset Troubleshooting

## No Examples Were Collected

You created a Dataset, sent traces, but the Examples list stays empty.

### 1. Wait For Ingestion And Collection

Automatic collection is asynchronous. After a conversation is inactive for roughly ten minutes, the
runtime evaluates eligible evidence. Check again after the delay plus a few minutes for job
processing.

### 2. Verify Collection Is Enabled

A Dataset only collects when `collectionEnabled` is `true` and it has a `collectionPolicy`.

```bash
provon datasets get <dataset-id>
```

If collection is disabled, re-enable it:

```bash
provon datasets update <dataset-id> --collection-enabled true
```

### 3. Check The Objective Requirements

| Objective                      | Required conversation state                                                         |
| ------------------------------ | ----------------------------------------------------------------------------------- |
| `preserve_successful_behavior` | Conversation must be `completed`.                                                   |
| `correct_failed_conversations` | Conversation must have a diagnosed failure and a corrected response.                |
| `learn_from_user_corrections`  | Conversation must contain a rejected answer and a verified correction.              |
| `teach_project_knowledge`      | Active Knowledge Goals must exist and the conversation must contain matching items. |

Not every trace matches every objective. A completed, uneventful conversation will not produce
Examples for `correct_failed_conversations`.

### 4. Confirm The Trace Was Ingested

Look up the conversation or trace in the Workbench under **Tracing**. If the trace is missing,
check:

- the OTLP endpoint or Gateway configuration;
- whether the project id matches the Dataset's project;
- trace attribute requirements for the producer you are using.

### 5. Verify The Schema

If the Dataset schema does not match the producer output, the Example will be discarded. Currently
only `chat` schema v1 is supported.

## Export Or Fine-Tuning Fails

### `input must be chat`

Formats such as `openai_chat_sft` and `preference_chat_dpo` require `payload.input.type` to be
`chat`. Convert the Example or choose a different format.

### `expectedOutput must be an assistant chat_message`

SFT and DPO formats need an assistant message as the expected output. If your Example stores text or
JSON output, either change the Example or use `chat_prompt_completion` / `alpaca_instruction` /
`evaluation_jsonl`.

### `rejectedOutput must be an assistant chat_message`

`preference_chat_dpo` requires a rejected assistant message in addition to the chosen one. Add a
valid `rejectedOutput` or use a non-preference format.

### `expectedOutput and rejectedOutput must differ`

For preference formats, the chosen and rejected messages cannot be identical. Verify the Example and
update one of the outputs.

### `expectedOutput is required`

`chat_prompt_completion` and `alpaca_instruction` require an expected output. Add one or exclude the
Example from the export.

## API Errors

### `Legacy top-level input/expectedOutput/metadata fields are not accepted`

Example create and update requests must nest `input`, `expectedOutput`, and `rejectedOutput` under
`payload`. Do not send them at the top level of the request body.

### `source.conversationId is required for conversation sources`

A `conversation` source must include `conversationId` and non-empty `traceIds`.

### `source.externalRef is required for import sources`

An `import` source must include an `externalRef` so the Example remains auditable.

### `tags must be an array of non-empty strings`

Tags cannot be empty strings and cannot contain duplicates. Remove empty entries before sending.

## Fine-Tuning Job Does Not See New Examples

Fine-tuning jobs materialize an immutable manifest at creation time. Adding or editing Examples after
the job starts does not affect it. Cancel the job and create a new one, or wait for it to finish and
start a follow-up job with the updated Dataset.

## Example Count Looks Wrong

The `exampleCount` field is updated in the same transaction as Example create and delete operations.
If it seems stale:

- Refresh the Dataset detail page or re-fetch via the API.
- Check whether automatic collection produced idempotent updates that did not change the count.
- Verify that no failed API requests left partial state.

## Still Stuck?

- Review the [Dataset quickstart](./quickstart.md) for a working end-to-end example.
- Read the [export format reference](./formats.md) to confirm your Examples match the chosen format.
- Inspect the conversation in **Tracing** to confirm it satisfies the objective requirements.