Review and Improve Predictions

This is the step that makes a function actually get better over time. Reviewing predictions converts each one into a training sample. As samples accumulate, Nyckel trains a private model on your data in the background and swaps it in automatically when it outperforms the current model.

You don’t trigger any of this. You just review.

Open the review queue

In the console, go to your function’s Review tab. Predictions made by the endpoint — from the console, from your application, from anywhere — appear here. While a function is new that’s every prediction; at production volume, invoke capture queues the most informative ones rather than everything.

For each prediction you’ll see:

• The input
• The predicted label and confidence
• Buttons to confirm (the prediction is right) or correct (change the label to the right one)

Confirm or correct

Click through the queue. For each prediction:

• If the predicted label is correct, confirm it. The prediction becomes a sample with the predicted label.
• If the predicted label is wrong, correct it to the right label. The prediction becomes a sample with the corrected label.

Corrections are a much stronger training signal than confirms, because they tell the model exactly where its current decision boundary is wrong. If you only have time to review some predictions, prioritize the ones with low confidence — they’re the ones the model is most likely to have gotten wrong.

What happens behind the scenes

Once you have 2 confirmed examples per label, Nyckel automatically begins building a private model trained on your data. You’ll see a training indicator in the console. The zero-shot classifier continues serving predictions while the private model trains in the background — the endpoint stays live the whole time.

When the private model is ready, Nyckel benchmarks it against the zero-shot baseline on held-out samples. If it performs better, the endpoint automatically switches to it. You don’t trigger this manually.

The same thing happens every time you’ve added enough new samples to potentially improve the current model. Better models replace worse ones; worse models never ship. There’s no manual retraining and no deploy step.

How much to review, and how often

A few practical guidelines:

• Aim for 5–10 confirmed samples per label to get a first private model with meaningfully better accuracy than the zero-shot baseline.
• Review continuously, not in batches. A steady drip of 10–20 reviewed predictions per day usually beats one big labeling sprint, because the model improves between batches and you see the effect.
• Prioritize low confidence and corrections. These are the predictions that, once labeled, move the model the most.

Bulk-add samples (optional)

If you already have a CSV or JSONL of labeled data, you can skip ahead. Open the Import tab and upload the file — each row an input and its correct label. This is the fastest way to bootstrap if you’ve already done a labeling project.

Annotate from the API

The Review tab is all most teams need — Nyckel picks which predictions are worth your time. But if your application learns true labels on its own (“this is wrong” buttons, support agent corrections, etc.), you can feed those back directly: submit the input together with its correct label as a new sample. Reuse the externalId you sent when you invoked, so feedback for the same record isn’t double-counted:

POST https://www.nyckel.com/v1/functions/{functionId}/samples
Authorization: Bearer {your_access_token}
Content-Type: application/json

{
  "data": "My card was charged twice for order 4471",
  "externalId": "ticket-4471",
  "annotation": { "labelName": "Refund request" }
}

The deeper guide on this is Build a Feedback Loop under Developer Platform, and the API Quickstart runs the whole loop in curl.

What you just did

Next

Next Steps — what to do once your function is live in production.