Build a Feedback Loop
A feedback loop connects your application’s outcomes back to Nyckel. When a prediction is confirmed or corrected, you send that information as an annotation. Nyckel uses annotations to retrain and improve the model.
Why feedback loops matter
Without feedback, the model stays static. With feedback:
- Accuracy improves as the model learns from real-world corrections.
- Edge cases that were hard at first become easy over time.
- You have a clear record of model performance in production.
Step-by-step: Implementing a feedback loop
1. Store the sampleId from each invoke response
When you call /invoke, the response includes a sampleId. Store this alongside the prediction in your database.
{
"sampleId": "smp_abc123",
"labelName": "Billing",
"confidence": 0.74
}
2. Surface the prediction to users (optional)
If your application shows the prediction to a user (for example, an auto-tagged support ticket), give them a way to correct it.
3. Submit the annotation when you know the correct answer
POST https://www.nyckel.com/v1/functions/{functionId}/samples/{sampleId}/annotation
Authorization: Bearer {token}
Content-Type: application/json
{ "labelName": "Technical" }
This tells Nyckel the correct label for that input.
4. Automate or batch annotation where possible
You do not need users to do this manually. If your workflow has a downstream step that determines the correct label (for example, a human agent resolves a ticket and tags it), that resolution can trigger the annotation automatically.
When to submit annotations
- Always: When a user or agent explicitly corrects a prediction.
- Selectively: When a prediction was confirmed as correct (these are also useful training signals).
- Avoid: Submitting annotations you are not confident about — noisy annotations degrade accuracy.