Build a Feedback Loop

A feedback loop connects your application’s outcomes back to Nyckel. When your workflow learns the true label for an input, you send it back as a labeled sample. Nyckel uses those samples to retrain and improve the model.

This is the hands-on path — in effect, you manage your own active learning, deciding which invokes are worth feeding back. If you’d rather not build anything, Nyckel already runs a managed version of this loop: invoke capture selects informative invokes and queues them in the console’s Review tab for annotation. Build your own loop when your application learns true labels on its own — an agent resolves a ticket, a user flags a wrong answer — so model errors flow into training data without waiting for a human reviewer in the console.

Why feedback loops matter

Without feedback, the model stays static. With feedback:

• Accuracy improves as the model learns from real-world corrections.
• Edge cases that were hard at first become easy over time.
• You have a clear record of model performance in production.

What happens to an invoke

One fact shapes the whole design: an invoke is not stored as a training sample. Nyckel answers it, counts it, and — for the most informative ones — queues it for console review via invoke capture. Capture is selective, so your application cannot assume any given invoke is waiting on Nyckel’s side to be annotated later.

That is why a feedback loop submits the labeled sample itself: your application already holds the input and, once the outcome is known, the correct label. Sending both creates the training sample in one call.

Step-by-step: Implementing a feedback loop

1. Send an externalId with each invoke

Pass your own identifier for the record — a ticket number, an order id, a row primary key. Nyckel echoes it back, attaches it to the capture if the invoke is selected for review, and uses it to deduplicate the feedback you submit in step 3.

POST https://www.nyckel.com/v1/functions/{functionId}/invoke?externalId=ticket-4471

{
  "labelName": "Billing",
  "labelId": "label_abc123",
  "confidence": 0.74,
  "externalId": "ticket-4471"
}

Because the id is one you already have, there is nothing new to store — your existing record is the join key.

2. Surface the prediction to users (optional)

If your application shows the prediction to a user (for example, an auto-routed support ticket), give them a way to correct it.

3. Submit the input with its correct label

When you know the correct answer, create a labeled sample from the input, reusing the record’s externalId:

POST https://www.nyckel.com/v1/functions/{functionId}/samples
Authorization: Bearer {token}
Content-Type: application/json

{
  "data": "<the same input you invoked with>",
  "externalId": "ticket-4471",
  "annotation": { "labelName": "Technical" }
}

This tells Nyckel the correct label for that input, and the sample joins the training data immediately.

4. Handle the already-exists case

If a sample with the same content or the same externalId already exists — you submitted feedback for this record before, or the invoke was captured and annotated in the console — the POST returns 409 Conflict with the existing sample’s id. Update that sample’s annotation instead:

PUT https://www.nyckel.com/v1/functions/{functionId}/samples/{sampleId}/annotation
Authorization: Bearer {token}
Content-Type: application/json

{ "labelName": "Technical" }

Note the verb is PUT — an annotation is set or replaced, not appended.

5. Automate or batch annotation where possible

You do not need users to do this manually. If your workflow has a downstream step that determines the correct label (for example, a human agent resolves a ticket and tags it), that resolution can trigger the annotation automatically.

Pulling the capture queue into your own tools

The feedback loop above covers inputs whose true label your workflow learns on its own. For everything else, invoke capture maintains a queue of the invokes most worth a human’s attention. If your team reviews in its own labeling tool rather than the Nyckel console, pull that queue and route it through your system (note the v0.9 prefix):

GET https://www.nyckel.com/v0.9/functions/{functionId}/captures?batchSize=20
Authorization: Bearer {token}

Each capture includes the input, the predicted label and confidence, the reason it was captured, and the externalId from the invoke. Once your reviewer picks the correct label, annotate the capture in place:

PUT https://www.nyckel.com/v0.9/functions/{functionId}/captures/{captureId}/annotation
Authorization: Bearer {token}
Content-Type: application/json

{ "labelId": "label_abc123" }

Annotating a capture converts it into a labeled training sample and clears it from the Review queue. To dismiss a capture without annotating it, DELETE it instead.

When to submit annotations

• Always: When a user or agent explicitly corrects a prediction.
• Selectively: When a prediction was confirmed as correct (these are also useful training signals).
• Avoid: Submitting annotations you are not confident about — noisy annotations degrade accuracy.