How Functions Improve from Feedback

A Nyckel function doesn’t get better the way a typical ML model does — there’s no “retrain” button, no notebook, no model versioning to manage. It gets better because feedback flows back into the same store the model is trained on, and Nyckel retrains continuously in the background.

This page explains the mechanics so you know what to expect.

The feedback signal

The only signal that makes a function better is labeled examples — inputs paired with the correct output. The more you have, and the more accurately they reflect what your endpoint sees in production, the better the function gets.

There are three ways labeled examples are added:

  1. You confirm or correct predictions (for classification functions). Every confirmed prediction adds a sample with the predicted label; every correction adds a sample with the corrected label.
  2. You upload training data in the UI — from the console.
  3. You upload training samples via the API — useful when your labels live in another system, or when you want labeling to happen inside your own application. For more on using the API, see the Developer Platform.

For classification functions, the first source is the most powerful in the long run, because it’s an automatic byproduct of using the function — the model starts learning from the inputs your endpoint is actually being asked to classify in production. The more your application invokes the endpoint and the more you (or your team) review the results, the more training data you accumulate, with no separate labeling project.

The other two sources are great too, especially for seeding a function with known-good data up front, or when labels are produced somewhere else in your stack.

What happens after you add samples

Nyckel watches your samples store and retrains in the background when there’s a meaningful change. You don’t trigger this and you don’t wait for it. While retraining runs, the existing model keeps serving requests — the endpoint never goes down.

When a candidate model is ready, Nyckel benchmarks it against the current model on held-out data. If the candidate is better, the endpoint switches over automatically. If it isn’t, the current model stays. Models that don’t improve never reach production.

The benchmark uses held-out samples so improvements are measured on examples the candidate model hasn’t seen — that’s the only way to get an honest read on whether it’s actually better.

How fast does it improve?

A few rules of thumb:

What you should do

You don’t manage retraining, but you do influence how fast and how well a function improves. The two highest-leverage actions are:

The deeper guide on this is Review and Improve Predictions under Best Practices.

Box Detect functions improve the same way — by adding more annotated training images with bounding boxes — but the review workflow is different: you don’t review every invoke, you add training images with boxes when the function misses an instance or finds something that isn’t there.

Search functions don’t have a “right answer” to correct — they improve by curating the reference data. Adding, removing, or re-tagging items changes the search results immediately.

Next