Use cases

Evaluate and compare models

Score a detector against ground truth, and choose between several with the same evaluation.

Evaluation in OvalBee compares two datasets: the ground truth and the model’s predictions on the same images. That makes it the same whether the model came from a training run in this flow, from the pretrained catalog or from somewhere else entirely — and it’s why a prediction node always sits in front of an evaluation node.

Evaluate one detector

Labeled test set Predict Evaluate object detection

Nodes: a Dataset with ground truth, a prediction node such as Predict EdgeCrafter, and Evaluate object detection. Template: Detector evaluation.

1

Pick the test set

Use labeled images the model has never trained on, and pin the dataset version so the benchmark doesn’t move under you.

2

Predict on it

The prediction node runs the model over the test set and writes a predictions dataset.

3

Evaluate

Bind the evaluation node’s ground-truth input to the test set and its predictions input to the prediction node’s output, then run it.

The evaluation report opens with a verdict and the headline numbers — mAP, AP50, AP75, precision, recall, F1 — and then breaks them down: per class, by object size, the class pairs the model confuses, and example images of the worst mistakes. Use Split by tags to get the same breakdown per scene, camera or any other tag your images carry.

When the evaluated model is in your registry, the result is recorded on that model version and appears in the Benchmarks tab, so every model scored on this test set lands on one leaderboard.

Check the class names first. If the model says traffic light and your ground truth says traffic_light, the class isn’t matched and counts as missed. The report’s per-class table shows it immediately — a class with zero matches is usually a naming problem, not a blind model.

See it running: live example · case study: what a headline mAP hides

Compare several detectors

Labeled test set Predict EdgeCrafter Predict YOLO Predict DEIM Evaluate Evaluate Evaluate Compare object detectors

The Detector comparison template runs several detectors over the same labeled set, evaluates each with the same settings, and hands the evaluations to Compare object detectors. It takes between two and five evaluations, checks that they’re comparable — same ground truth, same classes — and picks a winner.

The comparison report puts the models side by side: overall metrics, per-class results, and the same images with each model’s predictions next to the ground truth. The winning model is also an output of the node, so a later step — a deployment, a prediction on new data — can reference whichever model won.

The same shape compares two architectures you trained yourself: two training branches on the same split, each followed by predict and evaluate, joined by one comparison.

See it running: live example · case studies on three pretrained detectors and two trained architectures

Make it yours

  • Re-score on every retrain. When the prediction node uses the model’s @latest, a new training run leads straight to a new evaluation, and the Benchmarks leaderboard grows by itself.
  • Measure pre-labeling. Evaluate prelabeling efficiency tells you how much of a model’s output labelers kept, fixed or deleted — the number that decides whether pre-labeling is worth it.
  • Mine the errors. Send the images the evaluation flagged into a labeling stage with Review suspected label errors. Some of the model’s “mistakes” are label mistakes.