> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.ovalbee.com/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.ovalbee.com/_mcp/server. # Review suspected label errors > Collects the images an object detection evaluation report flagged as probable ground-truth label mistakes into a dataset, ready to send for re-labeling. ## Overview Collects the images an object detection evaluation report flagged as probable ground-truth label mistakes into a dataset a labeler can open. When a model confidently disagrees with a label, one of the two is wrong, and it is not always the model. [Evaluate object detection](/node-library/nodes/evaluate-evaluate-object-detection) already records those disagreements in its report, and this node reads them from there. It selects images, not objects: a reviewer opens an image once however many things were flagged on it, so an image carrying five loose boxes is ranked by its most confident one rather than by the count. Each rerun writes a new version of the node's own hidden `LAST_RUN` dataset holding that run's selection, and the badge shows how many images it selected and opens it. Set a destination dataset to accumulate across runs as well. ```mermaid flowchart LR n1["Evaluate object detection"] --> n2["Review suspected label errors"] --> n3["Labeling and review"] ``` ## Prerequisites | Requirement | What you need | | ------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **An evaluation report** | An object detection evaluation report from the Evaluate object detection node. Reports produced before this node existed do not record suspected label errors, so run the evaluation again for those. | | **The dataset the report evaluated** | Images and annotations are copied from there, so point **Source dataset** at the same ground-truth dataset the report was built from. | ## How it works Reads the suspected label errors from the report — the candidates the evaluation already ranked, not a fresh analysis. Keeps the issue types you asked for, drops anything the model was less sure about than the minimum confidence, and reduces the rest to one entry per image. Copies up to **Sample size** of the most confident images, with their annotations, into the node's own hidden dataset, and into your destination dataset if you set one. ## FAQ #### What does each issue type mean? | Issue type | What the model saw | | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Missing labels** | A confident prediction where the ground truth has nothing at all. Either an object nobody labeled, or a false positive. | | **Wrong classes** | A well-placed box whose class disagrees with the label on it. | | **Loose boxes** | A prediction that overlaps a labeled object of the same class only loosely. Noisier than the other two, because a merely imprecise model produces these as well. | #### Does a selected image mean its label is wrong? No. These are candidates, not verdicts. Every one of them is equally consistent with the model being wrong, which is why the node hands them to a reviewer instead of editing anything. #### How far can a lower Minimum confidence widen the selection? The report applies its own confidence floor while collecting candidates, and caps how many it records per issue type. Lowering **Minimum confidence** here therefore cannot widen the selection past what the report already recorded. #### Why did the run fail with no issue type selected? At least one of **Missing labels**, **Wrong classes**, or **Loose boxes** must be on. #### What happens when nothing in the report matches my settings? The run succeeds with a count of zero, and `LAST_RUN` is a new, empty version. Include more issue types or lower **Minimum confidence**, within what the report recorded. #### What if Source dataset is not the dataset the report evaluated? The flagged images are still added, but they arrive without annotations, because only annotations found in **Source dataset** are copied. ## Inputs **`Evaluation report`** `report` — required Object detection evaluation report the suspected label errors are read from. Generic reports are not supported. Key: `REPORT_ID`. --- **`Source dataset`** `dataset` — required Dataset the report evaluated. The selected images and their annotations are copied from here. Key: `SRC_DATASET`. --- **`To dataset`** `dataset` Dataset that also receives the selection and keeps collecting it across runs. Leave empty to write only the node's hidden last-run dataset. Key: `DST_DATASET`. --- **`Missing labels`** `boolean` — required, default: true Include images where the model confidently found an object nobody labeled. Key: `INCLUDE_MISSING_LABEL`. --- **`Wrong classes`** `boolean` — required, default: true Include images where the model disagrees with the class on a well-placed box. Key: `INCLUDE_WRONG_CLASS`. --- **`Loose boxes`** `boolean` — required, default: true Include images where the model's box overlaps the labeled one only loosely. Noisier than the other two. Key: `INCLUDE_LOOSE_BOX`. --- **`Sample size`** `integer` — required, default: 100 Maximum number of images to select. Fewer are returned when the report flagged fewer. Minimum: `1`. Maximum: `10000`. Step: `1`. Key: `SAMPLE_SIZE`. --- **`Minimum confidence`** `number` — required, default: 0.5 Ignore flagged objects the model was less sure about than this. The report already applies its own floor, so lowering this cannot widen the selection past what it recorded. Minimum: `0`. Maximum: `1`. Step: `0.01`. Key: `MIN_CONFIDENCE`. --- ## Outputs **`Dataset`** `dataset` Key: `DATASET`. --- **`Last run`** `dataset` Hidden dataset holding this run's selection. A rerun writes a new version of it. Shown as an artifact. Key: `LAST_RUN`. --- **`Selected images`** `string` Number of image assets attached to the selection. Shown on the node as a badge. Selecting the badge opens `LAST_RUN`. Key: `SELECTED_IMAGES`. --- ## JSON config Machine-readable node interface for automation and advanced usage. #### Show JSON ```json { "name": "Review suspected label errors", "description": "Collects the images an object detection evaluation report flagged as probable ground-truth label mistakes into a dataset, ready to send for re-labeling.", "category": "Sample", "namespace": "ovalbee", "templateKey": "sample/review_suspected_label_errors", "version": "v1", "inputs": [ { "key": "REPORT_ID", "label": "Evaluation report", "type": "report", "kind": "object_detection_evaluation_report_v1", "description": "Object detection evaluation report the suspected label errors are read from. Generic reports are not supported.", "required": true, "default": null, "visibleWhen": null }, { "key": "SRC_DATASET", "label": "Source dataset", "type": "dataset", "description": "Dataset the report evaluated. The selected images and their annotations are copied from here.", "required": true, "default": null, "visibleWhen": null }, { "key": "DST_DATASET", "label": "To dataset", "type": "dataset", "description": "Dataset that also receives the selection and keeps collecting it across runs. Leave empty to write only the node's hidden last-run dataset.", "required": false, "default": null, "visibleWhen": null }, { "key": "INCLUDE_MISSING_LABEL", "label": "Missing labels", "type": "boolean", "description": "Include images where the model confidently found an object nobody labeled.", "required": true, "default": true, "visibleWhen": null }, { "key": "INCLUDE_WRONG_CLASS", "label": "Wrong classes", "type": "boolean", "description": "Include images where the model disagrees with the class on a well-placed box.", "required": true, "default": true, "visibleWhen": null }, { "key": "INCLUDE_LOOSE_BOX", "label": "Loose boxes", "type": "boolean", "description": "Include images where the model's box overlaps the labeled one only loosely. Noisier than the other two.", "required": true, "default": true, "visibleWhen": null }, { "key": "SAMPLE_SIZE", "label": "Sample size", "type": "number", "description": "Maximum number of images to select. Fewer are returned when the report flagged fewer.", "required": true, "default": 100, "visibleWhen": null, "options": { "type": "integer", "min": 1, "max": 10000, "step": 1, "slider": false } }, { "key": "MIN_CONFIDENCE", "label": "Minimum confidence", "type": "number", "description": "Ignore flagged objects the model was less sure about than this. The report already applies its own floor, so lowering this cannot widen the selection past what it recorded.", "required": true, "default": 0.5, "visibleWhen": null, "options": { "type": "float", "min": 0, "max": 1, "step": 0.01 } } ], "outputs": [ { "key": "DATASET", "label": "Dataset", "type": "dataset", "description": "", "artifact": false, "badge": false, "badgeOpens": null, "hidden": false }, { "key": "LAST_RUN", "label": "Last run", "type": "dataset", "description": "Hidden dataset holding this run's selection. A rerun writes a new version of it.", "artifact": true, "badge": false, "badgeOpens": null, "hidden": false }, { "key": "SELECTED_IMAGES", "label": "Selected images", "type": "string", "description": "Number of image assets attached to the selection.", "artifact": false, "badge": true, "badgeOpens": "LAST_RUN", "hidden": false } ], "automation": { "isolated": true, "requires_configuration": true, "manual_run": true, "discard_history": "on_success" } } ```