> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.ovalbee.com/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.ovalbee.com/_mcp/server. # Sample images similar to failures > Take the images an evaluation report scored worst and sample visually similar ones from a candidate pool. Dockerfile > **Note** > > First run can take longer while the worker pulls the configured Docker image. ## Overview Closes the loop between evaluating a model and improving its training data. It reads which images the model scored worst on, then searches a pool of candidate images for ones that look like them — so the next round of labelling is spent on the material the model actually struggles with rather than on more of what it already handles. It embeds nothing itself. The **CLIP Index** node has already turned every indexed image into a vector, and the worst images from the report are among them, so this node only queries. Each rerun writes a new version of the node's own hidden `LAST_RUN` dataset holding that run's sample, and the badge shows how many images it selected and opens it. Set a destination dataset to accumulate samples across runs as well. ```mermaid flowchart LR n1["Evaluate"] --> n2["Sample images similar to failures"] --> n3["Labeling and review"] n4["CLIP Index"] --> n2 ``` ## Prerequisites | Requirement | What you need | | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **An evaluation report** | An object detection report from the [Evaluate](/node-library/nodes/evaluate-evaluate-object-detection) node. Reports produced before this node existed do not carry the per-image file it reads, so re-run the evaluation if you see it complain about that. | | **An indexed pool** | A **CLIP Index** node pointed at your candidate pool. It runs on a schedule, so give it a few minutes on a freshly wired dataset. | ## How it works Ranks the evaluated images by the metric you choose and takes the worst few as seeds. Images with no ground-truth objects are skipped — a zero score on an empty frame says nothing about the model. Asks the CLIP index which pool images sit nearest each seed, ignoring anything below the similarity threshold and anything in the **Exclude images in** dataset. Point that at your labeled set and the pool never has to be edited to keep picked images out of the next sample. Folds the per-seed results together, keeping a candidate that is near several seeds once at its best score, and drops the seeds themselves along with anything the report already evaluated. ## Inputs **`Evaluation report`** `report` — required Object detection evaluation report the failures are read from. Generic reports are not supported. Key: `REPORT_ID`. --- **`Candidate pool`** `dataset` — required Dataset the similar images are drawn from. Its images must already be in the CLIP index, so run a CLIP Index node on it first. Key: `POOL_DATASET`. --- **`Exclude images in`** `dataset` Images in this dataset are never sampled, so the pool itself never has to change. In an active learning loop, point it at the labeled training set and each sample holds only images nobody has picked yet. Key: `EXCLUDE_DATASET`. --- **`To dataset`** `dataset` Destination for the sample. Leave empty to keep it only in the node's own Last run dataset. Key: `DST_DATASET`. --- **`Worst by`** `select` — required, default: f1 Which per-image metric decides the failures the sample is seeded from. Key: `WORST_BY`. Options: * F1 (`f1`) * Recall (`recall`) * Precision (`precision`) --- **`Sample size`** `integer` — required, default: 100 Maximum number of images to select. Fewer are returned when the pool holds nothing else similar enough. Minimum: `1`. Maximum: `10000`. Step: `1`. Key: `SAMPLE_SIZE`. --- **`Failures used as seeds`** `integer` — required, default: 25 How many of the worst-scoring evaluated images the search starts from. Minimum: `1`. Maximum: `500`. Step: `1`. Key: `SEED_COUNT`. --- **`Similarity threshold`** `number` — required, default: 0.25 Ignore candidates less similar than this to every seed. Raise it when the sample drifts off-topic. Minimum: `0`. Maximum: `1`. Step: `0.01`. Key: `SIMILARITY_THRESHOLD`. --- **`Exclude evaluated images`** `boolean` — required, default: true Leave out images the report already evaluated, so the sample is only new material. Key: `EXCLUDE_EVALUATED`. --- ## Outputs **`Dataset`** `dataset` Key: `DATASET`. --- **`Last run`** `dataset` Hidden dataset holding this run's sample. A rerun writes a new version of it. Shown as an artifact. Key: `LAST_RUN`. --- **`Selected images`** `string` Number of image assets attached to the sample. Shown on the node as a badge. Selecting the badge opens `LAST_RUN`. Key: `SELECTED_IMAGES`. --- ## JSON config Machine-readable node interface for automation and advanced usage. #### Show JSON ```json { "name": "Sample images similar to failures", "description": "Take the images an evaluation report scored worst and sample visually similar ones from a candidate pool.", "category": "Sample", "namespace": "ovalbee", "templateKey": "sample/sample_images_similar_to_failures", "version": "v1", "inputs": [ { "key": "REPORT_ID", "label": "Evaluation report", "type": "report", "kind": "object_detection_evaluation_report_v1", "description": "Object detection evaluation report the failures are read from. Generic reports are not supported.", "required": true, "default": null, "visibleWhen": null }, { "key": "POOL_DATASET", "label": "Candidate pool", "type": "dataset", "description": "Dataset the similar images are drawn from. Its images must already be in the CLIP index, so run a CLIP Index node on it first.", "required": true, "default": null, "visibleWhen": null }, { "key": "EXCLUDE_DATASET", "label": "Exclude images in", "type": "dataset", "description": "Images in this dataset are never sampled, so the pool itself never has to change. In an active learning loop, point it at the labeled training set and each sample holds only images nobody has picked yet.", "required": false, "default": null, "visibleWhen": null }, { "key": "DST_DATASET", "label": "To dataset", "type": "dataset", "description": "Destination for the sample. Leave empty to keep it only in the node's own Last run dataset.", "required": false, "default": null, "visibleWhen": null }, { "key": "WORST_BY", "label": "Worst by", "type": "select", "description": "Which per-image metric decides the failures the sample is seeded from.", "required": true, "default": "f1", "visibleWhen": null, "options": { "control": "tabs", "options": [ { "label": "F1", "value": "f1" }, { "label": "Recall", "value": "recall" }, { "label": "Precision", "value": "precision" } ] } }, { "key": "SAMPLE_SIZE", "label": "Sample size", "type": "number", "description": "Maximum number of images to select. Fewer are returned when the pool holds nothing else similar enough.", "required": true, "default": 100, "visibleWhen": null, "options": { "type": "integer", "min": 1, "max": 10000, "step": 1, "slider": false } }, { "key": "SEED_COUNT", "label": "Failures used as seeds", "type": "number", "description": "How many of the worst-scoring evaluated images the search starts from.", "required": true, "default": 25, "visibleWhen": null, "options": { "type": "integer", "min": 1, "max": 500, "step": 1, "slider": false } }, { "key": "SIMILARITY_THRESHOLD", "label": "Similarity threshold", "type": "number", "description": "Ignore candidates less similar than this to every seed. Raise it when the sample drifts off-topic.", "required": true, "default": 0.25, "visibleWhen": null, "options": { "type": "float", "min": 0, "max": 1, "step": 0.01 } }, { "key": "EXCLUDE_EVALUATED", "label": "Exclude evaluated images", "type": "boolean", "description": "Leave out images the report already evaluated, so the sample is only new material.", "required": true, "default": true, "visibleWhen": null } ], "outputs": [ { "key": "DATASET", "label": "Dataset", "type": "dataset", "description": "", "artifact": false, "badge": false, "badgeOpens": null, "hidden": false }, { "key": "LAST_RUN", "label": "Last run", "type": "dataset", "description": "Hidden dataset holding this run's sample. A rerun writes a new version of it.", "artifact": true, "badge": false, "badgeOpens": null, "hidden": false }, { "key": "SELECTED_IMAGES", "label": "Selected images", "type": "string", "description": "Number of image assets attached to the sample.", "artifact": false, "badge": true, "badgeOpens": "LAST_RUN", "hidden": false } ], "runtime": { "type": "docker", "devices": [ "cpu" ], "dockerImage": "ovalbee/node-clip-index:0.0.32" }, "automation": { "isolated": true, "requires_configuration": true, "manual_run": true, "discard_history": "on_success" } } ```