Node LibraryModels

Embedding anomalies

Dockerfile
First run can take longer while the worker pulls the configured Docker image.

Overview

Splits a dataset into anomalies and typical images by how far each image sits from its nearest neighbours in the embedding index, without being told what to look for.

Every image is scored by its mean cosine distance to its nearest neighbours in the reference set. Everything at or above the threshold goes to Anomalies, the rest to Typical. A clean batch therefore yields no anomalies, and a badly drifted one yields as many as it really has. Nothing is deleted and the input dataset is left untouched. Badges show the anomaly count, the typical count and the anomaly share, and the first two open their dataset.

There are two ways to use it:

  • With a reference dataset — the incoming batch is compared against a baseline you trust, usually the pool your model already trained on. An anomaly means “unlike anything the model has seen”, which is domain drift.
  • Without one — images are compared against the rest of their own dataset. This only holds while anomalies are rare in it: if half the dataset was shot at night, night is the norm and won’t surface.

The report can be passed to Dataset insights through its Findings from other nodes input.

Dataset Embedding anomalies Reference dataset Anomalies Typical

An anomaly is not garbage. Rain and night shots are rare conditions, not defects — dropping them from a training set makes the model fail in rain. Send them to labeling or oversampling rather than to the bin.

Prerequisites

RequirementWhat you need
Image embeddingsImages indexed by AI index embeddings with the model chosen in Embedding model. Vectors are keyed by image, so images already indexed as part of another dataset are covered.
Reference dataset (optional)An existing Dataset to compare against, indexed with the same model.

How it works

1

Lists the images of the input dataset and looks up their vectors in the index of the selected model. Images with no vector yet are reported and left in the typical dataset.

2

Scores each image by its mean cosine distance to the nearest reference images.

3

Routes every image scoring at or above the threshold to the anomalies dataset and the rest to the typical one, each with its annotation.

4

Clusters the anomalies into visual groups when Group anomalies is on, and builds the report with the score at each percentile of the run.

FAQ

Read it off the report rather than guessing. The score at the 50th, 90th, 95th and 99th percentile of the run is printed there, so setting the threshold to the p95 value flags the top twentieth of that batch. A raw distance is not comparable between datasets or models, so check it again when the reference or the model changes.

A share always hands back the same count. If a batch holds twice as many anomalies as usual, half of them would quietly land in the typical dataset, and a perfectly clean batch would still come back with a full quota of ordinary images labelled anomalous. A threshold reports what is actually there, including nothing.

Set Neighbours to fix it. Left empty, it’s 1% of the reference size, kept between 5 and 50, and never more than the reference holds minus one.

It’s not an error. An accumulating pool is empty on its first run, so the node says so in the log and the report, then scores the dataset against itself for that run.

With no destination set, the first run creates one dataset for each side, and every rerun adds a new version to it, so earlier results stay in its history. A destination you pick gets the images added to what it already holds.

The clustering is a fact about the data, but any name put on a cluster would be a guess, and a fixed vocabulary of guesses only fits one domain. Four thumbnails per group say what it is faster and more accurately.

Not reliably. These embeddings are semantic — they separate weather, place, and scene type, not low-level image quality.

They can’t be scored, so they’re passed through to the typical dataset and their count is reported in the log and the report. Run AI index embeddings with the same model on the input dataset to cover them.

An input dataset with no images stops the flow with zero counts. The run fails when the workspace has no index for the selected model, or when none of the images, in the input or the reference, is indexed.

No. Both outputs are new dataset memberships over the same images and annotations, and the source keeps every image.

Inputs

Dataset
datasetRequired

Dataset whose images are scored. Images not yet indexed by AI index embeddings with the selected model cannot be scored and are passed through as typical. Key: DATASET.

Embedding model
selectDefaults to clipRequired

Model the images were indexed with. Choosing one they were not indexed with stops the run. Key: EMBEDDING_MODEL.

Options:

  • CLIP (clip)
  • PE-Core (pecore)
  • SigLIP 2 (siglip2)
Reference dataset
dataset

Baseline the images are compared against, usually the pool the model already trained on. Leave empty to look for anomalies inside the scored dataset itself, which only works while anomalies are rare in it. Key: REFERENCE_DATASET.

Anomaly score threshold
numberDefaults to 0.25Required

Images scoring at or above this distance go to the anomalies dataset. Every image above it is returned, so a clean batch yields none. The report prints the score at each percentile of the run, which is what to set this from. Minimum: 0. Maximum: 1. Step: 0.01. Key: SCORE_THRESHOLD.

Neighbours
integer

How many nearest reference images each score averages over. Derived from the reference size when empty. Minimum: 1. Maximum: 200. Step: 1. Key: NEIGHBORS.

Anomalies dataset
dataset

Destination for the anomalies. Leave empty to use the node’s own output dataset. Key: DST_ANOMALIES_DATASET.

Typical dataset
dataset

Destination for the remaining images. Leave empty to use the node’s own output dataset. Key: DST_TYPICAL_DATASET.

Group anomalies
booleanDefaults to trueRequired

Clusters of similar anomalies previewed in the report, so it is clear what kinds of anomaly turned up. Which images are routed where is unchanged either way. Key: GROUP_ANOMALIES.

Outputs

Anomalies
dataset

Dataset holding the images that sit farthest from the reference. Shown as an artifact. Key: ANOMALIES.

Typical
dataset

Dataset holding the remaining scored images. Shown as an artifact. Key: TYPICAL.

Anomalies count
string

Number of images routed to the anomalies dataset. Shown on the node as a badge. Selecting the badge opens ANOMALIES. Key: ANOMALIES_COUNT.

Typical count
string

Number of images routed to the typical dataset. Shown on the node as a badge. Selecting the badge opens TYPICAL. Key: TYPICAL_COUNT.

Anomalies ratio
string

Share of the scored images that were routed to the anomalies dataset. Shown on the node as a badge. Key: ANOMALIES_RATIO.

Report
report

Score distribution, anomaly groups, and previews of the most and least typical images. Shown as an artifact. Key: REPORT.

JSON config

Machine-readable node interface for automation and advanced usage.

{
  "name": "Embedding anomalies",
  "description": "Score indexed images by how far they sit from their nearest neighbours and split the dataset into anomalies and typical images.",
  "category": "Models",
  "namespace": "ovalbee",
  "templateKey": "models/clip/embedding_anomalies",
  "version": "v1",
  "inputs": [
    {
      "key": "DATASET",
      "label": "Dataset",
      "type": "dataset",
      "description": "Dataset whose images are scored. Images not yet indexed by AI index embeddings with the selected model cannot be scored and are passed through as typical.",
      "required": true,
      "default": null,
      "visibleWhen": null,
      "options": {
        "creatable": false
      }
    },
    {
      "key": "EMBEDDING_MODEL",
      "label": "Embedding model",
      "type": "select",
      "description": "Model the images were indexed with. Choosing one they were not indexed with stops the run.",
      "required": true,
      "default": "clip",
      "visibleWhen": null,
      "options": {
        "options": [
          {
            "label": "CLIP",
            "value": "clip"
          },
          {
            "label": "PE-Core",
            "value": "pecore"
          },
          {
            "label": "SigLIP 2",
            "value": "siglip2"
          }
        ]
      }
    },
    {
      "key": "REFERENCE_DATASET",
      "label": "Reference dataset",
      "type": "dataset",
      "description": "Baseline the images are compared against, usually the pool the model already trained on. Leave empty to look for anomalies inside the scored dataset itself, which only works while anomalies are rare in it.",
      "required": false,
      "default": null,
      "visibleWhen": null,
      "options": {
        "creatable": false
      }
    },
    {
      "key": "SCORE_THRESHOLD",
      "label": "Anomaly score threshold",
      "type": "number",
      "description": "Images scoring at or above this distance go to the anomalies dataset. Every image above it is returned, so a clean batch yields none. The report prints the score at each percentile of the run, which is what to set this from.",
      "required": true,
      "default": 0.25,
      "visibleWhen": null,
      "options": {
        "type": "float",
        "min": 0,
        "max": 1,
        "step": 0.01
      }
    },
    {
      "key": "NEIGHBORS",
      "label": "Neighbours",
      "type": "number",
      "description": "How many nearest reference images each score averages over. Derived from the reference size when empty.",
      "required": false,
      "default": null,
      "visibleWhen": null,
      "options": {
        "type": "integer",
        "min": 1,
        "max": 200,
        "step": 1,
        "slider": false
      }
    },
    {
      "key": "DST_ANOMALIES_DATASET",
      "label": "Anomalies dataset",
      "type": "dataset",
      "description": "Destination for the anomalies. Leave empty to use the node's own output dataset.",
      "required": false,
      "default": null,
      "visibleWhen": null
    },
    {
      "key": "DST_TYPICAL_DATASET",
      "label": "Typical dataset",
      "type": "dataset",
      "description": "Destination for the remaining images. Leave empty to use the node's own output dataset.",
      "required": false,
      "default": null,
      "visibleWhen": null
    },
    {
      "key": "GROUP_ANOMALIES",
      "label": "Group anomalies",
      "type": "boolean",
      "description": "Clusters of similar anomalies previewed in the report, so it is clear what kinds of anomaly turned up. Which images are routed where is unchanged either way.",
      "required": true,
      "default": true,
      "visibleWhen": null
    }
  ],
  "outputs": [
    {
      "key": "ANOMALIES",
      "label": "Anomalies",
      "type": "dataset",
      "description": "Dataset holding the images that sit farthest from the reference.",
      "artifact": true,
      "badge": false,
      "badgeOpens": null,
      "hidden": false
    },
    {
      "key": "TYPICAL",
      "label": "Typical",
      "type": "dataset",
      "description": "Dataset holding the remaining scored images.",
      "artifact": true,
      "badge": false,
      "badgeOpens": null,
      "hidden": false
    },
    {
      "key": "ANOMALIES_COUNT",
      "label": "Anomalies count",
      "type": "string",
      "description": "Number of images routed to the anomalies dataset.",
      "artifact": false,
      "badge": true,
      "badgeOpens": "ANOMALIES",
      "hidden": false
    },
    {
      "key": "TYPICAL_COUNT",
      "label": "Typical count",
      "type": "string",
      "description": "Number of images routed to the typical dataset.",
      "artifact": false,
      "badge": true,
      "badgeOpens": "TYPICAL",
      "hidden": false
    },
    {
      "key": "ANOMALIES_RATIO",
      "label": "Anomalies ratio",
      "type": "string",
      "description": "Share of the scored images that were routed to the anomalies dataset.",
      "artifact": false,
      "badge": true,
      "badgeOpens": null,
      "hidden": false
    },
    {
      "key": "REPORT",
      "label": "Report",
      "type": "report",
      "kind": "dataset_intelligence_findings_v1",
      "description": "Score distribution, anomaly groups, and previews of the most and least typical images.",
      "artifact": true,
      "badge": false,
      "badgeOpens": null,
      "hidden": false
    }
  ],
  "runtime": {
    "type": "docker",
    "requiresGpu": false,
    "dockerImage": "cr.internal.supervisely.com/ovalbee-internal/nodes/clip-index:0.0.24"
  },
  "automation": {
    "isolated": true,
    "requires_configuration": true,
    "manual_run": true,
    "discard_history": "on_success"
  },
  "widgets": {
    "widget": {
      "id": "report-preview",
      "settings": {
        "reportAsset": {
          "type": "variable",
          "value": "self.outputs.REPORT"
        },
        "name": {
          "type": "input",
          "value": "Open anomalies"
        }
      }
    }
  }
}