Node LibraryModels

Embedding map

GPU Dockerfile
First run can take longer while the worker pulls the configured Docker image.

Overview

Clusters the images of a dataset by their embeddings, names each cluster with a vision-language model, and lays the result out as an interactive 2D map. It answers “what does this dataset actually contain?” without anyone looking through it image by image.

Clustering runs on DINOv3 by default, which groups images by how they look. Pick CLIP, PE-Core, or SigLIP 2 to group them by meaning instead. Two modes, chosen by whether Group by tag is set:

  • Dataset map (default) — clusters every image and lays the whole dataset out on one map, so scenes that look alike sit close together.
  • Rare subtypes — clusters separately inside every value of an image-level tag (for example, a crop’s class) and flags only the small clusters: a handful of images that look like each other and unlike the rest of their group. They are highlighted on the map itself.

The badge counts the clusters on the image map, or the rare subtypes found across all groups. The Open map button opens the map, and images you select on it are added to a dataset.

Dataset AI index embeddings Embedding map

Prerequisites

RequirementWhat you need
Embedding indexImages indexed by AI index embeddings with the model chosen in Embedding model. DINOv3, the default, needs visual embedding on in that node. Vectors are keyed by image, so images already indexed as part of another dataset are covered.
Image-level tag (optional)For rare subtypes, images tagged with the tag named in Group by tag, at least 30 images per tag value.
GPU workerA worker with a GPU. The node always asks for one, and the cluster naming model runs on it.

How it works

1

Lists the dataset’s images and reads their vectors from the index of the selected model.

2

Splits the images by the value of Group by tag when it is set, dropping images without the tag and values under 30 images. Otherwise the whole dataset is one group.

3

Clusters each group with k-means, using Clusters when set or picking a count automatically.

4

Names every cluster in map mode, or only the rare ones in rare-subtypes mode, from the images nearest each cluster’s centre.

5

Projects every group onto a 2D map, gives each point an outlier score, and saves the map with a preview for the node’s canvas.

FAQ

With Clusters empty, the count with the best silhouette score between 3 and 15 is chosen, and the upper end shrinks to a tenth of the image count on small datasets. Very small datasets get 8, or fewer when there are fewer images. A count you set is capped at the number of images.

They are left off the map, and the log says how many. When none of the dataset’s images is indexed, the run fails and asks for AI index embeddings to run first.

Only tag values with at least 30 images are clustered. When no value reaches that, the log says so and the map has no points.

Every run writes a new map and re-fits the layout. A selection drawn on the old map is refused, so reopen the map and select again. Images already added to the anomalies dataset stay there, and later selections keep adding to the same dataset.

The flow stops and the node keeps the previous map, its badge, and its anomalies dataset.

Yes. Turn off Name clusters and every cluster gets a numbered name instead, with no call to the vision-language model.

Inputs

Dataset
datasetRequired

Dataset whose images are clustered. Must already be indexed by an AI Index Embeddings node with the model selected below enabled. Key: DATASET.

Embedding model
selectDefaults to dinov3Required

Which indexed embedding family the map clusters on. DINOv3 (visual-only) is the default because it groups images by how they look, the semantic models (CLIP, PE-Core, SigLIP 2) group more by meaning/content. Key: EMBEDDING_MODEL.

Options:

  • CLIP (clip)
  • PE-Core (pecore)
  • SigLIP 2 (siglip2)
  • DINOv3 (dinov3)
Clusters
integer

Number of groups the images are split into. Leave empty to let the node pick a count that fits the data. Minimum: 2. Maximum: 200. Step: 1. Key: CLUSTERS.

Group by tag
string

Image-level tag that splits the dataset before clustering, so every value of it is clustered on its own and the map calls out rare subtypes - small clusters that stand apart from the rest of their group. Leave empty for a single map of the whole dataset. Key: GROUP_BY_TAG.

Rare subtype factor
numberDefaults to 0.4Required

Size a cluster must stay under to count as a rare subtype, as a fraction of an average cluster in its own tag group. Only used when “Group by tag” is set. Minimum: 0.05. Maximum: 1. Step: 0.05. Key: RARE_FACTOR.

Naming model
selectDefaults to moondream2Required

Vision-language model used to name each cluster from its representative images. Key: LLM_MODEL.

Options:

  • Moondream 2 (moondream2)
Anomalies dataset
dataset

Dataset that images selected on the interactive map are added to. Leave empty to have the first selection create one, which later selections keep adding to. Key: DST_ANOMALIES_DATASET.

Name clusters
booleanDefaults to trueRequired

Ask the vision-language model to name every cluster. Turn off to skip the naming call and use the numbered fallback name instead, for a faster run with no VLM cost. Key: NAME_CLUSTERS.

Outputs

Map data
asset

Point-cloud data behind the interactive map (image and object projections). Shown as an artifact. Key: MAP_ASSET.

Map preview URL
string

Signed URL to the map’s preview image, for the node’s canvas widget. Hidden from the flow editor. Key: MAP_PREVIEW_URL.

Clusters found
string

Number of clusters in the map, or rare subtypes found across all groups. Shown on the node as a badge. Key: CLUSTERS_COUNT.

Anomalies dataset
dataset

Dataset the images selected on the map accumulate into. Shown as an artifact. Key: ANOMALIES_DATASET.

Widgets

The canvas shows a small preview of the map, coloured by cluster, and Open map opens the interactive map.

On the map, Images and Objects switch between the image map and a separate map of annotated objects. The object map exists only when the index holds object embeddings for the dataset’s images. It is clustered the same way, on its own layout. Automatic outliers selects the points by outlier score, and you can also draw a selection or pick points one by one. Add to dataset adds the selected images, with their annotations from the input dataset, to the Anomalies dataset: the one set on the node, or else a dataset the first selection creates and later ones keep adding to.

Runtime

The node runs in its own GPU image. The embeddings are read from the index, so no embedding model runs here. Moondream 2 runs on the GPU to name clusters. The outlier score is each point’s rank by the distance to its 15th nearest neighbour, computed on the embeddings rather than on the 2D layout.

JSON config

Machine-readable node interface for automation and advanced usage.

{
  "name": "Embedding map",
  "description": "Cluster a dataset by visual similarity and name each cluster with a vision-language model, either as a 2D map of the whole dataset or as rare subtypes inside one image-level tag.",
  "category": "Models",
  "namespace": "ovalbee",
  "templateKey": "models/clip/embedding_map",
  "version": "v1",
  "inputs": [
    {
      "key": "DATASET",
      "label": "Dataset",
      "type": "dataset",
      "description": "Dataset whose images are clustered. Must already be indexed by an AI Index Embeddings node with the model selected below enabled.",
      "required": true,
      "default": null,
      "visibleWhen": null,
      "options": {
        "creatable": false
      }
    },
    {
      "key": "EMBEDDING_MODEL",
      "label": "Embedding model",
      "type": "select",
      "description": "Which indexed embedding family the map clusters on. DINOv3 (visual-only) is the default because it groups images by how they look; the semantic models (CLIP, PE-Core, SigLIP 2) group more by meaning/content.",
      "required": true,
      "default": "dinov3",
      "visibleWhen": null,
      "options": {
        "options": [
          {
            "label": "CLIP",
            "value": "clip"
          },
          {
            "label": "PE-Core",
            "value": "pecore"
          },
          {
            "label": "SigLIP 2",
            "value": "siglip2"
          },
          {
            "label": "DINOv3",
            "value": "dinov3"
          }
        ]
      }
    },
    {
      "key": "CLUSTERS",
      "label": "Clusters",
      "type": "number",
      "description": "Number of groups the images are split into. Leave empty to let the node pick a count that fits the data.",
      "required": false,
      "default": null,
      "visibleWhen": null,
      "options": {
        "type": "integer",
        "min": 2,
        "max": 200,
        "step": 1
      }
    },
    {
      "key": "GROUP_BY_TAG",
      "label": "Group by tag",
      "type": "string",
      "description": "Image-level tag that splits the dataset before clustering, so every value of it is clustered on its own and the map calls out rare subtypes - small clusters that stand apart from the rest of their group. Leave empty for a single map of the whole dataset.",
      "required": false,
      "default": null,
      "visibleWhen": null
    },
    {
      "key": "RARE_FACTOR",
      "label": "Rare subtype factor",
      "type": "number",
      "description": "Size a cluster must stay under to count as a rare subtype, as a fraction of an average cluster in its own tag group. Only used when \"Group by tag\" is set.",
      "required": true,
      "default": 0.4,
      "visibleWhen": null,
      "options": {
        "type": "float",
        "min": 0.05,
        "max": 1,
        "step": 0.05
      }
    },
    {
      "key": "LLM_MODEL",
      "label": "Naming model",
      "type": "select",
      "description": "Vision-language model used to name each cluster from its representative images.",
      "required": true,
      "default": "moondream2",
      "visibleWhen": null,
      "options": {
        "options": [
          {
            "value": "moondream2",
            "label": "Moondream 2"
          }
        ]
      }
    },
    {
      "key": "DST_ANOMALIES_DATASET",
      "label": "Anomalies dataset",
      "type": "dataset",
      "description": "Dataset that images selected on the interactive map are added to. Leave empty to have the first selection create one, which later selections keep adding to.",
      "required": false,
      "default": null,
      "visibleWhen": null
    },
    {
      "key": "NAME_CLUSTERS",
      "label": "Name clusters",
      "type": "boolean",
      "description": "Ask the vision-language model to name every cluster. Turn off to skip the naming call and use the numbered fallback name instead, for a faster run with no VLM cost.",
      "required": true,
      "default": true,
      "visibleWhen": null
    }
  ],
  "outputs": [
    {
      "key": "MAP_ASSET",
      "label": "Map data",
      "type": "asset",
      "description": "Point-cloud data behind the interactive map (image and object projections).",
      "artifact": true,
      "badge": false,
      "badgeOpens": null,
      "hidden": false
    },
    {
      "key": "MAP_PREVIEW_URL",
      "label": "Map preview URL",
      "type": "string",
      "description": "Signed URL to the map's preview image, for the node's canvas widget.",
      "artifact": false,
      "badge": false,
      "badgeOpens": null,
      "hidden": true
    },
    {
      "key": "CLUSTERS_COUNT",
      "label": "Clusters found",
      "type": "string",
      "description": "Number of clusters in the map, or rare subtypes found across all groups.",
      "artifact": false,
      "badge": true,
      "badgeOpens": null,
      "hidden": false
    },
    {
      "key": "ANOMALIES_DATASET",
      "label": "Anomalies dataset",
      "type": "dataset",
      "description": "Dataset the images selected on the map accumulate into.",
      "artifact": true,
      "badge": false,
      "badgeOpens": null,
      "hidden": false
    }
  ],
  "runtime": {
    "type": "docker",
    "requiresGpu": true,
    "dockerImage": "cr.internal.supervisely.com/ovalbee-internal/nodes/clip-map:0.0.9"
  },
  "automation": {
    "isolated": true,
    "requires_configuration": true,
    "manual_run": true
  },
  "widgets": {
    "widget": {
      "id": "image-preview",
      "settings": {
        "imageUrl": {
          "type": "variable",
          "value": "self.outputs.MAP_PREVIEW_URL"
        }
      }
    },
    "secondaryWidget": {
      "id": "open-map",
      "settings": {
        "name": {
          "type": "input",
          "value": "Open map"
        }
      }
    }
  }
}