Node LibraryModels

AI index embeddings

Dockerfile
First run can take longer while the worker pulls the configured Docker image.

Overview

Embeds the images of one dataset, and the objects annotated on them, into the workspace’s embedding index, then keeps that index current as the dataset changes. Similar pairs, Embedding map, Label diagnostics and Embedding anomalies all read what this node wrote, so run it on a dataset before any of them.

Two kinds of embedding are built side by side. The semantic one (CLIP, PE-Core or SigLIP 2) also understands text, so a written prompt can find matching images and objects. The visual one (DINOv3) compares how things look rather than what they mean, which is what near-duplicate and outlier work needs.

The badge on the node counts the images added in the latest run.

Dataset AI index embeddings Similar pairs

Prerequisites

RequirementWhat you need
Annotated images (optional)Images that already carry annotations, for object-level embeddings. Without them only whole images are indexed.

How it works

1

Reads every image of the dataset together with its annotation.

2

Embeds the images and annotated objects that the workspace index is still missing, a batch at a time.

3

Writes the new embeddings, once for the semantic model and once more for DINOv3 when visual embedding is on.

4

Drops the objects this dataset no longer annotates, and deletes the ones left over in no dataset at all.

5

Records what it indexed, so a later run over an unchanged dataset finishes without rescanning.

FAQ

Each reader looks in the index of the model you pick on it, so index with that model first. Embedding map clusters on DINOv3 by default, which needs Enable visual embedding on. Label diagnostics works on objects, so it needs Enable object indexing on, and it only counts objects indexed by a node pointed at that same dataset.

Nothing is recomputed while the dataset is untouched: the run ends right away and the badge reads 0 images. Once images or annotations change, only what is missing gets embedded, everything already indexed is reused, and objects whose annotation is gone are removed from the index.

No. Once it is configured, the node checks its dataset every ten minutes on its own. You can also run it by hand.

No. The first run binds the node to that dataset, that embedding model, the visual embedding setting and the object embedding method, and changing any of them afterwards stops the run with an error. Add a second node for a second dataset or a different model. Turning object indexing off later is allowed: it stops new objects from being embedded and leaves the ones already indexed in place.

The binding is to the exact dataset version of the first run. When the dataset input moves to a newer version, for example because an upstream node reran and wrote one, the run stops with the same error as for a different dataset. Index the new version with a new node.

No. Images and annotations are read only. The embeddings live next to the dataset, in an index shared by the whole workspace. An image already indexed through another dataset is not embedded again.

Every object of the image’s annotation, taken from the box around its shape, whatever the shape is. An object whose box reaches past the edge of its image is skipped, and so is an annotation that does not record the image size.

Images only. Videos and other assets are not indexed. And because the workspace keeps one DINOv3 index, every node writing visual embeddings into it has to use the same object embedding method.

The service that stores the embeddings is unreachable or not configured for this instance. The dataset is left as it was, so the node picks up where it stopped once the service answers again.

Inputs

Dataset
datasetRequired

Dataset to index, with its images and their annotated objects. A node stays bound to the exact dataset version it first indexes, so indexing another dataset or version takes another node. Key: DATASET.

Embedding model
selectDefaults to clipRequired

Model used for image, object, and text-compatible embeddings. CLIP is the fastest and the weakest at matching objects to a text prompt. PE-Core and SigLIP 2 are both much stronger, but which one leads depends on the imagery, and PE-Core costs an order of magnitude more time per object. Index a sample both ways and compare before committing a large dataset. Key: EMBEDDING_MODEL.

Options:

  • CLIP (clip)
  • PE-Core (pecore)
  • SigLIP 2 (siglip2)
Enable visual embedding
booleanDefaults to trueRequired

Extra embeddings that compare images by how they look rather than by what they mean, using DINOv3. Similar pairs, Embedding map and Label diagnostics can then search on it. Turn off only when you need a faster run. Key: ENABLE_VISUAL_EMBEDDING.

Enable object indexing
booleanDefaults to trueRequired

Also embed every annotated object, not just whole images, for both the semantic and visual (DINOv3) collections. Turn off to index only whole images. Key: ENABLE_OBJECT_INDEXING.

Object embedding method
selectDefaults to cropRequired

Crop gives every bounding box its own full-resolution pass. Regions runs DINOv3 once per whole image and pools patch features inside each box, costing one pass per image instead of one per object, but leaving each object only the feature cells its area covers. Crop separated classes better at every object size measured, and by a wide margin for small objects, so prefer Regions only to trade accuracy for indexing speed on object-dense images. All DINOv3 nodes in one workspace write to the same index, so they all have to use the same method. Visible when ENABLE_OBJECT_INDEXING is True. Key: OBJECT_EMBEDDING_MODE.

Options:

  • Regions (ROI pooling) (regions)
  • Crop (crop)
Batch size
integerDefaults to 32Required

Number of images downloaded and embedded together. Minimum: 1. Maximum: 256. Step: 1. Key: BATCH_SIZE.

Outputs

Indexed images
string

Number of new image vectors written during this run. Shown on the node as a badge. Key: INDEXED_IMAGES.

Internal state
object

Internal scan state. Do not edit. Hidden from the flow editor. Key: STATE.

JSON config

Machine-readable node interface for automation and advanced usage.

{
  "name": "AI index embeddings",
  "description": "Embed a dataset's images and annotated objects into the workspace index, and keep it current as the dataset changes.",
  "category": "Models",
  "namespace": "ovalbee",
  "templateKey": "models/clip/ai_index_embeddings",
  "version": "v1",
  "inputs": [
    {
      "key": "DATASET",
      "label": "Dataset",
      "type": "dataset",
      "description": "Dataset to index, with its images and their annotated objects. A node stays bound to the exact dataset version it first indexes, so indexing another dataset or version takes another node.",
      "required": true,
      "default": null,
      "visibleWhen": null,
      "options": {
        "creatable": false
      }
    },
    {
      "key": "EMBEDDING_MODEL",
      "label": "Embedding model",
      "type": "select",
      "description": "Model used for image, object, and text-compatible embeddings. CLIP is the fastest and the weakest at matching objects to a text prompt. PE-Core and SigLIP 2 are both much stronger, but which one leads depends on the imagery, and PE-Core costs an order of magnitude more time per object. Index a sample both ways and compare before committing a large dataset.",
      "required": true,
      "default": "clip",
      "visibleWhen": null,
      "options": {
        "options": [
          {
            "label": "CLIP",
            "value": "clip"
          },
          {
            "label": "PE-Core",
            "value": "pecore"
          },
          {
            "label": "SigLIP 2",
            "value": "siglip2"
          }
        ]
      }
    },
    {
      "key": "ENABLE_VISUAL_EMBEDDING",
      "label": "Enable visual embedding",
      "type": "boolean",
      "description": "Extra embeddings that compare images by how they look rather than by what they mean, using DINOv3. Similar pairs, Embedding map and Label diagnostics can then search on it. Turn off only when you need a faster run.",
      "required": true,
      "default": true,
      "visibleWhen": null
    },
    {
      "key": "ENABLE_OBJECT_INDEXING",
      "label": "Enable object indexing",
      "type": "boolean",
      "description": "Also embed every annotated object, not just whole images, for both the semantic and visual (DINOv3) collections. Turn off to index only whole images.",
      "required": true,
      "default": true,
      "visibleWhen": null
    },
    {
      "key": "OBJECT_EMBEDDING_MODE",
      "label": "Object embedding method",
      "type": "select",
      "description": "Crop gives every bounding box its own full-resolution pass. Regions runs DINOv3 once per whole image and pools patch features inside each box, costing one pass per image instead of one per object, but leaving each object only the feature cells its area covers. Crop separated classes better at every object size measured, and by a wide margin for small objects, so prefer Regions only to trade accuracy for indexing speed on object-dense images. All DINOv3 nodes in one workspace write to the same index, so they all have to use the same method.",
      "required": true,
      "default": "crop",
      "visibleWhen": {
        "key": "ENABLE_OBJECT_INDEXING",
        "operator": "equals",
        "value": true
      },
      "options": {
        "options": [
          {
            "label": "Regions (ROI pooling)",
            "value": "regions"
          },
          {
            "label": "Crop",
            "value": "crop"
          }
        ]
      }
    },
    {
      "key": "BATCH_SIZE",
      "label": "Batch size",
      "type": "number",
      "description": "Number of images downloaded and embedded together.",
      "required": true,
      "default": 32,
      "visibleWhen": null,
      "options": {
        "type": "integer",
        "min": 1,
        "max": 256,
        "step": 1
      }
    }
  ],
  "outputs": [
    {
      "key": "INDEXED_IMAGES",
      "label": "Indexed images",
      "type": "string",
      "description": "Number of new image vectors written during this run.",
      "artifact": false,
      "badge": true,
      "badgeOpens": null,
      "hidden": false
    },
    {
      "key": "STATE",
      "label": "Internal state",
      "type": "object",
      "description": "Internal scan state. Do not edit.",
      "artifact": false,
      "badge": false,
      "badgeOpens": null,
      "hidden": true
    }
  ],
  "runtime": {
    "type": "docker",
    "requiresGpu": false,
    "dockerImage": "cr.internal.supervisely.com/ovalbee-internal/nodes/clip-index:0.0.24"
  },
  "automation": {
    "requires_configuration": true,
    "isolated": true,
    "manual_run": true,
    "schedule": "*/10 * * * *",
    "discard_history": "on_success"
  }
}