Skip to navigation
Node libraryModels

AI index embeddings

Dockerfile
First run can take longer while the worker pulls the configured Docker image.

Overview

Embeds the images of one dataset, and the objects annotated on them, into the workspace’s embedding index, then keeps that index current as the dataset changes. Similar pairs, Embedding map, Label diagnostics and Embedding anomalies all read what this node wrote, so run it on a dataset before any of them.

Two kinds of embedding are built side by side. The semantic one (CLIP, PE-Core or SigLIP 2) also understands text, so a written prompt can find matching images and objects. The visual one (DINOv3) compares how things look rather than what they mean, which is what near-duplicate and outlier work needs.

Built with DINOv3: the visual embedding is Meta’s DINOv3, used under the DINOv3 License.

The badge on the node counts the dataset’s images in the index.

Dataset AI index embeddings Similar pairs

Prerequisites

RequirementWhat you need
Annotated images (optional)Images that already carry annotations, for object-level embeddings. Without them only whole images are indexed.

How it works

1

Reads every image of the dataset together with its annotation.

2

Embeds the images and annotated objects that the workspace index is still missing, a batch at a time.

3

Writes the new embeddings, once for the semantic model and once more for DINOv3 when visual embedding is on.

4

Marks the objects as used by this dataset, and takes the dataset off objects that none of its versions annotates any more. An object no dataset uses is deleted.

5

Records what it indexed, so a later run over an unchanged dataset finishes without rescanning.

FAQ

Each reader looks in the index of the model you pick on it, so index with that model first. Embedding map clusters on DINOv3 by default, which needs Enable visual embedding on. Label diagnostics works on objects, so it needs Enable object indexing on.

Nothing is recomputed while the dataset is untouched: the run ends right away and the badge keeps its count. Once images or annotations change, only what is missing gets embedded, everything already indexed is reused, and objects whose annotation is gone from every version of the dataset are removed from the index.

No. Once it is configured, the node checks its dataset every ten minutes on its own. You can also run it by hand.

No. The first run binds the node to that dataset, that embedding model, the visual embedding setting and the object embedding method, and changing any of them afterwards stops the run with an error. Add a second node for a second dataset or a different model. Turning object indexing off later is allowed: it stops new objects from being embedded and leaves the ones already indexed in place. Turning it back on makes the next run a full one, which embeds the objects added in the meantime.

The node follows it. When the dataset input moves to a newer version, for example because an upstream import reran and committed one, the next run indexes that version, and only images and objects the index is still missing get embedded. Objects already indexed stay in the index while an older version still has them, so AI search and Label diagnostics pointed at an older version the node indexed still find them. A version the node never indexed, for example one committed and replaced between two of its runs, may be missing objects.

When no version of the dataset annotates the object any more, the dataset lets go of it, and once no dataset uses it the object is deleted from the index. An object stays while an older version still has the annotation it comes from. Every version holding an annotation shares it, so an object deleted from the annotation itself leaves all of them at once. Deleting the dataset removes its objects as well, unless another dataset still uses them. That happens only when the dataset is erased: after it has left the trash and the erase period set in Platform configuration has passed. Objects indexed before this node started following dataset versions are never removed. With object indexing off, objects are neither added nor removed.

The node checks that a new input is a version of the dataset it indexed before, and it reads that by the version it indexed last. Once the dataset is deleted, that version cannot be read and the node cannot tell whether the input is the same dataset. Add a new node for the dataset you want indexed now.

No. Images and annotations are read only. The embeddings live next to the dataset, in an index shared by the whole workspace. An image already indexed through another dataset is not embedded again.

Every object of the image’s annotation, taken from the box around its shape, whatever the shape is. An object whose box reaches past the edge of its image is skipped, and so is an annotation that does not record the image size.

Images only. Videos and other assets are not indexed. And because the workspace keeps one DINOv3 index, every node writing visual embeddings into it has to use the same object embedding method.

The service that stores the embeddings is unreachable or not configured for this instance. The dataset is left as it was, so the node picks up where it stopped once the service answers again.

Inputs

Dataset
datasetRequired

Dataset to index, with its images and their annotated objects. A node stays bound to the dataset it first indexes and follows that dataset’s versions, so indexing another dataset takes another node. Key: DATASET.

Embedding model
selectDefaults to clipRequired

Model used for image, object, and text-compatible embeddings. CLIP is the fastest and the weakest at matching objects to a text prompt. PE-Core and SigLIP 2 are both much stronger, but which one leads depends on the imagery, and PE-Core costs an order of magnitude more time per object. Index a sample both ways and compare before committing a large dataset. Key: EMBEDDING_MODEL.

Options:

  • CLIP (clip)
  • PE-Core (pecore)
  • SigLIP 2 (siglip2)
Enable visual embedding
booleanDefaults to trueRequired

Also embed each image with DINOv3, and each annotated object when object indexing is on. DINOv3 matches by appearance (shape, texture, colour) rather than meaning, and cannot be searched with text. Embedding map reads these by default, Label diagnostics and Similar pairs read them only when their Embedding model is DINOv3. No other node uses them, so turn this off when none of those three will read this index: it saves one model pass per image and per object. Fixed at the node’s first run. Key: ENABLE_VISUAL_EMBEDDING.

Enable object indexing
booleanDefaults to trueRequired

Also embed every annotated object, not just whole images, for both the semantic and visual (DINOv3) collections. Turn off to index only whole images. Key: ENABLE_OBJECT_INDEXING.

Object embedding method
selectDefaults to cropRequired

Crop gives every bounding box its own full-resolution pass. Regions runs DINOv3 once per whole image and pools patch features inside each box, costing one pass per image instead of one per object, but leaving each object only the feature cells its area covers. Crop separated classes better at every object size measured, and by a wide margin for small objects, so prefer Regions only to trade accuracy for indexing speed on object-dense images. All DINOv3 nodes in one workspace write to the same index, so they all have to use the same method. Visible when ENABLE_OBJECT_INDEXING is True. Key: OBJECT_EMBEDDING_MODE.

Options:

  • Regions (ROI pooling) (regions)
  • Crop (crop)
Batch size
integerDefaults to 32Required

Number of images downloaded and embedded together. Minimum: 1. Maximum: 256. Step: 1. Key: BATCH_SIZE.

Outputs

Indexed images
string

Total number of the dataset’s images in the index. Shown on the node as a badge. Key: INDEXED_IMAGES.

Last indexed
number

Number of images the last run embedded. An image already in the workspace index, for example from another dataset, is reused and not counted. 0 when the dataset did not change since the run before. Key: LAST_INDEXED.

Internal state
object

Internal scan state. Do not edit. Hidden from the flow editor. Key: STATE.

JSON config

Machine-readable node interface for automation and advanced usage.

{
  "name": "AI index embeddings",
  "description": "Embed a dataset's images and annotated objects into the workspace index, and keep it current as the dataset changes. Built with DINOv3.",
  "category": "Models",
  "namespace": "ovalbee",
  "templateKey": "models/clip/ai_index_embeddings",
  "version": "v1",
  "inputs": [
    {
      "key": "DATASET",
      "label": "Dataset",
      "type": "dataset",
      "description": "Dataset to index, with its images and their annotated objects. A node stays bound to the dataset it first indexes and follows that dataset's versions, so indexing another dataset takes another node.",
      "required": true,
      "default": null,
      "visibleWhen": null,
      "options": {
        "creatable": false
      }
    },
    {
      "key": "EMBEDDING_MODEL",
      "label": "Embedding model",
      "type": "select",
      "description": "Model used for image, object, and text-compatible embeddings. CLIP is the fastest and the weakest at matching objects to a text prompt. PE-Core and SigLIP 2 are both much stronger, but which one leads depends on the imagery, and PE-Core costs an order of magnitude more time per object. Index a sample both ways and compare before committing a large dataset.",
      "required": true,
      "default": "clip",
      "visibleWhen": null,
      "options": {
        "options": [
          {
            "label": "CLIP",
            "value": "clip"
          },
          {
            "label": "PE-Core",
            "value": "pecore"
          },
          {
            "label": "SigLIP 2",
            "value": "siglip2"
          }
        ]
      }
    },
    {
      "key": "ENABLE_VISUAL_EMBEDDING",
      "label": "Enable visual embedding",
      "type": "boolean",
      "description": "Also embed each image with DINOv3, and each annotated object when object indexing is on. DINOv3 matches by appearance (shape, texture, colour) rather than meaning, and cannot be searched with text. Embedding map reads these by default; Label diagnostics and Similar pairs read them only when their Embedding model is DINOv3. No other node uses them, so turn this off when none of those three will read this index: it saves one model pass per image and per object. Fixed at the node's first run.",
      "required": true,
      "default": true,
      "visibleWhen": null
    },
    {
      "key": "ENABLE_OBJECT_INDEXING",
      "label": "Enable object indexing",
      "type": "boolean",
      "description": "Also embed every annotated object, not just whole images, for both the semantic and visual (DINOv3) collections. Turn off to index only whole images.",
      "required": true,
      "default": true,
      "visibleWhen": null
    },
    {
      "key": "OBJECT_EMBEDDING_MODE",
      "label": "Object embedding method",
      "type": "select",
      "description": "Crop gives every bounding box its own full-resolution pass. Regions runs DINOv3 once per whole image and pools patch features inside each box, costing one pass per image instead of one per object, but leaving each object only the feature cells its area covers. Crop separated classes better at every object size measured, and by a wide margin for small objects, so prefer Regions only to trade accuracy for indexing speed on object-dense images. All DINOv3 nodes in one workspace write to the same index, so they all have to use the same method.",
      "required": true,
      "default": "crop",
      "visibleWhen": {
        "key": "ENABLE_OBJECT_INDEXING",
        "operator": "equals",
        "value": true
      },
      "options": {
        "options": [
          {
            "label": "Regions (ROI pooling)",
            "value": "regions"
          },
          {
            "label": "Crop",
            "value": "crop"
          }
        ]
      }
    },
    {
      "key": "BATCH_SIZE",
      "label": "Batch size",
      "type": "number",
      "description": "Number of images downloaded and embedded together.",
      "required": true,
      "default": 32,
      "visibleWhen": null,
      "options": {
        "type": "integer",
        "min": 1,
        "max": 256,
        "step": 1
      }
    }
  ],
  "outputs": [
    {
      "key": "INDEXED_IMAGES",
      "label": "Indexed images",
      "type": "string",
      "description": "Total number of the dataset's images in the index.",
      "artifact": false,
      "badge": true,
      "badgeOpens": null,
      "hidden": false
    },
    {
      "key": "LAST_INDEXED",
      "label": "Last indexed",
      "type": "number",
      "description": "Number of images the last run embedded. An image already in the workspace index, for example from another dataset, is reused and not counted. 0 when the dataset did not change since the run before.",
      "artifact": false,
      "badge": false,
      "badgeOpens": null,
      "hidden": false
    },
    {
      "key": "STATE",
      "label": "Internal state",
      "type": "object",
      "description": "Internal scan state. Do not edit.",
      "artifact": false,
      "badge": false,
      "badgeOpens": null,
      "hidden": true
    }
  ],
  "runtime": {
    "type": "docker",
    "devices": [
      "cpu"
    ],
    "dockerImage": "ovalbee/node-clip-index:0.0.35"
  },
  "automation": {
    "requires_configuration": true,
    "isolated": true,
    "manual_run": true,
    "schedule": "*/10 * * * *",
    "discard_history": "on_success"
  }
}