Skip to navigation
Node libraryModels

AI search

Dockerfile
First run can take longer while the worker pulls the configured Docker image.

Overview

Searches images that are already indexed and collects the matches into a dataset.

There are two modes. AI Search ranks images by how well they match a text prompt you write, for example a red car at night. Diverse takes no prompt and spreads the selection across a dataset instead, returning one image from each group of visually similar ones.

Every rerun writes a new version of the node’s own hidden dataset, LAST_RUN, holding that run’s matches, and the badge on the node shows how many images were selected and opens it. Pick a destination dataset when you want the matches to stay somewhere permanent.

Dataset AI index embeddings AI search Matching images

Prerequisites

RequirementWhat you need
Indexed imagesImages indexed by AI index embeddings with CLIP, PE-Core, or SigLIP 2. Images added after the last indexing run are invisible to the search until they are indexed too.
Annotated objects (optional)Objects annotated before the dataset was indexed, needed when Search in is set to Objects only.

How it works

1

Resolves which embeddings to search, using the model you picked or, on Auto, the first indexed model that covers the images in scope.

2

Selects the matches: AI Search ranks images against your prompt and keeps those clearing the confidence threshold, while Diverse groups the images and takes the one closest to the middle of each group.

3

Writes a new version of the node’s own hidden dataset for this run and adds the selected images to it, together with their annotations and classes.

4

Adds the same images to the destination dataset when you set one, and shows the number selected on the node.

FAQ

Only for Diverse and for object search, which both refuse to run without one. AI Search over whole images works without a source dataset and then searches every indexed image in the workspace.

The images the matching objects sit in, not the objects themselves. An image that holds several matching objects is still returned once.

Leave Auto unless you indexed the same images several times and want a specific one searched. Auto prefers PE-Core, then SigLIP 2, then CLIP, and settles on the first of them that actually covers the images in scope. Choosing a model the images were never indexed with stops the run.

Every image comes back, because there is nothing left to spread the selection over.

The usual reasons are an empty text prompt in AI Search, or no embeddings covering the images you asked for. Index the dataset first and the search finds it.

No. Images and their annotations are added to the new datasets and the source dataset stays as it was.

Inputs

From dataset
dataset

Source dataset the search runs over. Leave empty to search every indexed image in the workspace, which only AI Search over images can do. Key: DATASET.

To dataset
dataset

Destination dataset the matches are added to. Leave empty to keep them only in the short-lived dataset this run creates. Key: DST_DATASET.

Search mode
selectDefaults to ai_searchRequired

AI Search ranks indexed images by how closely they match your text prompt. Diverse instead spreads the selection across the dataset, taking one image from each group of similar ones. Key: SEARCH_MODE.

Options:

  • AI Search (ai_search)
  • Diverse (diverse)
Search in
selectDefaults to imagesRequired

Whole images matched against your prompt, or the individual annotated objects inside them. Object search returns the images those objects sit in and needs a source dataset. Visible when SEARCH_MODE is ai_search. Key: ENTITY_MODE.

Options:

  • Images only (images)
  • Objects only (objects)
Text prompt
textRequired

What you are looking for, in plain words, for example a red car at night. AI Search fails when it is empty. Visible when SEARCH_MODE is ai_search. Key: TEXT_PROMPT.

Confidence threshold
numberDefaults to 0.25Required

Lowest match score an image needs to be returned. Raise it for fewer but closer matches. Minimum: 0. Maximum: 1. Step: 0.01. Visible when SEARCH_MODE is ai_search. Key: CONFIDENCE_THRESHOLD.

Maximum results
integerDefaults to 100

Most images AI Search returns. Fewer come back when not enough of them clear the confidence threshold. Minimum: 1. Maximum: 10000. Step: 1. Visible when SEARCH_MODE is ai_search. Key: TOP_K.

Maximum results
integerDefaults to 100Required

Number of images Diverse selects, each one standing in for a different group of visually similar images. Minimum: 1. Step: 1. Visible when SEARCH_MODE is diverse. Key: DIVERSE_TOP_K.

Embedding model
selectDefaults to autoRequired

Model whose embeddings are searched, which has to be one the images were already indexed with. Auto uses whichever indexed model covers them, preferring PE-Core, then SigLIP 2, then CLIP. Key: EMBEDDING_MODEL.

Options:

  • Auto (auto)
  • CLIP (clip)
  • PE-Core (pecore)
  • SigLIP 2 (siglip2)

Outputs

Dataset
dataset

Key: DATASET.

Last run
dataset

Hidden dataset containing matches from this run. A rerun writes a new version of it. Shown as an artifact. Key: LAST_RUN.

Selected images
string

Number of image assets attached to the new dataset. Shown on the node as a badge. Selecting the badge opens LAST_RUN. Key: SELECTED_IMAGES.

JSON config

Machine-readable node interface for automation and advanced usage.

{
  "name": "AI search",
  "description": "Select embedded images by text relevance or k-means centroids.",
  "category": "Models",
  "namespace": "ovalbee",
  "templateKey": "models/clip/ai_search",
  "version": "v1",
  "inputs": [
    {
      "key": "DATASET",
      "label": "From dataset",
      "type": "dataset",
      "description": "Source dataset the search runs over. Leave empty to search every indexed image in the workspace, which only AI Search over images can do.",
      "required": false,
      "default": null,
      "visibleWhen": null,
      "options": {
        "creatable": false
      }
    },
    {
      "key": "DST_DATASET",
      "label": "To dataset",
      "type": "dataset",
      "description": "Destination dataset the matches are added to. Leave empty to keep them only in the short-lived dataset this run creates.",
      "required": false,
      "default": null,
      "visibleWhen": null
    },
    {
      "key": "SEARCH_MODE",
      "label": "Search mode",
      "type": "select",
      "description": "AI Search ranks indexed images by how closely they match your text prompt. Diverse instead spreads the selection across the dataset, taking one image from each group of similar ones.",
      "required": true,
      "default": "ai_search",
      "visibleWhen": null,
      "options": {
        "control": "tabs",
        "options": [
          {
            "label": "AI Search",
            "value": "ai_search"
          },
          {
            "label": "Diverse",
            "value": "diverse"
          }
        ]
      }
    },
    {
      "key": "ENTITY_MODE",
      "label": "Search in",
      "type": "select",
      "description": "Whole images matched against your prompt, or the individual annotated objects inside them. Object search returns the images those objects sit in and needs a source dataset.",
      "required": true,
      "default": "images",
      "visibleWhen": {
        "key": "SEARCH_MODE",
        "operator": "equals",
        "value": "ai_search"
      },
      "options": {
        "control": "tabs",
        "options": [
          {
            "label": "Images only",
            "value": "images"
          },
          {
            "label": "Objects only",
            "value": "objects"
          }
        ]
      }
    },
    {
      "key": "TEXT_PROMPT",
      "label": "Text prompt",
      "type": "text",
      "description": "What you are looking for, in plain words, for example a red car at night. AI Search fails when it is empty.",
      "required": true,
      "default": "",
      "visibleWhen": {
        "key": "SEARCH_MODE",
        "operator": "equals",
        "value": "ai_search"
      }
    },
    {
      "key": "CONFIDENCE_THRESHOLD",
      "label": "Confidence threshold",
      "type": "number",
      "description": "Lowest match score an image needs to be returned. Raise it for fewer but closer matches.",
      "required": true,
      "default": 0.25,
      "visibleWhen": {
        "key": "SEARCH_MODE",
        "operator": "equals",
        "value": "ai_search"
      },
      "options": {
        "type": "float",
        "min": 0,
        "max": 1,
        "step": 0.01
      }
    },
    {
      "key": "TOP_K",
      "label": "Maximum results",
      "type": "number",
      "description": "Most images AI Search returns. Fewer come back when not enough of them clear the confidence threshold.",
      "required": false,
      "default": 100,
      "visibleWhen": {
        "key": "SEARCH_MODE",
        "operator": "equals",
        "value": "ai_search"
      },
      "options": {
        "type": "integer",
        "min": 1,
        "max": 10000,
        "step": 1,
        "slider": false
      }
    },
    {
      "key": "DIVERSE_TOP_K",
      "label": "Maximum results",
      "type": "number",
      "description": "Number of images Diverse selects, each one standing in for a different group of visually similar images.",
      "required": true,
      "default": 100,
      "visibleWhen": {
        "key": "SEARCH_MODE",
        "operator": "equals",
        "value": "diverse"
      },
      "options": {
        "type": "integer",
        "min": 1,
        "step": 1,
        "slider": false
      }
    },
    {
      "key": "EMBEDDING_MODEL",
      "label": "Embedding model",
      "type": "select",
      "description": "Model whose embeddings are searched, which has to be one the images were already indexed with. Auto uses whichever indexed model covers them, preferring PE-Core, then SigLIP 2, then CLIP.",
      "required": true,
      "default": "auto",
      "visibleWhen": null,
      "options": {
        "options": [
          {
            "label": "Auto",
            "value": "auto"
          },
          {
            "label": "CLIP",
            "value": "clip"
          },
          {
            "label": "PE-Core",
            "value": "pecore"
          },
          {
            "label": "SigLIP 2",
            "value": "siglip2"
          }
        ]
      }
    }
  ],
  "outputs": [
    {
      "key": "DATASET",
      "label": "Dataset",
      "type": "dataset",
      "description": "",
      "artifact": false,
      "badge": false,
      "badgeOpens": null,
      "hidden": false
    },
    {
      "key": "LAST_RUN",
      "label": "Last run",
      "type": "dataset",
      "description": "Hidden dataset containing matches from this run. A rerun writes a new version of it.",
      "artifact": true,
      "badge": false,
      "badgeOpens": null,
      "hidden": false
    },
    {
      "key": "SELECTED_IMAGES",
      "label": "Selected images",
      "type": "string",
      "description": "Number of image assets attached to the new dataset.",
      "artifact": false,
      "badge": true,
      "badgeOpens": "LAST_RUN",
      "hidden": false
    }
  ],
  "runtime": {
    "type": "docker",
    "devices": [
      "cpu"
    ],
    "dockerImage": "cr.internal.supervisely.com/ovalbee-internal/nodes/clip-index:0.0.28"
  },
  "automation": {
    "isolated": true,
    "requires_configuration": true,
    "manual_run": true,
    "discard_history": "on_success"
  }
}