Skip to navigation
Node libraryModels

Dataset variance coverage

Dockerfile
First run can take longer while the worker pulls the configured Docker image.

Overview

Measures how many images a subset needs to capture the visual variety of a dataset, and whether choosing them carefully beats picking them at random. It answers questions such as “how much variety would labeling a random 100 images cover?” and “how many fewer images would do the same job if they were chosen deliberately?”.

For subset sizes from 5 images up to Largest subset size, the node chooses subsets five ways: random, k-means representatives, farthest point, D-optimal design and leverage score sampling. It then plots each strategy’s coverage curve against the best any subset could reach. The badge shows how many images the best strategy needs for 90% coverage, next to the number random selection needs. Nothing is written to any dataset.

AI index embeddings Dataset variance coverage Dataset Report

Coverage is measured in embedding space and is evidence of redundancy, not a promise about accuracy. A subset can cover 99% of the variance and still miss a rare class, so check a pruned dataset by training on it.

Prerequisites

RequirementWhat you need
Image embeddingsImages indexed by AI index embeddings with the model chosen in Embedding model.

How it works

1

Loads the vector of every image in the dataset from the selected model’s index. Images with no vector are reported and skipped.

2

Finds the dataset’s principal directions of variation and keeps the number set in Principal components, or as many as explain 90% of the variance.

3

Chooses subsets of each size with every strategy, averaging random and leverage score sampling over Random repeats draws.

4

Fits PCA on each subset alone and reconstructs every image of the dataset from it. Coverage is the variance that fit explains, as a share of what the dataset’s own components explain.

5

Builds the report with the curves, the gain over random and the subset size each strategy needs per coverage level.

FAQ

A subset at 100% recovers the dataset’s main directions of variation as well as PCA on the whole dataset does with the same number of components. A subset of k images spans at most k - 1 directions, so below that count even the best subset stays under 100%. The report draws that ceiling as its own line.

Random draws images uniformly. k-means representatives clusters the images into k groups and takes the image nearest each centre, a cheap stand-in for k-medoids. Farthest point (greedy k-center) keeps adding the image farthest from everything picked so far, so it reaches outliers first. D-optimal design keeps adding the image that most increases the volume the picks span in the principal directions, which matches greedy PCA subset selection until the picks span them all. Leverage score sampling draws images in proportion to how strongly each one alone pins down those directions.

Coverage is floored at zero. Farthest point and D-optimal design start at the edges of the dataset, so their first few picks put the subset’s centre far from the dataset’s, and the fit reconstructs the images worse than the dataset’s plain mean would.

The dataset is evenly spread, so a random sample of that size is already representative. Deliberate selection pays off on uneven datasets, where rare conditions take up a small share of the images and a random sample misses them.

Leave Principal components empty unless you need curves comparable across datasets. Fewer components measure only the coarsest variation, and every strategy reaches it quickly. More components bring in fine detail, which needs larger subsets before any curve flattens.

Yes. Each model spreads the same images differently, so compare runs only within one model.

Inputs

Dataset
datasetRequired

Dataset whose coverage is measured. Images not yet indexed by AI index embeddings with the selected model are skipped. Key: DATASET.

Embedding model
selectDefaults to clipRequired

Model the images were indexed with. Choosing one they were not indexed with stops the run. Key: EMBEDDING_MODEL.

Options:

  • CLIP (clip)
  • PE-Core (pecore)
  • SigLIP 2 (siglip2)
Largest subset size
integerDefaults to 1000Required

The curves are measured from 5 images up to this many. Larger values take longer, mostly in k-means. Minimum: 10. Maximum: 20000. Step: 10. Key: MAX_SUBSET_SIZE.

Principal components
integer

How many of the dataset’s principal directions coverage is measured on. Leave empty to use as many as explain 90% of the dataset’s variance. Minimum: 1. Maximum: 2048. Step: 1. Key: COMPONENTS.

Random repeats
integerDefaults to 5Required

How many independent draws the random and leverage score curves average over. The report shows their spread. Minimum: 1. Maximum: 20. Step: 1. Key: RANDOM_REPEATS.

Outputs

Coverage summary
string

Images the best strategy needs to reach 90% coverage, next to what random selection needs. Shown on the node as a badge. Key: COVERAGE_SUMMARY.

Report
report

Coverage curves for every strategy, the gain over random, and the images each needs per coverage level. Shown as an artifact. Key: REPORT.

Report

The report opens with the conclusion: which strategy reaches 90% coverage with the fewest images, and how that compares with random selection. It then lets you read the subset size you need for the coverage you want.

  • Overview - the run summary, the verdict and what coverage measures.
  • Coverage curves - every strategy’s coverage by subset size against the whole-dataset ceiling, a table of the values with the spread across random draws, and the gain of each strategy over random in percentage points.
  • Images needed - the subset size each strategy needs for 80%, 90%, 95% and 99% coverage.
  • How the strategies choose - one line per strategy.

The report can be passed to Dataset insights through its Findings from other nodes input.

JSON config

Machine-readable node interface for automation and advanced usage.

{
  "name": "Dataset variance coverage",
  "description": "Measure how much of a dataset's visual variety a subset of k images captures, for random and deliberately chosen subsets.",
  "category": "Models",
  "namespace": "ovalbee",
  "templateKey": "models/clip/dataset_variance_coverage",
  "version": "v1",
  "inputs": [
    {
      "key": "DATASET",
      "label": "Dataset",
      "type": "dataset",
      "description": "Dataset whose coverage is measured. Images not yet indexed by AI index embeddings with the selected model are skipped.",
      "required": true,
      "default": null,
      "visibleWhen": null,
      "options": {
        "creatable": false
      }
    },
    {
      "key": "EMBEDDING_MODEL",
      "label": "Embedding model",
      "type": "select",
      "description": "Model the images were indexed with. Choosing one they were not indexed with stops the run.",
      "required": true,
      "default": "clip",
      "visibleWhen": null,
      "options": {
        "options": [
          {
            "label": "CLIP",
            "value": "clip"
          },
          {
            "label": "PE-Core",
            "value": "pecore"
          },
          {
            "label": "SigLIP 2",
            "value": "siglip2"
          }
        ]
      }
    },
    {
      "key": "MAX_SUBSET_SIZE",
      "label": "Largest subset size",
      "type": "number",
      "description": "The curves are measured from 5 images up to this many. Larger values take longer, mostly in k-means.",
      "required": true,
      "default": 1000,
      "visibleWhen": null,
      "options": {
        "type": "integer",
        "min": 10,
        "max": 20000,
        "step": 10,
        "slider": false
      }
    },
    {
      "key": "COMPONENTS",
      "label": "Principal components",
      "type": "number",
      "description": "How many of the dataset's principal directions coverage is measured on. Leave empty to use as many as explain 90% of the dataset's variance.",
      "required": false,
      "default": null,
      "visibleWhen": null,
      "options": {
        "type": "integer",
        "min": 1,
        "max": 2048,
        "step": 1,
        "slider": false
      }
    },
    {
      "key": "RANDOM_REPEATS",
      "label": "Random repeats",
      "type": "number",
      "description": "How many independent draws the random and leverage score curves average over. The report shows their spread.",
      "required": true,
      "default": 5,
      "visibleWhen": null,
      "options": {
        "type": "integer",
        "min": 1,
        "max": 20,
        "step": 1,
        "slider": false
      }
    }
  ],
  "outputs": [
    {
      "key": "COVERAGE_SUMMARY",
      "label": "Coverage summary",
      "type": "string",
      "description": "Images the best strategy needs to reach 90% coverage, next to what random selection needs.",
      "artifact": false,
      "badge": true,
      "badgeOpens": null,
      "hidden": false
    },
    {
      "key": "REPORT",
      "label": "Report",
      "type": "report",
      "kind": "dataset_intelligence_findings_v1",
      "description": "Coverage curves for every strategy, the gain over random, and the images each needs per coverage level.",
      "artifact": true,
      "badge": false,
      "badgeOpens": null,
      "hidden": false
    }
  ],
  "runtime": {
    "type": "docker",
    "devices": [
      "cpu"
    ],
    "dockerImage": "cr.internal.supervisely.com/ovalbee-internal/nodes/clip-index:0.0.27"
  },
  "automation": {
    "isolated": true,
    "requires_configuration": true,
    "manual_run": true,
    "discard_history": "on_success"
  },
  "widgets": {
    "widget": {
      "id": "report-preview",
      "settings": {
        "reportAsset": {
          "type": "variable",
          "value": "self.outputs.REPORT"
        },
        "name": {
          "type": "input",
          "value": "Open coverage"
        }
      }
    }
  }
}