Skip to navigation
Node libraryDatasets

Compute stats

Overview

Parses a dataset’s annotations once and stores what they contain — object classes, geometries, areas, box sides, and tags — as columnar files in workspace storage. Other nodes, such as Data Quality Control and Filter images, read those files instead of downloading and parsing the annotations again.

Statistics are keyed to the annotation, and annotations are immutable, so every annotation is parsed once ever. A rerun over an unchanged dataset computes nothing, and datasets that share annotations — copies, splits, merges — share the stored rows.

Running this node is never required: consumers compute whatever the store is missing on their own. Place it in a flow to pay the one-time parsing cost up front, for example on a schedule or before a large filtering step.

Dataset Compute stats Statistics store

How it works

1

Lists the dataset’s images with their annotation records, without downloading annotation files.

2

Asks the statistics store which of those annotation ids it has never seen.

3

Downloads and parses only the missing annotations, in one batch.

4

Appends the new rows as parquet files and folds the accumulated files together when too many have piled up.

FAQ

No. Consumers compute whatever the store is missing on their own. Running this node first only moves the one-time parsing cost to a moment you choose.

Editing labels produces a new annotation with a new id, so the next run computes rows for the new annotations only. Rows of annotations that left the dataset stay in the store, and consumers ignore rows that no current annotation refers to.

Every annotation of the dataset is already in the store. That is the normal state after the first run.

Inputs

Dataset
datasetRequired

Dataset whose annotations are measured. Annotations measured by an earlier run are skipped, so a rerun over an unchanged dataset does no work. Key: DATASET.

Outputs

Statistics summary
string

How many annotations were computed on this run. Shown on the node as a badge. Key: STATS_SUMMARY.

JSON config

Machine-readable node interface for automation and advanced usage.

{
  "name": "Compute stats",
  "description": "Compute per-annotation and per-object statistics for a dataset and store them as columnar files that other nodes can query cheaply.",
  "category": "Datasets",
  "namespace": null,
  "templateKey": "dataset_ops/compute_stats",
  "version": "v1",
  "inputs": [
    {
      "key": "DATASET",
      "label": "Dataset",
      "type": "dataset",
      "description": "Dataset whose annotations are measured. Annotations measured by an earlier run are skipped, so a rerun over an unchanged dataset does no work.",
      "required": true,
      "default": null,
      "visibleWhen": null,
      "options": {
        "creatable": false
      }
    }
  ],
  "outputs": [
    {
      "key": "STATS_SUMMARY",
      "label": "Statistics summary",
      "type": "string",
      "description": "How many annotations were computed on this run.",
      "artifact": false,
      "badge": true,
      "badgeOpens": null,
      "hidden": false
    }
  ]
}