Skip to navigation
Node libraryDatasets

Dataset insights

Overview

Reads the statistics of a dataset and writes a short report saying what they mean for training: what is wrong, what follows from it, and what is still unknown. Each finding gets a severity and is backed by a chart, a table, or images from the report.

Annotation quality charts the same numbers. What it cannot do is judge them, because the judgement depends on the dataset: a 6:1 frame threatens small objects, a 16:9 frame is ordinary. This node makes that call per run instead of baking it into a template. Connect the reports of other analysis nodes, such as Annotation quality, Image quality, Find annotation gaps, Similar pairs, Label diagnostics, Embedding anomalies, Evaluate object detection or Evaluate prelabeling efficiency, to Findings from other nodes, and their findings are ranked and written up in the same report. When several of them count the same problem, for example an evaluation and a prelabeling check both finding unlabeled objects, it gets one section with the other counts named in it.

Currently supports only image datasets. The badge shows how many findings the report holds.

Dataset Annotation quality Dataset insights

Prerequisites

RequirementWhat you need
Workspace creditsThe report text is written by the language model of the OvalBee instance and charged to the workspace’s credits. On a self-hosted instance the admin must set an LLM token in the instance settings, and nothing is charged.
Analysis reports (optional)Reports from other analysis nodes that ran on the same dataset, for Findings from other nodes.

How it works

1

Measures the dataset’s annotations, reusing whatever the shared statistics store already holds — the same store Annotation quality uses, so nothing is computed twice.

2

Turns the statistics into findings. The questions are generated from the shape of the data, so a dataset with no tags is never asked about tags. Findings from the connected reports are added to the list.

3

Renders every finding that has a visual form into a chart, table, or image gallery.

4

When a connected evaluation says a model and the labels disagree, shows the model a few of those images, plus a few frames picked at random, with the labels and the predictions drawn side by side. It looks at the objects and says which side is wrong, so the report can tell bad labels from a weak model. Labels are called unreliable only when the random frames show errors too.

5

Asks the model to rank every finding by severity, then to write the report around those files in that order. It writes prose and references, never a number into a data file.

6

Checks the report and sends it back for a rewrite when something fails, for example a number that no finding supports. Drops the unused charts, then uploads.

FAQ

Statistics measured from the dataset, such as counts, class and tag names, and the findings and text of the connected reports. When a connected evaluation flags disagreements with the labels, up to six of those images and six random frames are sent too, with the labels and the predictions drawn on them. The request goes through the OvalBee API to the model the instance is configured with.

The report is checked before it is published. When the model’s text still fails the check after three attempts, the run fails rather than publish a wrong report. A failed model call also stops the run: the instance has no LLM token configured (its admin must add one in the instance settings), the workspace has no credits left for the month, or the model returned an error.

Only findings the node can read are folded in. A report whose findings cannot be read is skipped and shows a dash in the sources table, while a 0 there means that node ran and found nothing.

Inputs

Dataset
datasetRequired

Dataset to analyse. Its images and annotations are measured, and the report is written about it. Key: DATASET.

Findings from other nodes
report

Reports from other analysis nodes to fold into the findings list. Reports that carry no findings are skipped. Values come from DATASET. Key: REPORT_IDS.

Outputs

Report
report

Asset ID containing the dataset insights report. Shown as an artifact. Key: REPORT_ID.

Headline
string

Number of findings in this report, shown as a badge. Shown on the node as a badge. Key: HEADLINE.

Summary
object

Findings-per-severity table the node card shows. Hidden from the flow editor. Key: SUMMARY.

Report

The report opens on an Overview tab with a verdict on the dataset and the steps to take first, then metric cards and every finding in severity order: Critical stops training until it is fixed, Warning blocks something you are about to do, Notice is real and bounded, and Passed was checked and is not a problem. Each finding has its own section with the evidence behind it and a line saying why it got its level. Findings that come only from a connected report get a tab named after that node, and a Questions tab can appear when a finding depends on how the dataset will be used. A table lists where the findings came from.

JSON config

Machine-readable node interface for automation and advanced usage.

{
  "name": "Dataset insights",
  "description": "Turns the dataset's statistics into findings written in plain language, each one backed by a chart in the report.",
  "category": "Datasets",
  "namespace": null,
  "templateKey": "dataset_ops/dataset_insights",
  "version": "v1",
  "inputs": [
    {
      "key": "DATASET",
      "label": "Dataset",
      "type": "dataset",
      "description": "Dataset to analyse. Its images and annotations are measured, and the report is written about it.",
      "required": true,
      "default": null,
      "visibleWhen": null,
      "options": {
        "creatable": false
      }
    },
    {
      "key": "REPORT_IDS",
      "label": "Findings from other nodes",
      "type": "report",
      "kind": "dataset_intelligence_findings_v1",
      "description": "Reports from other analysis nodes to fold into the findings list. Reports that carry no findings are skipped.",
      "required": false,
      "default": null,
      "visibleWhen": null,
      "options": {
        "minItems": 0,
        "ref": "DATASET"
      }
    }
  ],
  "outputs": [
    {
      "key": "REPORT_ID",
      "label": "Report",
      "type": "report",
      "description": "Asset ID containing the dataset insights report.",
      "artifact": true,
      "badge": false,
      "badgeOpens": null,
      "hidden": false
    },
    {
      "key": "HEADLINE",
      "label": "Headline",
      "type": "string",
      "description": "Number of findings in this report, shown as a badge.",
      "artifact": false,
      "badge": true,
      "badgeOpens": null,
      "hidden": false
    },
    {
      "key": "SUMMARY",
      "label": "Summary",
      "type": "object",
      "description": "Findings-per-severity table the node card shows.",
      "artifact": false,
      "badge": false,
      "badgeOpens": null,
      "hidden": true
    }
  ],
  "widgets": {
    "widget": {
      "id": "table",
      "settings": {
        "data": {
          "type": "variable",
          "value": "self.outputs.SUMMARY"
        }
      }
    },
    "secondaryWidget": {
      "id": "report-preview",
      "settings": {
        "reportAsset": {
          "type": "variable",
          "value": "self.outputs.REPORT_ID"
        },
        "name": {
          "type": "input",
          "value": "Open insights"
        }
      }
    }
  }
}