Node LibraryDatasets

Image quality

Overview

Measures every image of a dataset from its pixels: how sharp it is, how it is exposed, whether it is a single flat colour, and whether the file is broken or cut short. Exposure is read from the two ends of each image’s brightness — the lightest and darkest parts of the frame, on the brightest of the three colour channels — so neither a dark background nor a dark-coloured subject is mistaken for a badly exposed shot. None of that needs annotations, so images nobody labelled are covered exactly like the rest.

No single image can tell a dark subject from too little light, so the report ranks and compares rather than pronouncing a defect. The frames it does flag outright are the ones where information is demonstrably gone (highlights clipped across a large part of the frame) or where the image is exposed unlike the rest of its own dataset, which in a fixed inspection rig is a sign the capture conditions changed.

The result is one report with the distributions, the counts and previews of the images at each extreme. It can be passed to Dataset insights through its Findings from other nodes input. The badge counts the images measured on this run. Reach for this node when a dataset is new, partly labelled, or scraped, and you want to know what the files themselves are worth before paying anyone to label them.

Dataset Image quality Report

How it works

1

Lists the dataset’s images, without downloading any file and without reading a single annotation.

2

Checks which of those images have never been measured before.

3

Downloads only those images in batches and measures each one in a single pass over a downscaled copy, keeping the numbers per image for later runs.

4

Reads back the numbers for the whole dataset, earlier measurements included, and builds the report.

FAQ

It downloads every image of the dataset. Measurements are kept per image, so later runs download only images that were never measured, and a rerun over an unchanged dataset downloads nothing and shows 0 processed on the badge.

No. The cutoffs are applied to the stored measurements when the report is built, so a rerun with other values reuses them.

The run fails when the dataset holds no images. Videos and other assets in the dataset are ignored.

Inputs

Dataset
datasetRequired

Dataset whose images are measured. Key: DATASET.

Dark frame cutoff
integerDefaults to 100Required

List an image among the darkest frames when the brightness of its lightest 5% of pixels (the 95th percentile, 0-255) falls below this. Lower it to report fewer images. Brightness is read from the brightest of the three colour channels, so a dark red or dark brown subject is not mistaken for a dark photograph, and from the lightest part of the frame, so a black background is not either. Minimum: 0. Maximum: 255. Step: 1. Key: LOW_EXPOSURE_THRESHOLD.

Blown highlight cutoff
integerDefaults to 90Required

Second half of the blown-highlight rule. An image is reported when more than a tenth of it has reached the top of the scale and lost its detail there, and the brightness of its darkest 5% of pixels (the 5th percentile, 0-255) is also above this - meaning nothing in the frame is left in shadow. Raise it to report only the most washed-out images. Clipping on its own is not enough: a product shot on a white backdrop clips without the photograph being wrong. Minimum: 0. Maximum: 255. Step: 1. Key: HIGH_EXPOSURE_THRESHOLD.

Outputs

Report
report

Sharpness and exposure distributions, broken and single-colour image counts, and previews of the images at each extreme. Shown as an artifact. Key: REPORT_ID.

Images processed
string

How many images were measured on this run. Shown on the node as a badge. Key: IMAGES_PROCESSED.

Report

Overview shows three counts: the images, the unusable ones (broken, cut short or a single flat colour), and the ones worth a look — dark, blown or blurred, each image counted once however many of those it is. Under them a ranked list names every kind of flag with its image count and share of the dataset, and opens the images behind it. Possible problems shows the broken and blank files with a gallery. Those are certain, and everything else in the report is a measurement to read rather than a verdict. Sharpness and Exposure each show a distribution and, under it, one section per rule: the images that crossed the rule, or - when none did — the ones nearest to it, ranked, so a section is never a bare “nothing found”. The two exposure cutoffs are settings, but neither the blur cutoff nor the exposure fence is, because neither sharpness nor brightness has an absolute scale — each is a robust fence computed in log space from the dataset being measured, and the report prints what it came out as. The exposure fence is placed only on a dataset lit consistently enough for one, and says so when it declines. Resolutions plots every image by its pixel size, which is the one thing no annotation-based report can show for an unlabelled image. All images is the per-image table. No section is ever hidden.

JSON config

Machine-readable node interface for automation and advanced usage.

{
  "name": "Image quality",
  "description": "Measure sharpness, exposure and broken files from the pixels alone, for every image including the ones nobody labelled. The first run downloads every image of the dataset. What it measured is kept per image, so later runs download only new files.",
  "category": "Datasets",
  "namespace": null,
  "templateKey": "dataset_ops/image_quality",
  "version": "v1",
  "inputs": [
    {
      "key": "DATASET",
      "label": "Dataset",
      "type": "dataset",
      "description": "Dataset whose images are measured.",
      "required": true,
      "default": null,
      "visibleWhen": null,
      "options": {
        "creatable": false
      }
    },
    {
      "key": "LOW_EXPOSURE_THRESHOLD",
      "label": "Dark frame cutoff",
      "type": "number",
      "description": "List an image among the darkest frames when the brightness of its lightest 5% of pixels (the 95th percentile, 0-255) falls below this. Lower it to report fewer images. Brightness is read from the brightest of the three colour channels, so a dark red or dark brown subject is not mistaken for a dark photograph, and from the lightest part of the frame, so a black background is not either.",
      "required": true,
      "default": 100,
      "visibleWhen": null,
      "options": {
        "type": "integer",
        "min": 0,
        "max": 255,
        "step": 1,
        "slider": false
      }
    },
    {
      "key": "HIGH_EXPOSURE_THRESHOLD",
      "label": "Blown highlight cutoff",
      "type": "number",
      "description": "Second half of the blown-highlight rule. An image is reported when more than a tenth of it has reached the top of the scale and lost its detail there, and the brightness of its darkest 5% of pixels (the 5th percentile, 0-255) is also above this - meaning nothing in the frame is left in shadow. Raise it to report only the most washed-out images. Clipping on its own is not enough: a product shot on a white backdrop clips without the photograph being wrong.",
      "required": true,
      "default": 90,
      "visibleWhen": null,
      "options": {
        "type": "integer",
        "min": 0,
        "max": 255,
        "step": 1,
        "slider": false
      }
    }
  ],
  "outputs": [
    {
      "key": "REPORT_ID",
      "label": "Report",
      "type": "report",
      "kind": "dataset_intelligence_findings_v1",
      "description": "Sharpness and exposure distributions, broken and single-colour image counts, and previews of the images at each extreme.",
      "artifact": true,
      "badge": false,
      "badgeOpens": null,
      "hidden": false
    },
    {
      "key": "IMAGES_PROCESSED",
      "label": "Images processed",
      "type": "string",
      "description": "How many images were measured on this run.",
      "artifact": false,
      "badge": true,
      "badgeOpens": null,
      "hidden": false
    }
  ],
  "widgets": {
    "widget": {
      "id": "report-preview",
      "settings": {
        "reportAsset": {
          "type": "variable",
          "value": "self.outputs.REPORT_ID"
        },
        "name": {
          "type": "input",
          "value": "Open quality report"
        }
      }
    }
  }
}

References