Node LibraryLabel

Labeling gate

Overview

Checks every image’s annotations in the source dataset against a set of rules, then moves the image into a passed or failed dataset depending on the result.

Place it right after a labeling step to gate submissions before human review: images that pass move on to review, images that fail can go back to labeling.

Annotated images Labeling gate Reviewing Failed images

How it works

1

Reads annotations from the source dataset.

2

Flags an image carrying no labels at all, or labels that fall outside the configured bounds.

3

Compares each label’s area and each class’s object count against the median for that class across the images of the same run, and flags whatever sits further away than the configured deviation.

4

Moves every image, with its annotations, into the passed or failed dataset and reports how many landed in each.

FAQ

A dataset holds one annotation per image. If an image ends up with more than one anyway, the newest is checked and the others are ignored.

The other images checked in the same run. The node takes the median area and object count per class over the whole batch it is processing, so the image that stands out from its own submission is the one that fails. Nothing outside the run is read, and the passed dataset is only a destination. The trade-off: a submission that is uniformly wrong sets its own norm and passes.

Each is off until you give it a deviation. Leaving Max area deviation or Max count deviation empty skips that check. A class is also skipped until it reaches Minimum reference samples measurements in the same run — labels for the area check, images for the count check — so a small batch checks little.

It always goes to the failed dataset. Fail images with no objects governs images whose annotation is empty — a legitimate negative sample — not images that carry no annotation at all, so turning it off never lets unannotated work through.

Yes. Min box side and Max duplicate IoU compare the bounding box around each object, whatever its shape.

It checks whatever is in the source dataset at that moment. Checked images have already left the source, so a rerun only sees images that arrived since. The passed and failed datasets are written as you picked them and keep growing, with no new version per run.

The image is routed to the failed dataset and the error is written to the run log.

It only checks images. Videos in the source dataset are left in place. It doesn’t record per-image failure reasons beyond the run log, and it doesn’t notify anyone or create a report.

Inputs

Source dataset
datasetRequired

Source dataset the freshly submitted images are drawn from. Every checked image leaves it for the passed or the failed dataset. Key: SOURCE_DATASET.

Passed dataset
datasetRequired

Destination dataset for images that clear every enabled check. Key: PASSED_DATASET.

Failed dataset
datasetRequired

Destination dataset for images that fail at least one enabled check. Key: FAILED_DATASET.

Fail images with no objects
booleanDefaults to trueRequired

Fails an image whose annotation holds no objects. An image left without any annotation fails whatever the rules say. Key: CHECK_NO_OBJECTS.

Allowed objects per image
range

Fails an image whose number of annotated objects falls outside this range. Empty means the check is off. Minimum: 0. Step: 1. Key: OBJECTS_RANGE.

Allowed area (%)
range

Fails an image holding an object that covers a smaller or larger share of it than this range allows. Empty means the check is off. Minimum: 0. Maximum: 100. Step: 0.5. Key: AREA_RANGE.

Min box side (px)
number

Fails an image holding a bounding box narrower or shorter than this many pixels. Empty means the check is off. Minimum: 1. Key: MIN_BOX_SIDE_PX.

Max duplicate IoU
number

Fails an image where two bounding boxes of one class overlap more than this, usually the same object annotated twice. Empty means the check is off. Minimum: 0.01. Maximum: 1. Key: MAX_DUPLICATE_IOU.

Max area deviation (%)
integer

Fails an image holding an object more than this percent away from the median area of its class in the run. Empty means the check is off. Minimum: 1. Step: 1. Key: MAX_AREA_DEVIATION.

Max count deviation (%)
integer

Fails an image holding more or fewer objects of a class than this percent away from the median count for that class in the run. Empty means the check is off. Minimum: 1. Step: 1. Key: MAX_COUNT_DEVIATION.

Minimum reference samples
integerDefaults to 5Required

Number of examples a class needs in the run before the deviation checks judge it. A median taken over fewer of them is too easy to skew. Minimum: 1. Key: MIN_REFERENCE_SAMPLES.

Outputs

QC Summary
string

Compact pass/fail count for the latest run. Shown on the node as a badge. Key: QC_SUMMARY.

Passed Dataset
dataset

ID of the dataset that received the passing images. Shown as an artifact. Key: PASSED_DATASET.

Failed Dataset
dataset

ID of the dataset that received the failing images. Shown as an artifact. Key: FAILED_DATASET.

JSON config

Machine-readable node interface for automation and advanced usage.

{
  "name": "Labeling gate",
  "description": "Run rule-based checks on freshly submitted annotations and route each image to a passed or failed dataset before human review.",
  "category": "Label",
  "namespace": "ovalbee",
  "templateKey": "label/labeling_gate",
  "version": "v1",
  "inputs": [
    {
      "key": "SOURCE_DATASET",
      "label": "Source dataset",
      "type": "dataset",
      "description": "Source dataset the freshly submitted images are drawn from. Every checked image leaves it for the passed or the failed dataset.",
      "required": true,
      "default": null,
      "visibleWhen": null,
      "options": {
        "creatable": false
      }
    },
    {
      "key": "PASSED_DATASET",
      "label": "Passed dataset",
      "type": "dataset",
      "description": "Destination dataset for images that clear every enabled check.",
      "required": true,
      "default": null,
      "visibleWhen": null
    },
    {
      "key": "FAILED_DATASET",
      "label": "Failed dataset",
      "type": "dataset",
      "description": "Destination dataset for images that fail at least one enabled check.",
      "required": true,
      "default": null,
      "visibleWhen": null
    },
    {
      "key": "CHECK_NO_OBJECTS",
      "label": "Fail images with no objects",
      "type": "boolean",
      "description": "Fails an image whose annotation holds no objects. An image left without any annotation fails whatever the rules say.",
      "required": true,
      "default": true,
      "visibleWhen": null
    },
    {
      "key": "OBJECTS_RANGE",
      "label": "Allowed objects per image",
      "type": "range",
      "description": "Fails an image whose number of annotated objects falls outside this range. Empty means the check is off.",
      "required": false,
      "default": null,
      "visibleWhen": null,
      "options": {
        "type": "integer",
        "min": 0,
        "step": 1
      }
    },
    {
      "key": "AREA_RANGE",
      "label": "Allowed area (%)",
      "type": "range",
      "description": "Fails an image holding an object that covers a smaller or larger share of it than this range allows. Empty means the check is off.",
      "required": false,
      "default": null,
      "visibleWhen": null,
      "options": {
        "min": 0,
        "max": 100,
        "step": 0.5
      }
    },
    {
      "key": "MIN_BOX_SIDE_PX",
      "label": "Min box side (px)",
      "type": "number",
      "description": "Fails an image holding a bounding box narrower or shorter than this many pixels. Empty means the check is off.",
      "required": false,
      "default": null,
      "visibleWhen": null,
      "options": {
        "min": 1
      }
    },
    {
      "key": "MAX_DUPLICATE_IOU",
      "label": "Max duplicate IoU",
      "type": "number",
      "description": "Fails an image where two bounding boxes of one class overlap more than this, usually the same object annotated twice. Empty means the check is off.",
      "required": false,
      "default": null,
      "visibleWhen": null,
      "options": {
        "min": 0.01,
        "max": 1,
        "slider": false
      }
    },
    {
      "key": "MAX_AREA_DEVIATION",
      "label": "Max area deviation (%)",
      "type": "number",
      "description": "Fails an image holding an object more than this percent away from the median area of its class in the run. Empty means the check is off.",
      "required": false,
      "default": null,
      "visibleWhen": null,
      "options": {
        "type": "integer",
        "min": 1,
        "step": 1
      }
    },
    {
      "key": "MAX_COUNT_DEVIATION",
      "label": "Max count deviation (%)",
      "type": "number",
      "description": "Fails an image holding more or fewer objects of a class than this percent away from the median count for that class in the run. Empty means the check is off.",
      "required": false,
      "default": null,
      "visibleWhen": null,
      "options": {
        "type": "integer",
        "min": 1,
        "step": 1
      }
    },
    {
      "key": "MIN_REFERENCE_SAMPLES",
      "label": "Minimum reference samples",
      "type": "number",
      "description": "Number of examples a class needs in the run before the deviation checks judge it. A median taken over fewer of them is too easy to skew.",
      "required": true,
      "default": 5,
      "visibleWhen": null,
      "options": {
        "type": "integer",
        "min": 1
      }
    }
  ],
  "outputs": [
    {
      "key": "QC_SUMMARY",
      "label": "QC Summary",
      "type": "string",
      "description": "Compact pass/fail count for the latest run.",
      "artifact": false,
      "badge": true,
      "badgeOpens": null,
      "hidden": false
    },
    {
      "key": "PASSED_DATASET",
      "label": "Passed Dataset",
      "type": "dataset",
      "description": "ID of the dataset that received the passing images.",
      "artifact": true,
      "badge": false,
      "badgeOpens": null,
      "hidden": false
    },
    {
      "key": "FAILED_DATASET",
      "label": "Failed Dataset",
      "type": "dataset",
      "description": "ID of the dataset that received the failing images.",
      "artifact": true,
      "badge": false,
      "badgeOpens": null,
      "hidden": false
    }
  ]
}