> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.ovalbee.com/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.ovalbee.com/_mcp/server. # Labeling > People label images or videos here. Each person gets a small set from the pool. A button in the tool, such as Submit or Skip, sends the item to its dataset. ## Overview Manages your labeling work, like a team lead. You give it a dataset of images or videos (assets) and a team of labelers, and it runs the work for you: * creates a small personal queue for each labeler, so nobody picks work by hand and two people never get the same asset; * refills a queue as soon as its labeler finishes an asset; * sends each finished asset to the right dataset, based on the button the labeler pressed; * takes back work that someone holds for too long, or that a removed labeler left behind. You decide which buttons labelers see. In **Decision buttons**, each row is a button name and the dataset it sends assets to. A new node starts with two: **Submit** for finished work and **Skip** for assets the labeler sets aside. While an asset is in someone's queue, it isn't in the source dataset, so the source only holds work that nobody has taken yet. Once every required input is set, the node runs by itself every 15 minutes to take back unfinished work. New assets in the source dataset don't wait for that run: they go straight to queues with free space. To check annotations that already exist without changing them, use [Review](/node-library/nodes/label-review). ```mermaid flowchart LR source["Source dataset"] --> queue["Labeler queue"] --> outcome["Outcome dataset of the clicked status"] ``` ## How it works Creates a personal queue for each person in **Labelers** and fills it up to **Batch size** from the source dataset, in the order the assets arrived there, or highest anomaly score first when **Queue order** asks for it. Moves an asset and its annotation to the button's dataset as soon as the labeler presses a decision button in the annotation tool. The labeler's queue refills right away. Returns assets that stayed in a queue longer than **Return unclaimed work after (minutes)** to the source dataset. Sends the queue of anyone you remove from **Labelers** back to the source dataset. Adds the classes being labeled to each outcome dataset's ontology. The **Submit** button's dataset, or the only one when there's a single button, becomes the **Labeled dataset** output, so later nodes can train on it or send more work to it. ## FAQ #### Which datasets does a new node create? A button without a dataset gets a new one on the node's first run. It's named **Labeled** for Submit, **Skipped** for Skip, and after the button name for any other. The node then saves that dataset in the button's row. You can also pick an existing dataset, or type a new name in the dataset picker to create one. #### Why did the run fail with a Classes error? Each queue is tied to an ontology (the set of classes and tags), so every submission is saved against it. The run fails when **Classes** has no ontology. This happens when nothing is picked and the source dataset has no ontology, when class names are typed without an ontology, or when the picked ontology was deleted. #### Why did the run fail on Decision buttons? Every button needs a name. The run also fails when: * two buttons have the same name (letter case doesn't matter); * two buttons use the same dataset; * there are no buttons at all; * a destination dataset no longer exists. #### Why doesn't a button I added show up in the annotation tool? The buttons come from the node's last run. After you change **Decision buttons**, run the node or wait for the next scheduled run. Then reload the tool. #### What happens to the ontology of an outcome dataset? Each run looks for classes being labeled that the outcome dataset's ontology doesn't have yet. These are the ticked classes, or all of them when nothing is ticked. The run adds them as a new ontology version and switches the dataset to it. An outcome dataset without an ontology gets the one picked in **Classes**. This way all outcome datasets share the same classes. #### When does unfinished work go back to the source dataset? The timer starts when an asset enters a labeler's queue, not at their last edit. The 15-minute run returns expired assets, so an asset can stay up to 15 minutes past the limit. Annotations the labeler saved but didn't submit go back with the asset. The returned asset is placed after the assets already waiting in the source dataset. #### Why is a labeler's queue not full? A queue refills when its labeler submits an asset and when new assets arrive in the source dataset. It can only take what the source dataset has. The 15-minute run is a backup, not the normal way work arrives. #### When does letting labelers add classes and tags take effect? As soon as the labeler reloads the annotation tool. No node run is needed. New classes and tags are added as a new version to the ontology of the labeler's queue and of each outcome dataset, and those datasets switch to it. The source dataset's ontology doesn't change. #### What happens when I delete the node? Every asset still in a labeler's queue goes back to the source dataset. The outcome datasets keep everything already submitted. #### Can one step label images and videos together? No. The run fails when the source dataset and the labelers' queues hold both images and videos. ## Inputs **`Source dataset`** `dataset` — required Dataset with the assets to label. Assets a labeler doesn't submit come back here. Key: `SOURCE_DATASET`. --- **`Decision buttons`** `labeling_outcomes` — required, default: \[\{'id': 'submit', 'label': 'Submit', 'dataset': None}, \{'id': 'skip', 'label': 'Skip', 'dataset': None}] Buttons shown in the annotation tool, each with a destination dataset. Clicking a button moves the current item there, with its annotations. Flow variables aren't accepted. Key: `OUTCOMES`. --- **`Labelers`** `users` — required Workspace members who label. Each person gets a personal queue from the source dataset. Key: `LABELERS`. --- **`Classes`** `ontology` — required Classes and tags to label with. Defaults to the source dataset's ontology. You can pick another one or create one here. Tick some entries to show only those to labelers, or tick nothing to show all of them. The queue still uses the whole ontology, so the annotations stay valid in it, and entries added to the ontology later are shown. Values come from `SOURCE_DATASET`. Key: `CLASSES`. --- **`Labelers can add classes and tags`** `boolean` — required, default: false Let labelers create classes and tags in the annotation tool. They're added to the ontology as a new version, and this node's queues and outcome datasets switch to it. Flow variables aren't accepted. Key: `ALLOW_NEW_CLASSES_AND_TAGS`. --- **`Queue order`** `select` — required, default: arrival Which assets go into a labeler's queue first, and the order the tool shows them in. Highest anomaly score first reads the score Embedding anomalies saves to each image, so the hardest cases come first. Images without a score follow, oldest first. Flow variables aren't accepted. Key: `QUEUE_ORDER`. Options: * Oldest first (`arrival`) * Highest anomaly score first (`anomaly_score`) --- **`Batch size`** `integer` — required, default: 10 How many assets a labeler holds at once. The queue refills to this number after each submit and on every node run. Minimum: `1`. Key: `BATCH_SIZE`. --- **`Return unclaimed work after (minutes)`** `integer` — required, default: 120 Assets that stay in a labeler's queue longer than this without being submitted go back to the source dataset. This keeps unfinished work from getting stuck. Minimum: `1`. Key: `CLAIM_TTL_MINUTES`. --- ## Outputs **`User queues`** `object` Internal map of user IDs to their hidden labeling queue datasets. Hidden from the flow editor. Key: `USERS_DATASETS`. --- **`Outcome datasets`** `object` Internal list of allowed outcome dataset IDs and button labels. Hidden from the flow editor. Key: `OUTCOME_DATASETS`. --- **`Labeled dataset`** `dataset` Dataset where submitted assets land: the Submit button's dataset, or the only one when there's a single button. Empty when there are several buttons and none is Submit. Key: `LABELED_DATASET`. --- **`Status`** `object` Compact per-labeler queue status table object. Hidden from the flow editor. Key: `STATUS`. --- ## Widgets The node card shows a live chart of the work: assets in progress in labelers' queues, assets labeled and waiting for review, and assets a review accepted or rejected. Below the chart is a list of labelers with their acceptance rate and the number of items each one labeled. Click a labeler to see only their numbers. The chart updates a few seconds after every **Submit**, **Accept** or **Reject**, without a node run. **Open annotation tool** opens your own queue. It works only if you're in **Labelers**, and only after the node has run once since you were added. Labelers can also open their queue from the **Labeling** page. The tool opens the image or video interface to match what the queue holds. ## JSON config Machine-readable node interface for automation and advanced usage. #### Show JSON ```json { "name": "Labeling", "description": "People label images or videos here. Each person gets a small set from the pool. A button in the tool, such as Submit or Skip, sends the item to its dataset.", "category": "Label", "namespace": "ovalbee", "templateKey": "label/labeling", "version": "v2", "inputs": [ { "key": "SOURCE_DATASET", "label": "Source dataset", "type": "dataset", "description": "Dataset with the assets to label. Assets a labeler doesn't submit come back here.", "required": true, "default": null, "visibleWhen": null, "options": { "creatable": false } }, { "key": "OUTCOMES", "label": "Decision buttons", "type": "labeling_outcomes", "description": "Buttons shown in the annotation tool, each with a destination dataset. Clicking a button moves the current item there, with its annotations.", "required": true, "default": [ { "id": "submit", "label": "Submit", "dataset": null }, { "id": "skip", "label": "Skip", "dataset": null } ], "visibleWhen": null, "options": { "allowVariables": false } }, { "key": "LABELERS", "label": "Labelers", "type": "users", "description": "Workspace members who label. Each person gets a personal queue from the source dataset.", "required": true, "default": null, "visibleWhen": null }, { "key": "CLASSES", "label": "Classes", "type": "ontology", "description": "Classes and tags to label with. Defaults to the source dataset's ontology. You can pick another one or create one here. Tick some entries to show only those to labelers, or tick nothing to show all of them. The queue still uses the whole ontology, so the annotations stay valid in it, and entries added to the ontology later are shown.", "required": true, "default": null, "visibleWhen": null, "options": { "ref": "SOURCE_DATASET", "selectable": true, "creatable": true } }, { "key": "ALLOW_NEW_CLASSES_AND_TAGS", "label": "Labelers can add classes and tags", "type": "boolean", "description": "Let labelers create classes and tags in the annotation tool. They're added to the ontology as a new version, and this node's queues and outcome datasets switch to it.", "required": true, "default": false, "visibleWhen": null, "options": { "allowVariables": false } }, { "key": "QUEUE_ORDER", "label": "Queue order", "type": "select", "description": "Which assets go into a labeler's queue first, and the order the tool shows them in. Highest anomaly score first reads the score Embedding anomalies saves to each image, so the hardest cases come first. Images without a score follow, oldest first.", "required": true, "default": "arrival", "visibleWhen": null, "options": { "allowVariables": false, "options": [ { "label": "Oldest first", "value": "arrival" }, { "label": "Highest anomaly score first", "value": "anomaly_score" } ] } }, { "key": "BATCH_SIZE", "label": "Batch size", "type": "number", "description": "How many assets a labeler holds at once. The queue refills to this number after each submit and on every node run.", "required": true, "default": 10, "visibleWhen": null, "options": { "type": "integer", "min": 1 } }, { "key": "CLAIM_TTL_MINUTES", "label": "Return unclaimed work after (minutes)", "type": "number", "description": "Assets that stay in a labeler's queue longer than this without being submitted go back to the source dataset. This keeps unfinished work from getting stuck.", "required": true, "default": 120, "visibleWhen": null, "options": { "type": "integer", "min": 1 } } ], "outputs": [ { "key": "USERS_DATASETS", "label": "User queues", "type": "object", "description": "Internal map of user IDs to their hidden labeling queue datasets.", "artifact": false, "badge": false, "badgeOpens": null, "hidden": true }, { "key": "OUTCOME_DATASETS", "label": "Outcome datasets", "type": "object", "description": "Internal list of allowed outcome dataset IDs and button labels.", "artifact": false, "badge": false, "badgeOpens": null, "hidden": true }, { "key": "LABELED_DATASET", "label": "Labeled dataset", "type": "dataset", "description": "Dataset where submitted assets land: the Submit button's dataset, or the only one when there's a single button. Empty when there are several buttons and none is Submit.", "artifact": false, "badge": false, "badgeOpens": null, "hidden": false }, { "key": "STATUS", "label": "Status", "type": "object", "description": "Compact per-labeler queue status table object.", "artifact": false, "badge": false, "badgeOpens": null, "hidden": true } ], "automation": { "requires_configuration": true, "schedule": "*/15 * * * *", "discard_history": "on_success" }, "widgets": { "widget": { "id": "labeling-stats", "settings": {} }, "secondaryWidget": { "id": "open-labeling-tool", "settings": { "name": { "type": "input", "value": "Open annotation tool" } } } } } ```