> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.ovalbee.com/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.ovalbee.com/_mcp/server. # Consensus review > Show every incoming item to every reviewer and route it to the status they agree on. An item the panel cannot settle goes to the unresolved dataset. ## Overview Shows every incoming image or video to every reviewer, who each route it to a configured status by pressing a button, and files the item under the status their votes decide. A new node starts with **Accept** and **Reject**. Each status either gets a new visible dataset on the first run, named **Accepted** for Accept, **Rejected** for Reject and after the status itself for any other, or routes to a dataset chosen in the flow editor. Reviewers never edit annotations: use [Labeling](/node-library/nodes/label-labeling) for that, and [Review](/node-library/nodes/label-review) for a step where the reviewers split the work and each item is judged once. ```mermaid flowchart LR source["Source dataset"] --> pool["Awaiting consensus"] pool --> q1["Reviewer 1 queue"] --> b1["Reviewer 1 ballot"] pool --> q2["Reviewer 2 queue"] --> b2["Reviewer 2 ballot"] pool --> q3["Reviewer 3 queue"] --> b3["Reviewer 3 ballot"] b1 & b2 & b3 -.-> tally{"Agreement reached?"} tally -->|yes| outcome["Winning status dataset"] tally -->|no| none["Unresolved dataset"] ``` One reviewer is a legitimate setup, and is how an expert step settles what a larger panel could not. ## How it works An item's position is dataset membership and nothing else, so a vote needs no state of its own. An item being voted on sits on a hidden pool that belongs to no reviewer and is a member of every reviewer's queue at the same time. Counting the ballots that hold an item counts its votes. Claims a batch from the source dataset onto the pool, and from there into every reviewer's queue at once. The item belongs to several datasets and nothing is copied. A reviewer pressing a status button casts a vote: the item leaves their queue for a hidden ballot recording the status they chose, while the pool still holds it and the others still see it. Each scheduled run tallies the ballots. A status that satisfies the **Agreement rule** wins and the item moves there from the pool. If everyone has voted without a clear leader, or the item passed the time limit, it goes to the **Unresolved dataset** instead. Either way the item is dropped from every queue and ballot, so a reviewer who has not voted stops seeing an item that is already decided. > **Note** > > **Queue size** is how many items one reviewer holds, and also the total taken from the source, because everyone is given the same items. > **Note** > > An item that passes the time limit goes to the **Unresolved dataset**, whether or not any votes were cast — returning it to the source would re-offer it to the same panel that already failed to settle it, and the votes cast so far are dropped with it. > **Note** > > The reviewer queues take the ontology of the source dataset, so each vote is recorded against it. A source dataset with no ontology still runs, and its votes are recorded without one. ## Inputs **`Source dataset`** `dataset` — required Source dataset the assets put up for review are drawn from. Whatever the panel settles leaves it for the matching status dataset. Key: `SOURCE_DATASET`. --- **`Outcome statuses`** `labeling_outcomes` — required, default: \[\{'id': 'accept', 'label': 'Accept', 'dataset': None}, \{'id': 'reject', 'label': 'Reject', 'dataset': None}] The verdict buttons shown in the review tool, each with the dataset it sends assets to. A verdict left without a dataset gets a new one on the node's first run. Flow variables aren't accepted. Key: `OUTCOMES`. --- **`Unresolved dataset`** `dataset` — required Destination dataset for assets the panel could not settle - reviewers disagreed, or the time limit passed. Key: `UNRESOLVED_DATASET`. --- **`Reviewers`** `users` — required Workspace members who review. Each sees every asset put up for review and votes on it. Key: `REVIEWERS`. --- **`Agreement rule`** `select` — required, default: majority Votes one status needs to win - more than half of the reviewers, or all of them. Key: `AGREEMENT_RULE`. Options: * Majority (`majority`) * Unanimous (`all`) --- **`Queue size`** `integer` — required, default: 10 Number of assets up for review at once. Every reviewer sees all of them, and the next asset is drawn only after one of them is routed. Minimum: `1`. Key: `BATCH_SIZE`. --- **`Time limit per item (minutes)`** `integer` — required, default: 120 How long an item may wait for a decision before it goes to the unresolved dataset. Minimum: `1`. Key: `CLAIM_TTL_MINUTES`. --- ## Outputs **`Reviewer queues`** `object` Internal map of user IDs to their hidden review queue datasets. Hidden from the flow editor. Key: `USERS_DATASETS`. --- **`Outcome datasets`** `object` Internal list of allowed status dataset IDs and button labels. Hidden from the flow editor. Key: `OUTCOME_DATASETS`. --- **`Consensus datasets`** `object` Internal voting pool and per-reviewer ballot datasets. Hidden from the flow editor. Key: `CONSENSUS_DATASETS`. --- **`Status`** `object` Compact per-reviewer status table object. Hidden from the flow editor. Key: `STATUS`. --- ## JSON config Machine-readable node interface for automation and advanced usage. #### Show JSON ```json { "name": "Consensus review", "description": "Show every incoming item to every reviewer and route it to the status they agree on. An item the panel cannot settle goes to the unresolved dataset.", "category": "Label", "namespace": "ovalbee", "templateKey": "label/consensus_review", "version": "v2", "inputs": [ { "key": "SOURCE_DATASET", "label": "Source dataset", "type": "dataset", "description": "Source dataset the assets put up for review are drawn from. Whatever the panel settles leaves it for the matching status dataset.", "required": true, "default": null, "visibleWhen": null, "options": { "creatable": false } }, { "key": "OUTCOMES", "label": "Outcome statuses", "type": "labeling_outcomes", "description": "The verdict buttons shown in the review tool, each with the dataset it sends assets to. A verdict left without a dataset gets a new one on the node's first run.", "required": true, "default": [ { "id": "accept", "label": "Accept", "dataset": null }, { "id": "reject", "label": "Reject", "dataset": null } ], "visibleWhen": null, "options": { "allowVariables": false } }, { "key": "UNRESOLVED_DATASET", "label": "Unresolved dataset", "type": "dataset", "description": "Destination dataset for assets the panel could not settle - reviewers disagreed, or the time limit passed.", "required": true, "default": null, "visibleWhen": null }, { "key": "REVIEWERS", "label": "Reviewers", "type": "users", "description": "Workspace members who review. Each sees every asset put up for review and votes on it.", "required": true, "default": null, "visibleWhen": null }, { "key": "AGREEMENT_RULE", "label": "Agreement rule", "type": "select", "description": "Votes one status needs to win - more than half of the reviewers, or all of them.", "required": true, "default": "majority", "visibleWhen": null, "options": { "options": [ { "label": "Majority", "value": "majority" }, { "label": "Unanimous", "value": "all" } ] } }, { "key": "BATCH_SIZE", "label": "Queue size", "type": "number", "description": "Number of assets up for review at once. Every reviewer sees all of them, and the next asset is drawn only after one of them is routed.", "required": true, "default": 10, "visibleWhen": null, "options": { "type": "integer", "min": 1 } }, { "key": "CLAIM_TTL_MINUTES", "label": "Time limit per item (minutes)", "type": "number", "description": "How long an item may wait for a decision before it goes to the unresolved dataset.", "required": true, "default": 120, "visibleWhen": null, "options": { "type": "integer", "min": 1 } } ], "outputs": [ { "key": "USERS_DATASETS", "label": "Reviewer queues", "type": "object", "description": "Internal map of user IDs to their hidden review queue datasets.", "artifact": false, "badge": false, "badgeOpens": null, "hidden": true }, { "key": "OUTCOME_DATASETS", "label": "Outcome datasets", "type": "object", "description": "Internal list of allowed status dataset IDs and button labels.", "artifact": false, "badge": false, "badgeOpens": null, "hidden": true }, { "key": "CONSENSUS_DATASETS", "label": "Consensus datasets", "type": "object", "description": "Internal voting pool and per-reviewer ballot datasets.", "artifact": false, "badge": false, "badgeOpens": null, "hidden": true }, { "key": "STATUS", "label": "Status", "type": "object", "description": "Compact per-reviewer status table object.", "artifact": false, "badge": false, "badgeOpens": null, "hidden": true } ], "automation": { "requires_configuration": true, "schedule": "*/15 * * * *", "discard_history": "on_success" }, "widgets": { "widget": { "id": "table", "settings": { "data": { "type": "variable", "value": "self.outputs.STATUS" } } }, "secondaryWidget": { "id": "open-labeling-tool", "settings": { "name": { "type": "input", "value": "Open review tool" } } } } } ```