Skip to main content
Consensus enables you to measure inter-annotator agreement for quality assurance. Configure consensus thresholds and compute agreement scores to identify items that need review.

​
Get consensus summary

Retrieve a summary of consensus scores for a project. 
const summary = await avala.consensus.getSummary('project-uid');

​
Parameters

​
projectUid
string
required
The unique identifier of the project

​
Returns

​
meanScore
number
Mean consensus score across all items
​
medianScore
number
Median consensus score
​
minScore
number
Minimum consensus score
​
maxScore
number
Maximum consensus score
​
totalItems
number
Total number of items evaluated
​
itemsWithConsensus
number
Number of items with consensus agreement
​
scoreDistribution
object | null
Distribution of scores across ranges
​
byTaskName
unknown[] | null
Consensus breakdown by task name

​
List consensus scores

Retrieve paginated consensus scores for individual items. 
const scores = await avala.consensus.listScores('project-uid', {
  limit: 50,
  cursor: 'next-cursor'
});

​
Parameters

​
projectUid
string
required
The unique identifier of the project
​
options.limit
number
Maximum number of scores to return per page
​
options.cursor
string
Cursor for pagination

​
Returns

Returns a cursor-paginated list of consensus score objects.
​
items
ConsensusScore[]
Array of consensus score objects
​
items[].uid
string
Unique identifier for the score record
​
items[].datasetItemUid
string
Dataset item UID being evaluated
​
items[].taskName
string
Task name
​
items[].scoreType
string | null
Type of score (e.g., ‘iou’, ‘agreement’)
​
items[].score
number
Consensus score value
​
items[].annotatorCount
number
Number of annotators who labeled this item
​
items[].details
object | null
Additional details about the score
​
items[].createdAt
string | null
ISO 8601 timestamp of when the score was computed
​
nextCursor
string | null
Cursor for the next page of results
​
previousCursor
string | null
Cursor for the previous page of results
​
hasMore
boolean
Whether more results are available

​
Compute consensus

Trigger consensus computation for a project. 
const result = await avala.consensus.compute('project-uid');

​
Parameters

​
projectUid
string
required
The unique identifier of the project

​
Returns

​
status
string
Status of the computation request
​
message
string
Description of the computation result

​
Get consensus config

Retrieve the consensus configuration for a project. 
const config = await avala.consensus.getConfig('project-uid');

​
Parameters

​
projectUid
string
required
The unique identifier of the project

​
Returns

​
uid
string
Unique identifier for the config
​
iouThreshold
number
Intersection over Union threshold for object matching
​
minAgreementRatio
number
Minimum agreement ratio to consider consensus (0.0 to 1.0)
​
minAnnotations
number
Minimum number of annotations required for consensus
​
createdAt
string | null
ISO 8601 timestamp of creation
​
updatedAt
string | null
ISO 8601 timestamp of last update

​
Update consensus config

Update the consensus configuration for a project. 
const config = await avala.consensus.updateConfig('project-uid', {
  iouThreshold: 0.5,
  minAgreementRatio: 0.8,
  minAnnotations: 3
});

​
Parameters

​
projectUid
string
required
The unique identifier of the project
​
options.iouThreshold
number
New IOU threshold for object matching (0.0 to 1.0)
​
options.minAgreementRatio
number
New minimum agreement ratio (0.0 to 1.0)
​
options.minAnnotations
number
New minimum number of annotations required

​
Returns

Returns the updated consensus config object.

Build docs developers (and LLMs) love

Get started for freeTalk to us