Shapes and Objects

​

Annotation Types Overview

​

Rectangle (Bounding Box)

​

Description

​

When to Use

• Object detection tasks
• When exact boundaries aren’t critical
• Quick initial annotation
• Objects with roughly rectangular extent
• YOLO, Faster R-CNN, and similar detector training

​

Drawing Methods

1. Activate rectangle tool (Shift+B)
2. Click and drag from one corner to the opposite corner
3. Release to finish

1. Select “By 4 Points” in the tool options
2. Click the four extreme points:
   • Topmost point of the object
   • Bottommost point
   • Leftmost point
   • Rightmost point
3. CVAT fits a bounding box automatically

​

Properties

• Coordinates: (x1, y1, x2, y2) where (x1, y1) is top-left and (x2, y2) is bottom-right
• Rotation: Can be rotated after creation (not axis-aligned anymore)
• Attributes: Supports all attribute types
• In tracks: Interpolates position and size

​

Best Practices

​

Polygon

​

Description

​

When to Use

• Instance segmentation tasks
• Objects with irregular boundaries
• Precise boundary annotation required
• Semantic segmentation with discrete objects
• Mask R-CNN and similar model training

​

Drawing

1. Activate polygon tool (Shift+P)
2. Click to place each vertex along the object boundary
3. Place vertices at corners and direction changes
4. Close the polygon:
   • Double-click the first vertex, or
   • Press N, or
   • Click the first point again

​

Editing

1. Select the polygon
2. Click on an edge where you want to insert a point
3. A new vertex is created

1. Select the polygon
2. Drag any vertex to reposition it

1. Select the polygon
2. Right-click a vertex → “Delete point”, or
3. Select vertex and press Del

• Right-click edge → Add point
• Right-click vertex → Delete point

​

Properties

• Coordinates: Array of (x, y) vertices
• Minimum points: 3 (triangle)
• Attributes: Supports all attribute types
• In tracks: Interpolates vertex positions (can morph shape)

​

Best Practices

• Zoom in for pixel-accurate boundaries
• Follow edges precisely, especially for training segmentation models
• Close gaps carefully at occlusion boundaries
• Optimize points: Use enough for accuracy, not excessive
• Clockwise/counterclockwise: Maintain consistent direction for all objects

​

Polyline

​

Description

​

When to Use

• Lane markings
• Trajectories and paths
• Borders and boundaries
• Linear infrastructure (roads, power lines)
• Centerlines
• Motion paths

​

Drawing

1. Activate polyline tool (Shift+L)
2. Click to place vertices along the line
3. Finish the line:
   • Press N, or
   • Double-click the last point

​

Editing

• Add/remove vertices
• Drag vertices to reposition
• Extend or shorten the line

​

Properties

• Coordinates: Array of (x, y) vertices (ordered)
• Minimum points: 2 (line segment)
• Open shape: No interior, just the path
• In tracks: Interpolates as a flexible path

​

Best Practices

• Place vertices at direction changes and curves
• For lanes, annotate the center line unless specified otherwise
• Maintain consistent direction (e.g., always left-to-right)
• Extend to frame boundaries if the feature continues beyond the visible area

​

Points

​

Description

​

When to Use

• Very small objects (< 5 pixels)
• Object centers or key points (single point per object)
• Counting (e.g., cells, people in crowds)
• Sparse annotation
• Click-based detection

​

Drawing

1. Activate points tool (Shift+.)
2. Optionally set the number of points to place
3. Click to place each point
4. Press N after the last point (or auto-completes after specified count)

​

Multiple Points in One Annotation

• Each click adds a point to the same annotation
• Useful for objects with multiple markers
• All points share the same label and attributes

​

Properties

• Coordinates: Array of (x, y) positions
• Minimum points: 1
• In tracks: Interpolates point positions

​

Best Practices

​

Ellipse

​

Description

​

When to Use

• Circular or oval objects (balls, wheels, heads)
• When rotation matters
• Faster than polygons, more accurate than rectangles
• Objects with elliptical cross-sections

​

Drawing

1. Activate ellipse tool
2. Click and drag to define center and initial size
3. Move mouse to adjust shape and rotation
4. Click to finish

​

Editing

• Drag center: Move the ellipse
• Drag edge handles: Resize radii
• Rotation handles: Rotate the ellipse

​

Properties

• Coordinates: Center (cx, cy), radii (rx, ry), and rotation angle
• Shape: Always elliptical
• In tracks: Interpolates position, size, and rotation

​

Best Practices

• Fit tightly to the object boundary
• Use for circular objects where rectangles waste space
• Consider rotation for tilted objects
• Alternative to polygons when approximation is acceptable

​

Cuboid

​

Description

​

When to Use

• Vehicles (cars, trucks, buses)
• Buildings and structures
• Furniture
• Packages and boxes
• Any object where 3D extent matters
• Monocular 3D object detection

​

Drawing Methods

1. Activate cuboid tool
2. Draw the front face as a rectangle
3. Drag to extend the depth (rear face)
4. Adjust perspectives and edges

1. Select “By 4 Points” method
2. Click four corners that define the visible faces
3. CVAT constructs the 3D cuboid

​

Structure

• Front face: 4 points
• Rear face: 4 points
• Edges: Connecting lines

​

Editing

• Drag faces: Move front or rear face
• Drag vertices: Adjust individual corners
• Drag edges: Adjust perspective
• Whole cuboid: Drag center to move entire object

​

Properties

• Coordinates: 8 vertices (x, y) for each corner
• Projection: Perspective projection of 3D box onto 2D image
• Attributes: Supports dimensions (width, height, length)
• In tracks: Interpolates all 8 vertices

​

Best Practices

​

Skeleton

​

Description

​

When to Use

• Human pose estimation
• Animal pose
• Articulated objects (robots, machinery)
• Hand keypoints
• Facial landmarks
• Any structured keypoint annotation

​

Setup

1. Keypoints: List of named points (e.g., “nose”, “left_eye”, “right_shoulder”)
2. Edges: Connections between keypoints (e.g., “left_shoulder” to “left_elbow”)

{
  "name": "person",
  "type": "skeleton",
  "sublabels": [
    {"name": "nose"},
    {"name": "left_eye"},
    {"name": "right_eye"},
    {"name": "left_ear"},
    {"name": "right_ear"},
    {"name": "left_shoulder"},
    {"name": "right_shoulder"},
    // ... more keypoints
  ],
  "svg": "<skeleton edges definition>"
}

​

Drawing

1. Activate skeleton tool
2. Select the skeleton label
3. Click to place each keypoint in order:
   • Follow the defined keypoint sequence
   • Place each point at the anatomical location
4. CVAT automatically draws edges between connected keypoints

​

Editing

• Move keypoints: Drag any keypoint to reposition
• Visibility states: Mark keypoints as visible/occluded/not visible
• Add attributes: Each keypoint can have visibility attributes
• Edges: Automatically update when keypoints move

​

Properties

• Structure: N keypoints with M edges
• Keypoint states: Visible, occluded, outside frame
• Coordinates: (x, y) for each keypoint
• In tracks: Interpolates keypoint positions
• Topology: Fixed edge structure defined in label

​

Common Skeleton Formats

• Full body pose with major joints
• Nose, eyes, ears, shoulders, elbows, wrists, hips, knees, ankles

• Similar to COCO, slight variations

• Wrist + 4 fingers × 5 joints each

• Facial landmarks for expression analysis

​

Best Practices

​

Mask

​

Description

​

When to Use

• Pixel-perfect segmentation required
• Complex, irregular boundaries
• Objects with holes or intricate details
• Semantic segmentation
• Medical imaging
• High-precision applications

​

Drawing

1. Activate mask tool
2. Select a label
3. The brush toolbox appears with tools:

• Paint to add mask pixels
• Adjustable size and shape (circle/square)
• Click and drag to paint

• Remove mask pixels
• Same size/shape controls as brush

• Add a polygonal region to the mask
• Click vertices, close polygon
• Region is filled and added to mask

• Remove a polygonal region from the mask
• Useful for quickly removing large areas

​

Brush Settings

• Size: 1-100 pixels
• Form: Circle or square
• Shortcuts:
  • [ - Decrease brush size
  • ] - Increase brush size

​

Editing

1. Select the mask annotation
2. The brush toolbox reappears
3. Use brush/eraser to modify
4. Click “Done” or press N to finish editing

​

Properties

• Storage: Run-length encoded (RLE) for efficiency
• Resolution: Pixel-level
• Binary: Each pixel is either in or out of the mask
• In tracks: Each keyframe stores a full mask, interpolation is not applied (masks appear/disappear)

​

Performance

• Large masks: May slow down the canvas
• Many masks: Consider using polygons if possible
• Video masks: Require keyframes on each annotated frame

​

Best Practices

​

Tag

​

Description

​

When to Use

• Image classification
• Scene classification
• Frame attributes (weather, time of day)
• Event detection (frame contains event X)
• Multi-label frame classification
• Metadata annotation

​

Creating Tags

1. Click the Tag tool in the controls sidebar
2. Select a tag label
3. Click Tag to apply to current frame
4. The tag appears in the objects sidebar

​

Properties

• No spatial extent: Tags have no position or shape
• Frame-specific: Each tag applies to one frame
• Attributes: Tags support attributes
• Label-based: Each tag has a label (tag type)

​

Tag Annotation Workspace

1. Select workspace → “Tag Annotation”
2. Interface shows:
   • Large image/frame display
   • Tag buttons for quick application
   • Keyboard shortcuts for each tag
3. Navigate frames quickly with arrow keys
4. Apply tags with number keys or clicks

​

Best Practices

• Use consistent criteria for tag application
• Document tag definitions in project guidelines
• Use attributes for tag variants (e.g., “weather” tag with “sunny”/“rainy” attribute)
• Leverage keyboard shortcuts for rapid annotation

​

Object Types: Shape vs Track

​

Shape

• Exists on a single frame only
• Used for static images or one-time appearances
• No temporal continuity
• ObjectType: SHAPE

​

Track

• Exists across multiple frames
• Represents the same object over time
• Supports interpolation between keyframes
• Maintains a single client ID across frames
• ObjectType: TRACK

​

Track Keyframes

• Keyframes: Frames where you manually set the object’s position/shape
• Interpolated frames: Frames between keyframes where CVAT automatically estimates position
• Mark a frame as keyframe with K

​

Choosing the Right Annotation Type

​

Export Formats

​

Next Steps