Skip to main content
Tank has comprehensive test coverage with 461 TypeScript tests and 16 Python tests across unit, integration, end-to-end, and performance testing.

​
Running Tests

​
All Tests

pnpm test
Runs all unit and integration tests across the monorepo (TypeScript + Python).

​
Workspace-Specific Tests

# CLI tests only
pnpm test --filter=cli

# Web tests only
pnpm test --filter=web

# Shared package tests only
pnpm test --filter=shared

​
End-to-End Tests

pnpm test:e2e
Requirements:
  • .env.local with real credentials
  • PostgreSQL database accessible
  • Supabase storage configured
E2E tests run sequentially and spawn real CLI processes to test the full user flow.

​
BDD Tests (Playwright)

pnpm test:bdd
Runs behavior-driven development tests using Playwright and Gherkin syntax.

​
Performance Tests

pnpm test:perf
Requirements:
  • Real PostgreSQL 17 database
  • Supabase local or production instance
  • No cached builds (tests measure cold performance)
What it tests:
  • API route latency (p95 budgets)
  • Web route TTFB and FCP
  • Security scan throughput
  • Database query performance
See Performance Testing for methodology.

​
Test Structure

​
TypeScript Tests (Vitest)

Tank uses Vitest for all TypeScript testing. Location pattern: __tests__/*.test.ts colocated with source code Examples:
  • apps/cli/src/__tests__/install.test.ts — CLI install command tests
  • apps/web/lib/__tests__/audit-score.test.ts — Audit scoring logic tests
  • packages/shared/src/__tests__/permissions.test.ts — Permission schema tests
Test counts by area:
  • CLI: 50+ tests (commands, packer, lockfile, linker)
  • Web: 30+ tests (API routes, auth, DB, audit scoring)
  • Shared: 10+ tests (schemas, resolver, validation)
  • E2E: 5 test suites (producer, consumer, admin, integration, on-prem)
  • Performance: 2 test suites (API routes, web routes)

​
Python Tests (pytest)

Tank uses pytest for Python testing. Location pattern: test_*.py files in test directories Examples:
  • python-api/api/analyze/tests/test_analyze.py — Security scan endpoint tests
  • python-api/lib/scan/test_snyk_scanner.py — Snyk integration tests
  • python-api/tests/test_skills/test_skill_corpus.py — Skill corpus validation tests
Test count: 16 Python tests

​
Writing Tests

​
TypeScript Tests

1

Create test file

Place test files in __tests__/ directory next to source:
src/
├── commands/
│   ├── install.ts
│   └── __tests__/
│       └── install.test.ts
2

Write test using Vitest

import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import { installCommand } from '../install';

describe('install command', () => {
  beforeEach(() => {
    // Setup
  });

  afterEach(() => {
    // Cleanup
  });

  it('should install skill from lockfile', async () => {
    // Arrange
    const manifest = { skills: { '@test/skill': '1.0.0' } };
    
    // Act
    const result = await installCommand(manifest);
    
    // Assert
    expect(result.success).toBe(true);
  });

  it('should fail if lockfile hash mismatch', async () => {
    // Test error case
    await expect(installCommand({ invalid: true }))
      .rejects.toThrow('Hash mismatch');
  });
});
3

Run your test

pnpm test --filter=cli -- install.test.ts

​
Python Tests

1

Create test file

Name files with test_ prefix:
lib/scan/
├── stage2.py
└── test_stage2.py
2

Write test using pytest

import pytest
from lib.scan.stage2 import analyze_ast

def test_analyze_ast_detects_subprocess():
    # Arrange
    code = "import subprocess\nsubprocess.run(['ls'])"
    
    # Act
    findings = analyze_ast(code)
    
    # Assert
    assert len(findings) > 0
    assert findings[0]['type'] == 'subprocess_usage'

def test_analyze_ast_handles_invalid_syntax():
    # Test error case
    with pytest.raises(SyntaxError):
        analyze_ast("invalid python code {{{")
3

Run your test

cd python-api
pytest lib/scan/test_stage2.py

​
Test Guidelines

​
General Principles

Follow the RED → GREEN → REFACTOR cycle:
  1. RED — Write a failing test first
  2. GREEN — Write minimal code to make it pass
  3. REFACTOR — Clean up the code while keeping tests green
This ensures all code is tested and testable.
Structure tests in three clear phases:
it('should do something', () => {
  // Arrange — Set up test data
  const input = { foo: 'bar' };
  
  // Act — Execute the code under test
  const result = doSomething(input);
  
  // Assert — Verify the outcome
  expect(result).toBe('expected');
});
Use clear, readable test names that describe what’s being tested:Good:
  • should reject skill if permissions exceed budget
  • should return 404 when skill not found
  • should generate lockfile with SHA-512 hashes
Bad:
  • test1
  • it works
  • testInstall
Each test should be independent and not rely on other tests:
  • Use beforeEach for setup instead of running tests in order
  • Clean up after tests with afterEach
  • Don’t share mutable state between tests
Focus each test on one behavior:
// Good — focused
it('should validate skill name format', () => {
  expect(validateSkillName('@org/skill')).toBe(true);
});

it('should reject invalid skill names', () => {
  expect(validateSkillName('invalid')).toBe(false);
});

// Avoid — testing multiple things
it('should validate skills', () => {
  expect(validateSkillName('@org/skill')).toBe(true);
  expect(validateSkillName('invalid')).toBe(false);
  expect(validateVersion('1.0.0')).toBe(true);
});

​
Mocking

Vitest mocking: 
import { vi } from 'vitest';
import * as fs from 'node:fs';

// Mock module
vi.mock('node:fs');

// Mock implementation
vi.mocked(fs.readFileSync).mockReturnValue('mock content');

// Verify calls
expect(fs.readFileSync).toHaveBeenCalledWith('path/to/file');
pytest mocking: 
from unittest.mock import Mock, patch

def test_with_mock():
    with patch('module.function') as mock_fn:
        mock_fn.return_value = 'mocked'
        result = call_function_that_uses_it()
        assert result == 'mocked'
        mock_fn.assert_called_once()

​
Testing API Routes

Next.js Route Handlers: 
import { GET } from '../route';
import { NextRequest } from 'next/server';

it('should return skill metadata', async () => {
  const request = new NextRequest('http://localhost/api/v1/skills/test');
  const response = await GET(request, { params: { name: 'test' } });
  
  expect(response.status).toBe(200);
  const data = await response.json();
  expect(data.name).toBe('test');
});

​
Testing CLI Commands

Spawning real CLI: 
import { execSync } from 'node:child_process';

it('should install skill', () => {
  const output = execSync('tank install @test/skill', { 
    encoding: 'utf-8' 
  });
  expect(output).toContain('Installed @test/skill');
});

​
Testing Database Queries

Drizzle ORM tests: 
import { db } from '../db';
import { skills } from '../db/schema';

it('should create skill', async () => {
  const [skill] = await db.insert(skills).values({
    name: '@test/skill',
    description: 'Test skill',
  }).returning();
  
  expect(skill.name).toBe('@test/skill');
});

​
Performance Testing

​
Running Performance Tests

1

Seed test data

pnpm --filter=web scripts/perf-seed.ts
Creates 500+ test skills with realistic data.
2

Run performance tests

pnpm test:perf
Measures p95 latency for API and web routes.
3

Analyze results

pnpm --filter=web scripts/perf-analyze-runs.ts
Compares against budgets and generates reports.

​
Performance Budgets

API routes (p95 latency):
  • /api/v1/skills (search) — < 100ms
  • /api/v1/skills/[name] (metadata) — < 50ms
  • /api/v1/skills/[name]/[version] — < 50ms
Web routes (TTFB):
  • Homepage — < 200ms
  • Skill detail page — < 300ms
What happens on regression:
  • CI fails if p95 exceeds budget
  • Merge blocked until performance fixed

​
Coverage

Tank doesn’t enforce code coverage metrics, but aims for:
  • Critical paths — 100% coverage (auth, permissions, security)
  • Business logic — High coverage (audit scoring, publish pipeline)
  • Utilities — Moderate coverage (helpers, formatters)
Check coverage: 
pnpm test --coverage

​
CI Testing

GitHub Actions workflow:
  1. Test job — Runs all unit and integration tests with fake credentials
  2. Performance job — Runs performance tests with real PostgreSQL 17 + Supabase local
When tests run:
  • On every push to a branch
  • On every pull request
  • Before merge to main
When tests fail:
  • PR checks fail
  • Merge is blocked
  • Review requires fixes

​
Debugging Tests

​
Vitest Debugging

# Run specific test file
pnpm test --filter=cli -- install.test.ts

# Run single test by name
pnpm test --filter=cli -- -t "should install skill"

# Watch mode
pnpm test --filter=cli -- --watch

# Enable verbose output
pnpm test --filter=cli -- --reporter=verbose

​
pytest Debugging

# Run specific test file
pytest test_analyze.py

# Run single test
pytest test_analyze.py::test_stage2_detects_subprocess

# Show print statements
pytest -s

# Show verbose output
pytest -v

# Drop into debugger on failure
pytest --pdb

​
Common Testing Patterns

​
Testing Error Cases

it('should throw error on invalid input', async () => {
  await expect(doSomething({ invalid: true }))
    .rejects.toThrow('Expected error message');
});

​
Testing Async Code

it('should handle async operations', async () => {
  const result = await fetchData();
  expect(result).toBeDefined();
});

​
Testing Timeouts

it('should timeout after 5 seconds', async () => {
  await expect(longRunningOperation())
    .rejects.toThrow('Timeout');
}, 6000); // Test timeout

​
Snapshot Testing

import { expect } from 'vitest';

it('should match snapshot', () => {
  const output = generateOutput();
  expect(output).toMatchSnapshot();
});

​
Next Steps

Setup

Set up your local development environment

Contributing

Learn how to contribute to Tank

Architecture

Understand the system design

Build docs developers (and LLMs) love

Get started for freeTalk to us