Testing Strategy

Testing Philosophy

Autonome follows a pragmatic testing approach focused on:

Critical path coverage: Test core trading logic, calculations, and data integrity
No bandaid fixes: Tests should validate real behavior, not workarounds
Edge case exhaustion: Think through failure scenarios before finalizing code
Type safety first: Strict TypeScript and Zod validation catch many issues at compile time

From AGENTS.md: “Before finalizing any code, strictly ‘stress test’ your solution mentally. Recursively generate failure scenarios and fix them immediately. Do not stop until you cannot find a way for the code to fail.”

Test Framework

Autonome uses Vitest as its test runner, chosen for:

Native ESM support
Fast execution with Bun runtime
Compatible with Vite ecosystem
Jest-like API for familiarity
Built-in TypeScript support

Running Tests

# Run all tests
bun run test

# Watch mode (auto-rerun on changes)
bun run test --watch

# Run specific test file
bun run test src/core/shared/trading/calculations.test.ts

# Run with coverage
bun run test --coverage

Test Structure

Tests are colocated with source code using the .test.ts or .test.tsx suffix:

src/
├── core/
│   └── shared/
│       └── trading/
│           ├── calculations.ts
│           └── calculations.test.ts      # Unit tests
├── server/
│   └── features/
│       └── trading/
│           ├── fillTracker.ts
│           └── fillTracker.test.ts       # Integration tests
└── components/
    └── ui/
        ├── Button.tsx
        └── Button.test.tsx                # Component tests

Testing Patterns

Unit Tests: Trading Calculations

Test pure functions with deterministic outputs:

import { describe, it, expect } from "vitest";
import {
	calculateUnrealizedPnl,
	calculateWinRate,
	calculateExpectancy,
} from "./calculations";

describe("calculateUnrealizedPnl", () => {
	it("calculates profit for long position", () => {
		const result = calculateUnrealizedPnl({
			side: "BUY",
			quantity: "100",
			averagePrice: "50.00",
			currentPrice: "55.00",
		});

		expect(result).toBe(500); // (55 - 50) * 100 = $500 profit
	});

	it("calculates loss for short position", () => {
		const result = calculateUnrealizedPnl({
			side: "SELL",
			quantity: "50",
			averagePrice: "100.00",
			currentPrice: "110.00",
		});

		expect(result).toBe(-500); // (100 - 110) * 50 = -$500 loss
	});

	it("handles zero quantity", () => {
		const result = calculateUnrealizedPnl({
			side: "BUY",
			quantity: "0",
			averagePrice: "50.00",
			currentPrice: "55.00",
		});

		expect(result).toBe(0);
	});
});

describe("calculateWinRate", () => {
	it("calculates win rate from trade history", () => {
		const trades = [
			{ pnl: 100 },
			{ pnl: -50 },
			{ pnl: 200 },
			{ pnl: -30 },
		];

		const winRate = calculateWinRate(trades);

		expect(winRate).toBe(0.5); // 2 wins out of 4 trades = 50%
	});

	it("returns 0 for empty trade array", () => {
		expect(calculateWinRate([])).toBe(0);
	});

	it("returns 1 for all winning trades", () => {
		const trades = [{ pnl: 100 }, { pnl: 50 }, { pnl: 75 }];
		expect(calculateWinRate(trades)).toBe(1);
	});
});

Integration Tests: Database Operations

Test repository patterns and database queries:

import { describe, it, expect, beforeEach, afterEach } from "vitest";
import { db } from "@/db";
import { Models, Orders } from "@/db/schema";
import { getOpenPositions, getClosedTrades } from "./tradingRepository";

describe("Trading Repository", () => {
	beforeEach(async () => {
		// Seed test data
		await db.insert(Models).values({
			id: "test-model",
			name: "Apex",
			variant: "Apex",
		});

		await db.insert(Orders).values([
			{
				id: "order-1",
				modelId: "test-model",
				status: "OPEN",
				side: "BUY",
				quantity: "100",
				averagePrice: "50.00",
			},
			{
				id: "order-2",
				modelId: "test-model",
				status: "CLOSED",
				side: "BUY",
				quantity: "50",
				averagePrice: "45.00",
				realizedPnl: "250.00",
			},
		]);
	});

	afterEach(async () => {
		// Clean up test data
		await db.delete(Orders).where(eq(Orders.modelId, "test-model"));
		await db.delete(Models).where(eq(Models.id, "test-model"));
	});

	it("fetches only open positions", async () => {
		const positions = await getOpenPositions("test-model");

		expect(positions).toHaveLength(1);
		expect(positions[0].id).toBe("order-1");
		expect(positions[0].status).toBe("OPEN");
	});

	it("fetches only closed trades", async () => {
		const trades = await getClosedTrades("test-model");

		expect(trades).toHaveLength(1);
		expect(trades[0].id).toBe("order-2");
		expect(trades[0].status).toBe("CLOSED");
	});
});

Component Tests: React Testing Library

Test UI components with React Testing Library:

import { describe, it, expect } from "vitest";
import { render, screen } from "@testing-library/react";
import { Button } from "./Button";

describe("Button", () => {
	it("renders button text", () => {
		render(<Button>Click me</Button>);
		expect(screen.getByText("Click me")).toBeInTheDocument();
	});

	it("applies variant styles", () => {
		const { container } = render(<Button variant="destructive">Delete</Button>);
		const button = container.querySelector("button");
		expect(button).toHaveClass("bg-destructive");
	});

	it("calls onClick handler", async () => {
		const handleClick = vi.fn();
		render(<Button onClick={handleClick}>Click</Button>);

		await userEvent.click(screen.getByText("Click"));
		expect(handleClick).toHaveBeenCalledOnce();
	});
});

Testing Critical Business Logic

Trading Calculations

These functions are critical and require comprehensive test coverage:

calculateUnrealizedPnl() - Position P&L calculation
calculateSharpeRatio() - Risk-adjusted return metrics
calculateMaxDrawdown() - Portfolio risk measurement
calculateWinRate() - Trading performance metrics
calculateExpectancy() - Expected value per trade

Test cases must include:

Positive and negative values
Zero/empty inputs
Edge cases (division by zero, very large numbers)
Different position sides (BUY/SELL)
Precision handling for monetary values

Data Validation

Use Zod schemas for input validation:

import { z } from "zod";

const orderInputSchema = z.object({
	side: z.enum(["BUY", "SELL"]),
	quantity: z.string().regex(/^\d+(\.\d+)?$/),
	averagePrice: z.string().regex(/^\d+(\.\d+)?$/),
	symbol: z.string().min(1),
});

// In tests
describe("Order validation", () => {
	it("accepts valid order", () => {
		const input = {
			side: "BUY",
			quantity: "100",
			averagePrice: "50.00",
			symbol: "BTC-USD",
		};

		expect(() => orderInputSchema.parse(input)).not.toThrow();
	});

	it("rejects invalid quantity format", () => {
		const input = {
			side: "BUY",
			quantity: "invalid",
			averagePrice: "50.00",
			symbol: "BTC-USD",
		};

		expect(() => orderInputSchema.parse(input)).toThrow();
	});
});

Manual QA Workflow

Pre-Deployment Checklist

Before deploying changes, perform manual testing:

1. Environment Validation

# Verify all required environment variables
bun run scripts/validate-env.ts

2. Simulated Trading Test

# Set trading mode to simulated
echo "TRADING_MODE=simulated" >> .env

# Run dev servers
bun run dev:all

Test scenarios:

Create positions via simulator
Verify position appears in dashboard
Check P&L calculations update with price changes
Test exit plan execution (stop loss, take profit)
Confirm trade history records closed positions

3. Database Schema Verification

# Open Drizzle Studio
bun run db:studio

Verify:

Table names use quoted identifiers ("Models", "Orders")
Monetary fields stored as TEXT
IDs are TEXT (UUID format)
Exit plans stored as JSONB
Foreign key relationships intact

4. API Endpoint Testing

Test oRPC procedures:

// In browser console or test file
import { orpc } from "@/server/orpc/client";

// Test trading procedures
const positions = await orpc.trading.getPositions.mutate({ input: {} });
console.log("Positions:", positions);

const trades = await orpc.trading.getTrades.mutate({ input: {} });
console.log("Trades:", trades);

const portfolio = await orpc.trading.getPortfolioHistory.mutate({
	input: { timeRange: "7d" },
});
console.log("Portfolio:", portfolio);

5. Real-Time Updates (SSE)

Verify Server-Sent Events:

# Terminal 1: Start dev servers
bun run dev:all

# Terminal 2: Monitor SSE streams
curl http://localhost:8081/api/events/trading
curl http://localhost:8081/api/events/trades

Verify:

Position updates stream every 3 seconds
New trades trigger immediate events
Frontend invalidates TanStack Query cache on events

Browser Testing

Test in multiple browsers:

Chrome/Chromium (primary)
Firefox (secondary)
Safari (if on macOS)

Test key flows:

Dashboard loads with portfolio summary
Position filters work (All/Apex/Trendsurfer/Contrarian/Sovereign)
Charts render correctly (portfolio history, drawdown)
Model chat tab shows AI reasoning
Exit plans display stop/target/invalidation levels
Dark mode toggle works

Performance Testing

Check for regressions:

# Build production bundle
bun run build

# Check bundle sizes
du -sh .output/public/_build/*

Metrics to monitor:

Initial page load < 2s
Time to Interactive (TTI) < 3s
SSE update latency < 100ms
TanStack Query cache hit rate > 80%

Testing Best Practices

Do’s

✅ Test behavior, not implementation

// Good: Test what the function does
expect(calculatePnl(position)).toBe(500);

// Bad: Test internal implementation details
expect(calculatePnl.calledWith).toBe(true);

✅ Use descriptive test names

// Good
it("calculates profit when current price exceeds entry price", () => {});

// Bad
it("test PnL", () => {});

✅ Test edge cases explicitly

it("returns 0 for zero quantity", () => {});
it("handles negative prices gracefully", () => {});
it("throws error for invalid side", () => {});

✅ Use test data builders

const createTestPosition = (overrides = {}) => ({
	id: "test-position",
	side: "BUY",
	quantity: "100",
	averagePrice: "50.00",
	...overrides,
});

Don’ts

❌ Don’t write defensive tests for non-issues

// Bad: Testing TypeScript's type safety
it("rejects non-string quantity", () => {
	// TypeScript already prevents this
});

❌ Don’t test external libraries

// Bad: Testing Zod's validation (already tested)
it("Zod schema validates correctly", () => {});

❌ Don’t use mocks unless necessary

// Good: Use real functions when possible
const result = calculatePnl(position);

// Bad: Over-mocking
const mockCalculate = vi.fn();

❌ Don’t skip cleanup in integration tests

// Bad: Leaves test data in database
afterEach(() => {
	// No cleanup
});

// Good: Clean up after each test
afterEach(async () => {
	await db.delete(Orders).where(...);
});

CI/CD Integration

GitHub Actions Workflow

name: Test

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: oven-sh/setup-bun@v1
      
      - name: Install dependencies
        run: bun install
      
      - name: Run type check
        run: bun run check
      
      - name: Run tests
        run: bun run test
        env:
          DATABASE_URL: postgres://test:test@localhost:5432/test_db
      
      - name: Upload coverage
        uses: codecov/codecov-action@v3

Coverage Goals

Category	Target	Priority
Trading calculations	100%	Critical
Database repositories	80%	High
oRPC procedures	70%	High
UI components	60%	Medium
Utility functions	80%	Medium

Coverage is not a goal in itself. Focus on testing critical paths and edge cases rather than chasing 100% coverage everywhere.

Debugging Tests

Debug in VS Code

Add to .vscode/launch.json:

{
  "type": "node",
  "request": "launch",
  "name": "Debug Vitest Tests",
  "runtimeExecutable": "bun",
  "runtimeArgs": ["run", "test", "--no-coverage"],
  "console": "integratedTerminal"
}

Console Logging

it("debugs calculation", () => {
	const position = createTestPosition();
	console.log("Position:", position);

	const result = calculatePnl(position);
	console.log("Result:", result);

	expect(result).toBe(500);
});

Isolate Failing Tests

// Run only this test
it.only("calculates profit correctly", () => {
	// ...
});

// Skip this test
it.skip("flaky test", () => {
	// ...
});

Next Steps

Write your first test: Follow the patterns above to add tests for new features
Run tests regularly: Use bun run test --watch during development
Learn code style: See Code Style for Biome configuration
Contribute: Review Contributing Guidelines for the full workflow

When in doubt, ask yourself: “What could break?” Then write a test that proves it won’t.

Architecture

Database

Configuration

Development

Testing Philosophy

Test Framework

Running Tests

Test Structure

Testing Patterns

Unit Tests: Trading Calculations

Integration Tests: Database Operations

Component Tests: React Testing Library

Testing Critical Business Logic

Trading Calculations

Data Validation

Manual QA Workflow

Pre-Deployment Checklist

1. Environment Validation

2. Simulated Trading Test

3. Database Schema Verification

4. API Endpoint Testing

5. Real-Time Updates (SSE)

Browser Testing

Performance Testing

Testing Best Practices

Do’s

Don’ts

CI/CD Integration

GitHub Actions Workflow

Coverage Goals

Debugging Tests

Debug in VS Code

Console Logging

Isolate Failing Tests

Next Steps

Build docs developers (and LLMs) love

Architecture

Database

Configuration

Development

​Testing Philosophy

​Test Framework

​Running Tests

​Test Structure

​Testing Patterns

​Unit Tests: Trading Calculations

​Integration Tests: Database Operations

​Component Tests: React Testing Library

​Testing Critical Business Logic

​Trading Calculations

​Data Validation

​Manual QA Workflow

​Pre-Deployment Checklist

​1. Environment Validation

​2. Simulated Trading Test

​3. Database Schema Verification

​4. API Endpoint Testing

​5. Real-Time Updates (SSE)

​Browser Testing

​Performance Testing

​Testing Best Practices

​Do’s

​Don’ts

​CI/CD Integration

​GitHub Actions Workflow

​Coverage Goals

​Debugging Tests

​Debug in VS Code

​Console Logging

​Isolate Failing Tests

​Next Steps

Build docs developers (and LLMs) love

Testing Philosophy

Test Framework

Running Tests

Test Structure

Testing Patterns

Unit Tests: Trading Calculations

Integration Tests: Database Operations

Component Tests: React Testing Library

Testing Critical Business Logic

Trading Calculations

Data Validation

Manual QA Workflow

Pre-Deployment Checklist

1. Environment Validation

2. Simulated Trading Test

3. Database Schema Verification

4. API Endpoint Testing

5. Real-Time Updates (SSE)

Browser Testing

Performance Testing

Testing Best Practices

Do’s

Don’ts

CI/CD Integration

GitHub Actions Workflow

Coverage Goals

Debugging Tests

Debug in VS Code

Console Logging

Isolate Failing Tests

Next Steps