Testing

Kayston’s Forge uses Vitest with jsdom to ensure all tools work correctly and handle edge cases gracefully. The test suite includes 157 tests across three categories.

Test Structure

Tests are organized in the tests/unit/ directory:

tests/
└── unit/
    ├── engine.test.ts      # 57 tool engine tests
    ├── bugfixes.test.ts    # 28 regression tests
    └── fuzz.test.ts        # 72 robustness tests

Running Tests

# Run entire test suite (157 tests)
npm run test

Test Configuration

Vitest is configured in vitest.config.ts:

import { defineConfig } from 'vitest/config';
import path from 'node:path';

export default defineConfig({
  resolve: {
    alias: {
      '@': path.resolve(__dirname, '.'),
    },
  },
  test: {
    environment: 'jsdom',    // Browser-like environment
    globals: true,           // Global test APIs (describe, it, expect)
    include: ['tests/**/*.test.ts'],
  },
});

Key Configuration

jsdom environment: Simulates a browser for client-side code
Path alias @/: Matches Next.js import paths
Globals enabled: No need to import describe, it, expect

Writing Tests

Basic Tool Test

Test a simple tool transformation:

import { describe, expect, it } from 'vitest';
import { processTool } from '@/lib/tools/engine';

describe('my new tool', () => {
  it('transforms input correctly', async () => {
    const result = await processTool('my-new-tool', 'test input', {
      action: 'default',
    });
    
    expect(result.output).toBe('expected output');
  });
});

Testing Actions

Test different actions for a tool:

describe('json formatter', () => {
  it('formats json', async () => {
    const out = await processTool('json-format-validate', '{"a":1}', {
      action: 'default',
    });
    expect(out.output).toContain('\n');
    expect(out.output).toContain('"a"');
  });

  it('minifies json', async () => {
    const out = await processTool('json-format-validate', '{"a":1, "b":2}', {
      action: 'minify',
    });
    expect(out.output).toBe('{"a":1,"b":2}');
  });
});

Testing Error Handling

Ensure tools handle invalid input gracefully:

it('handles invalid json', async () => {
  const out = await processTool('json-format-validate', '{invalid}');
  expect(out.output).toContain('error');
});

it('handles empty input', async () => {
  const out = await processTool('my-tool', '');
  expect(out.output).toBeDefined();
});

Testing with Secondary Input

For tools that accept two inputs (like diff):

it('compares two texts', async () => {
  const out = await processTool('text-diff', 'line 1\nline 2', {
    action: 'default',
    secondInput: 'line 1\nline 3',
  });
  expect(out.output).toContain('-');
  expect(out.output).toContain('+');
});

Test Categories

1. Engine Tests (engine.test.ts)

Covers core functionality for all 51 tools:

describe('tool engine', () => {
  it('formats json', async () => { /* ... */ });
  it('encodes base64', async () => { /* ... */ });
  it('parses urls', async () => { /* ... */ });
  it('generates hashes', async () => { /* ... */ });
  // ... 53 more tests
});

Focus: Happy path testing for each tool

2. Regression Tests (bugfixes.test.ts)

Prevents previously fixed bugs from returning:

describe('regression tests', () => {
  it('handles trailing commas in json', async () => {
    const out = await processTool('json-format-validate', '{"a":1,}');
    expect(out.output).not.toContain('error');
  });

  it('preserves unicode in base64', async () => {
    const encoded = await processTool('base64-string', '你好');
    const decoded = await processTool('base64-string', encoded.output, {
      action: 'decode',
    });
    expect(decoded.output).toBe('你好');
  });
});

Focus: Edge cases and previously reported bugs

3. Fuzz Tests (fuzz.test.ts)

Tests parser robustness with adversarial inputs:

describe('fuzz tests', () => {
  it('handles deeply nested json', async () => {
    const deep = '{'.repeat(100) + '"a":1' + '}'.repeat(100);
    const out = await processTool('json-format-validate', deep);
    expect(out.output).toBeDefined();
  });

  it('handles malformed xml', async () => {
    const out = await processTool('xml-beautify', '<tag>unclosed');
    expect(out.output).toContain('error');
  });

  it('rejects oversized input', async () => {
    const huge = 'x'.repeat(10 * 1024 * 1024); // 10 MB
    const out = await processTool('json-format-validate', huge);
    expect(out.output).toContain('too large');
  });
});

Focus: Security, performance, and edge case handling

Testing Best Practices

Test all actions

If your tool has multiple actions, test each one:

describe('string case converter', () => {
  it('converts to camelCase', async () => { /* ... */ });
  it('converts to snake_case', async () => { /* ... */ });
  it('converts to kebab-case', async () => { /* ... */ });
  it('converts to PascalCase', async () => { /* ... */ });
});

Test error paths

Don’t just test successful transformations:

it('rejects invalid input', async () => {
  const out = await processTool('my-tool', 'invalid');
  expect(out.output).toContain('error');
});

Use descriptive test names

Test names should clearly describe what they verify:

// Good
it('preserves unicode characters in base64 encoding', async () => { /* ... */ });

// Bad
it('test 1', async () => { /* ... */ });

Test idempotency

For reversible operations, test round-trips:

it('encodes and decodes without data loss', async () => {
  const original = 'test data';
  const encoded = await processTool('base64-string', original);
  const decoded = await processTool('base64-string', encoded.output, {
    action: 'decode',
  });
  expect(decoded.output).toBe(original);
});

Test metadata and previews

If your tool returns metadata or previews, test those too:

it('returns preview html', async () => {
  const out = await processTool('markdown-preview', '# Heading');
  expect(out.previewHtml).toContain('<h1>');
});

it('includes metadata', async () => {
  const out = await processTool('hash-generator', 'data', {
    action: 'sha256',
  });
  expect(out.meta).toContain('SHA256');
});

Common Testing Patterns

// For complex output, use snapshots
it('formats sql correctly', async () => {
  const out = await processTool('sql-formatter', 
    'SELECT * FROM users WHERE id=1');
  expect(out.output).toMatchSnapshot();
});

Continuous Integration

Tests run automatically on every push and pull request via GitHub Actions:

# .github/workflows/test.yml
name: Test

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: 18
      - run: npm install
      - run: npm run test

Debugging Failed Tests

Run in watch mode

npx vitest

Vitest will re-run tests as you save files.

Add console.log

it('debugging test', async () => {
  const out = await processTool('my-tool', 'input');
  console.log('Output:', out.output);
  console.log('Meta:', out.meta);
  expect(out.output).toBe('expected');
});

Run single test

Use .only to run just one test:

it.only('this test only', async () => {
  // Only this test will run
});

Check test isolation

Ensure tests don’t depend on each other:

# Run tests in random order
npx vitest --sequence.shuffle

Performance Testing

Test that tools handle large inputs efficiently:

it('handles large json input', async () => {
  const large = JSON.stringify({
    items: Array.from({ length: 10000 }, (_, i) => ({
      id: i,
      name: `Item ${i}`,
    })),
  });
  
  const start = performance.now();
  const out = await processTool('json-format-validate', large);
  const duration = performance.now() - start;
  
  expect(out.output).toBeDefined();
  expect(duration).toBeLessThan(1000); // Should process in < 1s
});

The engine rejects inputs larger than 5 MB to prevent UI freezing. Test this boundary:

it('rejects oversized input', async () => {
  const huge = 'x'.repeat(6 * 1024 * 1024); // 6 MB
  const out = await processTool('my-tool', huge);
  expect(out.output).toContain('too large');
});

Test Coverage Goals

Tool coverage: All tools must have at least one test
Action coverage: All actions must be tested
Error coverage: All error paths must be tested
Edge cases: Empty input, invalid input, boundary conditions

Troubleshooting

Tests Timeout

// Increase timeout for slow tests
it('slow operation', async () => {
  // Test code
}, 10000); // 10 second timeout

Import Errors

Ensure path alias is correct:

// Correct
import { processTool } from '@/lib/tools/engine';

// Incorrect
import { processTool } from '../../../lib/tools/engine';

jsdom Errors

Some browser APIs aren’t available in jsdom. Mock them:

import { vi } from 'vitest';

global.crypto = {
  getRandomValues: vi.fn((arr) => {
    for (let i = 0; i < arr.length; i++) {
      arr[i] = Math.floor(Math.random() * 256);
    }
    return arr;
  }),
} as any;

Next Steps

Review deployment process
Learn about adding new tools
Study existing tests in tests/unit/ for examples

Architecture

Development

Security

Test Structure

Running Tests

Test Configuration

Key Configuration

Writing Tests

Basic Tool Test

Testing Actions

Testing Error Handling

Testing with Secondary Input

Test Categories

1. Engine Tests (engine.test.ts)

2. Regression Tests (bugfixes.test.ts)

3. Fuzz Tests (fuzz.test.ts)

Testing Best Practices

Common Testing Patterns

Continuous Integration

Debugging Failed Tests

Performance Testing

Test Coverage Goals

Troubleshooting

Tests Timeout

Import Errors

jsdom Errors

Next Steps

Build docs developers (and LLMs) love

Architecture

Development

Security

​Test Structure

​Running Tests

​Test Configuration

​Key Configuration

​Writing Tests

​Basic Tool Test

​Testing Actions

​Testing Error Handling

​Testing with Secondary Input

​Test Categories

​1. Engine Tests (engine.test.ts)

​2. Regression Tests (bugfixes.test.ts)

​3. Fuzz Tests (fuzz.test.ts)

​Testing Best Practices

​Common Testing Patterns

​Continuous Integration

​Debugging Failed Tests

​Performance Testing

​Test Coverage Goals

​Troubleshooting

​Tests Timeout

​Import Errors

​jsdom Errors

​Next Steps

Build docs developers (and LLMs) love

Test Structure

Running Tests

Test Configuration

Key Configuration

Writing Tests

Basic Tool Test

Testing Actions

Testing Error Handling

Testing with Secondary Input

Test Categories

1. Engine Tests (engine.test.ts)

2. Regression Tests (bugfixes.test.ts)

3. Fuzz Tests (fuzz.test.ts)

Testing Best Practices

Common Testing Patterns

Continuous Integration

Debugging Failed Tests

Performance Testing

Test Coverage Goals

Troubleshooting

Tests Timeout

Import Errors

jsdom Errors

Next Steps