Testing - Ora Browser

Test Infrastructure

Ora Browser uses Swift Testing for unit and UI tests. The test suite is organized into:

oraTests: Unit tests for core functionality
oraUITests: UI automation and integration tests

Running Tests

In Xcode

Run the complete test suite:

Select the ora scheme
Press ⌘U or choose Product → Test

Run individual tests:

Click the diamond icon next to any test function
Right-click a test and select “Run Test”

Tests run in the Debug configuration by default. You can view test results in the Test Navigator (⌘6).

From Command Line

Run all tests via xcodebuild:

xcodebuild test \
  -scheme ora \
  -destination "platform=macOS"

For prettier output, pipe through xcbeautify:

xcodebuild test \
  -scheme ora \
  -destination "platform=macOS" | xcbeautify

Pre-Push Testing

Tests are automatically run before pushing to ensure code quality:

lefthook.yml

pre-push:
  commands:
    build:
      run: ./scripts/xcbuild-debug.sh

If the build fails in the pre-push hook, your push will be blocked. Fix any build errors before pushing.

Writing Tests

Unit Tests

Ora uses Swift Testing framework for writing tests:

oraTests/oraTests.swift

@testable import ora
import Testing

struct OraTests {
    @Test func example() {
        // Write your test here and use APIs like `#expect(...)` to check expected conditions.
    }
}

Test Structure

Follow this pattern for unit tests:

import Testing
@testable import ora

struct BrowserViewModelTests {
    @Test func testNavigationToURL() {
        let viewModel = BrowserViewModel()
        let url = URL(string: "https://example.com")!
        
        viewModel.navigate(to: url)
        
        #expect(viewModel.currentURL == url)
    }
    
    @Test func testInvalidURLHandling() {
        let viewModel = BrowserViewModel()
        let invalidURL = "not a url"
        
        #expect(throws: URLError.self) {
            try viewModel.navigate(to: URL(string: invalidURL)!)
        }
    }
}

UI Tests

UI tests verify the user interface and interactions:

oraUITests/oraUITests.swift

import XCTest

final class oraUITests: XCTestCase {
    override func setUpWithError() throws {
        continueAfterFailure = false
    }

    func testLaunchPerformance() throws {
        measure(metrics: [XCTApplicationLaunchMetric()]) {
            XCUIApplication().launch()
        }
    }
}

Test Guidelines

When to Write Tests

New features

All new features should include appropriate unit tests covering:

Happy path functionality
Edge cases and error handling
Invalid input scenarios

Bug fixes

When fixing bugs:

Write a failing test that reproduces the bug
Fix the bug
Verify the test passes

Refactoring

Ensure existing tests continue to pass after refactoring:

xcodebuild test -scheme ora -destination "platform=macOS"

Test Requirements

Before submitting a pull request:

Test Organization

Directory Structure

browser/
├── oraTests/
│   ├── oraTests.swift           # Unit test examples
│   └── Info.plist
├── oraUITests/
│   ├── oraUITests.swift         # UI test suite
│   ├── oraUITestsLaunchTests.swift  # Launch tests
│   └── Info.plist

Test Target Configuration

Tests are configured in project.yml:

schemes:
  ora:
    test:
      config: Debug

Continuous Integration

Local Validation

Before pushing code, ensure your changes pass local validation:

# Run formatter
swiftformat . --quiet

# Run linter
swiftlint --quiet

# Build debug
./scripts/xcbuild-debug.sh

# Run tests
xcodebuild test -scheme ora -destination "platform=macOS"

Git Hooks

Git hooks automatically enforce quality checks:

pre-commit:
  parallel: true
  commands:
    swiftformat:
      glob: "**/*.swift"
      run: swiftformat {staged_files} --quiet
      stage_fixed: true
    swiftlint:
      glob: "**/*.swift"
      run: swiftlint lint --fix --use-alternative-excluding {staged_files}
      stage_fixed: true

Testing Best Practices

Keep Tests Fast

Unit tests should run in milliseconds
Avoid network calls in unit tests
Use mocks for external dependencies
Keep UI tests focused and minimal

Write Meaningful Tests

// Good: Clear test name and assertion
@Test func testSearchEngineSelectionPersistsAcrossLaunches() {
    let settings = BrowserSettings()
    settings.searchEngine = .google
    settings.save()
    
    let newSettings = BrowserSettings()
    #expect(newSettings.searchEngine == .google)
}

// Bad: Vague test name
@Test func testSettings() {
    // What is this testing?
}

Test One Thing at a Time

// Good: Each test has a single focus
@Test func testNavigationUpdatesCurrentURL() {
    // ...
}

@Test func testNavigationAddsHistoryEntry() {
    // ...
}

// Bad: Testing multiple behaviors
@Test func testNavigation() {
    // Tests URL update, history, and favorites
}

Use Descriptive Assertions

// Good: Clear assertion with context
#expect(
    viewModel.tabs.count == 3,
    "Expected 3 tabs after opening new tab"
)

// Bad: Vague assertion
#expect(viewModel.tabs.count == 3)

Debugging Tests

Test Failures

When a test fails:

Read the failure message: Swift Testing provides detailed error context
Run the test in isolation: Focus on the failing test
Use breakpoints: Add breakpoints in the test and implementation
Check test data: Verify test inputs and expected outputs

Debugging in Xcode

@Test func testComplexBehavior() {
    let viewModel = BrowserViewModel()
    
    // Add breakpoint here
    viewModel.performAction()
    
    // Inspect state in debugger
    #expect(viewModel.state == .ready)
}

Console Output

Add logging to understand test execution:

@Test func testWithLogging() {
    print("Starting test with initial state: \(viewModel.state)")
    viewModel.performAction()
    print("After action, state: \(viewModel.state)")
    
    #expect(viewModel.state == .ready)
}

Performance Testing

Measuring Performance

Test launch performance:

func testLaunchPerformance() throws {
    measure(metrics: [XCTApplicationLaunchMetric()]) {
        XCUIApplication().launch()
    }
}

Performance Baselines

Run performance tests multiple times
Xcode will establish a baseline
Future runs compare against the baseline
Investigate significant regressions

Troubleshooting

Tests Not Running

Ensure test target is selected in scheme
Clean build folder (⇧⌘K)
Reset test data: Product → Test → Delete Test Data

Flaky Tests

Add waits for asynchronous operations
Avoid hardcoded timeouts
Use XCTest expectations for async code

Build Failures

If tests fail to build:

# Regenerate project
xcodegen

# Clean and rebuild
xcodebuild clean
xcodebuild build -scheme ora -destination "platform=macOS"

Contributing

Building

Advanced

​Test Infrastructure

​Running Tests

​In Xcode

​From Command Line

​Pre-Push Testing

​Writing Tests

​Unit Tests

​Test Structure

​UI Tests

​Test Guidelines

​When to Write Tests

​Test Requirements

​Test Organization

​Directory Structure

​Test Target Configuration

​Continuous Integration

​Local Validation

​Git Hooks

​Testing Best Practices

​Keep Tests Fast

​Write Meaningful Tests

​Test One Thing at a Time

​Use Descriptive Assertions

​Debugging Tests

​Test Failures

​Debugging in Xcode

​Console Output

​Performance Testing

​Measuring Performance

​Performance Baselines

​Troubleshooting

​Tests Not Running

​Flaky Tests

​Build Failures

​Next Steps

Building

Releases

Build docs developers (and LLMs) love

Test Infrastructure

Running Tests

In Xcode

From Command Line

Pre-Push Testing

Writing Tests

Unit Tests

Test Structure

UI Tests

Test Guidelines

When to Write Tests

Test Requirements

Test Organization

Directory Structure

Test Target Configuration

Continuous Integration

Local Validation

Git Hooks

Testing Best Practices

Keep Tests Fast

Write Meaningful Tests

Test One Thing at a Time

Use Descriptive Assertions

Debugging Tests

Test Failures

Debugging in Xcode

Console Output

Performance Testing

Measuring Performance

Performance Baselines

Troubleshooting

Tests Not Running

Flaky Tests

Build Failures

Next Steps