Platform-Specific Testing

Each platform has unique testing requirements, tools, and workflows. This guide covers platform-specific testing considerations for Mullvad VPN.

Desktop Testing (Linux, macOS, Windows)

Prerequisites

All Platforms

Latest stable Rust from https://rustup.rs/
Node.js 16+ and npm 8.3+
Protocol Buffers compiler

macOS

Install dependencies:

brew install cirruslabs/cli/tart wireguard-tools pkg-config openssl protobuf

Wireshark for packet capture:

Install Wireshark from https://www.wireshark.org/download.html
Install wireshark-chmodbpf component
Add user to access_bpf group:

dseditgroup -o edit -a $USER -t user access_bpf

VM setup with Tart:

# Download VM image
tart clone ghcr.io/cirruslabs/macos-ventura-base:latest ventura-base

# Configure test manager
cargo run --bin test-manager set macos-ventura tart ventura-base macos \
    --architecture aarch64 \
    --provisioner ssh --ssh-user admin --ssh-password admin

# Verify setup
cargo run -p test-manager run-vm macos-ventura

Linux

Fedora:

dnf install git gcc protobuf-devel libpcap-devel qemu \
    podman golang-github-rootless-containers-rootlesskit slirp4netns dnsmasq \
    dbus-devel pkgconf-pkg-config swtpm edk2-ovmf \
    wireguard-tools

Debian/Ubuntu:

apt install qemu-utils qemu-system-x86 libpcap-dev slirp4netns \
    rootlesskit dnsmasq nftables

Note for Debian: sysctl is only invokable by root by default. VM setup with QEMU:

# Create configuration
cargo run --bin test-manager set debian11 qemu ./os-images/debian11.qcow2 linux \
    --package-type deb --architecture x64 \
    --provisioner ssh --ssh-user test --ssh-password test

# Verify setup
cargo run --bin test-manager run-vm debian11

Windows

Windows VMs are typically run from Linux hosts using QEMU. Build test-runner for Windows:

./scripts/container-run.sh ./scripts/build/test-runner.sh windows

Running Desktop Tests

Unit Tests

JavaScript/TypeScript tests:

cd desktop/packages/mullvad-vpn
npm test

Run specific test file:

npm test -- account-number.spec.ts

Watch mode:

npm test -- --watch

E2E Tests

Mocked backend tests:

cd desktop/packages/mullvad-vpn
npm run e2e

Run specific test suite:

npm run e2e:no-build -- login.spec.ts

Run with UI visible:

npm run e2e:no-build -- --headed

Debug mode:

npm run e2e:no-build -- --debug

Installed app tests:

npm run e2e:sequential

VM-Based Integration Tests

Setup test environment:

# Build test runner
./scripts/build/test-runner.sh macos  # or linux

# Build application package
./build.sh --dev-build

# Build GUI test executable
cd desktop/packages/mullvad-vpn
npm run build-test-executable

Run tests on Linux VM:

cargo run --bin test-manager run-tests --vm debian11 \
    --display \
    --account 1234567890123456 \
    --app-package <git-hash-or-tag> \
    --app-package-to-upgrade-from 2023.2

Run tests on macOS VM:

cargo run --bin test-manager run-tests --vm macos-ventura \
    --display \
    --account 1234567890123456 \
    --app-package <git-hash-or-tag>

Run specific test:

cargo run --bin test-manager run-tests --vm debian11 \
    --display \
    --account 1234567890123456 \
    --test install::test_install

Platform-Specific Test Areas

macOS-Specific Tests

Tests in test-manager/src/tests/macos.rs:

Launch daemon functionality
Keychain integration
System extension loading
Network extension behavior
Notification permissions

Windows-Specific Tests

Tests in test-manager/src/tests/windows.rs:

Windows Filtering Platform (WFP) integration
Service installation and management
TAP adapter functionality
Driver loading and unloading
Windows firewall integration

Linux-Specific Tests

Tests in test-manager/src/tests/split_tunnel.rs:

Split tunneling via cgroups
Network namespace management
iptables/nftables rules
systemd integration
D-Bus communication

Android Testing

Prerequisites

Android Studio or Android SDK command-line tools
Java Development Kit (JDK) 17+
Android device or emulator
Valid Mullvad account for e2e tests

Test Structure

Android tests are organized by module in android/lib/*/src/test/:

app/src/test - Main application unit tests
lib/billing/src/test - Billing functionality tests
lib/feature/*/src/test - Feature module tests
test/e2e - End-to-end instrumented tests

Running Android Tests

Unit Tests

Run all unit tests:

cd android
./gradlew testDebugUnitTest

Run tests for specific module:

./gradlew :app:testDebugUnitTest
./gradlew :lib:billing:testDebugUnitTest
./gradlew :lib:feature:account:impl:testDebugUnitTest

Run with coverage:

./gradlew testDebugUnitTest jacocoTestReport

Instrumented Tests

Run on connected device:

./gradlew connectedDebugAndroidTest

End-to-End Tests

Configure test accounts: Add to ~/.gradle/gradle.properties:

mullvad.test.e2e.prod.accountNumber.valid=1234567890123456
mullvad.test.e2e.prod.accountNumber.invalid=9999999999

Run e2e tests:

./gradlew :test:e2e:connectedDebugAndroidTest \
    -Pmullvad.test.e2e.prod.accountNumber.valid=1234567890123456 \
    -Pmullvad.test.e2e.prod.accountNumber.invalid=9999999999

Run via ADB (for manual APK installation):

adb shell 'CLASSPATH=$(pm path androidx.test.services) app_process / \
    androidx.test.services.shellexecutor.ShellMain am instrument -w \
    -e clearPackageData true \
    -e mullvad.test.e2e.prod.accountNumber.valid 1234567890123456 \
    -e mullvad.test.e2e.prod.accountNumber.invalid 9999999999 \
    -e targetInstrumentation net.mullvad.mullvadvpn.test.e2e/androidx.test.runner.AndroidJUnitRunner \
    androidx.test.orchestrator/.AndroidTestOrchestrator'

Firebase Test Lab

Run tests on multiple devices using Firebase Test Lab: Setup:

# Install gcloud CLI
# Follow: https://firebase.google.com/docs/test-lab/android/command-line

# Authenticate
gcloud auth login

Run tests:

cd android
gcloud firebase test android run \
    --type instrumentation \
    --app ./app/build/outputs/apk/debug/app-debug.apk \
    --test ./test/e2e/build/outputs/apk/debug/e2e-debug.apk \
    --device model=redfin,version=30,locale=en,orientation=portrait \
    --use-orchestrator \
    --environment-variables clearPackageData=true,ORG_GRADLE_PROJECT_mullvad.test.e2e.prod.accountNumber.valid=1234567890123456

Android Test Artifacts

Test artifacts are stored on device:

adb pull /sdcard/Download/test-attachments ./test-results/

Artifacts include:

Screenshots
Logcat output
Network traffic logs
Database dumps

Android-Specific Test Areas

Feature tests:

Account management (lib/feature/account/impl/src/test)
Login flow (lib/feature/login/impl/src/test)
Device management (lib/feature/managedevices/impl/src/test)
Split tunneling (lib/feature/splittunneling/impl/src/test)
Multi-hop configuration (lib/feature/multihop/impl/src/test)
DAITA support (lib/feature/daita/impl/src/test)

Compose UI tests: Android uses Jetpack Compose with UI testing support:

@Test
fun testLoginScreen() {
    composeTestRule.setContent {
        LoginScreen()
    }
    composeTestRule.onNodeWithText("Login").assertIsDisplayed()
}

iOS Testing

Prerequisites

macOS with Xcode installed
iOS Simulator or physical iOS device
Xcode Command Line Tools
CocoaPods (if used)

Test Structure

iOS tests are organized by framework:

MullvadRESTTests - REST API client tests
MullvadRustRuntimeTests - Rust FFI integration tests
MullvadPostQuantumTests - Post-quantum cryptography tests
MullvadVPNTests - Main app logic tests
MullvadVPNScreenshotTests - UI screenshot tests

Running iOS Tests

Via Xcode

Open ios/MullvadVPN.xcodeproj
Select test scheme (e.g., MullvadVPN)
Press Cmd + U to run tests

Via Command Line

Run all tests:

xcodebuild test \
    -project ios/MullvadVPN.xcodeproj \
    -scheme MullvadVPN \
    -destination 'platform=iOS Simulator,name=iPhone 15'

Run specific test target:

xcodebuild test \
    -project ios/MullvadVPN.xcodeproj \
    -scheme MullvadRESTTests \
    -destination 'platform=iOS Simulator,name=iPhone 15'

Run on physical device:

xcodebuild test \
    -project ios/MullvadVPN.xcodeproj \
    -scheme MullvadVPN \
    -destination 'platform=iOS,id=<device-udid>'

List available simulators:

xcrun simctl list devices

Via xcodebuild with JSON output

xcodebuild test \
    -project ios/MullvadVPN.xcodeproj \
    -scheme MullvadVPN \
    -destination 'platform=iOS Simulator,name=iPhone 15' \
    -resultBundlePath ./test-results \
    2>&1 | tee test-output.log

iOS-Specific Test Areas

REST API Tests (`MullvadRESTTests`)

Test files:

AppVersionServiceTests.swift - App version API
DefaultLocationServiceTests.swift - Location service
ServerRelayTests.swift - Relay server selection
RetryStrategyTests.swift - Network retry logic
ShadowsocksCacheCleanerTests.swift - Cache management

Rust Runtime Tests (`MullvadRustRuntimeTests`)

Test files:

EphemeralPeerExchangeActorTests.swift - Peer exchange protocol
TunnelObfuscationTests.swift - Tunnel obfuscation

Network Extension Tests

iOS Network Extension requires special testing considerations:

Must test on actual iOS device for full functionality
Simulator has limited VPN capabilities
Requires provisioning profiles and certificates

UI Testing on iOS

Screenshot tests:

xcodebuild test \
    -project ios/MullvadVPN.xcodeproj \
    -scheme MullvadVPNScreenshotTests \
    -destination 'platform=iOS Simulator,name=iPhone 15'

XCUITest framework:

func testLoginFlow() {
    let app = XCUIApplication()
    app.launch()
    
    let loginButton = app.buttons["Login"]
    XCTAssertTrue(loginButton.exists)
    loginButton.tap()
}

Rust Core Testing

All platforms share Rust core components that require testing.

Running Rust Tests

All workspace tests:

cargo test --workspace

Specific crate:

cargo test -p mullvad-daemon
cargo test -p talpid-core
cargo test -p mullvad-api

With verbose output:

cargo test -- --nocapture --test-threads=1

Documentation tests:

cargo test --doc

Platform-Specific Rust Tests

Test only on specific platform:

#[cfg(target_os = "linux")]
#[test]
fn test_linux_specific_feature() {
    // Linux-only test
}

#[cfg(target_os = "macos")]
#[test]
fn test_macos_specific_feature() {
    // macOS-only test
}

#[cfg(target_os = "windows")]
#[test]
fn test_windows_specific_feature() {
    // Windows-only test
}

Cross-Platform Test Automation

GitHub Actions

Tests run automatically on pull requests and scheduled builds. View test results:

Check the “Checks” tab on pull requests
Review GitHub Actions workflow runs

Test workflows:

android-app.yml - Android unit and instrumented tests
desktop-*.yml - Desktop tests for each platform
ios-*.yml - iOS build and test workflows

Test by Version Script

Automate testing of specific versions:

./test-by-version.sh --help

This script:

Downloads pre-built application packages
Sets up test VMs
Runs the test suite
Reports results

Debugging Platform-Specific Issues

Desktop Debugging

Enable debug logging:

RUST_LOG=debug ./mullvad-daemon

Electron debug mode:

cd desktop/packages/mullvad-vpn
npm run develop

Android Debugging

Logcat filtering:

adb logcat | grep Mullvad

Debug specific test:

./gradlew :app:testDebugUnitTest --tests "*LoginViewModelTest.testLoginSuccess"

iOS Debugging

Console output:

log stream --predicate 'subsystem == "net.mullvad.vpn"'

Simulator logs:

xcrun simctl spawn booted log stream --predicate 'subsystem == "net.mullvad.vpn"'

Best Practices

Test on actual hardware - Especially for VPN functionality
Use platform-specific CI - Each platform has its own test requirements
Mock network dependencies - For faster, more reliable unit tests
Test permission flows - Each platform handles permissions differently
Verify platform integration - System extensions, services, and permissions
Test upgrade paths - Ensure smooth updates from previous versions
Monitor test flakiness - Platform-specific timing issues can cause flaky tests

Setup

Contributing

Testing

Advanced

​Desktop Testing (Linux, macOS, Windows)

​Prerequisites

​All Platforms

​macOS

​Linux

​Windows

​Running Desktop Tests

​Unit Tests

​E2E Tests

​VM-Based Integration Tests

​Platform-Specific Test Areas

​macOS-Specific Tests

​Windows-Specific Tests

​Linux-Specific Tests

​Android Testing

​Prerequisites

​Test Structure

​Running Android Tests

​Unit Tests

​Instrumented Tests

​End-to-End Tests

​Firebase Test Lab

​Android Test Artifacts

​Android-Specific Test Areas

​iOS Testing

​Prerequisites

​Test Structure

​Running iOS Tests

​Via Xcode

​Via Command Line

​Via xcodebuild with JSON output

​iOS-Specific Test Areas

​REST API Tests (MullvadRESTTests)

​Rust Runtime Tests (MullvadRustRuntimeTests)

​Network Extension Tests

​UI Testing on iOS

​Rust Core Testing

​Running Rust Tests

​Platform-Specific Rust Tests

​Cross-Platform Test Automation

​GitHub Actions

​Test by Version Script

​Debugging Platform-Specific Issues

​Desktop Debugging

​Android Debugging

​iOS Debugging

​Best Practices

​Related Documentation

Build docs developers (and LLMs) love

Desktop Testing (Linux, macOS, Windows)

Prerequisites

All Platforms

macOS

Linux

Windows

Running Desktop Tests

Unit Tests

E2E Tests

VM-Based Integration Tests

Platform-Specific Test Areas

macOS-Specific Tests

Windows-Specific Tests

Linux-Specific Tests

Android Testing

Prerequisites

Test Structure

Running Android Tests

Unit Tests

Instrumented Tests

End-to-End Tests

Firebase Test Lab

Android Test Artifacts

Android-Specific Test Areas

iOS Testing

Prerequisites

Test Structure

Running iOS Tests

Via Xcode

Via Command Line

Via xcodebuild with JSON output

iOS-Specific Test Areas

REST API Tests (`MullvadRESTTests`)

Rust Runtime Tests (`MullvadRustRuntimeTests`)

Network Extension Tests

UI Testing on iOS

Rust Core Testing

Running Rust Tests

Platform-Specific Rust Tests

Cross-Platform Test Automation

GitHub Actions

Test by Version Script

Debugging Platform-Specific Issues

Desktop Debugging

Android Debugging

iOS Debugging

Best Practices

Related Documentation