Skip to main content
Windows Calculator includes comprehensive test coverage with both unit tests and UI automation tests. This guide shows you how to run them.

​
Test Projects

Calculator has two test projects:
ProjectTypeFrameworkDescription
CalculatorUnitTestsC++VSTestUnit tests for CalcManager and CalcViewModel
CalculatorUITestsC#Appium/WinAppDriverEnd-to-end UI automation tests

​
Unit Tests

Unit tests verify the core calculation logic and view models without requiring the full UI.

​
Running Unit Tests in Visual Studio

1

Open Test Explorer

In Visual Studio, go to Test > Test Explorer or press Ctrl+E, T.
2

Build the Test Project

Build the CalculatorUnitTests project:
  • Right-click CalculatorUnitTests in Solution Explorer
  • Select Build
3

Run Tests

In Test Explorer:
  • Click Run All to run all unit tests
  • Or right-click specific tests to run individually
  • View results in the Test Explorer window

​
Running Unit Tests from Command Line

1

Build the Test Package

msbuild ./src/Calculator.sln `
  -p:Platform=x64 `
  -p:Configuration=Release `
  -t:CalculatorUnitTests
2

Install Test Certificate

The test app requires a certificate to install:
# Sign the test package
./build/scripts/SignTestApp.ps1 `
  -AppToSign ./output/CalculatorUnitTests/AppPackages/CalculatorUnitTests_Test/CalculatorUnitTests.msix
3

Run with VSTest Console

vstest.console.exe `
  ./output/CalculatorUnitTests/AppPackages/CalculatorUnitTests_Test/CalculatorUnitTests.msix `
  /Platform:x64
Unit tests are run automatically in the CI pipeline for both x64 and x86 platforms on non-PR builds.

​
UI Tests

UI tests use Windows Application Driver (WinAppDriver) to automate the Calculator UI and verify end-to-end functionality.

​
Prerequisites for UI Tests

1

Install WinAppDriver

Download and install Windows Application Driver.
# Download the latest release
Invoke-WebRequest 'https://github.com/microsoft/WinAppDriver/releases/download/v1.2.1/WindowsApplicationDriver_1.2.1.msi' `
  -OutFile 'WinAppDriver.msi'

# Install silently
msiexec.exe /i WinAppDriver.msi /quiet /passive
2

Set Display Resolution

UI tests require a specific screen resolution to run reliably:
Set-DisplayResolution -Width 1920 -Height 1080 -Force
This is especially important for CI/CD environments where the display resolution may vary.

​
Running UI Tests

1

Build and Install Calculator

# Build the app package
msbuild ./src/Calculator.sln `
  -p:Platform=x64 `
  -p:Configuration=Release `
  -t:Calculator

# Sign and install the app
./build/scripts/SignTestApp.ps1 `
  -AppToSign './output/Calculator/AppPackages/Calculator*_Test/Calculator_*.msixbundle'

# Run the installation script
./output/Calculator/AppPackages/Calculator*_Test/Add-AppDevPackage.ps1 -Force
2

Build the UI Test Project

msbuild ./src/Calculator.sln `
  -p:Platform=x64 `
  -p:Configuration=Release `
  -p:GenerateProjectSpecificOutputFolder=true `
  -t:Publish `
  -p:PublishDir=./output/publish/
3

Run the UI Tests

vstest.console.exe ./output/publish/CalculatorUITests.dll `
  /Platform:x64 `
  /Settings:./output/publish/CalculatorUITests.ci.runsettings

​
UI Test Framework

The CalculatorUITestFramework project provides:
  • Page object models for Calculator screens
  • Reusable test utilities and helpers
  • WinAppDriver session management
  • Common assertions and validations

​
Manual Tests

Before each release, the Calculator team runs extensive manual tests. These tests cover:

​
Smoke Tests

  • Math operations in Standard, Scientific, and Programmer calculators
  • Converter functionality
  • Always-on-Top mode behavior
  • Memory and history features

​
Verification Tests

  • Launch and basic app functionality
  • All number input methods (mouse, keyboard, number pad)
  • Arithmetic functions across all calculator modes
  • Memory functions (MS, M+, M-, MR, MC)
  • Trigonometric functions (Scientific mode)
  • Logical operations (Programmer mode)
  • Graphing mode features
  • Date calculation
  • Currency conversion

​
Accessibility Tests

  • Narrator compatibility
  • Keyboard navigation
  • High contrast themes
  • Screen reader support

​
Localization Tests

  • Right-to-left (RTL) language support
  • Localized strings and error messages
  • Number format preferences
The complete manual test suite is available in ManualTests.md.

​
Test Coverage

Calculator aims for high test coverage:
  • Unit tests cover core calculation logic, view models, and utilities
  • UI tests verify end-to-end user scenarios
  • Manual tests ensure quality for features that are difficult to automate
✓ CalcManager (calculation engine)
✓ CalcViewModel (view models)
✓ Converter logic
✓ History management
✓ Memory operations
✓ Number formatting

​
Running Tests in CI

The GitHub Actions workflow automatically runs tests: 
# Unit tests run on x64 and x86 (x64 only for PRs)
vstest.console.exe CalculatorUnitTests.msix /Platform:x64

# UI tests run on x64 only
vstest.console.exe CalculatorUITests.dll /Platform:x64
CI tests run on every:
  • Push to main, release/**, or feature/** branches
  • Pull request to main, release/**, or feature/** branches
  • Manual workflow dispatch

​
Writing Tests

When contributing to Calculator, follow these guidelines:
1

Include Tests with Changes

Your pull request should include tests whenever possible. Code should be structured to be unit testable independently of the UI.
2

Use Manual Tests When Needed

For features that cannot be easily automated, document test cases in ManualTests.md.
3

Follow Existing Patterns

Structure your tests similarly to existing tests in the codebase. Use the CalculatorUITestFramework for UI tests.
4

Verify on Multiple Platforms

Ensure tests pass on both x64 and x86 platforms before submitting.

​
Troubleshooting Tests

​
UI Tests Fail to Start

WinAppDriver not running 
Error: Could not connect to WinAppDriver
Ensure WinAppDriver is installed and the service is running: 
Get-Service | Where-Object {$_.Name -like "*WinAppDriver*"}
App not installed 
Error: Application not found
Make sure Calculator is installed before running UI tests: 
Get-AppxPackage | Where-Object {$_.Name -like "*Calculator*"}

​
Unit Tests Fail to Run

Test certificate not trusted 
Error: Package signature validation failed
Run the SignTestApp.ps1 script to install the test certificate. Wrong platform 
Error: Platform mismatch
Ensure the test platform matches the build platform (x64, x86, or ARM64).

​
Next Steps

Debugging

Learn debugging techniques and tips

Contributing

Start contributing to Calculator

Build docs developers (and LLMs) love

Get started for freeTalk to us