Testing Dalamud

Learn more about Mintlify

Enter your email to receive updates about new features and product releases.

Test Framework
Running Tests
Using Build Scripts
NUKE Build Targets
Test Structure
Writing Unit Tests
Basic Test Structure
Test Attributes
Assertions
Integration Testing
Testing Strategies
Testing with Dalamud.CorePlugin
CI/CD Testing
GitHub Actions Workflow
API Compatibility Checks
Manual Testing Checklist
Before Submitting a PR
Debugging Tests
Visual Studio
Command Line
Test Coverage
Measuring Coverage
Performance Testing
Benchmarking with BenchmarkDotNet
Troubleshooting Test Failures
Best Practices
Next Steps

Testing is crucial for maintaining Dalamud’s stability. This guide covers testing approaches for both development and contributions.

Test Framework

Dalamud uses xunit v3 for unit testing.

Dalamud.Test/Dalamud.Test.csproj

<ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" />
    <PackageReference Include="xunit.v3" />
    <PackageReference Include="xunit.v3.assert" />
    <PackageReference Include="xunit.runner.visualstudio" />
</ItemGroup>

Test projects target net10.0-windows and reference the main Dalamud project.

Running Tests

Using Build Scripts

Windows
Command Line

# Run tests only
./build.ps1 Test

# Build and test (CI target)
./build.ps1 CI

# Using dotnet CLI
dotnet test Dalamud.Test/Dalamud.Test.csproj --configuration Release

# With verbose output
dotnet test Dalamud.Test/Dalamud.Test.csproj -v detailed

# Run specific test
dotnet test --filter "FullyQualifiedName~LocalizationTests"

NUKE Build Targets

The Test target in the build system:

build/DalamudBuild.cs

Target Test => _ => _
    .DependsOn(Compile)
    .Executes(() =>
    {
        DotNetTasks.DotNetTest(s => s
            .SetProjectFile(TestProjectFile)
            .SetConfiguration(Configuration)
            .AddProperty("WarningLevel", "0")
            .EnableNoRestore());
    });

This ensures:

Full compilation before testing
Tests run with same configuration (Debug/Release)
No package restore during test execution

Test Structure

Tests are organized in the Dalamud.Test project:

Dalamud.Test/
├── Game/
│   ├── Text/
│   │   ├── SeStringHandling/
│   │   │   ├── SeStringTests.cs
│   │   │   └── SeStringManagerTests.cs
│   │   └── Sanitizer/
│   │       └── SanitizerTests.cs
│   ├── GameVersionTests.cs
│   └── GameVersionConverterTests.cs
├── Storage/
│   └── ReliableFileStorageTests.cs
└── LocalizationTests.cs

Writing Unit Tests

Basic Test Structure

using System.IO;
using System.Reflection;
using Xunit;

namespace Dalamud.Test
{
    public class LocalizationTests
    {
        private readonly Localization localization;
        private string currentLangCode;

        public LocalizationTests()
        {
            // Setup in constructor
            var workingDir = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
            this.localization = new Localization(workingDir, "dalamud_");
            this.localization.LocalizationChanged += code => this.currentLangCode = code;
        }

        [Fact]
        public void SetupWithFallbacks_EventInvoked()
        {
            // Arrange & Act
            this.localization.SetupWithFallbacks();
            
            // Assert
            Assert.Equal("en", this.currentLangCode);
        }
    }
}

Test Attributes

Attribute	Purpose	Example
`[Fact]`	Single test case	`[Fact] public void Test() { }`
`[Theory]`	Data-driven test	`[Theory] [InlineData(1)] public void Test(int x) { }`
`[InlineData]`	Provides test data	`[InlineData("input", "expected")]`
`[Skip]`	Skip test	`[Fact(Skip = "Not implemented")]`

Assertions

Common Assertions

using Xunit;

// Equality
Assert.Equal(expected, actual);
Assert.NotEqual(value1, value2);

// Nullability
Assert.Null(value);
Assert.NotNull(value);

// Boolean
Assert.True(condition);
Assert.False(condition);

// Collections
Assert.Empty(collection);
Assert.NotEmpty(collection);
Assert.Contains(item, collection);
Assert.All(collection, item => Assert.NotNull(item));

// Exceptions
Assert.Throws<InvalidOperationException>(() => MethodThatThrows());

// Types
Assert.IsType<MyType>(obj);
Assert.IsAssignableFrom<IMyInterface>(obj);

Integration Testing

Dalamud cannot be fully tested without FFXIV running due to its nature as an in-game framework.

Testing Strategies

Unit test pure logic

Test components that don’t require game context:

String parsing (SeString, game versions)
Data structures and algorithms
Configuration serialization
Localization loading

Use mocking for game dependencies

For code that interacts with game state:

// Create mock implementations of game interfaces
public class MockGameGui : IGameGui
{
    public bool IsHovered { get; set; }
    // Implement other interface members
}

Manual testing in-game

Critical for:

UI components and overlays
Game hooks and detours
Plugin loading and unloading
Chat and network functionality

Testing with Dalamud.CorePlugin

The Dalamud.CorePlugin project serves as an internal testbed:

Dalamud.CorePlugin/Dalamud.CorePlugin.csproj

<ItemGroup>
    <ProjectReference Include="..\Dalamud\Dalamud.csproj" />
</ItemGroup>

Purpose:

Access Dalamud internals during development
Prototype new API features before making them public
Test plugin-like functionality with privileged access

CI/CD Testing

GitHub Actions Workflow

Tests run automatically on every push and PR:

.github/workflows/main.yml

- name: Build and Test Dalamud
  run: .\build.ps1 ci

API Compatibility Checks

Pull requests automatically verify API compatibility:

API Compatibility Workflow

- name: "Verify Compatibility"
  run: |
    $FILES_TO_VALIDATE = "Dalamud.dll","FFXIVClientStructs.dll","Lumina.dll","Lumina.Excel.dll"
    
    foreach ($file in $FILES_TO_VALIDATE) {
        apicompat -l "left\${file}" -r "right\${file}" --noWarn "CP0006"
    }

This compares:

Left: Current staging build from distribution
Right: Proposed changes in the PR

API breaking changes will cause the compatibility check to fail.

Manual Testing Checklist

Before Submitting a PR

Core Functionality

Dalamud loads successfully via XIVLauncher
All existing plugins continue to work
No crashes during normal gameplay
Game hooks function correctly
Plugin installer works

UI Components

Settings window displays properly
Plugin windows render correctly
No UI scaling issues
ImGui overlays work in windowed/fullscreen
Color schemes apply correctly

Plugin API Changes

Sample plugin compiles against changes
Breaking changes documented
Migration path provided for developers
API documentation updated

Performance

No significant FPS drops
Memory usage is reasonable
No memory leaks over extended play
Network performance unchanged

Debugging Tests

Visual Studio

Open Test Explorer

View → Test Explorer (Ctrl+E, T)

Set breakpoints

Add breakpoints in your test code

Debug test

Right-click test → Debug

Command Line

# Debug with verbose output
dotnet test --logger "console;verbosity=detailed"

# Run with debugger attached (VSCode)
dotnet test --logger "console" --environment VSTEST_HOST_DEBUG=1

# Generate code coverage
dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura

Test Coverage

While Dalamud doesn’t enforce coverage requirements, aim for:

Coverage Goals

Critical paths: 80%+ coverage
Public APIs: All major code paths tested
Bug fixes: Regression tests for each fix
Utilities: High coverage for pure functions

Measuring Coverage

Using Coverlet:

dotnet test /p:CollectCoverage=true \
            /p:CoverletOutputFormat=cobertura \
            /p:CoverletOutput=./coverage/

# Generate HTML report
reportgenerator -reports:./coverage/coverage.cobertura.xml \
                -targetdir:./coverage/report \
                -reporttypes:Html

Performance Testing

Benchmarking with BenchmarkDotNet

For performance-critical code:

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

[MemoryDiagnoser]
public class SeStringBenchmarks
{
    private readonly byte[] testData;
    
    [Benchmark]
    public void ParseSeString()
    {
        var result = SeString.Parse(testData);
    }
}

// Run benchmarks
BenchmarkRunner.Run<SeStringBenchmarks>();

Troubleshooting Test Failures

Test discovery fails

Symptoms: No tests appear in Test ExplorerSolutions:

Rebuild the solution
Clear Test Explorer cache
Verify xunit.v3 packages are restored
Check test class is public

Tests pass locally but fail in CI

Common causes:

Environment differences (paths, culture, timezone)
Missing test data files
Platform-specific behavior

Debug approach:

Check CI logs for error details
Ensure test data is included in project
Use [Fact(Skip = "CI only")] for problematic tests

Intermittent test failures

Common causes:

Race conditions in async code
Shared state between tests
Time-dependent assertions

Solutions:

Use proper async/await patterns
Ensure test isolation
Avoid Thread.Sleep, use proper synchronization

Best Practices

Fast Tests

Keep unit tests fast (<1s each). Move slow tests to integration category.

Isolated Tests

Each test should be independent. Don’t rely on test execution order.

Clear Names

Use descriptive test names: MethodName_Scenario_ExpectedBehavior

Arrange-Act-Assert

Structure tests clearly: setup, execute, verify.

Next Steps

Building Dalamud

Learn how to build from source

Contributing

Submit your tested changes

Contributing to Dalamud

Troubleshooting

⌘I

Build docs developers (and LLMs) love

Get started for free Talk to us

Get Started

Core Concepts

Plugin Development

UI & ImGui

Game Integration

IPC & Shared Data

Advanced Topics

Guides

​Test Framework

​Running Tests

​Using Build Scripts

​NUKE Build Targets

​Test Structure

​Writing Unit Tests

​Basic Test Structure

​Test Attributes

​Assertions

​Integration Testing

​Testing Strategies

​Testing with Dalamud.CorePlugin

​CI/CD Testing

​GitHub Actions Workflow

​API Compatibility Checks

​Manual Testing Checklist

​Before Submitting a PR

​Debugging Tests

​Visual Studio

​Command Line

​Test Coverage

Coverage Goals

​Measuring Coverage

​Performance Testing

​Benchmarking with BenchmarkDotNet

​Troubleshooting Test Failures

​Best Practices

Fast Tests

Isolated Tests

Clear Names

Arrange-Act-Assert

​Next Steps

Building Dalamud

Contributing

Build docs developers (and LLMs) love

Test Framework

Running Tests

Using Build Scripts

NUKE Build Targets

Test Structure

Writing Unit Tests

Basic Test Structure

Test Attributes

Assertions

Integration Testing

Testing Strategies

Testing with Dalamud.CorePlugin

CI/CD Testing

GitHub Actions Workflow

API Compatibility Checks

Manual Testing Checklist

Before Submitting a PR

Debugging Tests

Visual Studio

Command Line

Test Coverage

Measuring Coverage

Performance Testing

Benchmarking with BenchmarkDotNet

Troubleshooting Test Failures

Best Practices

Next Steps