Allure Reports

Allure is a flexible, lightweight test reporting tool that provides clear and detailed test reports with rich visualizations. This guide shows you how to integrate Allure reporting with your Patrol tests.

This integration is currently Android-only. iOS support may be added in future releases.

Overview

Allure integration enhances your Patrol test reports with:

Automatic screenshot capture at test completion
Window hierarchy dumps for UI debugging
Logcat logs embedded in reports
Rich test execution timeline
Categorized test results
Trend analysis across test runs

Before you proceed, make sure you’ve completed the native setup guide.

Why Allure?

Allure provides a significantly more detailed view of your test execution compared to standard JUnit reports. It’s particularly useful for:

Debugging test failures with screenshots and logs
Understanding test execution flow
Tracking test trends over time
Sharing results with non-technical stakeholders

Prerequisites

This guide assumes basic familiarity with Allure. If you’re new to Allure:

Setup

Add dependencies

Update your app-level build.gradle file to add Allure dependencies:

android {
  // ...
  defaultConfig {
    // ...
    
    // Replace the existing testInstrumentationRunner
    testInstrumentationRunner "pl.leancode.patrol.example.AllurePatrolJUnitRunner"
  }
  // ...
}

dependencies {
  // ... existing dependencies ...
  
  // Allure dependencies
  androidTestImplementation "io.qameta.allure:allure-kotlin-model:2.4.0"
  androidTestImplementation "io.qameta.allure:allure-kotlin-commons:2.4.0"
  androidTestImplementation "io.qameta.allure:allure-kotlin-junit4:2.4.0"
  androidTestImplementation "io.qameta.allure:allure-kotlin-android:2.4.0"
}

Important: Replace pl.leancode.patrol.example with your app’s actual package name. You’ll find this at the top of your MainActivityTest.java file.

For more details, see the allure-kotlin README.

Create custom test runner

Create a new Kotlin file named AllurePatrolJUnitRunner.kt in the same directory as your MainActivityTest.java file (typically android/app/src/androidTest/java/your/package/):

AllurePatrolJUnitRunner.kt

package pl.leancode.patrol.example // Replace with your app's package name

import android.os.Bundle
import io.qameta.allure.android.AllureAndroidLifecycle
import io.qameta.allure.android.listeners.ExternalStoragePermissionsListener
import io.qameta.allure.android.writer.TestStorageResultsWriter
import io.qameta.allure.kotlin.Allure
import io.qameta.allure.kotlin.junit4.AllureJunit4
import io.qameta.allure.kotlin.util.PropertiesUtils
import pl.leancode.patrol.PatrolJUnitRunner

class AllurePatrolJUnitRunner : PatrolJUnitRunner() {
    override fun onCreate(arguments: Bundle) {
        Allure.lifecycle = createAllureAndroidLifecycle()
        val listenerArg = listOfNotNull(
            arguments.getCharSequence("listener"),
            AllureJunit4::class.java.name,
            ExternalStoragePermissionsListener::class.java.name.takeIf { useTestStorage }
        ).joinToString(separator = ",")
        arguments.putCharSequence("listener", listenerArg)
        super.onCreate(arguments)
    }

    private fun createAllureAndroidLifecycle() : AllureAndroidLifecycle {
      return createDefaultAllureAndroidLifecycle()
    }

    private fun createDefaultAllureAndroidLifecycle() : AllureAndroidLifecycle {
        if (useTestStorage) {
            return AllureAndroidLifecycle(TestStorageResultsWriter())
        }

        return AllureAndroidLifecycle()
    }

    private val useTestStorage: Boolean
        get() = PropertiesUtils.loadAllureProperties()
            .getProperty("allure.results.useTestStorage", "true")
            .toBoolean()
}

Make sure to replace package pl.leancode.patrol.example at the top of the file with your app’s package name!

This custom runner extends Patrol’s PatrolJUnitRunner and adds Allure’s reporting capabilities.

Create allure.properties

Create a properties file at android/app/src/main/res/allure.properties:

android/app/src/main/res/allure.properties

allure.results.useTestStorage=true

This configuration is required if you’re using Android Test Orchestrator with the clearPackageData option. Without it, test reports will be cleared after each test.

Update MainActivityTest

Modify your MainActivityTest.java file to add Allure rules for enhanced reporting:

MainActivityTest.java

package pl.leancode.patrol.example; // Replace with your app's package name

import androidx.test.platform.app.InstrumentationRegistry;

import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameters;

import io.qameta.allure.android.rules.LogcatRule;
import io.qameta.allure.android.rules.ScreenshotRule;
import io.qameta.allure.android.rules.WindowHierarchyRule;
import pl.leancode.patrol.PatrolJUnitRunner;

@RunWith(Parameterized.class)
public class MainActivityTest {
    @Rule
    public ScreenshotRule screenshotRule = new ScreenshotRule(ScreenshotRule.Mode.END, "ss_end");

    @Rule
    public WindowHierarchyRule windowHierarchyRule = new WindowHierarchyRule();

    @Rule
    public LogcatRule logcatRule = new LogcatRule();

    @Parameters(name = "{0}")
    public static Object[] testCases() {
        PatrolJUnitRunner instrumentation = (PatrolJUnitRunner) InstrumentationRegistry.getInstrumentation();
        instrumentation.setUp(MainActivity.class);
        instrumentation.waitForPatrolAppService();
        return instrumentation.listDartTests();
    }

    public MainActivityTest(String dartTestName) {
        this.dartTestName = dartTestName;
    }

    private final String dartTestName;

    @Test
    public void runDartTest() {
        PatrolJUnitRunner instrumentation = (PatrolJUnitRunner) InstrumentationRegistry.getInstrumentation();
        instrumentation.runDartTest(dartTestName);
    }
}

Remember to replace package pl.leancode.patrol.example with your app’s package name!

What these rules do:

ScreenshotRule - Automatically captures a screenshot at the end of each test
WindowHierarchyRule - Dumps the window hierarchy (view tree) at the end of each test
LogcatRule - Embeds logcat output into the test report

Sync Gradle

Sync your project with Gradle:

cd android && ./gradlew --refresh-dependencies

Running Tests

Run your tests as usual with Patrol:

patrol test --target integration_test/app_test.dart

The Allure data will be generated automatically during test execution and stored on the device.

Retrieving Reports

Create reports directory

Create a local directory for the reports:

mkdir -p ./build/reports

Download results from device

Use adb to retrieve the Allure results from the Android device:

adb exec-out sh -c 'cd /sdcard/googletest/test_outputfiles && tar cf - allure-results' | tar xvf - -C build/reports

This command:

Navigates to the test output directory on the device
Creates a tar archive of the allure-results folder
Streams it to your local machine
Extracts it to build/reports/

Generate and view the report

Use the Allure CLI to generate and serve the report:

allure serve ./build/reports/allure-results

This command generates the HTML report and opens it in your default browser.

If you don’t have Allure CLI installed:macOS (Homebrew):

brew install allure

Linux:

curl -o allure-2.24.0.tgz -L https://github.com/allure-framework/allure2/releases/download/2.24.0/allure-2.24.0.tgz
tar -zxvf allure-2.24.0.tgz
sudo mv allure-2.24.0 /opt/allure
sudo ln -s /opt/allure/bin/allure /usr/local/bin/allure

Windows: Download from Allure releases and add to PATH.

Understanding Allure Reports

Report Features

Allure reports provide rich information about your test execution:

Overview

Summary statistics, pass/fail rates, and test duration trends

Suites

Tests organized by suites and categories

Graphs

Visual representation of test results over time

Timeline

Chronological view of test execution

Behaviors

Tests grouped by features and stories

Packages

Tests organized by package structure

Test Details

For each test, Allure captures:

Execution time - How long the test took
Steps - Detailed test steps (if annotated)
Screenshots - Captured at test end
Window hierarchy - UI structure at test end
Logcat - Complete device logs
Parameters - Test input parameters
Categories - Custom test categorization

Advanced Configuration

Custom screenshot timing

Modify ScreenshotRule to capture screenshots at different points:

// Capture at start
@Rule
public ScreenshotRule screenshotRule = new ScreenshotRule(ScreenshotRule.Mode.START, "ss_start");

// Capture on failure only
@Rule
public ScreenshotRule screenshotRule = new ScreenshotRule(ScreenshotRule.Mode.FAILURE, "ss_failure");

// Multiple screenshots
@Rule
public ScreenshotRule screenshotStart = new ScreenshotRule(ScreenshotRule.Mode.START, "ss_start");

@Rule
public ScreenshotRule screenshotEnd = new ScreenshotRule(ScreenshotRule.Mode.END, "ss_end");

Filter logcat output

Customize logcat filtering in LogcatRule:

import io.qameta.allure.android.rules.LogcatRule;

@Rule
public LogcatRule logcatRule = new LogcatRule()
    .setMaxLines(500)  // Limit output
    .setFilter("MyApp:D *:S");  // Filter by tag and level

Generate static reports

Instead of serving the report, generate static HTML files:

# Generate report
allure generate ./build/reports/allure-results -o ./build/reports/allure-report

# Open in browser
open ./build/reports/allure-report/index.html

This is useful for:

Archiving reports
Hosting on web servers
Sharing via file systems

Integrate with CI/CD

Add Allure report generation to your CI pipeline:

GitHub Actions

- name: Run tests
  run: patrol test --target integration_test/app_test.dart

- name: Retrieve Allure results
  run: |
    mkdir -p ./build/reports
    adb exec-out sh -c 'cd /sdcard/googletest/test_outputfiles && tar cf - allure-results' | tar xvf - -C build/reports

- name: Generate Allure report
  uses: simple-elf/allure-report-action@master
  if: always()
  with:
    allure_results: build/reports/allure-results
    allure_history: allure-history

- name: Deploy to GitHub Pages
  uses: peaceiris/actions-gh-pages@v3
  if: always()
  with:
    github_token: ${{ secrets.GITHUB_TOKEN }}
    publish_dir: allure-history

Best Practices

Run on Real Devices

Allure works best with real devices where all logs and screenshots are available.

Clean Between Runs

Clear previous results before new test runs to avoid mixing data:

rm -rf build/reports/allure-results

Archive Reports

Save generated reports for historical comparison and trend analysis.

Review Failures

Use screenshots and logcat to quickly identify root causes of failures.

Troubleshooting

No results found

If you can’t retrieve results from the device:

Verify tests ran successfully
Check that allure.properties is in the correct location
Ensure the device is connected: adb devices

Try listing the directory manually:

adb shell ls -la /sdcard/googletest/test_outputfiles/

Permission denied on device

If you get permission errors when retrieving results:

adb shell pm grant YOUR_PACKAGE_NAME android.permission.WRITE_EXTERNAL_STORAGE
adb shell pm grant YOUR_PACKAGE_NAME android.permission.READ_EXTERNAL_STORAGE

Build errors after adding dependencies

If Gradle sync fails:

Verify all package names match your app
Check that Allure versions are compatible (all 2.4.0)
Clean and rebuild:
```
cd android && ./gradlew clean build
```
Invalidate caches in Android Studio

Report shows no data

If the report loads but shows no tests:

Ensure tests actually ran (check adb logcat)
Verify AllurePatrolJUnitRunner is being used
Check that rules are properly added to MainActivityTest
Confirm allure-results directory exists and has JSON files

Next Steps

Firebase Test Lab

Run tests with Allure reporting on Firebase Test Lab

CI/CD Integration

Automate Allure report generation in your pipeline

CI/CD

Integrations

Feature Guides

Best Practices

Allure Reports

Overview

Why Allure?

Prerequisites

Setup

Running Tests

Retrieving Reports

Understanding Allure Reports

Report Features

Overview

Suites

Graphs

Timeline

Behaviors

Packages

Test Details

Advanced Configuration

Best Practices

Run on Real Devices

Clean Between Runs

Archive Reports

Review Failures

Troubleshooting

Next Steps

Firebase Test Lab

CI/CD Integration

Resources

Build docs developers (and LLMs) love

CI/CD

Integrations

Feature Guides

Best Practices

​Overview

​Why Allure?

​Prerequisites

​Setup

​Running Tests

​Retrieving Reports

​Understanding Allure Reports

​Report Features

Overview

Suites

Graphs

Timeline

Behaviors

Packages

​Test Details

​Advanced Configuration

​Best Practices

Run on Real Devices

Clean Between Runs

Archive Reports

Review Failures

​Troubleshooting

​Next Steps

Firebase Test Lab

CI/CD Integration

​Resources

Build docs developers (and LLMs) love

Overview

Why Allure?

Prerequisites

Setup

Running Tests

Retrieving Reports

Understanding Allure Reports

Report Features

Test Details

Advanced Configuration

Best Practices

Troubleshooting

Next Steps

Resources