Skip to main content

​
Overview

Testing is critical for maintaining Helium’s stability, performance, and privacy guarantees. This guide covers the testing utilities, validation scripts, and best practices for ensuring code quality.

​
Testing Philosophy

Core Testing Principles

  • Privacy First - Ensure no data leakage or tracking
  • Cross-Platform - Test on Windows, macOS, and Linux
  • Regression Prevention - Validate patches don’t break existing functionality
  • Automated Validation - Use provided scripts before submitting changes

​
Development Utilities

Helium provides comprehensive testing utilities in the devutils/ directory:

​
Validation Scripts

validate_patches.py

Validates that all patches apply cleanly against the Chromium source tree

validate_config.py

Runs sanity checks over all configuration files

check_patch_files.py

Verifies patch file integrity and series consistency

check_gn_flags.py

Ensures GN flags are sorted and not duplicated

​
Running Tests

​
Complete Validation Suite

Run all validation checks before submitting changes: 
./devutils/validate_config.py
This comprehensive script checks:
  • All patches exist and are referenced
  • No duplicate patches in series
  • Patches are readable and properly formatted
  • GN flags in flags.gn are sorted and unique
  • downloads.ini has correct format and schema
Exit code 0 means no problems detected. Exit code 1 indicates warnings or errors.

​
Patch Validation

Validates patches against files downloaded from Google:
./devutils/validate_patches.py --remote
Features:
  • Downloads only required source files
  • Validates against exact Chromium version
  • No local source tree needed
Requirements:
  • Python requests module
  • Internet connection
Options:
# Specify custom series file
./devutils/validate_patches.py --remote \
  --series patches/series \
  --patches patches/

# Enable verbose output for debugging
./devutils/validate_patches.py --remote --verbose

​
Patch File Checks

Validate patch file integrity: 
./devutils/check_patch_files.py
Checks performed:
1

Patch Readability

Verifies all patches in the series file are:
  • Present on disk
  • Parseable as valid unified diff format
  • Free of syntax errors
2

Series Consistency

Ensures:
  • No duplicate entries in series file
  • All patches are referenced exactly once
3

Unused Patches

Identifies patch files that exist but aren’t referenced in series (excluding .md files).
Custom patches directory: 
./devutils/check_patch_files.py --patches /path/to/patches

​
GN Flags Validation

Verify GN build flags are properly formatted: 
./devutils/check_gn_flags.py
Validates:
  • Flags are sorted alphabetically
  • No duplicate flag definitions
  • Proper flag syntax
Custom flags file: 
./devutils/check_gn_flags.py --flags-gn /path/to/flags.gn

​
Python Code Quality

Helium includes utilities for maintaining Python code quality:

​
Linting

# Lint development utilities
./devutils/run_devutils_pylint.py

​
Code Formatting

Format code with YAPF (Yet Another Python Formatter): 
./devutils/run_devutils_yapf.sh

​
Unit Tests

Run Python unit tests: 
./devutils/run_devutils_tests.sh
Test coverage:
  • Patch validation logic
  • Domain substitution
  • Configuration parsing
  • File integrity checks

​
Pre-Commit Checklist

Before committing changes, run this comprehensive checklist:
1

Validate all code

./devutils/check_all_code.sh
Runs all linting, formatting, and testing utilities.
2

Validate configuration

./devutils/validate_config.py
Ensures all config files are valid.
3

Validate patches

./devutils/validate_patches.py --remote
Confirms patches apply cleanly.
4

Build and test locally

Build Helium with your changes and perform manual testing on your platform.

​
Platform-Specific Testing

​
Windows Testing

  • Verify builds complete without errors
  • Test installer generation
  • Validate file associations
  • Check update mechanisms (not yet auto-update)
  • Browser launch and stability
  • Extensions (uBlock Origin, etc.)
  • Privacy features
  • UI rendering and responsiveness

​
macOS Testing

  • Universal binary (Intel + Apple Silicon)
  • Code signing and notarization
  • DMG packaging
  • Auto-update functionality
  • Native menu integration
  • Keyboard shortcuts
  • Touch Bar support (if applicable)
  • macOS-specific privacy features

​
Linux Testing

  • AppImage generation
  • Dependencies bundling
  • Desktop integration
  • Auto-update via AppImageUpdate
  • Wayland and X11 compatibility
  • System theme integration
  • File picker integration
  • Clipboard functionality

​
Debugging Test Failures

​
Verbose Output

Enable detailed logging for debugging: 
./devutils/validate_patches.py --remote --verbose
Provides:
  • Line-by-line patch application details
  • Exact mismatch locations
  • Context from source files
  • patch --dry-run output for failures

​
Common Test Failures

Symptoms:
Patch failed validation: ungoogled-chromium/some-patch.patch
Line 'old_code' does not match removal line 'new_code' from patch
Solutions:
  1. Check Chromium version matches
  2. Refresh patch against current source
  3. Manually resolve conflicts
  4. Re-run validation
Symptoms:
In GN flags flags.gn, "flag_b" should be sorted before "flag_a"
Solution: Manually sort flags alphabetically in flags.gn
Symptoms:
Patch appears more than once in series: some-patch.patch
Solution: Remove duplicate entry from patches/series
Symptoms:
Unused patch: ungoogled-chromium/old-patch.patch
Solutions:
  • Add patch to series file if needed
  • Remove patch file if obsolete
  • Document reason if intentionally unused
Symptoms:
pylint: error: unrecognized arguments
Solution: Auto-format and fix linting issues:
./devutils/run_devutils_yapf.sh
./devutils/run_devutils_pylint.py

​
Continuous Integration

​
Automated Testing

When you submit a pull request:
  1. Validation runs automatically
    • Patch validation
    • Config validation
    • Code linting
    • Unit tests
  2. Platform builds triggered
    • Windows build
    • macOS build (Intel + ARM)
    • Linux AppImage
  3. Manual testing required
    • You must test locally before PR
    • Indicate tested platforms in PR template

​
Local CI Simulation

Simulate CI checks locally: 
# Run all validation
./devutils/check_all_code.sh
./devutils/validate_config.py
./devutils/validate_patches.py --remote

# Check for warnings
if [ $? -eq 0 ]; then
  echo "✓ All checks passed"
else
  echo "✗ Some checks failed - review output above"
fi

​
Manual Testing Guidelines

​
Privacy Testing

1

Network traffic analysis

Use tools like Wireshark or mitmproxy to verify:
  • No unexpected Google domains
  • No telemetry or analytics
  • No tracking beacons
2

Storage inspection

Check for:
  • No unexpected cookies
  • No tracking tokens in localStorage
  • Proper incognito mode isolation
3

Feature verification

Test that privacy features work:
  • uBlock Origin functionality
  • Privacy flags effectiveness
  • Referrer customization
  • WebRTC leak prevention

​
Functional Testing

  • Basic browsing - Load various websites
  • Extensions - Install and use extensions
  • Downloads - Test download functionality
  • Forms - Test autofill and form submission
  • Media - Play video and audio
  • PDF viewer - Open and view PDFs
  • Settings - Navigate and modify settings

​
Performance Testing

  • Startup time - Measure cold and warm starts
  • Memory usage - Monitor with multiple tabs
  • CPU usage - Check during heavy workloads
  • Responsiveness - Test UI lag and jank

​
Reporting Test Results

When reporting test results in PRs: 
## Test Results

### Automated Tests
- [x] validate_config.py - PASSED
- [x] validate_patches.py --remote - PASSED
- [x] check_all_code.sh - PASSED

### Manual Testing
Tested on:
- [x] macOS 14.2 (Apple Silicon)
- [ ] Windows 11
- [ ] Linux (Ubuntu 22.04)

### Functionality
- [x] Privacy features working
- [x] No regressions found
- [x] Performance acceptable
- [x] UI renders correctly

### Issues Found
- None
Comprehensive testing helps maintain Helium’s quality and privacy guarantees. Always test thoroughly before submitting changes.

Build docs developers (and LLMs) love

Get started for freeTalk to us