Skip to main content
Syft-Flwr uses pytest for testing, with separate test suites for unit tests, in-memory integration tests, and Google Drive integration tests.

​
Test Organization

Tests are organized into different categories: 
tests/
├── unit/                   # Fast unit tests (run in CI)
└── integration/
    ├── simulation/         # In-memory integration tests (run in CI)
    └── syft-client/        # Google Drive integration tests (manual)

​
Running Tests

​
Unit Tests

Unit tests are fast and run automatically in CI. They test individual components in isolation. 
just test-unit

​
In-Memory Integration Tests

These integration tests use in-memory simulation and don’t require Google OAuth credentials. They run in CI. 
just test-integration-inmemory

​
Google Drive Integration Tests

These tests require Google OAuth credentials and test the full federated learning workflow using Google Drive as the transport layer. They are marked as slow and must be run manually.
Google Drive integration tests require OAuth credentials. See the Google OAuth Setup section in the Development Setup guide for configuration instructions.
just test-integration-gdrive

​
All Tests

Run the complete test suite including unit and integration tests: 
just test

​
Test Markers

Tests use pytest markers to categorize and filter test execution:
  • slow: Integration tests with Google Drive (excluded from CI)

​
Running Tests by Marker

pytest -m slow -v -s

​
Writing Tests

​
Test Structure

Follow these conventions when writing tests: 
import pytest
from syft_flwr import YourModule

def test_basic_functionality():
    """Test description."""
    result = YourModule.function()
    assert result == expected_value

@pytest.mark.slow
def test_integration_with_gdrive():
    """Integration test requiring Google OAuth."""
    # Test implementation
    pass

@pytest.fixture
def sample_data():
    """Reusable test fixture."""
    return {"key": "value"}

def test_with_fixture(sample_data):
    """Test using a fixture."""
    assert sample_data["key"] == "value"

​
Best Practices

  • Test one thing at a time
  • Use descriptive test names that explain what is being tested
  • Keep tests fast and isolated
  • Mock external dependencies
  • Use fixtures for common test data
  • Mark slow tests with @pytest.mark.slow
  • Test real-world scenarios and workflows
  • Clean up resources after tests
  • Document any required setup or credentials
  • Use @pytest.mark.asyncio for async tests
  • Ensure proper cleanup of async resources
  • Use pytest-asyncio fixtures when needed

​
CI/CD Testing

​
Automated Tests (GitHub Actions)

Unit and in-memory integration tests run automatically on every push/PR to main branch:
  • Operating Systems: Ubuntu, Windows, macOS
  • Python Versions: 3.10, 3.11, 3.12

​
Manual Integration Tests

Google Drive integration tests can be triggered manually via GitHub Actions. Required secrets must be configured in repository settings:
Secret NameContent
GDRIVE_CRED_DO1Contents of do1.json
GDRIVE_CRED_DO2Contents of do2.json
GDRIVE_CRED_DSContents of ds.json
GDRIVE_TOKEN_DO1Contents of token_do1.json
GDRIVE_TOKEN_DO2Contents of token_do2.json
GDRIVE_TOKEN_DSContents of token_ds.json
EMAIL_DO1DO1 email address
EMAIL_DO2DO2 email address
EMAIL_DSDS email address
1

Navigate to Actions

Go to the GitHub repository → Actions tab
2

Select workflow

Select “Integration Tests (Google Drive)” workflow
3

Run workflow

Click “Run workflow” button to trigger manual execution

​
Test Configuration

​
pytest Configuration

The project’s pyproject.toml includes pytest configuration: 
[tool.pytest.ini_options]
markers = [
    "slow: marks tests as slow (integration tests with Google Drive)",
]

​
Parallel Test Execution

Unit tests use pytest-xdist for parallel execution to speed up test runs: 
# Automatically detect CPU count and parallelize
pytest -n auto

# Use specific number of workers
pytest -n 4

​
Coverage Reports

Generate code coverage reports to identify untested code: 
uv run pytest tests/ --cov=syft_flwr --cov-report=term-missing

​
Troubleshooting

Ensure you’ve installed the package in editable mode:
uv pip install -e .
uv sync --group dev
  1. Verify credentials/.env is configured correctly
  2. Check that OAuth credentials files exist in credentials/
  3. Ensure the OAuth app is published (not in testing mode)
  4. Re-authenticate by deleting token files and running tests again
  • Check for deadlocks in async code
  • Increase timeout values for slow operations
  • Ensure proper cleanup of resources
  • Run tests with verbose output: pytest -v -s
  • Check for shared state between tests
  • Use fixtures to isolate test data
  • Avoid using global variables
  • Run with -n 1 to disable parallelization for debugging

​
Next Steps

Build docs developers (and LLMs) love

Get started for freeTalk to us