Skip to main content
The Application SDK provides a comprehensive test framework to help you write end-to-end tests for your applications. The framework includes built-in test methods that verify everything from basic connectivity to full workflow execution and data validation.
See the atlan-postgres-app test directory for complete examples.

​
BaseTest Class

The BaseTest class provides a base implementation for end-to-end testing of workflows. It includes several key test methods that run in a specific order to validate your application comprehensively.

​
Test Execution Order

The tests are executed in a predefined sequence using pytest’s ordering:
1

Health Check

test_health_check (Order 1)Verifies that the server is running and responding to requests by making a GET request to the host.
2

Authentication

test_auth (Order 2)Tests the authentication and connection flow by calling test_connection with provided credentials and validating against expected responses.
3

Metadata Retrieval

test_metadata (Order 3)Tests metadata retrieval by calling get_metadata with credentials and comparing to expected responses.
4

Preflight Checks

test_preflight_check (Order 4)Executes preflight checks using credentials and metadata, validating against expected responses.
5

Workflow Execution

test_run_workflow (Order 5)Tests the full workflow execution:
  • Starts the workflow with credentials, metadata, and connection details
  • Monitors the workflow until completion
  • Raises a WorkflowExecutionError if workflow does not complete successfully
6

Data Validation

test_data_validation (Order 6)Validates the extracted source data against schema definitions using Pandera.
The execution order is specified using the @pytest.mark.order decorator. If you add custom test methods, use order values greater than 6 to run them after the built-in tests.

​
Required Properties

The BaseTest class requires several properties to be defined:
​
config_file_path
string
required
Path to the configuration file that defines test scenarios
​
extracted_output_base_path
string
required
Base path for extracted output files (set in the override file)
​
schema_base_path
string
required
Base path for schema files used in data validation
​
credentials
dict
required
Dictionary containing authentication credentials
​
metadata
dict
required
Dictionary containing metadata configuration
​
connection
dict
required
Dictionary containing connection details

​
Configuration

The test framework uses a YAML configuration file to configure tests and workflows.

Example Configuration

See a complete configuration example in the atlan-postgres-app repository.

​
Configuration Structure

Here’s a typical configuration structure: 
credentials:
  authType: basic
  host: localhost
  port: 5432
  username: postgres
  password: password
  database: test_db

connection:
  connection_name: test-postgres-connection
  connection_qualified_name: default/postgres/test

metadata:
  exclude-filter: "{}"
  include-filter: "{}"
  temp-table-regex: "^temp_"
  extraction-method: direct
  exclude_views: "false"
  exclude_empty_tables: "false"

tenant_id: test-tenant

schema_base_path: tests/e2e/test_postgres_workflow/schema
extracted_output_base_path: /tmp/test_output
Each configuration file corresponds to a test scenario. To add more test scenarios, copy the directory and modify the config file.

​
Data Validation

The test_data_validation method validates extracted data against schema definitions using Pandera, a powerful data validation library for pandas DataFrames.

​
Schema Files

Schema files define the expected structure and data types for extracted metadata: 
import pandera as pa
from pandera.typing import Series

class DatabaseSchema(pa.SchemaModel):
    database_name: Series[str] = pa.Field(nullable=False)
    
    class Config:
        strict = True

Example Schema Files

View complete schema examples in the atlan-postgres-app repository.

​
Custom Validation

You can override the test_data_validation method to add custom validation logic: 
import pandas as pd
import pandera as pa
from application_sdk.test_utils import BaseTest

class CustomTest(BaseTest):
    def test_data_validation(self):
        # Call parent validation
        super().test_data_validation()
        
        # Add custom validation logic
        table_df = pd.read_parquet(
            f"{self.extracted_output_base_path}/tables.parquet"
        )
        
        # Custom check: ensure no empty table names
        assert not table_df["table_name"].str.strip().eq("").any(), \
            "Found tables with empty names"
        
        # Custom check: validate table count
        assert len(table_df) > 0, "No tables extracted"
        
        print(f"Validated {len(table_df)} tables successfully")

​
Running Tests

Run tests using pytest from your project root: 
pytest tests/e2e/test_postgres_workflow

​
Test Output

The test framework provides detailed output for each test: 
tests/e2e/test_postgres_workflow/test_basic.py::TestPostgresWorkflow::test_health_check PASSED
tests/e2e/test_postgres_workflow/test_basic.py::TestPostgresWorkflow::test_auth PASSED
tests/e2e/test_postgres_workflow/test_basic.py::TestPostgresWorkflow::test_metadata PASSED
tests/e2e/test_postgres_workflow/test_basic.py::TestPostgresWorkflow::test_preflight_check PASSED
tests/e2e/test_postgres_workflow/test_basic.py::TestPostgresWorkflow::test_run_workflow PASSED
tests/e2e/test_postgres_workflow/test_basic.py::TestPostgresWorkflow::test_data_validation PASSED
Use the -s flag with pytest to see print statements and logs: pytest tests/e2e/test_postgres_workflow -s

​
Adding Custom Test Steps

To add additional test steps after the built-in ones: 
import pytest
from application_sdk.test_utils import BaseTest

class CustomTest(BaseTest):
    # ... existing test setup ...
    
    @pytest.mark.order(7)  # Run after test_data_validation (order 6)
    def test_custom_validation(self):
        """Custom validation logic."""
        # Your custom test logic here
        pass
    
    @pytest.mark.order(8)
    def test_cleanup(self):
        """Clean up test resources."""
        # Your cleanup logic here
        pass
Always use order values greater than 6 for custom test methods to ensure they run after the built-in validation tests.

​
Multiple Test Scenarios

You can create multiple test scenarios by copying the test directory structure: 
tests/e2e/
├── test_postgres_workflow_basic/
│   ├── config.yaml
│   ├── schema/
│   └── test_basic.py
├── test_postgres_workflow_filtered/
│   ├── config.yaml
│   ├── schema/
│   └── test_filtered.py
└── test_postgres_workflow_views/
    ├── config.yaml
    ├── schema/
    └── test_views.py
Each directory represents a different test scenario with its own configuration.

​
Best Practices

Create separate test directories for different scenarios (basic extraction, filtered extraction, view handling, etc.).
Name test files and methods descriptively to make test results easy to understand.
Define comprehensive schema files that validate all expected fields and data types.
Create test scenarios for edge cases like empty databases, special characters, or large datasets.
Add cleanup steps to remove temporary files and resources after tests complete.
Keep test configuration files in version control, but never commit actual credentials.

​
Troubleshooting

  • Verify the application server is running
  • Check the host and port in your configuration
  • Ensure firewall rules allow connections
  • Verify credentials in the configuration file
  • Check that the database user has necessary permissions
  • Ensure the authentication type matches your database setup
  • Check Temporal server logs for errors
  • Verify worker is running and registered
  • Review activity logs for specific error messages
  • Check schema files match the actual extracted data structure
  • Review Pandera error messages for specific validation failures
  • Verify extracted output files exist at the expected paths

​
Next Steps

Best Practices

Learn best practices for building scalable applications.

SQL Applications

Build comprehensive SQL metadata extraction applications.

PostgreSQL Example

Explore a complete production-ready example with tests.

Pandera Documentation

Learn more about data validation with Pandera.

Build docs developers (and LLMs) love

Get started for freeTalk to us