Getting Started

Prerequisites

Before you begin, ensure you have the following tools installed:

Docker

Required for running local databases and services

Doppler

Secret management for environment variables

AWS CLI

For AWS service integration and deployment

Just

Task runner for development commands

sqlx-cli

Database migration management

Pulumi CLI

Infrastructure as code management

Rust & Cargo

For building backend services

Bun

For frontend development

Initial Setup

Currently, we only support running services against dev-assets. This is a work in progress.

Step 1: Environment Configuration

Export the SOPS KMS ARN for secret decryption:

export SOPS_KMS_ARN="arn:aws:kms:us-east-1:569036502058:key/mrk-cab29bf948044eb79005a81f48d40e93,arn:aws:kms:us-west-1:569036502058:key/mrk-cab29bf948044eb79005a81f48d40e93"

You can skip this step if you’re using nix-shell

Step 2: One-Command Setup

Run the complete setup process:

just setup

This single command performs the following operations:

Environment Setup

Decrypts and loads environment variables from encrypted .env files

Network Creation

Creates Docker networks (databases and auth) for service communication

LocalStack Setup

Starts LocalStack for local AWS emulation and creates:

SQS queues for async processing
DynamoDB tables for connection tracking
S3 buckets for file storage

Database Initialization

Sets up PostgreSQL databases:

Creates MacroDB
Runs database migrations

FusionAuth Setup

Configures authentication service

Service Images

Builds Docker images for all dev services

Running the Application

Backend Services

Start all backend services using Docker Compose:

just run_local

By default, convert_service and search_processing_service are not run locally as they are not needed by the frontend when using dev assets.

Rebuilding Services

If you’ve updated a service and need to rebuild the Docker image:

just run_local --build

This triggers Docker Compose to update the containers with your changes.

Stopping Services

To stop all running services:

just stop-local

Frontend Application

Run the frontend pointed to local services:

cd js/app && bun i && just local

This will:

Install frontend dependencies
Start the development server
Connect to locally running backend services

Database Management

Setting Up Individual Databases

If you need to set up databases separately:

# Main database
just rust/cloud-storage/macro_db_client/create_db
just rust/cloud-storage/macro_db_client/migrate_db

Or use the helper commands:

just setup_local_dbs    # Sets up all local databases
just run_dbs            # Runs database containers
just stop-databases     # Stops database containers

Running Databases Only

To run just the databases without other services:

just run_dbs

To run them in detached mode:

just run_dbs -d

LocalStack (Local AWS)

Macro uses LocalStack to emulate AWS services locally:

Manual LocalStack Setup

just setup_localstack

This creates: SQS Queues:

notification-queue
push-delivery-queue
email-service-backfill-queue
delete-chat-handler-queue
contacts-queue
convert-service-queue
search-event-queue
And more…

DynamoDB Tables:

bulk-upload
connection-gateway-table
static-file-metadata

S3 Buckets:

macro-email-attachments
doc-storage
docx-upload
static-file-storage
bulk-upload-staging

Individual LocalStack Commands

just start_localstack      # Start LocalStack container
just create_local_queues   # Create SQS queues
just create_local_tables   # Create DynamoDB tables
just create_local_buckets  # Create S3 buckets

FusionAuth Setup

Manage the authentication service:

just setup_fusionauth   # Setup and configure FusionAuth
just stop_fusionauth    # Stop FusionAuth containers

Building Services

Build All Services

just build

Build Lambda Functions

just build_lambdas

Individual Lambda builds:

just build_document_text_extractor
just build_docx_unzip_handler
just build_delete_chat_handler
just build_upload_extractor_lambda_handler
just build_email_suppression
just build_deleted_item_poller

Type Check Without Building

just check

Testing

Setup Test Environment

Start Databases

docker-compose up -d macrodb

Setup Test Env Files

just setup_test_envs

Initialize Databases

just initialize_dbs

Run Tests

just test

Test Individual Services

Test a specific service from the rust/cloud-storage/ directory:

cargo test -p {service_name}

Example:

cargo test -p document-storage-service

Environment Management

Macro uses encrypted environment files managed by SOPS:

Get Environment Variables

just get_environment

This decrypts .env-local.enc and creates .env with your AWS credentials from ~/.aws/credentials.

Edit Environment

just edit_environment

Fix Environment (Re-encrypt)

just fix_environment

Docker Cache Management

Clear All Build Cache

just docker_cache_clear

This forces a full cold rebuild on the next build.

Clear Only Rust Target Caches

just docker_cache_clear_targets

This keeps downloaded crates but forces recompilation.

View Cache Usage

just docker_cache_usage

Cleanup

Destroy all local infrastructure:

just destroy

This will:

Destroy FusionAuth infrastructure
Stop and remove all Docker containers
Remove Docker volumes

Next Steps

Architecture

Learn about the system architecture

Directory Structure

Understand the codebase organization

Troubleshooting

Services Won’t Start

Ensure Docker networks are created:

just create_networks

Database Connection Errors

Verify databases are running:

docker ps | grep postgres

Restart databases if needed:

just stop-databases
just run_dbs -d

LocalStack Issues

Restart LocalStack:

docker rm -f localstack
just start_localstack

Build Failures

Clear Docker cache and rebuild:

just docker_cache_clear
just build

Overview

Frontend

Backend

Infrastructure

​Prerequisites

Docker

Doppler

AWS CLI

Just

sqlx-cli

Pulumi CLI

Rust & Cargo

Bun

​Initial Setup

​Step 1: Environment Configuration

​Step 2: One-Command Setup

​Running the Application

​Backend Services

​Rebuilding Services

​Stopping Services

​Frontend Application

​Database Management

​Setting Up Individual Databases

​Running Databases Only

​LocalStack (Local AWS)

​Manual LocalStack Setup

​Individual LocalStack Commands

​FusionAuth Setup

​Building Services

​Build All Services

​Build Lambda Functions

​Type Check Without Building

​Testing

​Setup Test Environment

​Test Individual Services

​Environment Management

​Get Environment Variables

​Edit Environment

​Fix Environment (Re-encrypt)

​Docker Cache Management

​Clear All Build Cache

​Clear Only Rust Target Caches

​View Cache Usage

​Cleanup

​Next Steps

Architecture

Directory Structure

​Troubleshooting

​Services Won’t Start

​Database Connection Errors

​LocalStack Issues

​Build Failures

Build docs developers (and LLMs) love

Prerequisites

Initial Setup

Step 1: Environment Configuration

Step 2: One-Command Setup

Running the Application

Backend Services

Rebuilding Services

Stopping Services

Frontend Application

Database Management

Setting Up Individual Databases

Running Databases Only

LocalStack (Local AWS)

Manual LocalStack Setup

Individual LocalStack Commands

FusionAuth Setup

Building Services

Build All Services

Build Lambda Functions

Type Check Without Building

Testing

Setup Test Environment

Test Individual Services

Environment Management

Get Environment Variables

Edit Environment

Fix Environment (Re-encrypt)

Docker Cache Management

Clear All Build Cache

Clear Only Rust Target Caches

View Cache Usage

Cleanup

Next Steps

Troubleshooting

Services Won’t Start

Database Connection Errors

LocalStack Issues

Build Failures