Catalog Database - AspNetRun Microservices

Overview

The Catalog service uses Marten as its data persistence layer. Marten is a .NET library that provides document database capabilities using PostgreSQL as the backing store.

Why Marten?

Document Database

Store .NET objects as JSON documents without ORM impedance mismatch

PostgreSQL Power

Leverage PostgreSQL’s JSONB storage and querying capabilities

LINQ Support

Write queries using familiar LINQ syntax

Event Sourcing

Built-in support for event sourcing patterns (not used in Catalog)

Configuration

Service Registration

Marten is configured in Program.cs:

Program.cs

builder.Services.AddMarten(opts =>
{
    opts.Connection(builder.Configuration.GetConnectionString("Database")!);
}).UseLightweightSessions();

if (builder.Environment.IsDevelopment())
    builder.Services.InitializeMartenWith<CatalogInitialData>();

Connection String

Configure the database connection in appsettings.json:

appsettings.json

{
  "ConnectionStrings": {
    "Database": "Host=localhost;Database=CatalogDb;Username=postgres;Password=postgres"
  }
}

For Docker Compose deployments:

appsettings.json

{
  "ConnectionStrings": {
    "Database": "Host=catalogdb;Database=CatalogDb;Username=postgres;Password=postgres"
  }
}

Lightweight Sessions

The service uses lightweight sessions for better performance:

.UseLightweightSessions()

Lightweight sessions:

No identity map caching
No automatic dirty checking
Better performance for stateless HTTP APIs
Lower memory footprint

Document Store

Product Document

Products are stored as JSON documents in PostgreSQL:

Models/Product.cs

public class Product
{
    public Guid Id { get; set; }
    public string Name { get; set; } = default!;
    public List<string> Category { get; set; } = new();
    public string Description { get; set; } = default!;
    public string ImageFile { get; set; } = default!;
    public decimal Price { get; set; }
}

Storage Format

Marten stores this as a JSONB document in PostgreSQL:

{
  "id": "5334c996-8457-4cf0-815c-ed2b77c4ff61",
  "name": "IPhone X",
  "category": ["Smart Phone"],
  "description": "This phone is the company's biggest change...",
  "imageFile": "product-1.png",
  "price": 950.00
}

Database Schema

Marten automatically creates the necessary tables:

CREATE TABLE public.mt_doc_product (
    id uuid NOT NULL,
    data jsonb NOT NULL,
    mt_last_modified timestamp without time zone DEFAULT now(),
    mt_version uuid NOT NULL,
    mt_dotnet_type character varying,
    CONSTRAINT pk_mt_doc_product PRIMARY KEY (id)
);

CREATE INDEX idx_product_data ON public.mt_doc_product USING gin (data);

Data Operations

Creating Documents

Products/CreateProduct/CreateProductHandler.cs

public async Task<CreateProductResult> Handle(
    CreateProductCommand command, 
    CancellationToken cancellationToken)
{
    var product = new Product
    {
        Name = command.Name,
        Category = command.Category,
        Description = command.Description,
        ImageFile = command.ImageFile,
        Price = command.Price
    };
    
    // Store the document
    session.Store(product);
    
    // Commit changes to database
    await session.SaveChangesAsync(cancellationToken);

    return new CreateProductResult(product.Id);
}

Key Points:

Store() adds the document to the session
SaveChangesAsync() commits to the database
ID is auto-generated if not provided

Reading Documents

Load by ID

Products/GetProductById/GetProductByIdHandler.cs

public async Task<GetProductByIdResult> Handle(
    GetProductByIdQuery query, 
    CancellationToken cancellationToken)
{
    // Load by primary key (optimized)
    var product = await session.LoadAsync<Product>(query.Id, cancellationToken);

    if (product is null)
    {
        throw new ProductNotFoundException(query.Id);
    }

    return new GetProductByIdResult(product);
}

LoadAsync:

Optimized for single document retrieval by ID
Returns null if not found
Uses PostgreSQL primary key index

Query with LINQ

Products/GetProducts/GetProductsHandler.cs

public async Task<GetProductsResult> Handle(
    GetProductsQuery query, 
    CancellationToken cancellationToken)
{
    // Query with pagination
    var products = await session.Query<Product>()
        .ToPagedListAsync(
            query.PageNumber ?? 1, 
            query.PageSize ?? 10, 
            cancellationToken
        );

    return new GetProductsResult(products);
}

Query Features:

Full LINQ support
Translated to SQL queries
Automatic JSONB field extraction

Filter by Property

Products/GetProductByCategory/GetProductByCategoryHandler.cs

public async Task<GetProductByCategoryResult> Handle(
    GetProductByCategoryQuery query, 
    CancellationToken cancellationToken)
{
    // Query with WHERE clause on JSON array
    var products = await session.Query<Product>()
        .Where(p => p.Category.Contains(query.Category))
        .ToListAsync(cancellationToken);

    return new GetProductByCategoryResult(products);
}

Generated SQL:

SELECT data 
FROM mt_doc_product 
WHERE data -> 'category' @> '"Smart Phone"'::jsonb

Updating Documents

Products/UpdateProduct/UpdateProductHandler.cs

public async Task<UpdateProductResult> Handle(
    UpdateProductCommand command, 
    CancellationToken cancellationToken)
{
    // Load existing document
    var product = await session.LoadAsync<Product>(command.Id, cancellationToken);

    if (product is null)
    {
        throw new ProductNotFoundException(command.Id);
    }

    // Modify properties
    product.Name = command.Name;
    product.Category = command.Category;
    product.Description = command.Description;
    product.ImageFile = command.ImageFile;
    product.Price = command.Price;

    // Mark as updated
    session.Update(product);
    
    // Commit changes
    await session.SaveChangesAsync(cancellationToken);

    return new UpdateProductResult(true);
}

Update Process:

Load existing document
Modify properties in memory
Call Update() to mark as modified
SaveChangesAsync() replaces the document

Deleting Documents

Products/DeleteProduct/DeleteProductHandler.cs

public async Task<DeleteProductResult> Handle(
    DeleteProductCommand command, 
    CancellationToken cancellationToken)
{
    // Mark document for deletion
    session.Delete<Product>(command.Id);
    
    // Commit deletion
    await session.SaveChangesAsync(cancellationToken);

    return new DeleteProductResult(true);
}

Delete Options:

Hard Delete: Removes document from database (default)
Soft Delete: Marks as deleted but keeps in database (requires configuration)

Data Seeding

Initial Data Configuration

The service seeds initial data in development mode:

Data/CatalogInitialData.cs

public class CatalogInitialData : IInitialData
{
    public async Task Populate(
        IDocumentStore store, 
        CancellationToken cancellation)
    {
        using var session = store.LightweightSession();

        // Check if data already exists
        if (await session.Query<Product>().AnyAsync())
            return;

        // Seed initial products
        session.Store<Product>(GetPreconfiguredProducts());
        await session.SaveChangesAsync();
    }

    private static IEnumerable<Product> GetPreconfiguredProducts() => new List<Product>()
    {
        new Product()
        {
            Id = new Guid("5334c996-8457-4cf0-815c-ed2b77c4ff61"),
            Name = "IPhone X",
            Description = "This phone is the company's biggest change...",
            ImageFile = "product-1.png",
            Price = 950.00M,
            Category = new List<string> { "Smart Phone" }
        },
        new Product()
        {
            Id = new Guid("c67d6323-e8b1-4bdf-9a75-b0d0d2e7e914"),
            Name = "Samsung 10",
            Description = "This phone is the company's biggest change...",
            ImageFile = "product-2.png",
            Price = 840.00M,
            Category = new List<string> { "Smart Phone" }
        },
        // ... more products
    };
}

Seed Data Products

The initial dataset includes 7 products:

Product	Category	Price
IPhone X	Smart Phone	$950.00
Samsung 10	Smart Phone	$840.00
Huawei Plus	White Appliances	$650.00
Xiaomi Mi 9	White Appliances	$470.00
HTC U11+ Plus	Smart Phone	$380.00
LG G7 ThinQ	Home Kitchen	$240.00
Panasonic Lumix	Camera	$240.00

Advanced Features

Pagination

Marten provides built-in pagination:

var products = await session.Query<Product>()
    .ToPagedListAsync(pageNumber, pageSize, cancellationToken);

This generates efficient SQL with OFFSET and LIMIT:

SELECT data 
FROM mt_doc_product 
ORDER BY id 
OFFSET 0 LIMIT 10

Indexing

Marten automatically creates indexes, but you can configure custom ones:

builder.Services.AddMarten(opts =>
{
    opts.Connection(connectionString);
    
    // Create custom index on Name property
    opts.Schema.For<Product>().Index(x => x.Name);
    
    // Create composite index
    opts.Schema.For<Product>().Index(x => x.Category, x => x.Price);
});

Transactions

Marten operations are transactional:

using var session = store.LightweightSession();

try
{
    session.Store(product1);
    session.Store(product2);
    session.Delete<Product>(oldProductId);
    
    // All operations committed atomically
    await session.SaveChangesAsync();
}
catch (Exception ex)
{
    // Transaction automatically rolled back
}

Concurrency Control

Marten tracks document versions:

var product = await session.LoadAsync<Product>(id);
product.Price = 999.99M;

// Will fail if document was modified by another request
session.Update(product);
await session.SaveChangesAsync(); // May throw ConcurrencyException

Performance Considerations

Connection Pooling

PostgreSQL connection pooling is enabled by default:

Host=localhost;Database=CatalogDb;Username=postgres;Password=postgres;Pooling=true;Minimum Pool Size=5;Maximum Pool Size=100

Query Optimization

Do:

Use LoadAsync() for single document retrieval by ID
Add indexes on frequently queried properties
Use pagination for large result sets
Leverage JSONB GIN indexes for complex queries

Don’t:

Load all documents without pagination
Perform complex calculations in LINQ queries
Use Include() excessively (not relevant for documents)

Batch Operations

For bulk inserts, use batch operations:

using var session = store.LightweightSession();

var products = GenerateProducts(1000);

// All stored in a single transaction
session.Store(products);
await session.SaveChangesAsync();

Monitoring

Database Health Check

The service includes PostgreSQL health monitoring:

builder.Services.AddHealthChecks()
    .AddNpgSql(builder.Configuration.GetConnectionString("Database")!);

Access at: GET /health

Query Logging

Enable query logging in development:

builder.Services.AddMarten(opts =>
{
    opts.Connection(connectionString);
    
    if (builder.Environment.IsDevelopment())
    {
        opts.Logger(new ConsoleMartenLogger());
    }
});

Backup and Migration

Database Backup

Use standard PostgreSQL backup tools:

pg_dump -h localhost -U postgres CatalogDb > catalog_backup.sql

Schema Migration

Marten auto-creates and updates schema on startup. For production:

builder.Services.AddMarten(opts =>
{
    opts.Connection(connectionString);
    
    // Generate schema changes without applying
    if (args.Contains("--schema"))
    {
        opts.AutoCreateSchemaObjects = AutoCreate.CreateOrUpdate;
    }
});

Troubleshooting

Common Issues

Connection Errors:

Npgsql.NpgsqlException: Connection refused

Solution: Verify PostgreSQL is running and connection string is correct. Missing Table:

PostgreSQL error 42P01: relation "mt_doc_product" does not exist

Solution: Ensure AutoCreateSchemaObjects is enabled for initial setup. Serialization Errors:

System.Text.Json.JsonException: A possible object cycle was detected

Solution: Configure JSON serialization options or simplify object graph.

References

Marten Documentation

Official Marten documentation

PostgreSQL JSONB

PostgreSQL JSON functions

Catalog Service

Basket Service

Discount Service

Ordering Service

​Overview

​Why Marten?

Document Database

PostgreSQL Power

LINQ Support

Event Sourcing

​Configuration

​Service Registration

​Connection String

​Lightweight Sessions

​Document Store

​Product Document

​Storage Format

​Database Schema

​Data Operations

​Creating Documents

​Reading Documents

​Load by ID

​Query with LINQ

​Filter by Property

​Updating Documents

​Deleting Documents

​Data Seeding

​Initial Data Configuration

​Seed Data Products

​Advanced Features

​Pagination

​Indexing

​Transactions

​Concurrency Control

​Performance Considerations

​Connection Pooling

​Query Optimization

​Batch Operations

​Monitoring

​Database Health Check

​Query Logging

​Backup and Migration

​Database Backup

​Schema Migration

​Troubleshooting

​Common Issues

​References