Skip to main content
Shipped is a self-hosted release tracking application built with a unique architecture designed for reliability, type safety, and real-time reactivity. This document explains the high-level design and key architectural decisions.

Core Philosophy

The architecture follows these principles:
  1. Never Fail on User Error - Configuration errors are isolated and reported, never crash the app
  2. Reactive E2E - Changes propagate from filesystem to UI without restarts
  3. Type Safety First - Runtime validation at every boundary
  4. Cache as Database - External API calls are expensive; cache aggressively
  5. Defense in Depth - Hash-based package IDs prevent arbitrary API requests

System Architecture

Key Architectural Patterns

1. Config-First Design

Unlike traditional apps that store configuration in a database, Shipped uses YAML files as runtime configuration. This enables:
  • Easy editing without UI complexity - just edit files
  • Atomic updates via file replacement
  • Fast startup - no database queries needed
  • Hot reloading - changes reflected immediately without restarts
The config files live in the config/ directory on your deployed server and are watched for changes.

Learn More

Deep dive into the configuration system architecture

2. Package Hashing

Packages are identified by a hash of their complete configuration, not by name. This provides:
  • Security - Only pre-configured packages can be queried
  • Cache Invalidation - Changing any config property changes the hash
  • Uniqueness - Same package with different provider settings = different hash

Learn More

Learn about the package system architecture

3. Effect TS on Server

The server uses Effect TS for all business logic, providing:
  • Typed errors - Every failure case is explicit
  • Composability - Effects chain and combine safely
  • Testability - Easy to mock and test
  • Resource safety - Automatic cleanup and error handling
Example from server/services/config/index.ts:12:
const initialView = yield* Effect.sync(() => UserConfigView.make(initialRawConfig)).pipe(
  Effect.tapError((error) => Effect.logError("Failed to create initial config view", error)),
  Effect.mapError((error) => new ConfigViewCreateError({ cause: error })),
);
Effect TS makes errors explicit in the type signature. Traditional try/catch is implicit and easy to miss.

4. Multi-Layer Caching

Package data is cached in two layers:
  • L1 (Memory) - Fast access, lost on restart
  • L2 (File) - Survives restarts, slower
Even “not found” results are cached (with shorter TTL) to avoid hammering external APIs.

5. Request Coalescing

Multiple concurrent requests for the same package result in a single external API call:
Request A ──┐
Request B ──┼──> Package Service ──> Single API Call
Request C ──┘
This prevents thundering herd problems during cache misses. Implementation in server/libs/cache/coalescing-cache.ts:48:
const getOrSetEither = <A, E, R>(opts: GetOrSetOptions<A, E, R>) =>
  Effect.suspend(() => {
    const keyHash = hash(opts.key);
    const key = opts.namespace ? `${opts.namespace}:${keyHash}` : keyHash;

    let deferred = MutableHashMap.get<string, Deferred.Deferred<A, E>>(inflight, key).pipe(
      Option.getOrUndefined,
    );

    if (deferred === undefined) {
      // First request - create deferred and fetch
      deferred = Deferred.unsafeMake<A, E>(fiberId);
      MutableHashMap.set(inflight, key, deferred);
      // ... fetch logic
    } else {
      // Subsequent requests - wait for deferred
      return Effect.gen(function* () {
        stats.track("deferred");
        return yield* Effect.either(Deferred.await(deferred!));
      });
    }
  });

Data Flow

Config Loading Flow

1. File System (YAML)

2. File Watcher (chokidar) detects change

3. Config Adapter parses file

4. Effect Schema validates

5. Individual errors isolated → Warnings

6. UserConfigView aggregates all configs

7. SubscriptionRef updated

8. SSE broadcasts to clients

9. Client updates reactive state

10. UI re-renders

Package Fetching Flow

1. Client requests package by hash

2. Package Service validates hash exists in config

3. Cache lookup (L1 → L2)

4. Cache miss → Request coalescing check

5. Provider adapter fetches from external API

6. Response cached (success or not-found)

7. Response returned to client

Project Structure

shipped/
├── app/                    # Vue application (components, pages, composables)
├── layers/                 # Nuxt layers (01-base/, 02-packages/)
├── libs/                   # Shared business logic (client + server safe)
│   ├── config/            # Config schemas and view classes
│   ├── packages/          # Package types and schemas
│   └── utils/             # General utilities
├── server/                 # Backend code (Effect TS)
│   ├── providers/         # External provider adapters
│   ├── services/          # Business services
│   └── rpc/               # RPC route handlers
├── shared/                 # Client boundary - re-exports from libs
└── config/                 # User configuration files (YAML)

Key Directories Explained

libs/ vs server/libs/
  • libs/ - Shared domain logic, no server dependencies, safe for client
  • server/libs/ - Server-only infrastructure (cache, file system, errors)
This split enables the client to use domain types and view classes without bundling server code. layers/ - Nuxt Layer Organization
  • 01-base/ - Core functionality, RPC, config system
  • 02-packages/ - Package-specific UI components
Layers provide clean separation while allowing shared configuration.

Technology Stack

LayerTechnologyPurpose
FrontendNuxt 4 + Vue 3SSR, routing, UI framework
StylingTailwind CSS + Nuxt UIConsistent design system
Data FetchingTanStack QueryClient-side data fetching, caching, and synchronization
BackendNuxt Server + NitroAPI layer
Server LogicEffect TSFunctional programming, error handling
ValidationEffect Schema & Zod v4Effect Schema for server logic, Zod for RPC boundaries
RPCORPCType-safe API calls
CachingBentoCacheMulti-layer caching
ConfigchokidarConfig file watching and hot reloading

Why This Architecture?

Why YAML over Database?

  • Easy editing without UI complexity
  • Hot reloading - changes reflect immediately
  • No database migrations for config schema changes
  • Simple backup/restore (just copy files)

Why Effect TS?

Traditional try/catch error handling is implicit and easy to miss. Effect makes errors:
  • Explicit in the type signature
  • Composable - errors propagate through chains
  • Recoverable - easy to catch and handle specific cases

Why Hash-Based Package IDs?

Security is paramount for self-hosted apps. By requiring packages to be pre-configured:
  • Users can’t probe arbitrary packages
  • API rate limits are predictable
  • Configuration drives all behavior

Why SSE over WebSockets?

Server-Sent Events provide:
  • Simple reconnection - automatic with EventSource
  • HTTP-based - works through proxies/firewalls
  • One-way streaming - perfect for config updates
  • Less overhead - no handshake for each client

Effect TS Patterns

Services and Dependency Injection

Effect TS uses a service-oriented architecture with compile-time dependency injection: Defining a Service (server/services/config/index.ts:10):
export class UserConfigService extends Effect.Service<UserConfigService>()(
  "userconfig-service",
  {
    accessors: true,
    scoped: Effect.gen(function* () {
      const loader = yield* UserConfigLoader;
      const { config } = loader;
      
      const initialRawConfig = yield* config.get;
      const initialView = yield* Effect.sync(() => UserConfigView.make(initialRawConfig));
      const configView = yield* SubscriptionRef.make<UserConfigView>(initialView);
      
      return {
        config,
        configView,
      };
    }),
  }
) {}

Error Handling

Effect TS makes all errors explicit in type signatures: From server/libs/provider/index.ts:7:
export type ProviderError = 
  | PackageNotFoundError 
  | NetworkError 
  | InvalidPackageNameError;

export type PackageProvider<T extends ProviderInfo = ProviderInfo> = {
  readonly getPackage: (
    opts: PackageConfigView
  ) => Effect.Effect<Package, ProviderError>;
};
The return type Effect.Effect<Package, ProviderError> explicitly declares:
  • Success type: Package
  • Error types: ProviderError (union of specific errors)
  • Dependencies: None in this simplified signature

Resource Management

Effect provides automatic cleanup with Effect.acquireRelease: From server/libs/chokidar/index.ts:18:
const stream = Stream.asyncScoped<{ event: Events; path: string }, unknown>((emit) =>
  Effect.acquireRelease(
    Effect.sync(() => {
      const watcher = chokidar.watch(paths, opts);
      watcher.on("all", (event, path) => {
        emit(Effect.succeed(Chunk.make({ event, path })));
      });
      return watcher;
    }),
    (watcher) =>
      Effect.promise(async () => {
        watcher.removeAllListeners();
        await watcher.close();
      })
  )
);
The watcher is automatically cleaned up when the stream ends, preventing resource leaks.

Next Steps

Config System

Deep dive into configuration loading, validation, and hot-reloading

Package System

Learn about package hashing, caching, and security model

Build docs developers (and LLMs) love