Package System Architecture

The package system is designed to securely fetch, cache, and serve package release information. It uses a unique hash-based identification system to prevent arbitrary API requests while enabling aggressive caching.

Design Goals

Security - Only pre-configured packages can be queried
Cache Efficiency - Maximize cache hits, minimize API calls
Type Safety - Full runtime validation of all data
Resilience - Handle provider failures gracefully
Performance - Request coalescing prevents duplicate calls

Architecture Overview

Package Identification

Why Hash-Based IDs?

Traditional REST APIs use resource IDs like /api/packages/vuejs/vue. This has problems:

Security - Anyone can query arbitrary packages
Cache Invalidation - Changing config does not change the URL
Ambiguity - Same package with different settings = same ID

Shipped uses content-addressed hashes:

Package ID = hash(name + provider + packageExtras + providerExtras)

What is in the Hash?

The hash includes everything that affects the package data:

// Package-level config
{
  name: "vuejs/vue",
  provider: "github",
  extra: {
    includePrereleases: true,
    maxReleases: 10
  }
}

// Provider-level config (from providers.yaml)
{
  github: {
    maxReleases: 50,  // Default for all GitHub packages
    includePrereleases: false
  }
}

// Combined and hashed
const packageId = hash({
  spec: { name, provider, extra },
  providerExtra: { maxReleases, includePrereleases }
});

This means:

vuejs/vue with maxReleases: 10 → Hash A
vuejs/vue with maxReleases: 20 → Hash B (different!)
Changing any provider setting → New hash → Cache miss

Hash Generation

Implementation in libs/config/views/package.ts:28:

class PackageConfigView extends Data.Class<
  PackageConfig & {
    providerConfig?: ProviderConfig;
  }
> {
  packageId = PackageConfigView.hash(this);

  static hash(config: PackageConfigView): string {
    if (import.meta.dev) {
      // Human-readable in development
      return `${config.spec.name}:${config.spec.provider}:${hash(config.spec.extra)}:${JSON.stringify(config.providerExtra)}`;
    }

    // Hashed in production
    return hash({
      spec: config.spec,
      extra: config.providerExtra,
    });
  }
}

Dev mode hash example:

"vuejs/vue:github:a1b2c3d4:e5f6g7h8"

Production hash example:

"a1b2c3d4e5f6..."

PackageConfigView

The view class encapsulates all package configuration:

class PackageConfigView extends Data.Class<{
  readonly spec: PackageSpec; // name, provider, package-level extras
  readonly providerExtra: Record<string, unknown>; // provider-level defaults
}> {
  // Unique identifier (computed hash)
  readonly packageId: string;

  // Provider name for routing
  get providerName(): string {
    return this.spec.provider;
  }

  // Package name (format varies by provider)
  get name(): string {
    return this.spec.name;
  }

  // Display name for UI
  get displayName(): string {
    return `${this.spec.provider}:${this.spec.name}`;
  }
}

The Package Map

UserConfigView creates a fast lookup map:

class UserConfigView {
  get packageMap(): ReadonlyMap<string, PackageConfigView> {
    const map = new Map<string, PackageConfigView>();

    for (const list of this.lists) {
      for (const pkg of list.packages) {
        map.set(pkg.id, pkg); // O(1) lookup by hash
      }
    }

    return map;
  }
}

This enables constant-time validation:

// Validate package exists
const pkg = userConfig.packageMap.get(packageId);
if (!pkg) {
  return Effect.fail(new PackageNotFoundError({ id: packageId }));
}

Package Service Flow

Cache Architecture

Multi-Layer Caching

Request
  ↓
┌─────────────────────────────────────────┐
│ L1 Cache (Memory)                       │
│ - In-memory Map                         │
│ - Fastest access                        │
│ - Lost on restart                       │
└─────────────────────────────────────────┘
  ↓ (miss)
┌─────────────────────────────────────────┐
│ L2 Cache (File)                         │
│ - BentoCache with file backend          │
│ - Survives restarts                     │
│ - Slower than L1                        │
└─────────────────────────────────────────┘
  ↓ (miss)
External API Call

Cache Key Structure

// Namespace includes provider and implementation versions
const namespace = `${providerName}-${providerVersion}-package_v${implVersion}`.replace(
  /[^a-zA-Z0-9]/g,
  "-"
);

// Key is the package hash
const key = packageId;

// Full cache key: "github-1-2-package-v-3" + 
":" + "abc123..."

Why include versions?

Provider schema changes → New namespace → Cache miss
Implementation changes → New namespace → Cache miss
Prevents stale data when code changes

TTL Strategy

const CACHE_TTL = {
  // Successful fetches cached longer
  success: 3 * 60 * 60 * 1000, // 3 hours

  // Not found cached shorter (might be temporary)
  notFound: 10 * 60 * 1000, // 10 minutes

  // Errors cached briefly (avoid hammering failing APIs)
  error: 60 * 1000, // 1 minute
};

Caching undefined (not found) is critical - prevents repeated failed API calls for:

Typos in package names
Deleted packages
Non-existent versions

Request Coalescing

When multiple requests hit the same cache miss:

Time →

Request A ─────┐
Request B ─────┼───> Single API Call ───> All Get Result
Request C ─────┘

Implementation in server/libs/cache/coalescing-cache.ts:48:

const getOrSetEither = <A, E, R>(opts: GetOrSetOptions<A, E, R>) =>
  Effect.suspend(() => {
    const keyHash = hash(opts.key);
    const key = opts.namespace ? `${opts.namespace}:${keyHash}` : keyHash;

    let deferred = MutableHashMap.get<string, Deferred.Deferred<A, E>>(
      inflight,
      key
    ).pipe(Option.getOrUndefined);

    if (deferred === undefined) {
      // First request - create deferred and execute factory
      deferred = Deferred.unsafeMake<A, E>(fiberId);
      MutableHashMap.set(inflight, key, deferred);

      return Effect.gen(function* () {
        const cachedValue = yield* backend.get<CachedValue<A>>({ key });

        if (cachedValue) {
          stats.track("hits");
          const either = Either.right(cachedValue.data);
          yield* completeDeferred(key, Exit.fromEither(either));
          return either;
        }

        stats.track("misses");
        const factoryEither = yield* Effect.either(opts.factory);
        yield* completeDeferred(key, Exit.fromEither(factoryEither));

        if (Either.isRight(factoryEither)) {
          const policy = opts.policy?.(factoryEither.right);
          if (policy?.cacheNil === true || !isNil(factoryEither.right)) {
            yield* backend.set({
              key,
              value: cacheValue(factoryEither.right),
              ttl: policy?.ttl ?? opts.ttl ?? Duration.seconds(5),
            });
          }
        }

        return factoryEither;
      });
    } else {
      // Subsequent requests - await the deferred
      return Effect.gen(function* () {
        stats.track("deferred");
        return yield* Effect.either(Deferred.await(deferred!));
      });
    }
  });

This prevents thundering herd when:

Multiple users open the same package simultaneously
Cache expires and multiple requests arrive
Server restarts and cache is cold

Provider Adapter Interface

Each provider implements a standard interface. From server/libs/provider/index.ts:15:

export type ProviderError = 
  | PackageNotFoundError 
  | NetworkError 
  | InvalidPackageNameError;

export type PackageProvider<T extends ProviderInfo = ProviderInfo> = {
  readonly info: T;
  readonly version: string | number;

  // Fetch package data
  readonly getPackage: (
    opts: PackageConfigView
  ) => Effect.Effect<Package, ProviderError>;
};

Data Flow Through Provider

PackageConfigView
       ↓
  + provider-specific parsing
       ↓
Provider Internal Config
       ↓
  + API call
       ↓
Raw API Response
       ↓
  + Validation with Provider Schema
       ↓
Package (unified format)

All providers return the same Package structure:

interface Package {
  overview: PackageOverview; // name, description, url, etc.
  releases: PackageRelease[]; // version, date, notes, etc.
}

Security Model

Hash Validation

The API never accepts arbitrary package identifiers:

// RPC Route
const getPackage = o.router({
  getOneById: o.procedure
    .input(z.object({ id: z.string() }))
    .handler(({ input }) =>
      Effect.gen(function* () {
        const config = yield* UserConfigService;

        // CRITICAL: Only allow pre-configured packages
        const pkg = config.getPackageById(input.id);
        if (pkg._tag === "None") {
          return yield* Effect.fail(
            new PackageNotFoundError({
              id: input.id,
            })
          );
        }

        // Proceed with fetching
        return yield* packageService.getOneById(input.id);
      })
    ),
});

Why This Matters

Without hash validation, users could:

Query any package on GitHub/NPM (security risk)
Exhaust API rate limits by requesting random packages
Probe for private package names

With hash validation:

Only packages in lists.yaml can be queried
Attacker cannot guess valid hashes (cryptographically secure)
API rate limits are predictable and controlled

Error Handling

Error Types

Defined in server/libs/provider/errors.ts:

// Package not found in config
class PackageNotFoundError extends Data.TaggedError("PackageNotFoundError")<{
  id: string;
}> {}

// Provider API error
class NetworkError extends Data.TaggedError("NetworkError")<{
  provider: string;
  message: string;
  cause?: unknown;
}> {}

// Invalid package name format
class InvalidPackageNameError extends Data.TaggedError(
  "InvalidPackageNameError"
)<{
  name: string;
  provider: string;
}> {}

Error Recovery

const getPackageSafe = (id: string) =>
  Effect.gen(function* () {
    const result = yield* getPackage(id).pipe(
      // Handle specific errors
      Effect.catchTag("PackageNotFoundError", (e) => 
        Effect.succeed(null)
      ),
      Effect.catchTag("NetworkError", (e) =>
        Effect.gen(function* () {
          // Try cache even on provider error
          const cached = yield* cache.getStale(id);
          if (cached._tag === "Some") {
            return cached.value;
          }
          return yield* Effect.fail(e);
        })
      ),
      // Generic fallback
      Effect.catchAll((e) => {
        logger.error("Unexpected error fetching package", e);
        return Effect.succeed(null);
      })
    );

    return result;
  });

Performance Considerations

Cache Hit Ratio

Target: >95% cache hit ratio Factors affecting hit rate:

TTL configuration
Number of unique packages
User browsing patterns

Memory Usage

L1 cache stores deserialized objects:

Memory ≈ avgPackageSize × numPackages × 2 (for JS overhead)

For 1000 packages at 10KB each: ~20MB

Cold Start

On server restart:

L1 cache is empty
L2 cache survives (file-based)
First requests populate L1 from L2
External APIs only hit on L2 miss

Monitoring

Metrics to Track

Cache Hit Rate - Should be >95%
API Call Rate - Watch for spikes
Package Fetch Latency - P95 < 100ms (cached)
Provider Error Rate - Alert on >1%

Cache Statistics

The coalescing cache tracks:

class Stats {
  hits = 0;      // Cache hits
  misses = 0;    // Cache misses (API calls)
  deferred = 0;  // Coalesced requests
}

Summary

The package system provides:

Security via content-addressing - Only configured packages accessible
Automatic cache invalidation - Config changes change hashes
Multi-layer caching - L1 (memory) + L2 (file)
Request coalescing - Prevents duplicate API calls
Graceful degradation - Stale cache on provider errors

This architecture enables serving thousands of package requests while minimizing external API calls and maintaining strict security boundaries.

Overview

Getting Started

Configuration

Providers

Features

Advanced

Package System Architecture

Design Goals

Architecture Overview

Package Identification

Why Hash-Based IDs?

What is in the Hash?

Hash Generation

PackageConfigView

The Package Map

Package Service Flow

Cache Architecture

Multi-Layer Caching

Cache Key Structure

TTL Strategy

Request Coalescing

Provider Adapter Interface

Data Flow Through Provider

Security Model

Hash Validation

Why This Matters

Error Handling

Error Types

Error Recovery

Performance Considerations

Cache Hit Ratio

Memory Usage

Cold Start

Monitoring

Metrics to Track

Cache Statistics

Summary

Build docs developers (and LLMs) love

Overview

Getting Started

Configuration

Providers

Features

Advanced

​Design Goals

​Architecture Overview

​Package Identification

​Why Hash-Based IDs?

​What is in the Hash?

​Hash Generation

​PackageConfigView

​The Package Map

​Package Service Flow

​Cache Architecture

​Multi-Layer Caching

​Cache Key Structure

​TTL Strategy

​Request Coalescing

​Provider Adapter Interface

​Data Flow Through Provider

​Security Model

​Hash Validation

​Why This Matters

​Error Handling

​Error Types

​Error Recovery

​Performance Considerations

​Cache Hit Ratio

​Memory Usage

​Cold Start

​Monitoring

​Metrics to Track

​Cache Statistics

​Summary

Build docs developers (and LLMs) love

Design Goals

Architecture Overview

Package Identification

Why Hash-Based IDs?

What is in the Hash?

Hash Generation

PackageConfigView

The Package Map

Package Service Flow

Cache Architecture

Multi-Layer Caching

Cache Key Structure

TTL Strategy

Request Coalescing

Provider Adapter Interface

Data Flow Through Provider

Security Model

Hash Validation

Why This Matters

Error Handling

Error Types

Error Recovery

Performance Considerations

Cache Hit Ratio

Memory Usage

Cold Start

Monitoring

Metrics to Track

Cache Statistics

Summary