GatewayService

Overview

The GatewayService provides a robust gateway to a custom language service with advanced resilience patterns including circuit breaker, retry budget, and idempotency support. This service demonstrates production-grade reliability patterns for external service integration. Base Path: /gateway.v1.GatewayService Authentication: Required - Bearer token via Authorization header Middleware: CORS, rate limiting

RPCs

InvokeCustom

Invokes the custom language service with the provided name, returning a custom-generated message. Includes circuit breaker protection, smart retry logic with retry budget, and synchronous database logging for both successes and failures. Endpoint: POST /gateway.v1.GatewayService/InvokeCustom Signature:

rpc InvokeCustom(InvokeCustomRequest) returns (InvokeCustomResponse)

Request

name

string

The name to pass to the custom language service. If empty, defaults to “World”.

Response

message

string

required

The custom message generated by the downstream language service

Connect-Query Example with Idempotency

import { createClient } from '@connectrpc/connect';
import { useMutation } from '@tanstack/react-query';
import { useState } from 'react';
import { GatewayService } from '../../../gen/gateway/v1/gateway_pb';
import { transport } from '../../../lib/transport';

export function GatewayDemo() {
  const [name, setName] = useState('World');
  const client = createClient(GatewayService, transport);

  const mutation = useMutation({
    mutationFn: async (nameInput: string) => {
      return client.invokeCustom(
        { name: nameInput },
        {
          headers: new Headers({
            'idempotency-key': crypto.randomUUID(),
          }),
        },
      );
    },
  });

  const handleSubmit = () => {
    mutation.mutate(name);
  };

  return (
    <div>
      <input
        value={name}
        onChange={(e) => setName(e.target.value)}
        placeholder="name"
      />
      <button onClick={handleSubmit} disabled={mutation.isPending}>
        {mutation.isPending ? 'Sending...' : 'Invoke Gateway'}
      </button>

      {mutation.data && (
        <pre>{JSON.stringify(mutation.data, null, 2)}</pre>
      )}

      {mutation.error && (
        <pre>{mutation.error.message}</pre>
      )}
    </div>
  );
}

cURL Example (h2c)

curl -X POST http://localhost:30081/gateway.v1.GatewayService/InvokeCustom \
  --http2-prior-knowledge \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
  -H "Connect-Protocol-Version: 1" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d '{"name": "Alice"}'

Example Response

{
  "message": "Greetings from the custom language service, Alice!"
}

Implementation Details

Circuit Breaker Pattern

The GatewayService uses the gobreaker library to implement circuit breaker protection: Circuit Breaker Configuration:

Name: custom-lang-service
Max Requests (half-open): 3
Interval: 10 seconds (failure count reset window)
Timeout: 30 seconds (time to wait before half-open state)
Trip Threshold: 5 consecutive failures

Circuit Breaker States:

Closed (Normal): All requests pass through
Open (Tripped): Requests fail immediately with CodeUnavailable error
Half-Open (Testing): Limited requests (max 3) to test service recovery

When the circuit is open, the service returns:

CodeUnavailable: circuit breaker is open

Retry Budget Pattern

The service implements a token bucket-based retry budget to prevent retry storms: Retry Budget Configuration:

Capacity: 20 tokens
Refill Rate: 10 tokens per second

Retries are only attempted if:

The error is retryable (see Retryable Errors below)
A retry token is available in the budget

Retryable Errors:

429 Too Many Requests
502 Bad Gateway
503 Service Unavailable
504 Gateway Timeout
Network timeout errors

Non-Retryable Errors:

400 Bad Request
401 Unauthorized
403 Forbidden
404 Not Found
Context deadline exceeded

Database Integration

The InvokeCustom RPC performs synchronous database writes to the invocations table for both successes and failures:

-- On success
INSERT INTO invocations (name, result_message, success) 
VALUES ($1, $2, true)

-- On failure
INSERT INTO invocations (name, result_message, success) 
VALUES ($1, $2, false)

This ensures complete audit trail of all invocation attempts, allowing for:

Success/failure rate analysis
Error pattern detection
Debugging failed invocations

Downstream Communication

The service makes HTTP POST requests to the custom language service: Endpoint: POST /invoke Request Body:

{
  "name": "Alice"
}

Timeout: 1 second per request Status Code Handling:

2xx: Success - extract message from response
4xx/5xx: Error - wrapped with appropriate Connect error code

Error Code Mapping

HTTP status codes are mapped to Connect RPC error codes:

HTTP Status	Connect Code	Description
400	`CodeInvalidArgument`	Bad request
401	`CodeUnauthenticated`	Authentication failed
403	`CodePermissionDenied`	Forbidden
404	`CodeNotFound`	Not found
409	`CodeAlreadyExists`	Conflict
429	`CodeResourceExhausted`	Too many requests
502, 503	`CodeUnavailable`	Service unavailable
504	`CodeDeadlineExceeded`	Gateway timeout
5xx (other)	`CodeInternal`	Internal server error

Authentication

All requests to the GatewayService must include a valid JWT token in the Authorization header:

Authorization: Bearer <JWT_TOKEN>

The authentication is enforced at the ingress layer via Traefik middleware. Requests without valid authentication will receive a 401 Unauthorized response.

Idempotency Support

The GatewayService supports idempotency through the Idempotency-Key header:

Idempotency-Key: <unique-uuid>

While the service doesn’t currently implement idempotency deduplication at the application level, the header is accepted and passed through the middleware chain, making it available for future implementation or downstream service consumption. Best Practice: Generate a new UUID for each unique invocation intent:

const idempotencyKey = crypto.randomUUID();

Service Dependencies

Custom Language Service: Target service at http://custom-lang-service.microservices:3000
PostgreSQL: Stores invocation records in the gateway_db.invocations table

Service Configuration

Environment Variable	Default	Description
`PORT`	`8082`	Service port
`CUSTOM_LANG_BASE_URL`	`http://custom-lang-service.microservices:3000`	Custom language service endpoint
`DATABASE_URL`	-	PostgreSQL connection string (optional)
`OTEL_EXPORTER_OTLP_ENDPOINT`	-	OpenTelemetry collector endpoint

High Availability

Pod Disruption Budget

The GatewayService is protected by a PodDisruptionBudget to ensure availability during cluster maintenance:

minAvailable: 1

This prevents all gateway pods from being evicted simultaneously during node drains or voluntary disruptions.

Resilience Features

Circuit Breaker: Prevents cascading failures when downstream service is unhealthy
Retry Budget: Limits retry attempts to prevent retry storms
Timeout Control: 1-second timeout per downstream request
OpenTelemetry: Full distributed tracing for debugging
Database Logging: Complete audit trail of all invocations

Monitoring & Observability

Key Metrics to Monitor

Circuit breaker state (closed/open/half-open)
Retry budget token availability
Success/failure rates from invocations table
Request latency (p50, p95, p99)
Downstream service response times

Circuit Breaker State

The circuit breaker state can be monitored through OpenTelemetry metrics. When the circuit opens, all requests will fail immediately until the timeout period (30 seconds) elapses.

Rate Limiting

The GatewayService is protected by rate limiting middleware configured in Traefik. Excessive requests may receive a 429 Too Many Requests response at the ingress layer.

Common Error Scenarios

Circuit Breaker Open

{
  "code": "unavailable",
  "message": "circuit breaker is open"
}

Resolution: Wait 30 seconds for circuit to enter half-open state, or investigate downstream service health.

Retry Budget Exhausted

When retry budget is exhausted, the service will not retry failed requests, returning the original error immediately. Resolution: Reduce request rate or investigate why so many requests are failing.

Downstream Timeout

{
  "code": "deadline_exceeded",
  "message": "context deadline exceeded"
}

Resolution: Check custom language service performance and network latency.

gRPC Services

REST APIs

Overview

RPCs

InvokeCustom

Request

Response

Connect-Query Example with Idempotency

cURL Example (h2c)

Example Response

Implementation Details

Circuit Breaker Pattern

Retry Budget Pattern

Database Integration

Downstream Communication

Error Code Mapping

Authentication

Idempotency Support

Service Dependencies

Service Configuration

High Availability

Pod Disruption Budget

Resilience Features

Monitoring & Observability

Key Metrics to Monitor

Circuit Breaker State

Rate Limiting

Common Error Scenarios

Circuit Breaker Open

Retry Budget Exhausted

Downstream Timeout

Build docs developers (and LLMs) love

gRPC Services

REST APIs

​Overview

​RPCs

​InvokeCustom

​Request

​Response

​Connect-Query Example with Idempotency

​cURL Example (h2c)

​Example Response

​Implementation Details

​Circuit Breaker Pattern

​Retry Budget Pattern

​Database Integration

​Downstream Communication

​Error Code Mapping

​Authentication

​Idempotency Support

​Service Dependencies

​Service Configuration

​High Availability

​Pod Disruption Budget

​Resilience Features

​Monitoring & Observability

​Key Metrics to Monitor

​Circuit Breaker State

​Rate Limiting

​Common Error Scenarios

​Circuit Breaker Open

​Retry Budget Exhausted

​Downstream Timeout

Build docs developers (and LLMs) love

Overview

RPCs

InvokeCustom

Request

Response

Connect-Query Example with Idempotency

cURL Example (h2c)

Example Response

Implementation Details

Circuit Breaker Pattern

Retry Budget Pattern

Database Integration

Downstream Communication

Error Code Mapping

Authentication

Idempotency Support

Service Dependencies

Service Configuration

High Availability

Pod Disruption Budget

Resilience Features

Monitoring & Observability

Key Metrics to Monitor

Circuit Breaker State

Rate Limiting

Common Error Scenarios

Circuit Breaker Open

Retry Budget Exhausted

Downstream Timeout