Background Jobs

Polaris uses Inngest to handle long-running operations like AI processing, GitHub imports, and exports. This keeps the UI responsive while complex tasks run in the background.

Why Inngest?

Inngest provides:

Non-blocking UI: Long operations run outside the request-response cycle
Retries: Automatic retry with exponential backoff on failures
Steps: Break jobs into resumable steps that don’t re-run on retry
Cancellation: Cancel running jobs (e.g., stop AI generation)
Monitoring: Built-in dashboard to track job status
Type Safety: Full TypeScript support

Setup

Inngest Client (`src/inngest/client.ts`)

import { Inngest } from "inngest";
import { sentryMiddleware } from "@inngest/middleware-sentry";

export const inngest = new Inngest({ 
  id: "polaris",
  middleware: [sentryMiddleware()],
});

The Sentry middleware automatically captures errors from Inngest functions and sends them to Sentry for monitoring.

Function Registry (`src/inngest/functions.ts`)

All Inngest functions must be exported here:

import { inngest } from "./client";
import { importGithubRepo } from "@/features/projects/inngest/import-github-repo";
import { exportToGithub } from "@/features/projects/inngest/export-to-github";
import { processMessage } from "@/features/conversations/inngest/process-message";

export const functions = [
  importGithubRepo,
  exportToGithub,
  processMessage,
];

API Endpoint (`src/app/api/inngest/route.ts`)

Inngest needs an HTTP endpoint to invoke functions:

import { serve } from "inngest/next";
import { inngest } from "@/inngest/client";
import { functions } from "@/inngest/functions";

export const { GET, POST, PUT } = serve({
  client: inngest,
  functions,
});

Job Patterns

Basic Job Structure

import { inngest } from "@/inngest/client";

export const myJob = inngest.createFunction(
  {
    id: "my-job",           // Unique identifier
    onFailure: async ({ event, step }) => {
      // Handle failures (optional)
    },
  },
  { event: "my/event" },    // Event trigger
  async ({ event, step }) => {
    // Job implementation
    const data = event.data;
    
    const result = await step.run("step-name", async () => {
      // Step logic
      return value;
    });
    
    return { success: true };
  }
);

Triggering Jobs

From a Next.js API route or server action:

import { inngest } from "@/inngest/client";

await inngest.send({
  name: "my/event",
  data: {
    userId: "user_123",
    input: "some data",
  },
});

Real-World Examples

GitHub Import Job

Imports a repository into a Polaris project:

import { inngest } from "@/inngest/client";
import { Octokit } from "octokit";
import { NonRetriableError } from "inngest";

export const importGithubRepo = inngest.createFunction(
  {
    id: "import-github-repo",
    onFailure: async ({ event, step }) => {
      const { projectId } = event.data.event.data;
      
      await step.run("set-failed-status", async () => {
        await convex.mutation(api.system.updateImportStatus, {
          internalKey: process.env.POLARIS_CONVEX_INTERNAL_KEY,
          projectId,
          status: "failed",
        });
      });
    },
  },
  { event: "github/import.repo" },
  async ({ event, step }) => {
    const { owner, repo, projectId, githubToken } = event.data;
    
    const internalKey = process.env.POLARIS_CONVEX_INTERNAL_KEY;
    if (!internalKey) {
      throw new NonRetriableError("POLARIS_CONVEX_INTERNAL_KEY not configured");
    }
    
    // Continue with import...
  }
);

Each step.run() is memoized - if the function fails and retries, completed steps don’t re-run. This prevents duplicate API calls and database writes.

AI Message Processing

Processes user messages with AI agents and tools:

import { createAgent, anthropic, createNetwork } from '@inngest/agent-kit';

export const processMessage = inngest.createFunction(
  {
    id: "process-message",
    cancelOn: [
      {
        event: "message/cancel",
        if: "event.data.messageId == async.data.messageId",
      },
    ],
    onFailure: async ({ event, step }) => {
      const { messageId } = event.data.event.data;
      
      await step.run("update-message-on-failure", async () => {
        await convex.mutation(api.system.updateMessageContent, {
          internalKey,
          messageId,
          content: "I encountered an error. Please try again!",
        });
      });
    }
  },
  { event: "message/sent" },
  async ({ event, step }) => {
    const { messageId, conversationId, projectId, message } = event.data;
    // Process message...
  }
);

GitHub Export Job

Exports a project to a new GitHub repository:

export const exportToGithub = inngest.createFunction(
  {
    id: "export-to-github",
    cancelOn: [{
      event: "github/export.cancel",
      if: "event.data.projectId == async.data.projectId"
    }],
  },
  { event: "github/export.repo" },
  async ({ event, step }) => {
    const { projectId, repoName, visibility, githubToken } = event.data;

    // Create repository
    const { data: repo } = await step.run("create-repo", async () => {
      const octokit = new Octokit({ auth: githubToken });
      return await octokit.rest.repos.createForAuthenticatedUser({
        name: repoName,
        private: visibility === "private",
        auto_init: true,
      });
    });

    // Wait for GitHub to initialize
    await step.sleep("wait-for-repo-init", "3s");

    // Fetch project files
    const files = await step.run("fetch-project-files", async () => {
      return await convex.query(api.system.getProjectFilesWithUrls, {
        internalKey,
        projectId,
      });
    });

    // Create Git blobs and commit
    const treeItems = await step.run("create-blobs", async () => {
      // Create blob for each file...
    });

    const { data: tree } = await step.run("create-tree", async () => {
      return await octokit.rest.git.createTree({
        owner: user.login,
        repo: repoName,
        tree: treeItems,
      });
    });

    const { data: commit } = await step.run("create-commit", async () => {
      return await octokit.rest.git.createCommit({
        owner: user.login,
        repo: repoName,
        message: "Initial commit from Polaris",
        tree: tree.sha,
        parents: [initialCommitSha],
      });
    });

    return { success: true, repoUrl: repo.html_url };
  }
);

Advanced Features

Step Sleep

Pause execution for async operations:

await step.sleep("wait-for-api", "3s");
await step.sleep("long-wait", "5m");

Error Handling

Retriable Errors

By default, all errors trigger retries:

await step.run("fetch-data", async () => {
  const response = await fetch(url);
  if (!response.ok) {
    throw new Error("API request failed"); // Will retry
  }
  return response.json();
});

Non-Retriable Errors

Use NonRetriableError for errors that shouldn’t retry:

import { NonRetriableError } from "inngest";

if (!process.env.API_KEY) {
  throw new NonRetriableError("API_KEY not configured");
}

Failure Handlers

Run cleanup on failure:

inngest.createFunction(
  {
    id: "my-job",
    onFailure: async ({ event, step }) => {
      await step.run("cleanup", async () => {
        // Rollback changes
        // Update status to failed
        // Send notification
      });
    },
  },
  { event: "my/event" },
  async ({ event, step }) => {
    // Main logic
  }
);

Job Cancellation

Cancel running jobs with events:

// Define cancellation condition
inngest.createFunction(
  {
    id: "process-message",
    cancelOn: [{
      event: "message/cancel",
      if: "event.data.messageId == async.data.messageId",
    }],
  },
  { event: "message/sent" },
  async ({ event, step }) => {
    // Long-running AI processing
  }
);

// Trigger cancellation
await inngest.send({
  name: "message/cancel",
  data: { messageId: "msg_123" },
});

Convex Integration

Inngest jobs interact with Convex using an internal API key:

// convex/system.ts
import { mutation } from "./_generated/server";
import { v } from "convex/values";

export const updateImportStatus = mutation({
  args: {
    internalKey: v.string(),
    projectId: v.id("projects"),
    status: v.union(v.literal("pending"), v.literal("completed"), v.literal("failed")),
  },
  handler: async (ctx, args) => {
    // Verify internal key
    if (args.internalKey !== process.env.POLARIS_CONVEX_INTERNAL_KEY) {
      throw new Error("Unauthorized");
    }
    
    await ctx.db.patch(args.projectId, {
      importStatus: args.status,
    });
  },
});

Set POLARIS_CONVEX_INTERNAL_KEY in both your .env.local and Inngest environment variables. Generate a random string: openssl rand -hex 32

Local Development

Run the Inngest dev server:

npx inngest-cli@latest dev

This starts a local dashboard at http://localhost:8288 where you can:

View running and completed jobs
Inspect step execution
Trigger test events
Debug failures

Use the Inngest dashboard to manually trigger events during development instead of going through the full UI flow.

Best Practices

Use Steps for Idempotency: Wrap API calls and database writes in step.run() so they don’t repeat on retry
NonRetriableError for Config Issues: If the job can’t succeed without config changes, use NonRetriableError
Failure Handlers for Cleanup: Always set status to “failed” in onFailure so users know something went wrong
Sleep for External APIs: Some APIs (like GitHub’s auto_init) need time to complete
Type Event Data: Define TypeScript interfaces for event payloads to catch errors early
Secure Internal APIs: Always validate internalKey in Convex mutations called from Inngest

Monitoring

Inngest provides built-in monitoring:

Function Metrics: Success rate, duration, throughput
Step Details: See which steps passed/failed
Event History: Track all triggered events
Error Tracking: View stack traces and error messages

Integrate with Sentry for additional error monitoring:

import { sentryMiddleware } from "@inngest/middleware-sentry";

export const inngest = new Inngest({ 
  id: "polaris",
  middleware: [sentryMiddleware()],
});

Configure Sentry tracing to see Inngest jobs alongside your Next.js requests in the performance dashboard.

Setup

Development

Integration

​Why Inngest?

​Setup

​Inngest Client (src/inngest/client.ts)

​Function Registry (src/inngest/functions.ts)

​API Endpoint (src/app/api/inngest/route.ts)

​Job Patterns

​Basic Job Structure

​Triggering Jobs

​Real-World Examples

​GitHub Import Job

​AI Message Processing

​GitHub Export Job

​Advanced Features

​Step Sleep

​Error Handling

​Job Cancellation

​Convex Integration

​Local Development

​Best Practices

​Monitoring

Build docs developers (and LLMs) love

Why Inngest?

Setup

Inngest Client (`src/inngest/client.ts`)

Function Registry (`src/inngest/functions.ts`)

API Endpoint (`src/app/api/inngest/route.ts`)

Job Patterns

Basic Job Structure

Triggering Jobs

Real-World Examples

GitHub Import Job

AI Message Processing

GitHub Export Job

Advanced Features

Step Sleep

Error Handling

Job Cancellation

Convex Integration

Local Development

Best Practices

Monitoring