Skip to main content
Jan provides a built-in, OpenAI-compatible API server that runs entirely on your computer, powered by llama.cpp. Use it as a drop-in replacement for cloud APIs to build private, offline-capable AI applications.

​
Quick Start

1

Start the Server

  1. Navigate to Settings > Local API Server
  2. Enter a custom API Key (e.g., secret-key-123). This is required for all requests
  3. Click Start Server
The server is ready when the logs show JAN API listening at http://127.0.0.1:1337
2

Test with cURL

Open a terminal and make a request. Replace YOUR_MODEL_ID with the ID of an available model in Jan.
curl http://127.0.0.1:1337/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer secret-key-123" \
  -d '{
    "model": "YOUR_MODEL_ID",
    "messages": [{"role": "user", "content": "Tell me a joke."}]
  }'

​
Server Configuration

These settings control the network accessibility and basic behavior of your local server.

​
Server Host

The network address the server listens on:
  • 127.0.0.1 (Default): The server is only accessible from your own computer. This is the most secure option for personal use.
  • 0.0.0.0: The server is accessible from other devices on your local network (e.g., your phone or another computer). Use this with caution.

​
Server Port

The port number for the API server:
  • 1337 (Default): A common alternative port
  • You can change this to any available port number (e.g., 8000)

​
API Prefix

The base path for all API endpoints:
  • /v1 (Default): Follows OpenAI’s convention. The chat completions endpoint would be http://127.0.0.1:1337/v1/chat/completions
  • You can change this or leave it empty if desired

​
API Key

A mandatory secret key to authenticate requests:
  • You must set a key. It can be any string (e.g., a-secure-password)
  • All API requests must include this key in the Authorization: Bearer YOUR_API_KEY header

​
Trusted Hosts

A comma-separated list of hostnames allowed to access the server. This provides an additional layer of security when the server is exposed on your network.

​
Advanced Settings

​
Cross-Origin Resource Sharing (CORS)

  • (Enabled by default) Allows web applications (like a custom web UI you are building) running on different domains to make requests to the API server
  • Disable this if your API will only be accessed by non-browser-based applications (e.g., scripts, command-line tools) for slightly improved security

​
Verbose Server Logs

  • (Enabled by default) Provides detailed, real-time logs of all incoming requests, responses, and server activity
  • This is extremely useful for debugging application behavior and understanding exactly what is being sent to the models

​
Integration Examples

​
Python Example

import openai

# Configure the OpenAI client to use Jan's local server
client = openai.OpenAI(
    api_key="secret-key-123",
    base_url="http://127.0.0.1:1337/v1"
)

# Make a chat completion request
response = client.chat.completions.create(
    model="YOUR_MODEL_ID",
    messages=[
        {"role": "user", "content": "What is the capital of France?"}
    ]
)

print(response.choices[0].message.content)

​
JavaScript Example

import OpenAI from 'openai';

// Configure the OpenAI client to use Jan's local server
const client = new OpenAI({
  apiKey: 'secret-key-123',
  baseURL: 'http://127.0.0.1:1337/v1'
});

// Make a chat completion request
const response = await client.chat.completions.create({
  model: 'YOUR_MODEL_ID',
  messages: [
    { role: 'user', content: 'What is the capital of France?' }
  ]
});

console.log(response.choices[0].message.content);

​
Streaming Example

curl http://127.0.0.1:1337/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer secret-key-123" \
  -d '{
    "model": "YOUR_MODEL_ID",
    "messages": [{"role": "user", "content": "Write a short story."}],
    "stream": true
  }'

​
Troubleshooting

Ensure Verbose Server Logs are enabled to get detailed error messages in the “Server Logs” view.
Connection Refused
  • The server is not running, or your application is pointing to the wrong host or port
401 Unauthorized
  • Your API Key is missing from the Authorization header or is incorrect
404 Not Found
  • The model ID in your request body does not match an available model in Jan
  • Your request URL is incorrect (check the API Prefix)
CORS Error (in a web browser)
  • Ensure the CORS toggle is enabled in Jan’s settings

​
API Compatibility

Jan’s local API server implements the OpenAI API specification, making it compatible with:
  • OpenAI’s official client libraries (Python, JavaScript, .NET, Go)
  • LangChain and LlamaIndex frameworks
  • Any tool or library that supports OpenAI-compatible APIs
Simply change the base_url or baseURL parameter to point to your Jan server instead of OpenAI’s servers.

Build docs developers (and LLMs) love

Get started for freeTalk to us