Custom Providers

While goose supports 30+ LLM providers out of the box, you may need to integrate with a custom or proprietary provider. Goose offers two approaches: declarative configuration for OpenAI-compatible APIs, and full Rust trait implementation for complete control.

When to Create a Custom Provider

Consider creating a custom provider when:

Your organization has a proprietary LLM deployment
You’re using an OpenAI-compatible API that isn’t built-in
You need special authentication or request preprocessing
You want to add custom retry logic or error handling
You’re using a provider with non-standard streaming formats

Before building a custom provider, check if your provider is OpenAI or Anthropic API-compatible. If so, the declarative approach is much simpler.

Declarative Providers (Recommended)

For OpenAI-compatible APIs, create a JSON configuration file:

Basic Structure

Create ~/.config/goose/providers/my-provider.json:

{
  "name": "my-provider",
  "engine": "openai",
  "host": "https://api.myprovider.com/v1",
  "models": [
    {
      "name": "my-model-7b",
      "context_window": 8192,
      "supports_tools": true,
      "supports_vision": false
    },
    {
      "name": "my-model-70b",
      "context_window": 32768,
      "supports_tools": true,
      "supports_vision": true
    }
  ],
  "auth": {
    "type": "header",
    "header_name": "Authorization",
    "header_value": "Bearer ${MY_PROVIDER_API_KEY}"
  }
}

Configuration Fields

name

string

required

Unique identifier for the provider

engine

string

required

API format to use: openai, anthropic, or ollama

host

string

required

Base URL for API requests

models

array

required

List of available models

models[].name

string

required

Model identifier

models[].context_window

integer

Maximum context size in tokens

models[].supports_tools

boolean

default:"true"

Whether the model supports tool/function calling

models[].supports_vision

boolean

default:"false"

Whether the model can process images

auth

object

Authentication configuration

auth.type

string

Authentication method: header, query, or oauth

Real-World Examples

Together.ai
OpenRouter
vLLM (Self-Hosted)

{
  "name": "together",
  "engine": "openai",
  "host": "https://api.together.xyz/v1",
  "models": [
    {
      "name": "meta-llama/Llama-3.3-70B-Instruct-Turbo",
      "context_window": 131072,
      "supports_tools": true
    },
    {
      "name": "Qwen/Qwen3.5-72B-Instruct-Turbo",
      "context_window": 32768,
      "supports_tools": true
    }
  ],
  "auth": {
    "type": "header",
    "header_name": "Authorization",
    "header_value": "Bearer ${TOGETHER_API_KEY}"
  }
}

Set your API key:

export TOGETHER_API_KEY="your-key-here"

{
  "name": "openrouter",
  "engine": "openai",
  "host": "https://openrouter.ai/api/v1",
  "models": [
    {
      "name": "anthropic/claude-4.5-sonnet",
      "context_window": 200000,
      "supports_tools": true
    },
    {
      "name": "google/gemini-4-flash-latest",
      "context_window": 1000000,
      "supports_tools": true,
      "supports_vision": true
    }
  ],
  "auth": {
    "type": "header",
    "header_name": "Authorization",
    "header_value": "Bearer ${OPENROUTER_API_KEY}"
  },
  "extra_headers": {
    "HTTP-Referer": "https://github.com/block/goose",
    "X-Title": "Goose AI Agent"
  }
}

{
  "name": "vllm",
  "engine": "openai",
  "host": "http://localhost:8000/v1",
  "models": [
    {
      "name": "Qwen/Qwen3.5-32B-Instruct",
      "context_window": 32768,
      "supports_tools": true
    }
  ]
}

No API key needed for local deployment.

Using Your Provider

After creating the configuration file, use it:

export GOOSE_PROVIDER="my-provider"
export GOOSE_MODEL="my-model-70b"
goose session

Or configure interactively:

goose configure
# Select your provider from the list

Goose automatically discovers provider JSON files in ~/.config/goose/providers/.

Programmatic Provider Implementation

For providers with non-standard APIs or special requirements, implement the Provider trait in Rust:

Provider Trait Overview

#[async_trait]
pub trait Provider: Send + Sync {
    // Core methods
    async fn complete(
        &self,
        messages: &[Message],
        config: &ModelConfig,
    ) -> Result<Response>;

    async fn stream(
        &self,
        messages: &[Message],
        config: &ModelConfig,
    ) -> Result<Pin<Box<dyn Stream<Item = Result<StreamChunk>> + Send>>>;

    fn metadata(&self) -> &ProviderMetadata;

    // Optional methods
    fn supports_tools(&self) -> bool { true }
    fn supports_vision(&self) -> bool { false }
    async fn list_models(&self) -> Result<Vec<String>> { Ok(vec![]) }
}

Basic Implementation

Create a custom provider in crates/goose/src/providers/my_provider.rs:

use anyhow::Result;
use async_trait::async_trait;
use crate::providers::base::{Provider, ProviderMetadata, Response};
use crate::model::{Message, ModelConfig};

pub struct MyProvider {
    client: reqwest::Client,
    api_key: String,
    base_url: String,
}

impl MyProvider {
    pub fn new(api_key: String) -> Result<Self> {
        Ok(Self {
            client: reqwest::Client::new(),
            api_key,
            base_url: "https://api.myprovider.com/v1".to_string(),
        })
    }
}

#[async_trait]
impl Provider for MyProvider {
    async fn complete(
        &self,
        messages: &[Message],
        config: &ModelConfig,
    ) -> Result<Response> {
        let request_body = serde_json::json!({
            "model": config.model,
            "messages": messages,
            "temperature": config.temperature,
            "max_tokens": config.max_tokens,
        });

        let response = self.client
            .post(&format!("{}/chat/completions", self.base_url))
            .header("Authorization", format!("Bearer {}", self.api_key))
            .json(&request_body)
            .send()
            .await?;

        let json: serde_json::Value = response.json().await?;
        
        // Parse response and return
        Ok(Response {
            content: json["choices"][0]["message"]["content"]
                .as_str()
                .unwrap_or("") 
                .to_string(),
            usage: extract_usage(&json),
            model: config.model.clone(),
        })
    }

    async fn stream(
        &self,
        messages: &[Message],
        config: &ModelConfig,
    ) -> Result<Pin<Box<dyn Stream<Item = Result<StreamChunk>> + Send>>> {
        // Implement streaming if supported
        todo!("Streaming implementation")
    }

    fn metadata(&self) -> &ProviderMetadata {
        &ProviderMetadata {
            name: "my-provider".to_string(),
            display_name: "My Custom Provider".to_string(),
            supported_models: vec!["model-7b".to_string(), "model-70b".to_string()],
            requires_auth: true,
        }
    }
}

Register the Provider

Add your provider to the registry in crates/goose/src/providers/provider_registry.rs:

use crate::providers::my_provider::MyProvider;

pub fn create_provider(name: &str, config: &Config) -> Result<Box<dyn Provider>> {
    match name {
        "my-provider" => {
            let api_key = config.get_secret("MY_PROVIDER_API_KEY")?;
            Ok(Box::new(MyProvider::new(api_key)?))
        },
        // ... other providers
    }
}

Advanced Features

OAuth Authentication

For providers requiring OAuth:

use crate::oauth::{OAuthClient, OAuthConfig};

pub struct MyOAuthProvider {
    oauth_client: OAuthClient,
    // ...
}

impl MyOAuthProvider {
    pub async fn new(config: &OAuthConfig) -> Result<Self> {
        let oauth_client = OAuthClient::new(config).await?;
        Ok(Self { oauth_client })
    }

    async fn get_access_token(&self) -> Result<String> {
        self.oauth_client.get_token().await
    }
}

See the OAuth Providers guide for a complete implementation.

Retry Logic

Implement custom retry behavior:

use crate::providers::retry::{RetryConfig, retry_with_backoff};

impl MyProvider {
    async fn request_with_retry<F, T>(
        &self,
        f: F,
    ) -> Result<T>
    where
        F: Fn() -> BoxFuture<'static, Result<T>>,
    {
        let retry_config = RetryConfig {
            max_retries: 3,
            initial_delay_ms: 1000,
            max_delay_ms: 60000,
            exponential_base: 2.0,
        };
        
        retry_with_backoff(f, &retry_config).await
    }
}

Model Discovery

Allow dynamic model listing:

#[async_trait]
impl Provider for MyProvider {
    async fn list_models(&self) -> Result<Vec<String>> {
        let response = self.client
            .get(&format!("{}/models", self.base_url))
            .header("Authorization", format!("Bearer {}", self.api_key))
            .send()
            .await?;

        let json: serde_json::Value = response.json().await?;
        let models = json["models"]
            .as_array()
            .unwrap_or(&vec![])
            .iter()
            .filter_map(|m| m["id"].as_str().map(String::from))
            .collect();

        Ok(models)
    }
}

Testing Your Provider

Unit Tests

Test provider functionality:

#[cfg(test)]
mod tests {
    use super::*;

    #[tokio::test]
    async fn test_complete() {
        let provider = MyProvider::new("test-key".to_string()).unwrap();
        let messages = vec![Message::user("Hello")];
        let config = ModelConfig::default();

        let response = provider.complete(&messages, &config).await;
        assert!(response.is_ok());
    }
}

Integration Tests

Test with goose:

export MY_PROVIDER_API_KEY="your-key"
export GOOSE_PROVIDER="my-provider"
goose session

Next Steps

Provider Interface

Complete guide to implementing the Provider trait

Declarative Providers

JSON configuration reference for OpenAI-compatible providers

OAuth Providers

Implement OAuth authentication flows

Providers Overview

API reference for the Provider trait

Getting Started

Core Concepts

User Guides

Advanced

Troubleshooting

Custom Providers

When to Create a Custom Provider

Declarative Providers (Recommended)

Basic Structure

Configuration Fields

Real-World Examples

Using Your Provider

Programmatic Provider Implementation

Provider Trait Overview

Basic Implementation

Register the Provider

Advanced Features

OAuth Authentication

Retry Logic

Model Discovery

Testing Your Provider

Unit Tests

Integration Tests

Next Steps

Provider Interface

Declarative Providers

OAuth Providers

Providers Overview

Build docs developers (and LLMs) love

Getting Started

Core Concepts

User Guides

Advanced

Troubleshooting

​When to Create a Custom Provider

​Declarative Providers (Recommended)

​Basic Structure

​Configuration Fields

​Real-World Examples

​Using Your Provider

​Programmatic Provider Implementation

​Provider Trait Overview

​Basic Implementation

​Register the Provider

​Advanced Features

​OAuth Authentication

​Retry Logic

​Model Discovery

​Testing Your Provider

​Unit Tests

​Integration Tests

​Next Steps

Provider Interface

Declarative Providers

OAuth Providers

Providers Overview

Build docs developers (and LLMs) love

When to Create a Custom Provider

Declarative Providers (Recommended)

Basic Structure

Configuration Fields

Real-World Examples

Using Your Provider

Programmatic Provider Implementation

Provider Trait Overview

Basic Implementation

Register the Provider

Advanced Features

OAuth Authentication

Retry Logic

Model Discovery

Testing Your Provider

Unit Tests

Integration Tests

Next Steps