Skip to main content

Overview

Golden examples are high-quality query-response pairs that demonstrate ideal assistant behavior. They are used to improve response quality through RAG (Retrieval-Augmented Generation) by providing the LLM with relevant examples during query processing.

Sources of Golden Examples

Golden examples can be created through multiple sources:
  1. Auto-approval - Automatically created from positive user feedback when auto-approval is enabled
  2. Manual Review - Created by admins when resolving feedback
  3. Manual Creation - Directly created by admins through the API

Example Lifecycle

  • Active - Examples are stored in the vector database and used for RAG retrieval
  • Inactive - Examples are archived but preserved for potential future reactivation
  • Deleted - Examples are permanently removed from the system

List Golden Examples

Requires golden_example.view permission

Query Parameters

limit
integer
default:"20"
Number of items to return (1-100)
offset
integer
default:"0"
Number of items to skip for pagination
source_type
string
Filter by source: feedback or manual
is_active
boolean
Filter by active status: true or false
Search in query or response text

Response

items
array
Array of golden example objects
total
integer
Total number of examples matching filters
limit
integer
Number of items returned
offset
integer
Number of items skipped

Example Request

curl -X GET "https://api.example.com/feedback/golden-examples/?source_type=manual&is_active=true&limit=10" \
  -H "Authorization: Bearer YOUR_TOKEN"

Example Response

{
  "items": [
    {
      "id": "8d8f7780-8536-51ef-b827-f18ed2091bf8",
      "feedback_id": null,
      "source_type": "manual",
      "approval_type": "manual",
      "original_query": "How do I escalate a P0 incident?",
      "original_response": "",
      "golden_response": "To escalate a P0 incident, follow these steps:\n1. Contact the on-call engineer immediately via PagerDuty\n2. Notify the incident commander\n3. Create a Slack channel #incident-[number]\n4. Update the incident status page",
      "qdrant_point_id": "point_12345",
      "created_by": "admin-123",
      "creator_email": "[email protected]",
      "is_active": true,
      "created_at": "2026-03-01T08:00:00Z",
      "updated_at": null
    }
  ],
  "total": 1,
  "limit": 10,
  "offset": 0
}

Get Golden Example

Requires golden_example.view permission

Path Parameters

example_id
string
required
ID of the golden example to retrieve

Response

Returns a GoldenExampleResponse object with all details.

Example Request

curl -X GET https://api.example.com/feedback/golden-examples/8d8f7780-8536-51ef-b827-f18ed2091bf8 \
  -H "Authorization: Bearer YOUR_TOKEN"

Example Response

{
  "id": "8d8f7780-8536-51ef-b827-f18ed2091bf8",
  "feedback_id": "7c9e6679-7425-40de-944b-e07fc1f90ae7",
  "source_type": "feedback",
  "approval_type": "admin_review",
  "original_query": "How do I escalate a P0 incident?",
  "original_response": "Contact the on-call engineer.",
  "golden_response": "To escalate a P0 incident, follow these steps:\n1. Contact the on-call engineer immediately via PagerDuty\n2. Notify the incident commander\n3. Create a Slack channel #incident-[number]\n4. Update the incident status page",
  "qdrant_point_id": "point_12345",
  "created_by": "admin-123",
  "creator_email": "[email protected]",
  "is_active": true,
  "created_at": "2026-03-01T08:00:00Z",
  "updated_at": "2026-03-01T09:15:00Z"
}

Create Golden Example

Requires golden_example.create permission

Request Body

original_query
string
required
The user’s original question or prompt
golden_response
string
required
The ideal response that should be provided
original_response
string
The original AI response (optional for manual creation)

Response

Returns the created GoldenExampleResponse object.

Example Request

curl -X POST https://api.example.com/feedback/golden-examples/ \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "original_query": "What are the SLA targets for P1 incidents?",
    "golden_response": "P1 incidents have the following SLA targets:\n- Response time: 30 minutes\n- Resolution time: 4 hours\n- Update frequency: Every 2 hours\n\nThese targets apply during business hours (9am-5pm EST). Outside business hours, add 1 hour to each target.",
    "original_response": "P1 incidents should be resolved within 4 hours."
  }'

Example Response

{
  "id": "9e9f8891-9647-62f0-c938-028fe3102cg9",
  "feedback_id": null,
  "source_type": "manual",
  "approval_type": "manual",
  "original_query": "What are the SLA targets for P1 incidents?",
  "original_response": "P1 incidents should be resolved within 4 hours.",
  "golden_response": "P1 incidents have the following SLA targets:\n- Response time: 30 minutes\n- Resolution time: 4 hours\n- Update frequency: Every 2 hours\n\nThese targets apply during business hours (9am-5pm EST). Outside business hours, add 1 hour to each target.",
  "qdrant_point_id": "point_67890",
  "created_by": "admin-123",
  "creator_email": "[email protected]",
  "is_active": true,
  "created_at": "2026-03-01T10:30:00Z",
  "updated_at": null
}
Manually created golden examples are immediately added to the vector database and available for RAG retrieval.

Update Golden Example

Requires golden_example.edit permission

Path Parameters

example_id
string
required
ID of the golden example to update

Request Body

golden_response
string
Updated ideal response text
is_active
boolean
Whether the example should be active and used for RAG
All fields are optional. Only provided fields will be updated.

Response

Returns the updated GoldenExampleResponse object.

Example Request

curl -X PUT https://api.example.com/feedback/golden-examples/9e9f8891-9647-62f0-c938-028fe3102cg9 \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "golden_response": "P1 incidents have the following SLA targets:\n- Response time: 30 minutes\n- Resolution time: 4 hours\n- Update frequency: Every 2 hours\n- Escalation: After 2 hours if no progress\n\nThese targets apply during business hours (9am-5pm EST). Outside business hours, add 1 hour to each target.",
    "is_active": true
  }'

Example Response

{
  "id": "9e9f8891-9647-62f0-c938-028fe3102cg9",
  "feedback_id": null,
  "source_type": "manual",
  "approval_type": "manual",
  "original_query": "What are the SLA targets for P1 incidents?",
  "original_response": "P1 incidents should be resolved within 4 hours.",
  "golden_response": "P1 incidents have the following SLA targets:\n- Response time: 30 minutes\n- Resolution time: 4 hours\n- Update frequency: Every 2 hours\n- Escalation: After 2 hours if no progress\n\nThese targets apply during business hours (9am-5pm EST). Outside business hours, add 1 hour to each target.",
  "qdrant_point_id": "point_67890_updated",
  "created_by": "admin-123",
  "creator_email": "[email protected]",
  "is_active": true,
  "created_at": "2026-03-01T10:30:00Z",
  "updated_at": "2026-03-01T11:45:00Z"
}
Updating the golden_response will re-embed the example in the vector database, which may take a few seconds.

Delete Golden Example

This action is permanent and cannot be undone. The example will be removed from the vector database.
Requires golden_example.delete permission

Path Parameters

example_id
string
required
ID of the golden example to delete

Response

Returns HTTP 204 No Content on success.

Example Request

curl -X DELETE https://api.example.com/feedback/golden-examples/9e9f8891-9647-62f0-c938-028fe3102cg9 \
  -H "Authorization: Bearer YOUR_TOKEN"

Common Error Responses

400 Bad Request

{
  "detail": "golden_response is required"
}

401 Unauthorized

{
  "detail": "Not authenticated"
}

403 Forbidden

{
  "detail": "Insufficient permissions. Required: golden_example.create"
}

404 Not Found

{
  "detail": "Golden example not found"
}

500 Internal Server Error

{
  "detail": "Failed to create golden example"
}

How Golden Examples Work

Golden examples enhance the assistant’s responses through RAG:
  1. Embedding - Each golden example’s query is embedded and stored in the Qdrant vector database
  2. Retrieval - When a user asks a question, similar golden examples are retrieved
  3. Context - Retrieved examples are provided to the LLM as context
  4. Generation - The LLM uses the examples to inform its response
Golden examples are most effective when they:
  • Cover common query patterns
  • Include detailed, accurate responses
  • Are kept up-to-date with current information
  • Demonstrate the desired tone and format

Approval Types

auto
string
Automatically approved from positive feedback when auto-approval is enabled
manual
string
Manually created by an admin through the API
admin_review
string
Created by an admin when resolving feedback

Best Practices

Quality Over Quantity

Focus on creating high-quality, detailed examples rather than many basic ones.

Keep Examples Current

Regularly review and update examples to ensure accuracy and relevance.

Cover Common Patterns

Create examples for frequently asked questions and common use cases.

Use Consistent Format

Maintain consistent formatting and structure across golden responses.

Include Context

Provide enough detail in responses to be useful as examples for the LLM.

Monitor Performance

Track which examples are being retrieved to identify gaps in coverage.

Archive, Don't Delete

Use is_active=false to archive outdated examples instead of deleting them.

Leverage Feedback

Use the feedback system to continuously identify opportunities for new examples.

Integration Example

Here’s a complete workflow for managing golden examples:
Python
import requests

BASE_URL = "https://api.example.com"
HEADERS = {"Authorization": "Bearer YOUR_TOKEN"}

# 1. Create a new golden example
response = requests.post(
    f"{BASE_URL}/feedback/golden-examples/",
    headers=HEADERS,
    json={
        "original_query": "How do I reset my password?",
        "golden_response": "To reset your password:\n1. Go to the login page\n2. Click 'Forgot Password'\n3. Enter your email\n4. Check your inbox for the reset link\n5. Follow the link and create a new password\n\nThe reset link expires in 24 hours."
    }
)
example = response.json()
print(f"Created example: {example['id']}")

# 2. List all active examples
response = requests.get(
    f"{BASE_URL}/feedback/golden-examples/",
    headers=HEADERS,
    params={"is_active": True, "limit": 50}
)
data = response.json()
print(f"Active examples: {data['total']}")

# 3. Update an example
response = requests.put(
    f"{BASE_URL}/feedback/golden-examples/{example['id']}",
    headers=HEADERS,
    json={
        "golden_response": "To reset your password:\n1. Go to the login page\n2. Click 'Forgot Password'\n3. Enter your email\n4. Check your inbox for the reset link (check spam folder)\n5. Follow the link and create a new password\n\nThe reset link expires in 24 hours. If you don't receive the email, contact support."
    }
)
print("Example updated")

# 4. Archive old example instead of deleting
response = requests.put(
    f"{BASE_URL}/feedback/golden-examples/{example['id']}",
    headers=HEADERS,
    json={"is_active": False}
)
print("Example archived")

Build docs developers (and LLMs) love