Prompting Strategies

CheckThat AI supports multiple prompting strategies to optimize claim extraction and normalization. Choose the right strategy based on your use case and desired output quality.

Available Prompting Strategies

CheckThat AI implements three core prompting approaches:

Zero-Shot

Direct claim extraction without examples

Few-Shot

Learning from provided examples

Chain-of-Thought

Step-by-step reasoning process

Zero-Shot Prompting

Zero-shot prompting provides direct instructions without examples. Best for simple, straightforward claims.

How It Works

The system uses the base instruction prompt defined in prompts.py:55:

instruction = """Identify the decontextualized, stand-alone, and verifiable central claim in the given post: """

The system prompt guides the model through a multi-step process:

Sentence Splitting: Break down the post into individual sentences
Selection: Identify verifiable information
Disambiguation: Resolve referential and structural ambiguity
Decomposition: Extract self-contained propositions

Example API Call

from checkthat import CheckThat

client = CheckThat(api_key="your-api-key")

response = client.chat.completions.create(
    model="gpt-5-2025-08-07",
    messages=[
        {
            "role": "user",
            "content": "Lieutenant Retired General Asif Mumtaz appointed as Chairman Pakistan Medical Commission PMC"
        }
    ]
)

print(response.choices[0].message.content)
# Output: "Pakistani government appoints former army general to head medical regulatory body."

When to Use Zero-Shot

Best For

Simple, clear statements
News headlines
Direct factual claims
Quick processing needs

Avoid When

Complex context required
Ambiguous language
Domain-specific jargon
Subtle misinformation

Few-Shot Prompting

Few-shot prompting provides examples to guide the model’s behavior. The system includes pre-configured examples in prompts.py:59-118.

Example Structure

CheckThat AI includes 5 carefully curated examples covering:

Medical/institutional claims
Celebrity/viral content
Health misinformation
Scientific false claims
COVID-19 related content

Implementation Details

The few-shot prompt includes examples with clear structure:

<user_query id="example-1">
**Identify the decontextualized, stand-alone, and verifiable central claim:**
[Original Post]

Let's think step by step.
</user_query>

<assistant_response id="example-1">
**Normalized claim:** [Extracted Claim]
</assistant_response>

Example API Call with Few-Shot

from checkthat import CheckThat

client = CheckThat(api_key="your-api-key")

# Construct few-shot prompt by prepending examples
few_shot_examples = """
Here are some examples:

Example 1:
Input: "Eating vaginal fluids makes you immune to cancer. Scientists at St. Austin University in North Carolina investigated..."
Output: "St.Austin University North Carolina says eating vaginal fluid makes you immune to cancer"

Example 2:
Input: "Corona virus before it reaches the lungs it remains in the throat for four days..."
Output: "Gargling water can protect against coronavirus"

Now extract the claim from:
"""

response = client.chat.completions.create(
    model="gpt-5-2025-08-07",
    messages=[
        {
            "role": "user",
            "content": few_shot_examples + "Hydrate YOURSELF. Water after waking up helps activate internal organs."
        }
    ]
)

print(response.choices[0].message.content)
# Output: "Drinking water at specific times can have different health benefits"

When to Use Few-Shot

Use few-shot prompting when you need consistent output format or have domain-specific examples

Ideal scenarios:

Establishing output format consistency
Domain-specific terminology
Complex extraction patterns
Training on edge cases

Chain-of-Thought (CoT) Reasoning

Chain-of-Thought prompting adds explicit reasoning steps before the final claim. Implemented with the trigger phrase from prompts.py:57:

chain_of_thought_trigger = "Let's think step by step."

How CoT Works

The enhanced few-shot CoT prompt (prompts.py:120-215) includes detailed reasoning:

Identify Key Elements

Spot the subject, action, and context

**Identify the actor:** "Lieutenant Retired General Asif Mumtaz."
**Find the action:** He is "appointed as Chairman."
**Clarify the institution:** "Pakistan Medical Commission (PMC)"

Infer Missing Context

Determine implicit information

4. **Infer the agent:** Such appointments are made by the Pakistani government.

Condense and Rephrase

Create self-contained claim

5. **Condense & rephrase:** Government appoints ex-army general to lead medical regulatory body.

Output Final Claim

Present the normalized claim

**Normalized claim:** Pakistani government appoints former army general to head medical regulatory body.

Example API Call with CoT

from checkthat import CheckThat

client = CheckThat(api_key="your-api-key")

response = client.chat.completions.create(
    model="gpt-5-2025-08-07",
    messages=[
        {
            "role": "user",
            "content": """Identify the claim in this post: 
            
            'A priceless clip of 1970 of Bruce Lee playing Table Tennis with his Nan-chak!!'
            
            Let's think step by step."""
        }
    ]
)

print(response.choices[0].message.content)
# The model will show reasoning steps:
# 1. **Spot the subject & date:** "Bruce Lee" and "1970."
# 2. **Extract the activity:** "playing Table Tennis with his Nan-chak" (nunchucks).
# 3. **Add clarifying detail:** Bruce Lee is now deceased ("late actor and martial artist").
# 4. **Formulate succinctly:** "Late actor and martial artist Bruce Lee playing table tennis with a set of nunchucks."
#
# **Normalized claim:** Late actor and martial artist Bruce Lee playing table tennis with a set of nunchucks.

When to Use Chain-of-Thought

CoT increases token usage and latency but significantly improves accuracy for complex claims

Best for:

Complex ambiguous statements
Multi-part claims requiring decomposition
Cases needing explicit context resolution
High-stakes fact-checking scenarios

Strategy Comparison

Performance
Use Cases
Code Examples

Strategy	Speed	Accuracy	Token Usage	Cost
Zero-Shot	Fast	Good	Low	$
Few-Shot	Medium	Better	Medium	$$
Chain-of-Thought	Slow	Best	High	$$$

# Quick comparison
from checkthat import CheckThat

client = CheckThat(api_key="your-api-key")
post = "Your social media post here"

# Zero-shot (fastest)
zero_shot = client.chat.completions.create(
    model="gpt-5-nano-2025-08-07",  # Faster model
    messages=[{"role": "user", "content": post}]
)

# Few-shot (balanced)
few_shot = client.chat.completions.create(
    model="gpt-5-2025-08-07",
    messages=[{
        "role": "user", 
        "content": few_shot_examples + post
    }]
)

# Chain-of-Thought (most accurate)
cot = client.chat.completions.create(
    model="gpt-5-2025-08-07",
    messages=[{
        "role": "user",
        "content": f"{post}\n\nLet's think step by step."
    }]
)

System Prompt Details

The core system prompt (ClaimNorm) from prompts.py:3-53 includes:

View Full System Prompt Structure

Identity:

ClaimNorm AI assistant
Expert in claim detection, extraction, and normalization

4-Step Process:

Sentence Splitting and Context Creation
- Split post into sentences
- Create context using 2 preceding and 2 following sentences
Selection
- Determine if sentence contains verifiable information
- Rewrite to retain only verifiable parts
Disambiguation
- Resolve referential ambiguity (unclear references)
- Resolve structural ambiguity (multiple interpretations)
- Discard if ambiguity cannot be resolved
Decomposition
- Identify specific, verifiable propositions
- Ensure decontextualization (self-contained)
- Create simplest discrete units of information

Fallback:

If no verifiable claims found, return extractive summary
Maximum 25 words
Preserve named entities
Return as news headline style

Advanced Techniques

Combining Strategies

You can combine strategies for optimal results:

from checkthat import CheckThat

client = CheckThat(api_key="your-api-key")

# First pass: Few-shot for structure
first_pass = client.chat.completions.create(
    model="gpt-5-nano-2025-08-07",
    messages=[{"role": "user", "content": few_shot_examples + original_post}]
)

extracted_claim = first_pass.choices[0].message.content

# Second pass: CoT for refinement
refined = client.chat.completions.create(
    model="gpt-5-2025-08-07",
    messages=[
        {
            "role": "user",
            "content": f"""Verify and refine this claim:
            
            Original: {original_post}
            Extracted: {extracted_claim}
            
            Is this claim accurate and self-contained? Let's think step by step."""
        }
    ]
)

Custom System Prompts

Override the default system prompt for specialized use cases:

custom_system_prompt = """
You are a medical fact-checker specialized in COVID-19 claims.
Focus on:
- Treatment claims
- Vaccine information
- Transmission mechanisms
- Official health guidance

Always preserve source attribution when present.
"""

response = client.chat.completions.create(
    model="gpt-5-2025-08-07",
    messages=[
        {"role": "system", "content": custom_system_prompt},
        {"role": "user", "content": "Gargling with salt water kills coronavirus"}
    ]
)

Best Practices

Start Simple

Begin with zero-shot, upgrade to few-shot if needed

Test Examples

Validate few-shot examples match your domain

Monitor Costs

CoT uses 2-3x more tokens than zero-shot

Measure Quality

Use G-Eval metrics to compare strategies

For production systems, consider implementing a hybrid approach: use zero-shot for initial filtering, then apply CoT to high-priority or ambiguous claims.

Get Started

Core Concepts

Web Application

Guides

Deployment

Available Prompting Strategies

Zero-Shot

Few-Shot

Chain-of-Thought

Zero-Shot Prompting

How It Works

Example API Call

When to Use Zero-Shot

Best For

Avoid When

Few-Shot Prompting

Example Structure

Implementation Details

Example API Call with Few-Shot

When to Use Few-Shot

Chain-of-Thought (CoT) Reasoning

How CoT Works

Example API Call with CoT

When to Use Chain-of-Thought

Strategy Comparison

System Prompt Details

Advanced Techniques

Combining Strategies

Custom System Prompts

Best Practices

Start Simple

Test Examples

Monitor Costs

Measure Quality

Next Steps

Self-Refine

Custom Evaluation

Build docs developers (and LLMs) love

Get Started

Core Concepts

Web Application

Guides

Deployment

​Available Prompting Strategies

Zero-Shot

Few-Shot

Chain-of-Thought

​Zero-Shot Prompting

​How It Works

​Example API Call

​When to Use Zero-Shot

Best For

Avoid When

​Few-Shot Prompting

​Example Structure

​Implementation Details

​Example API Call with Few-Shot

​When to Use Few-Shot

​Chain-of-Thought (CoT) Reasoning

​How CoT Works

​Example API Call with CoT

​When to Use Chain-of-Thought

​Strategy Comparison

​System Prompt Details

​Advanced Techniques

​Combining Strategies

​Custom System Prompts

​Best Practices

Start Simple

Test Examples

Monitor Costs

Measure Quality

​Next Steps

Self-Refine

Custom Evaluation

Build docs developers (and LLMs) love

Available Prompting Strategies

Zero-Shot Prompting

How It Works

Example API Call

When to Use Zero-Shot

Few-Shot Prompting

Example Structure

Implementation Details

Example API Call with Few-Shot

When to Use Few-Shot

Chain-of-Thought (CoT) Reasoning

How CoT Works

Example API Call with CoT

When to Use Chain-of-Thought

Strategy Comparison

System Prompt Details

Advanced Techniques

Combining Strategies

Custom System Prompts

Best Practices

Next Steps