Skip to main content

​
Overview

The HallucinationDetectorApp class provides a complete application wrapper around the PAS2 detector, including database storage for feedback, progress tracking, and Gradio UI integration.

​
Constructor

HallucinationDetectorApp()
Initialize the hallucination detector application. No parameters required.

​
Class attributes

​
pas2
PAS2 | None
The initialized PAS2 detector instance. Initially None until initialize_api is called.
​
data_dir
str
Directory path for persistent data storage (default: "/data").
​
db_path
str
Full path to the SQLite database file for storing feedback.
​
progress_callback
callable | None
Registered callback function for progress updates.

​
Methods

​
initialize_api

initialize_api(mistral_api_key: str, openai_api_key: str) -> str
Initialize the PAS2 detector with API credentials.
​
mistral_api_key
str
required
Mistral API key for accessing the Mistral API.
​
openai_api_key
str
required
OpenAI API key for accessing the OpenAI API.
​
return
str
Status message indicating success or failure of initialization.

​
process_query

process_query(query: str) -> Dict
Process a query using the PAS2 detector.
​
query
str
required
The user query to analyze for hallucinations.
​
return
Dict
Dictionary containing detection results with keys:
  • original_query: The input query
  • original_response: Response to the original query
  • paraphrased_queries: List of paraphrased queries
  • paraphrased_responses: List of responses to paraphrased queries
  • hallucination_detected: Boolean indicating detection result
  • confidence_score: Float between 0-1
  • conflicting_facts: List of detected conflicts
  • reasoning: Detailed reasoning
  • summary: Summary of analysis
  • error: Error message if processing failed
Returns a dictionary with an error key if the PAS2 instance is not initialized or if the query is empty.

​
save_feedback

save_feedback(results: Dict, feedback: str) -> str
Save detection results and user feedback to the SQLite database.
​
results
Dict
required
The results dictionary returned from process_query.
​
feedback
str
required
User feedback text to store with the results.
​
return
str
Status message indicating success or failure of the save operation.

​
get_feedback_stats

get_feedback_stats() -> Dict | None
Retrieve statistics about collected feedback from the database.
​
return
Dict | None
Dictionary containing statistics with keys:
  • total_feedback: Total number of feedback entries
  • hallucinations_detected: Count of detections with hallucinations
  • no_hallucinations: Count of detections without hallucinations
  • average_confidence: Average confidence score across all entries
Returns None if an error occurs.

​
set_progress_callback

set_progress_callback(callback: callable) -> None
Register a callback function to receive progress updates.
​
callback
callable
required
Function to call with progress updates. Should accept (stage: str, **kwargs).

​
Database schema

The app automatically creates a SQLite database with the following schema: 
CREATE TABLE IF NOT EXISTS feedback (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    timestamp TEXT,
    original_query TEXT,
    original_response TEXT,
    paraphrased_queries TEXT,
    paraphrased_responses TEXT,
    hallucination_detected INTEGER,
    confidence_score REAL,
    conflicting_facts TEXT,
    reasoning TEXT,
    summary TEXT,
    user_feedback TEXT
)

​
Example usage

from pas2 import HallucinationDetectorApp

# Initialize the app
app = HallucinationDetectorApp()

# Set up API keys
status = app.initialize_api(
    mistral_api_key="your-mistral-key",
    openai_api_key="your-openai-key"
)
print(status)  # "API keys set successfully! You can now use the application."

# Process a query
results = app.process_query("What is the capital of France?")

if "error" not in results:
    print(f"Hallucination: {results['hallucination_detected']}")
    print(f"Confidence: {results['confidence_score']}")
    
    # Save feedback
    feedback_status = app.save_feedback(results, "Accurate result")
    print(feedback_status)
    
    # Get stats
    stats = app.get_feedback_stats()
    print(f"Total feedback entries: {stats['total_feedback']}")

​
Example with progress callback

def progress_handler(stage, **kwargs):
    if stage == "generating_paraphrases":
        print("Generating paraphrases...")
    elif stage == "responses_progress":
        completed = kwargs.get('completed_responses', 0)
        total = kwargs.get('total_responses', 0)
        print(f"Fetching responses: {completed}/{total}")
    elif stage == "complete":
        print("Analysis complete!")

app = HallucinationDetectorApp()
app.set_progress_callback(progress_handler)
app.initialize_api(mistral_api_key="key", openai_api_key="key")

results = app.process_query("Who wrote 1984?")
The database is created in the /data directory by default for persistent storage in Hugging Face Spaces. If this directory is not accessible, the app falls back to a temporary directory.

Build docs developers (and LLMs) love

Get started for freeTalk to us