Skip to main content
The Question Generator creates targeted practice questions in two modes. Custom mode grounds questions in your knowledge base. Mimic mode uploads a reference exam and produces new questions that match its style, format, and difficulty. All questions go through a single-pass relevance analysis before being saved — no rejection loops, just a relevance classification (high or partial) to help you understand how well each question is anchored to your material.

​
Modes

Custom mode retrieves background knowledge from your knowledge base, plans a set of question focuses, then generates and analyses each question in parallel.

​
Workflow

User requirement
    ↓
RetrieveAgent    — generates RAG queries, retrieves background knowledge
    ↓
Plan generation  — creates one focus per question (topic, difficulty, type)
    ↓
GenerateAgent    — generates each question from knowledge + focus
    ↓
RelevanceAnalyzer — classifies each question as high or partial relevance
    ↓
Save results

​
How to use

1

Open the question generator

Navigate to http://localhost:3782/question.
2

Fill in the requirements

Enter a topic or knowledge point, select a difficulty level, choose a question type, and set the number of questions to generate.
3

Click Generate Questions

Click Generate Questions. Progress updates stream live as each question is generated and analysed.
4

Review the results

Each question displays alongside its relevance classification and a KB coverage explanation. Questions marked partial include extension notes describing how they go beyond the source material.

​
Supported question types

TypeDescription
Multiple choiceFour-option questions with a single correct answer and explanation
Fill-in-the-blankShort answer questions targeting a specific term or value
CalculationStep-by-step numerical problems
Written responseOpen-ended conceptual or analytical questions
The question type is specified as part of your requirement. In Mimic mode, the type is inferred from the reference question.

​
Relevance analysis

Every generated question is analysed by RelevanceAnalyzer after generation. This is a single-pass analysis — questions are never rejected.
Relevance levelMeaning
highThe question is fully grounded in your knowledge base content
partialThe question extends beyond the knowledge base; extension_points explains how
{
  "decision": "approve",
  "relevance": "high",
  "kb_coverage": "This question tests the definition of gradient descent covered in chapter 3.",
  "extension_points": ""
}

​
Python API

​
Custom mode

import asyncio
from src.agents.question import AgentCoordinator

async def main():
    coordinator = AgentCoordinator(
        kb_name="ai_textbook",
        output_dir="data/user/question"
    )

    # Generate multiple questions from text requirement
    result = await coordinator.generate_questions_custom(
        requirement_text="Generate 3 medium-difficulty questions about deep learning basics",
        difficulty="medium",
        question_type="choice",
        count=3
    )

    print(f"Generated {result['completed']}/{result['requested']} questions")
    for q in result['results']:
        print(f"- Relevance: {q['validation']['relevance']}")

asyncio.run(main())

​
Mimic mode

from src.agents.question.tools.exam_mimic import mimic_exam_questions

result = await mimic_exam_questions(
    pdf_path="exams/midterm.pdf",
    kb_name="calculus",
    output_dir="data/user/question/mimic_papers",
    max_questions=5
)

print(f"Generated {result['successful_generations']} questions")
print(f"Output: {result['output_file']}")

​
Output files

Each batch run creates a timestamped directory:
data/user/question/batch_YYYYMMDD_HHMMSS/
├── knowledge.json       # RAG queries issued and retrieval results
├── plan.json            # Question focuses (one per question)
├── q_1/
│   ├── result.json      # Question content + relevance analysis
│   └── question.md      # Human-readable Markdown version
├── q_2/
│   ├── result.json
│   └── question.md
└── summary.json         # Aggregate stats: requested, completed, failed
Each mimic run saves to a folder named after the source PDF:
data/user/question/mimic_papers/{paper_name}/
├── auto/{paper_name}.md                              # MinerU parsed Markdown
├── {paper_name}_YYYYMMDD_HHMMSS_questions.json       # Extracted reference questions
└── {paper_name}_YYYYMMDD_HHMMSS_generated.json       # Generated questions

Build docs developers (and LLMs) love

Get started for freeTalk to us