Skip to main content
The IdeaGen module processes notebook records (or a free-text description) through a multi-stage AI pipeline that extracts knowledge points, explores and filters research ideas, and generates a concise research statement for each idea.

​
Endpoints

​
WS /api/v1/ideagen/generate

Run the full idea generation workflow. The server streams typed status messages as each stage completes and delivers individual ideas as they are ready.

​
Initial message

​
notebook_id
string
ID of a notebook to use as source material. Provide either this or records.
​
record_ids
string[]
Filter to specific records within notebook_id. Omit to use all records.
​
records
object[]
List of record objects for cross-notebook mode. Provide either this or notebook_id.
​
user_thoughts
string
Optional free-text description of a research topic. Can be used without any notebook records to generate ideas from scratch.

​
Streaming messages

All messages share a common base structure. The type field distinguishes regular status messages from idea deliveries.
​
type
string
required
Either "status", "idea", "task_id", or "error".
For type: "task_id":
​
task_id
string
Unique identifier for this run.
For type: "status":
​
stage
string
Current pipeline stage. See the stage table below.
​
message
string
Human-readable description of what is happening.
​
timestamp
string
ISO 8601 timestamp.
​
data
object
Stage-specific data payload. Content varies by stage.
For type: "idea":
​
data
object
The generated idea.

​
Pipeline stages

The workflow emits a status message at each of the following stages:
StageDescription
initInitializing the workflow
extractingExtracting knowledge points from records
knowledge_extractedExtraction complete; data.knowledge_points contains the list
filteringApplying loose filter to knowledge points
filteredFiltering complete; data.filtered_points contains survivors
exploringExploring research ideas for a knowledge point
exploredExploration complete for one knowledge point
strict_filteringStrictly filtering candidate ideas
generatingGenerating the research statement
idea_readyA single idea is ready (followed immediately by a type: "idea" message)
completeAll ideas generated; data.ideas contains the full list
errorAn error occurred; data.error contains the message

​
Example

const ws = new WebSocket('ws://localhost:8001/api/v1/ideagen/generate');

ws.onopen = () => {
  ws.send(JSON.stringify({
    notebook_id: "nb_abc123"
  }));
};

ws.onmessage = (event) => {
  const msg = JSON.parse(event.data);

  if (msg.type === 'task_id') {
    console.log('Task:', msg.task_id);
  } else if (msg.type === 'status') {
    console.log(`[${msg.stage}] ${msg.message}`);
  } else if (msg.type === 'idea') {
    console.log('New idea:', msg.data.statement);
  }
};
Example streaming messages for a run with two knowledge points: 
{ "type": "task_id", "task_id": "ideagen_20250101_xyz" }
{ "type": "status", "stage": "init", "message": "Initializing idea generation workflow...", "timestamp": "2025-01-01T12:00:00" }
{ "type": "status", "stage": "extracting", "message": "Extracting knowledge points from 5 records...", "data": { "record_count": 5 }, "timestamp": "..." }
{ "type": "status", "stage": "knowledge_extracted", "message": "Extracted 3 knowledge points", "data": { "knowledge_points": [...], "count": 3 }, "timestamp": "..." }
{ "type": "status", "stage": "filtering", "message": "Filtering 3 knowledge points (loose criteria)...", "timestamp": "..." }
{ "type": "status", "stage": "filtered", "message": "Filtered to 2 knowledge points", "data": { "filtered": 2, "original": 3 }, "timestamp": "..." }
{ "type": "status", "stage": "exploring", "message": "Exploring research ideas for: Attention mechanisms (1/2)", "timestamp": "..." }
{ "type": "status", "stage": "explored", "message": "Generated 4 research ideas for: Attention mechanisms", "timestamp": "..." }
{ "type": "status", "stage": "strict_filtering", "message": "Strictly filtering 4 ideas for: Attention mechanisms", "timestamp": "..." }
{ "type": "status", "stage": "generating", "message": "Generating statement for: Attention mechanisms", "timestamp": "..." }
{ "type": "status", "stage": "idea_ready", "message": "Research idea ready: Attention mechanisms", "data": { "index": 1, "total": 2 }, "timestamp": "..." }
{ "type": "idea", "data": {
    "id": "idea-0",
    "knowledge_point": "Attention mechanisms",
    "description": "Mathematical foundations of self-attention in transformers",
    "research_ideas": [...],
    "statement": "Investigating efficient attention variants for long-sequence modeling in resource-constrained environments.",
    "expanded": false
  }
}
{ "type": "status", "stage": "complete", "message": "Successfully generated 2 research ideas", "data": { "ideas": [...], "count": 2 }, "timestamp": "..." }
You can pass only user_thoughts without any notebook records. In this case, IdeaGen treats your description as a single virtual knowledge point and generates ideas from it directly.

​
GET /api/v1/ideagen/test

Simple health check that confirms the IdeaGen API is running. 
curl http://localhost:8001/api/v1/ideagen/test

{ "status": "ok", "message": "IdeaGen API is working" }

Build docs developers (and LLMs) love

Get started for freeTalk to us