Models API The Models API provides functions for interacting with generative AI models.
generate() Generate calls a generative model based on the provided prompt and configuration.
Signature: generate < O extends z . ZodTypeAny = z . ZodTypeAny , CustomOptions extends z . ZodTypeAny = typeof GenerationCommonConfigSchema > ( registry : Registry , options : GenerateOptions < O , CustomOptions > | PromiseLike < GenerateOptions < O , CustomOptions >> ): Promise < GenerateResponse < z . infer < O >>>
Parameters Generation options The model to use (name string or ModelAction). If not provided, uses the default model.
User prompt - can be text or multipart content
System instructions for the model
Conversation history for multi-turn interactions
Tools the model can call. Model will automatically invoke tools unless returnToolRequests is true.
If true, return tool requests without executing them
Model configuration including:
temperature: Number (0-2) - Sampling temperaturemaxOutputTokens: Maximum tokens to generatetopK: Top-K sampling parametertopP: Top-P (nucleus) sampling parameterstopSequences: Array of sequences that stop generation Output schema specification
schema: Zod schema for structured outputformat: Output format (e.g., ‘json’, ‘text’) Documents for retrieval-augmented generation (RAG)
Returns Generation response object The generated text content
Parses and returns structured output according to the schema
Complete conversation including user prompts, model responses, and tool calls
Token usage statistics:
inputTokens: Number of input tokensoutputTokens: Number of generated tokenstotalTokens: Total tokens used Why generation stopped (e.g., ‘stop’, ‘length’, ‘safety’)
Additional details about finish reason
Example import { generate } from '@genkit-ai/ai' ; import { z } from 'zod' ; // Simple text generation const response = await generate ( registry , { model: 'googleai/gemini-2.0-flash-exp' , prompt: 'Write a haiku about code' , }); console . log ( response . text ); // Structured output const response = await generate ( registry , { model: 'googleai/gemini-2.0-flash-exp' , prompt: 'List 3 programming languages' , output: { schema: z . object ({ languages: z . array ( z . string ()), }), }, }); const { languages } = response . output (); // With tools const response = await generate ( registry , { model: 'googleai/gemini-2.0-flash-exp' , prompt: 'What is the weather in Paris?' , tools: [ weatherTool ], });
generateStream() Streaming version of generate() that yields response chunks as they are generated.
Signature: generateStream < O extends z . ZodTypeAny = z . ZodTypeAny , CustomOptions extends z . ZodTypeAny = typeof GenerationCommonConfigSchema > ( registry : Registry , options : GenerateStreamOptions < O , CustomOptions > | PromiseLike < GenerateStreamOptions < O , CustomOptions >> ): GenerateStreamResponse < z . infer < O >>
Parameters Accepts the same GenerateOptions as generate(), plus:
Callback invoked for each chunk: ( chunk : GenerateResponseChunk ) => void | Promise < void >
Returns Streaming response object response
Promise<GenerateResponse>
Promise that resolves to the final complete response
stream
Channel<GenerateResponseChunk>
Async iterable channel of response chunks. Each chunk contains:
text: Incremental textcontent: Array of content partsusage: Cumulative token usage Example import { generateStream } from '@genkit-ai/ai' ; const { response , stream } = generateStream ( registry , { model: 'googleai/gemini-2.0-flash-exp' , prompt: 'Write a long story' , }); // Process chunks as they arrive for await ( const chunk of stream ) { process . stdout . write ( chunk . text ); } // Get final response const final = await response ; console . log ( ' \n Usage:' , final . usage );
modelRef() Creates a reference to a model by name.
Signature: modelRef < ConfigSchema extends z . ZodTypeAny = z . ZodTypeAny >( options : { name : string ; config ?: z . infer < ConfigSchema >; info ?: ModelInfo ; }) : ModelReference < ConfigSchema >
Parameters Model name (e.g., ‘googleai/gemini-2.0-flash-exp’)
Default configuration for this model reference
Model metadata (capabilities, version, etc.)
Returns A reference object that can be passed to generate() or other model functions
Example import { modelRef } from '@genkit-ai/ai/model' ; const flash = modelRef ({ name: 'googleai/gemini-2.0-flash-exp' , config: { temperature: 0.7 , }, }); const response = await generate ( registry , { model: flash , prompt: 'Hello!' , });
Types GenerateRequest The raw request object sent to a model.
interface GenerateRequest < CustomOptions extends z . ZodTypeAny = z . ZodTypeAny > { messages : MessageData []; config ?: z . infer < CustomOptions >; tools ?: ToolDefinition []; output ?: { format ?: string ; schema ?: JSONSchema ; }; candidates ?: number ; }
GenerateResponse The response from a model.
interface GenerateResponse < O = any > { message ?: MessageData ; text : string ; output : () => O ; messages : MessageData []; usage ?: GenerationUsage ; finishReason ?: string ; finishMessage ?: string ; request : GenerateRequest ; }
Part Content part within a message.
type Part = TextPart | MediaPart | DataPart | ToolRequestPart | ToolResponsePart | CustomPart ; interface TextPart { text : string ; } interface MediaPart { media : { url : string ; contentType ?: string ; }; } interface DataPart { data : any ; } interface ToolRequestPart { toolRequest : { name : string ; ref ?: string ; input : any ; }; } interface ToolResponsePart { toolResponse : { name : string ; ref ?: string ; output : any ; }; }
MessageData A message in the conversation.
interface MessageData { role : 'user' | 'model' | 'system' | 'tool' ; content : Part []; metadata ?: Record < string , any >; }
See Also