The GeneratorParams class controls the behavior of text generation, including search strategies, sampling parameters, and output constraints.
Constructor
GeneratorParams(Model model)
Creates a new generator parameters instance.
The model to create parameters for
using Microsoft.ML.OnnxRuntimeGenAI;
using Model model = new Model("/path/to/model");
using GeneratorParams generatorParams = new GeneratorParams(model);
Search Option Methods
SetSearchOption(string searchOption, double value)
Sets a numeric search option (e.g., temperature, top_p, max_length).
The name of the search option to set
The numeric value for the option
generatorParams.SetSearchOption("max_length", 1024);
generatorParams.SetSearchOption("temperature", 0.7);
generatorParams.SetSearchOption("top_p", 0.9);
generatorParams.SetSearchOption("top_k", 50);
SetSearchOption(string searchOption, bool value)
Sets a boolean search option (e.g., do_sample).
The name of the search option to set
The boolean value for the option
generatorParams.SetSearchOption("do_sample", true);
GetSearchNumber(string searchOption)
Retrieves the value of a numeric search option.
The name of the search option to retrieve
double - The current value of the option
double temperature = generatorParams.GetSearchNumber("temperature");
Console.WriteLine($"Temperature: {temperature}");
GetSearchBool(string searchOption)
Retrieves the value of a boolean search option.
The name of the search option to retrieve
bool - The current value of the option
bool doSample = generatorParams.GetSearchBool("do_sample");
Console.WriteLine($"Do sample: {doSample}");
Guidance Methods
SetGuidance(string type, string data, bool enableFFTokens = false)
Sets structured output guidance (JSON schema or grammar) to constrain the model’s output.
The type of guidance: “json_schema” or “lark_grammar”
The guidance specification (JSON schema or LARK grammar string)
Whether to enable fast-forward tokens
string jsonSchema = @"
{
""type"": ""object"",
""properties"": {
""name"": { ""type"": ""string"" },
""age"": { ""type"": ""number"" }
},
""required"": [""name"", ""age""]
}
";
generatorParams.SetGuidance("json_schema", jsonSchema);
Common Search Options
Here are the most commonly used search options:
Generation Length
Minimum number of tokens to generate (including prompt)
Maximum number of tokens to generate (including prompt)
generatorParams.SetSearchOption("min_length", 10);
generatorParams.SetSearchOption("max_length", 1024);
Sampling Parameters
Enable random sampling. When false, uses greedy or beam search
Controls randomness. Higher values (e.g., 1.0) make output more random, lower values (e.g., 0.1) make it more deterministic
Nucleus sampling: only sample from tokens with cumulative probability mass of top_p
Only sample from the top K most likely tokens
generatorParams.SetSearchOption("do_sample", true);
generatorParams.SetSearchOption("temperature", 0.7);
generatorParams.SetSearchOption("top_p", 0.9);
generatorParams.SetSearchOption("top_k", 50);
Penalty Parameters
Penalty for repeating tokens. Values > 1.0 discourage repetition
generatorParams.SetSearchOption("repetition_penalty", 1.1);
Beam Search Parameters
Number of beams for beam search. 1 means greedy search
Number of sequences to return (must be ≤ num_beams)
generatorParams.SetSearchOption("num_beams", 4);
generatorParams.SetSearchOption("num_return_sequences", 1);
Batch Parameters
Batch size for processing multiple prompts
generatorParams.SetSearchOption("batch_size", 1);
Complete Examples
Basic Generation Configuration
using Microsoft.ML.OnnxRuntimeGenAI;
using Model model = new Model("/path/to/model");
using GeneratorParams generatorParams = new GeneratorParams(model);
// Set basic parameters
generatorParams.SetSearchOption("max_length", 1024);
generatorParams.SetSearchOption("temperature", 0.7);
generatorParams.SetSearchOption("top_p", 0.9);
generatorParams.SetSearchOption("do_sample", true);
// Use with generator
using Generator generator = new Generator(model, generatorParams);
Creative vs Deterministic Generation
// Creative generation (higher temperature, sampling enabled)
using GeneratorParams creativeParams = new GeneratorParams(model);
creativeParams.SetSearchOption("do_sample", true);
creativeParams.SetSearchOption("temperature", 1.0);
creativeParams.SetSearchOption("top_p", 0.95);
creativeParams.SetSearchOption("top_k", 50);
// Deterministic generation (low temperature or greedy)
using GeneratorParams deterministicParams = new GeneratorParams(model);
deterministicParams.SetSearchOption("do_sample", false); // Use greedy search
// Or use very low temperature:
// deterministicParams.SetSearchOption("do_sample", true);
// deterministicParams.SetSearchOption("temperature", 0.1);
Beam Search Configuration
using GeneratorParams beamParams = new GeneratorParams(model);
beamParams.SetSearchOption("num_beams", 4);
beamParams.SetSearchOption("num_return_sequences", 1);
beamParams.SetSearchOption("max_length", 512);
Structured Output with JSON Schema
examples/csharp/ModelChat/Program.cs:166-170
using Microsoft.ML.OnnxRuntimeGenAI;
using Model model = new Model("/path/to/model");
using GeneratorParams generatorParams = new GeneratorParams(model);
// Define JSON schema for structured output
string jsonSchema = @"
{
""type"": ""object"",
""properties"": {
""name"": {
""type"": ""string",
""description"": ""Person's full name""
},
""age"": {
""type"": ""number",
""description"": ""Person's age in years""
},
""email"": {
""type"": ""string",
""description"": ""Email address""
}
},
""required"": [""name"", ""age""]
}
";
generatorParams.SetGuidance("json_schema", jsonSchema);
// The model will now generate valid JSON matching this schema
using Generator generator = new Generator(model, generatorParams);
Complete Example from Source
examples/csharp/Common/Common.cs:134-159
using Microsoft.ML.OnnxRuntimeGenAI;
public static void SetSearchOptions(
GeneratorParams generatorParams,
GeneratorParamsArgs args,
bool verbose)
{
var options = new List<string>();
// Set do_sample if provided
if (args.do_sample.HasValue)
{
bool val = args.do_sample.Value;
options.Add($"do_sample: {val}");
generatorParams.SetSearchOption("do_sample", val);
}
// Set numeric options
if (args.min_length.HasValue)
{
double val = Convert.ToDouble(args.min_length.Value);
options.Add($"min_length: {val}");
generatorParams.SetSearchOption("min_length", val);
}
if (args.max_length.HasValue)
{
double val = Convert.ToDouble(args.max_length.Value);
options.Add($"max_length: {val}");
generatorParams.SetSearchOption("max_length", val);
}
if (args.temperature.HasValue)
{
double val = args.temperature.Value;
options.Add($"temperature: {val}");
generatorParams.SetSearchOption("temperature", val);
}
if (args.top_p.HasValue)
{
double val = args.top_p.Value;
options.Add($"top_p: {val}");
generatorParams.SetSearchOption("top_p", val);
}
if (args.top_k.HasValue)
{
double val = Convert.ToDouble(args.top_k.Value);
options.Add($"top_k: {val}");
generatorParams.SetSearchOption("top_k", val);
}
if (args.repetition_penalty.HasValue)
{
double val = args.repetition_penalty.Value;
options.Add($"repetition_penalty: {val}");
generatorParams.SetSearchOption("repetition_penalty", val);
}
if (verbose)
{
Console.WriteLine("GeneratorParams: {" + string.Join(", ", options) + "}");
}
}
See Also