C API Overview

The ONNX Runtime GenAI C API provides a complete interface for integrating text generation capabilities into C applications. This API is not thread-safe and is designed for single-threaded usage.

Key Features

Model loading and configuration
Text generation with customizable parameters
Tokenization and decoding
Multi-modal processing (images and audio)
Adapter support for model customization
Engine-based request scheduling

Memory Management

The C API follows a consistent pattern for memory management:

Object Lifecycle

Creation: Objects are created through OgaCreate*() functions that return a pointer to the object
Usage: Objects are passed to other API functions
Destruction: Objects must be explicitly destroyed using corresponding OgaDestroy*() functions

Example Pattern

// Create
OgaModel* model = NULL;
OgaResult* result = OgaCreateModel("path/to/model", &model);

// Check for errors
if (result != NULL) {
    const char* error = OgaResultGetError(result);
    printf("Error: %s\n", error);
    OgaDestroyResult(result);
    return -1;
}

// Use the model
// ...

// Destroy when done
OgaDestroyModel(model);

Error Handling

All API functions that can fail return an OgaResult* pointer:

Success: Returns NULL
Failure: Returns a pointer to an OgaResult object containing error information

Error Handling Pattern

OgaResultGetError

function

const char* OGA_API_CALL OgaResultGetError(const OgaResult* result);

Returns the error message from an OgaResult. The returned string is owned by the OgaResult and will be freed when the result is destroyed.

OgaDestroyResult

function

void OGA_API_CALL OgaDestroyResult(OgaResult* result);

Destroys an OgaResult object and frees its resources.

Example

OgaResult* result = OgaCreateModel("path/to/model", &model);
if (result != NULL) {
    const char* error = OgaResultGetError(result);
    fprintf(stderr, "Failed to create model: %s\n", error);
    OgaDestroyResult(result);
    return -1;
}

Basic Usage Pattern

Here’s a complete example of using the C API for text generation:

#include "ort_genai_c.h"
#include <stdio.h>
#include <stdlib.h>

int main() {
    OgaResult* result = NULL;
    
    // 1. Create model
    OgaModel* model = NULL;
    result = OgaCreateModel("path/to/model", &model);
    if (result != NULL) {
        fprintf(stderr, "Error: %s\n", OgaResultGetError(result));
        OgaDestroyResult(result);
        return -1;
    }
    
    // 2. Create tokenizer
    OgaTokenizer* tokenizer = NULL;
    result = OgaCreateTokenizer(model, &tokenizer);
    if (result != NULL) {
        fprintf(stderr, "Error: %s\n", OgaResultGetError(result));
        OgaDestroyResult(result);
        OgaDestroyModel(model);
        return -1;
    }
    
    // 3. Tokenize input
    OgaSequences* sequences = NULL;
    result = OgaCreateSequences(&sequences);
    if (result != NULL) {
        fprintf(stderr, "Error: %s\n", OgaResultGetError(result));
        OgaDestroyResult(result);
        OgaDestroyTokenizer(tokenizer);
        OgaDestroyModel(model);
        return -1;
    }
    
    const char* prompt = "Once upon a time";
    result = OgaTokenizerEncode(tokenizer, prompt, sequences);
    if (result != NULL) {
        fprintf(stderr, "Error: %s\n", OgaResultGetError(result));
        OgaDestroyResult(result);
        OgaDestroySequences(sequences);
        OgaDestroyTokenizer(tokenizer);
        OgaDestroyModel(model);
        return -1;
    }
    
    // 4. Create generator parameters
    OgaGeneratorParams* params = NULL;
    result = OgaCreateGeneratorParams(model, &params);
    if (result != NULL) {
        fprintf(stderr, "Error: %s\n", OgaResultGetError(result));
        OgaDestroyResult(result);
        OgaDestroySequences(sequences);
        OgaDestroyTokenizer(tokenizer);
        OgaDestroyModel(model);
        return -1;
    }
    
    // Set generation parameters
    OgaGeneratorParamsSetSearchNumber(params, "max_length", 100);
    OgaGeneratorParamsSetSearchNumber(params, "temperature", 0.7);
    
    // 5. Create generator
    OgaGenerator* generator = NULL;
    result = OgaCreateGenerator(model, params, &generator);
    if (result != NULL) {
        fprintf(stderr, "Error: %s\n", OgaResultGetError(result));
        OgaDestroyResult(result);
        OgaDestroyGeneratorParams(params);
        OgaDestroySequences(sequences);
        OgaDestroyTokenizer(tokenizer);
        OgaDestroyModel(model);
        return -1;
    }
    
    // Add input tokens
    result = OgaGenerator_AppendTokenSequences(generator, sequences);
    if (result != NULL) {
        fprintf(stderr, "Error: %s\n", OgaResultGetError(result));
        OgaDestroyResult(result);
        OgaDestroyGenerator(generator);
        OgaDestroyGeneratorParams(params);
        OgaDestroySequences(sequences);
        OgaDestroyTokenizer(tokenizer);
        OgaDestroyModel(model);
        return -1;
    }
    
    // 6. Generate tokens
    while (!OgaGenerator_IsDone(generator)) {
        result = OgaGenerator_GenerateNextToken(generator);
        if (result != NULL) {
            fprintf(stderr, "Error: %s\n", OgaResultGetError(result));
            OgaDestroyResult(result);
            break;
        }
    }
    
    // 7. Get the generated sequence
    size_t seq_length = OgaGenerator_GetSequenceCount(generator, 0);
    const int32_t* seq_data = OgaGenerator_GetSequenceData(generator, 0);
    
    // 8. Decode the tokens
    const char* output = NULL;
    result = OgaTokenizerDecode(tokenizer, seq_data, seq_length, &output);
    if (result == NULL) {
        printf("Generated text: %s\n", output);
        OgaDestroyString(output);
    } else {
        fprintf(stderr, "Error: %s\n", OgaResultGetError(result));
        OgaDestroyResult(result);
    }
    
    // 9. Cleanup
    OgaDestroyGenerator(generator);
    OgaDestroyGeneratorParams(params);
    OgaDestroySequences(sequences);
    OgaDestroyTokenizer(tokenizer);
    OgaDestroyModel(model);
    OgaShutdown();
    
    return 0;
}

Library Initialization and Shutdown

OgaShutdown

function

void OGA_API_CALL OgaShutdown();

Call this on process exit to cleanly shutdown the GenAI library and its ONNX Runtime usage. This should be the last GenAI function called in your application.

Logging Configuration

OgaSetLogBool

function

OgaResult* OGA_API_CALL OgaSetLogBool(const char* name, bool value);

Set a boolean logging option. See the logging.h ‘struct LogItems’ for available options.Parameters:

name: The name of the logging option
value: The boolean value to set

OgaSetLogString

function

OgaResult* OGA_API_CALL OgaSetLogString(const char* name, const char* value);

Set a string logging option. When called with name “filename”, the library will log to that file. Pass an empty string to revert to the default destination (std::cerr).Parameters:

name: The name of the logging option
value: The string value to set

OgaSetLogCallback

function

OgaResult* OGA_API_CALL OgaSetLogCallback(void (*callback)(const char* string, size_t length));

Register a callback function to receive log messages. The callback overrides any previously set logging destination. Pass nullptr to disable callback and revert to the default destination.Parameters:

callback: Function pointer to the logging callback (or nullptr to disable)

String Management

OgaDestroyString

function

void OGA_API_CALL OgaDestroyString(const char* str);

Destroys a string returned by the API. All strings returned by GenAI functions must be freed using this function.

GPU Device Management

OgaSetCurrentGpuDeviceId

function

OgaResult* OGA_API_CALL OgaSetCurrentGpuDeviceId(int device_id);

Sets the current GPU device ID for operations.Parameters:

device_id: The GPU device ID to use

OgaGetCurrentGpuDeviceId

function

OgaResult* OGA_API_CALL OgaGetCurrentGpuDeviceId(int* device_id);

Gets the current GPU device ID.Parameters:

device_id: Pointer to store the current GPU device ID

Execution Provider Registration

OgaRegisterExecutionProviderLibrary

function

void OGA_API_CALL OgaRegisterExecutionProviderLibrary(const char* registration_name, const char* library_path);

Registers an execution provider library with ONNX Runtime.Parameters:

registration_name: Name for the registration
library_path: Path to the provider library

OgaUnregisterExecutionProviderLibrary

function

void OGA_API_CALL OgaUnregisterExecutionProviderLibrary(const char* registration_name);

Unregisters a previously registered execution provider library.Parameters:

registration_name: Name of the registration to unregister

Next Steps

Model Functions

Learn about model creation, configuration, and management

Generator Functions

Explore text generation and inference capabilities

Python API

C++ API

C# API

C API

Key Features

Memory Management

Object Lifecycle

Example Pattern

Error Handling

Error Handling Pattern

Example

Basic Usage Pattern

Library Initialization and Shutdown

Logging Configuration

String Management

GPU Device Management

Execution Provider Registration

Next Steps

Model Functions

Generator Functions

Build docs developers (and LLMs) love

Python API

C++ API

C# API

C API

​Key Features

​Memory Management

​Object Lifecycle

​Example Pattern

​Error Handling

​Error Handling Pattern

​Example

​Basic Usage Pattern

​Library Initialization and Shutdown

​Logging Configuration

​String Management

​GPU Device Management

​Execution Provider Registration

​Next Steps

Model Functions

Generator Functions

Build docs developers (and LLMs) love

Key Features

Memory Management

Object Lifecycle

Example Pattern

Error Handling

Error Handling Pattern

Example

Basic Usage Pattern

Library Initialization and Shutdown

Logging Configuration

String Management

GPU Device Management

Execution Provider Registration

Next Steps