Skip to main content

​
C/C++ API Overview

ONNX Runtime provides both C and C++ APIs for high-performance inference and training of deep learning models.

​
API Design

​
C API

The C API (onnxruntime_c_api.h) is the foundation of ONNX Runtime:
  • Version: Current API version is ORT_API_VERSION 25
  • Error Handling: Functions return OrtStatus* (nullptr indicates success)
  • Memory Management: Manual memory management with explicit Release* functions
  • Thread Safety: Most objects are not thread-safe unless explicitly documented

​
C++ API

The C++ API (onnxruntime_cxx_api.h) is a header-only wrapper around the C API:
  • Exception Safety: Converts C error codes to C++ exceptions
  • RAII: Automatic resource management through destructors
  • Type Safety: Strong typing with C++ classes
  • Move Semantics: Owning objects support move-only semantics

​
Core Concepts

​
Entry Point

const OrtApiBase* OrtGetApiBase(void);
Get the API entry point to access all ONNX Runtime functions: 
const OrtApi* api = OrtGetApiBase()->GetApi(ORT_API_VERSION);

​
Key Object Types

C TypeC++ TypeDescription
OrtEnvOrt::EnvGlobal environment, holds logging state
OrtSessionOptionsOrt::SessionOptionsConfiguration for session creation
OrtSessionOrt::SessionInference session containing a loaded model
OrtValueOrt::ValueContainer for tensors and other ONNX types
OrtMemoryInfoOrt::MemoryInfoDescriptor for memory location
OrtAllocatorOrt::AllocatorMemory allocator interface

​
Basic Workflow

​
1. Initialize Environment

OrtEnv* env;
OrtStatus* status = OrtApi->CreateEnv(ORT_LOGGING_LEVEL_WARNING, "test", &env);
if (status != NULL) {
    // Handle error
}

​
2. Create Session Options

OrtSessionOptions* session_options;
OrtApi->CreateSessionOptions(&session_options);
OrtApi->SetIntraOpNumThreads(session_options, 1);
OrtApi->SetSessionGraphOptimizationLevel(session_options, ORT_ENABLE_ALL);

​
3. Load Model and Create Session

OrtSession* session;
const ORTCHAR_T* model_path = "model.onnx";
OrtApi->CreateSession(env, model_path, session_options, &session);

​
4. Prepare Input/Output

// Get input/output names and count
size_t num_input_nodes;
OrtApi->SessionGetInputCount(session, &num_input_nodes);

OrtAllocator* allocator;
OrtApi->GetAllocatorWithDefaultOptions(&allocator);

char* input_name;
OrtApi->SessionGetInputName(session, 0, allocator, &input_name);

​
5. Run Inference

const char* input_names[] = {"input"};
const char* output_names[] = {"output"};
OrtValue* input_tensor = /* create input tensor */;
OrtValue* output_tensor = NULL;

OrtApi->Run(session, NULL, input_names, &input_tensor, 1,
            output_names, 1, &output_tensor);

​
6. Cleanup

OrtApi->ReleaseValue(output_tensor);
OrtApi->ReleaseValue(input_tensor);
OrtApi->ReleaseSession(session);
OrtApi->ReleaseSessionOptions(session_options);
OrtApi->ReleaseEnv(env);

​
Error Handling

​
C API

All C API functions that can fail return OrtStatus*:
  • NULL indicates success
  • Non-NULL indicates an error
OrtStatus* status = OrtApi->CreateSession(...);
if (status != NULL) {
    const char* msg = OrtApi->GetErrorMessage(status);
    OrtErrorCode code = OrtApi->GetErrorCode(status);
    printf("Error: %s (code: %d)\n", msg, code);
    OrtApi->ReleaseStatus(status);
}

​
C++ API

Errors are thrown as Ort::Exception: 
try {
    Ort::Session session(env, "model.onnx", session_options);
} catch (const Ort::Exception& e) {
    std::cout << "Error: " << e.what() << std::endl;
    std::cout << "Error code: " << e.GetOrtErrorCode() << std::endl;
}

​
Platform Considerations

​
Path Handling

  • Windows: Use wchar_t* for file paths (ORTCHAR_T is wchar_t)
  • Linux/macOS: Use char* for file paths (ORTCHAR_T is char)
Use the ORT_TSTR macro for portable string literals: 
const ORTCHAR_T* model_path = ORT_TSTR("model.onnx");

​
Thread Safety

  • OrtEnv: Thread-safe
  • OrtSession: Can be used by multiple threads concurrently for inference
  • OrtSessionOptions: Not thread-safe (use during session creation only)
  • OrtValue: Not thread-safe

​
Data Types

​
Tensor Element Types

typedef enum ONNXTensorElementDataType {
    ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT,      // float
    ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT8,      // uint8_t
    ONNX_TENSOR_ELEMENT_DATA_TYPE_INT8,       // int8_t
    ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT16,     // uint16_t
    ONNX_TENSOR_ELEMENT_DATA_TYPE_INT16,      // int16_t
    ONNX_TENSOR_ELEMENT_DATA_TYPE_INT32,      // int32_t
    ONNX_TENSOR_ELEMENT_DATA_TYPE_INT64,      // int64_t
    ONNX_TENSOR_ELEMENT_DATA_TYPE_STRING,     // std::string
    ONNX_TENSOR_ELEMENT_DATA_TYPE_BOOL,
    ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT16,
    ONNX_TENSOR_ELEMENT_DATA_TYPE_DOUBLE,     // double
    ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT32,     // uint32_t
    ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT64,     // uint64_t
    ONNX_TENSOR_ELEMENT_DATA_TYPE_BFLOAT16,
    // ... additional float8 and int4/int2 types
} ONNXTensorElementDataType;

​
Next Steps

OrtApi Structure

Complete reference for all C API functions

Session Management

Loading models and running inference

Tensor Operations

Creating and manipulating tensors

Execution Providers

GPU acceleration and specialized hardware

​
See Also