Skip to main content
OpenInference is a set of semantic conventions and instrumentation libraries built on OpenTelemetry to enable comprehensive observability for AI applications. It standardizes how LLM calls, agent reasoning steps, tool invocations, retrieval operations, and other AI-specific workloads are represented as distributed traces.

Python

Instrumentation for OpenAI, LangChain, LlamaIndex, DSPy, and 30+ frameworks

JavaScript

Node.js instrumentations for OpenAI, LangChain, Anthropic, and more

Java

Auto-instrumentation for LangChain4j and Spring AI applications

Specification

Semantic conventions and attribute reference

​
Why OpenInference?

OpenTelemetry provides a universal wire format for distributed tracing, but AI applications have unique observability requirements that generic conventions don’t address:

Structured AI data

LLM calls carry multi-turn message arrays, system prompts, tool definitions, and multimodal content — a single input.value string is insufficient.

Token economics

Track prompt and completion token counts, cached tokens, and reasoning token breakdowns as first-class operational metrics.

Agentic control flow

Modern AI systems route through reasoning loops, delegate to sub-agents, and invoke tools. Each operation needs consistent identity and span-kind taxonomy.

Privacy controls

Prompts and completions frequently contain personal information. OpenInference supports per-field masking before export.
OpenInference solves these problems by defining a concrete attribute schema and span-kind taxonomy on top of OpenTelemetry spans. Every OpenInference trace is a valid OTLP trace — the conventions give attribute names their AI-specific meaning.

​
Key features

​
Multi-language support

OpenInference provides instrumentation libraries for the most popular AI development languages:
  • Python: 30+ instrumentations including OpenAI, Anthropic, LangChain, LlamaIndex, DSPy, CrewAI, Bedrock, VertexAI, and more
  • JavaScript/TypeScript: Node.js instrumentations for OpenAI, LangChain, Anthropic, Bedrock, BeeAI, and MCP
  • Java: Auto-instrumentation for LangChain4j and Spring AI

​
Comprehensive AI span kinds

OpenInference defines 10 span kinds that cover the full spectrum of AI operations:
Span kindDescription
LLMLanguage model API calls with messages, token counts, and parameters
AGENTAutonomous reasoning steps that spawn child spans for tools and retrievals
TOOLFunction or external API execution called by a language model
RETRIEVERVector store, search engine, or knowledge base queries
EMBEDDINGVector embedding generation from text or other content
CHAINDeterministic sequences like prompt formatting or orchestration
RERANKERReranking models that reorder documents by relevance
GUARDRAILInput/output moderation and safety checks
EVALUATORAutomated evaluation of model responses (LLM-as-judge)
PROMPTNamed prompt template invocations

​
Rich attribute schema

Capture everything you need to understand and reproduce AI application behavior:
  • LLM attributes: Input/output messages, model name, provider, token counts (prompt, completion, cached, reasoning), costs, invocation parameters
  • Tool attributes: Tool definitions, function calls, arguments, and results
  • Retrieval attributes: Documents with content, scores, IDs, and metadata
  • Multimodal attributes: Images, audio, and mixed-content messages
  • Context attributes: Session IDs, user IDs, metadata, tags, and prompt templates

​
Privacy and data masking

Control what sensitive data gets exported: 
from openinference.instrumentation import TraceConfig

config = TraceConfig(
    hide_inputs=True,
    hide_outputs=True,
    hide_input_messages=True,
    hide_output_messages=True,
)

​
Works with any OpenTelemetry backend

Send traces to any OTLP-compatible collector:

Arize Phoenix

Open source AI observability platform with native OpenInference support

Arize

Production AI monitoring and observability platform

OTLP Collectors

Any OpenTelemetry-compatible backend (Jaeger, Zipkin, Datadog, etc.)

​
Data model

​
Traces

A trace records the full execution path of a request — from the user’s initial input through every LLM call, tool invocation, and retrieval step to the final response. Traces are trees of spans connected by parent–child relationships.

​
Spans

A span is the atomic unit of work: one LLM call, one tool execution, one retrieval query. Every span carries:
  • Name: Human-readable operation name (e.g., ChatCompletion, web_search)
  • Timestamps: Start and end time with nanosecond precision
  • Span kind: The role of this operation (openinference.span.kind)
  • Attributes: Typed key/value pairs capturing inputs, outputs, configuration, and cost
  • Status: OK, ERROR, or UNSET

​
Attributes

Attributes are typed key/value pairs following a structured naming convention:
  • Dot-separated namespaces: llm.input_messages, llm.token_count.prompt
  • Flattened list indices: llm.input_messages.0.message.role
  • JSON strings for complex objects: llm.invocation_parameters
See the semantic conventions for the complete attribute reference.

​
Next steps

Quickstart

Get started with OpenInference in 5 minutes

Python guide

Explore Python instrumentation libraries

JavaScript guide

Explore JavaScript/TypeScript instrumentations

Specification

Semantic conventions and span attributes

JavaScript guide

Explore JavaScript/TypeScript instrumentations

Specification

Read the full semantic conventions

Build docs developers (and LLMs) love

Get started for freeTalk to us