Skip to main content

Build Reliable AI Agents

Learn to build production-ready AI agents with comprehensive observability, automated evaluation, and real-world deployment strategies using LangSmith.

Get StartedCore Concepts
Agent v5 running…
→
Tracing enabled
→
Evaluation active
✓
Production ready

Quick Start

Get up and running with your first reliable agent in minutes

1

Install Dependencies

Choose your preferred language and install the required packages.
git clone --depth 1 https://github.com/langchain-ai/lca-reliable-agents.git
cd lca-reliable-agents/python
uv sync
2

Configure API Keys

Set up your environment variables with your OpenAI and LangSmith API keys.
.env
OPENAI_API_KEY='your_openai_api_key_here'
LANGSMITH_API_KEY='your_langsmith_api_key_here'
LANGSMITH_TRACING=true
LANGSMITH_PROJECT=lca-reliable-agents
Get your OpenAI API key and create a free LangSmith account to get your LangSmith API key.
3

Run Your First Agent

Start the OfficeFlow customer support agent with full tracing enabled.
uv run python officeflow-agent/agent_v5.py
The agent will start an interactive session where you can ask questions about products, inventory, and company policies.
4

View Traces in LangSmith

Open LangSmith and navigate to your project to see detailed traces of every agent interaction, including LLM calls, tool usage, and decision paths.

What You’ll Learn

Master every aspect of building production-ready AI agents

Observability

Learn why traditional debugging falls short and how to instrument agents with comprehensive tracing.

Agent Development

Build the OfficeFlow customer support agent from scratch with progressive improvements.

Evaluation Methods

Master code-based evals, LLM-as-judge, and pairwise comparison techniques.

Tracing

Capture every LLM call, tool invocation, and decision in detailed execution traces.

Dataset Creation

Build robust test datasets for repeatable evaluation and regression testing.

Production Deployment

Scale your agents to production with trace upload and online evaluation.

Course Structure

Three comprehensive modules covering the full agent development lifecycle

1

Observation

Learn observability fundamentals, LangSmith tracing, and systematic agent analysis techniques.

Explore Module 1
2

Evaluation

Master dataset creation, experiment running, and multiple evaluation methodologies for agent quality.

Explore Module 2
3

Production

Deploy agents with confidence using trace upload, online evaluation, and automated insights at scale.

Explore Module 3

Ready to Build Reliable Agents?

Start with the Python or TypeScript setup guide and build your first production-ready AI agent with comprehensive observability and evaluation.

Python SetupTypeScript Setup

Build docs developers (and LLMs) love

Get started for freeTalk to us