Skip to main content

​
Requirements

  • Python 3.10 or later
  • PyTorch (installed automatically as a transitive dependency via Transformers)

​
Install from PyPI

Install TRL using pip or uv: 
pip install trl
This installs TRL along with its core dependencies: transformers, accelerate, and datasets.

​
Install from source

To use the latest unreleased features, install directly from the GitHub repository: 
pip install git+https://github.com/huggingface/trl.git
Alternatively, clone the repository and install in editable mode: 
git clone https://github.com/huggingface/trl.git
cd trl/
pip install -e .

​
Optional dependencies

TRL provides optional dependency groups for specific use cases. Install them using the extras syntax: 
pip install "trl[<extra>]"
ExtraPackages installedUse case
peftpeft>=0.8.0LoRA and QLoRA training via PEFT
vllmvllm, fastapi, pydantic, aiohttp, uvicorn, and othersFast online generation for RL-based trainers
deepspeeddeepspeed>=0.14.4DeepSpeed ZeRO distributed training
ligerliger-kernel>=0.7.0Liger fused kernels for faster and more memory-efficient training
quantizationbitsandbytes4-bit and 8-bit quantization
judgesopenai>=1.23.2, llm-blender>=0.0.2LLM-based reward judges
vlmPillow, torchvision, num2wordsVision-language model training
math_verifymath-verify>=0.5.2Mathematical answer verification for reward functions
For example, to install TRL with PEFT and quantization support: 
pip install "trl[peft,quantization]"
For most use cases, start with pip install trl and add optional dependencies as needed. The peft and quantization extras are recommended for training on consumer hardware.

​
Developer install

If you want to contribute to TRL or run the test suite, install the development dependencies: 
git clone https://github.com/huggingface/trl.git
cd trl/
pip install -e ".[dev]"
The dev extra includes testing tools (pytest, pytest-cov, pytest-xdist), code quality tools (pre-commit, hf-doc-builder), and most optional dependency groups.
vLLM is not included in the dev extra by default due to CUDA compatibility constraints. Install it separately with pip install "trl[vllm]" if needed.
See the contributing guide for more details on setting up a development environment.

Build docs developers (and LLMs) love

Get started for freeTalk to us