Skip to main content

​
Contributing to TensorRT-LLM

Thank you for your interest in contributing to TensorRT-LLM! This guide outlines the process for contributing code, documentation, and other improvements to the project.

​
Issue Tracking

All enhancement, bugfix, or change requests must begin with the creation of a TensorRT-LLM Issue Request.
  • The issue request must be reviewed by TensorRT-LLM engineers and approved prior to code review.
  • This helps ensure your contribution aligns with the project’s goals and roadmap.

​
Developer Workflow

Follow these steps to contribute code:

​
1. Fork the Repository

Developers must first fork the upstream TensorRT-LLM OSS repository.

​
2. Clone and Push Changes

Clone the forked repository and push changes to your personal fork: 
git clone https://github.com/YOUR_USERNAME/YOUR_FORK.git TensorRT-LLM
# Checkout the targeted branch and commit changes
# Push the commits to a branch on the fork (remote).
git push -u origin <local-branch>:<remote-branch>

​
3. Create a Pull Request

Once your code changes are staged on the fork and ready for review, create a Pull Request to merge the changes from a branch of your fork into a selected branch of upstream.
  • PRs should typically target the main branch.
  • Creation of a PR kicks off the code review process.
  • At least one TensorRT-LLM engineer will be assigned for the review.
  • When the PR is under review, the label Pending Review will be added.
  • If changes are requested, the reviewer will add the label Changes Requested.
  • Once changes are approved, CI will be launched to validate the change.
  • When CI passes, the reviewer will merge the PR.
  • If CI reports any failures, it’s up to the requester to fix any CI failures before requesting another review.

​
Pull Request Guidelines

​
PR Title Format

The naming of pull requests follows the Conventional Commits specification: Good PR Title Examples:
  • feat: Add support for starcoder-v2 FP8 base + FP16/BF16 LoRA
  • BREAKING CHANGE: Set default max batch size to 2048
  • chore: Remove version from plugins .so
  • None: Stringized enums for better error msgs
  • fix https://github.com/NVIDIA/TensorRT-LLM/issues/700: a Memory leak issue in C++ runtime
  • [TRTLLM-5516] perf: replicate dummy request for cuda graph padding (NVIDIAN only)
  • [nvbug/5334370] fix: Fix one model EAGLE3 (NVIDIAN only)
For NVIDIA developers: please include the JIRA number or NVBUG ID in the PR title whenever possible. Also, ensure your GitHub account displays your full name or NVIDIA account name in the Name field of your profile.
If the PR includes an API change that might break user code/API usage, consider adding “BREAKING CHANGE” in the title so that reviewers know what to expect. Additionally, if the PR is not related to any bug and task, consider using “chore” or None as the placeholder.

​
PR Description

In the PR description, please address these points:
  • Background or motivation: Why is the change necessary?
  • Summary: Summarize the changes in one paragraph, if possible.
  • Large PRs: If the PR is large, explain why it cannot be broken down into multiple PRs.
  • Impacts: Potential performance or functional impacts of the changes. If there are risks, please inform the reviewers.
  • Related PRs: Link to related PRs.
For NVIDIA developers: please submit feature or bug fixes to the dedicated branch specified in the nvbug Keywords field. For example, if a bug is reported on the release/v0.20 branch, please submit the fix to release/v0.20 instead of the main branch.Please add the “release blocker” label to any PRs that could potentially cause a release delay.

​
Keep PRs Concise

Please try to keep pull requests as concise as possible:
  • Avoid committing commented-out code.
  • Wherever possible, each PR should address a single concern.
  • If there are several otherwise-unrelated things that should be fixed to reach a desired endpoint, our recommendation is to open several PRs and indicate the dependencies in the description.
  • The more complex the changes are in a single PR, the more time it will take to review those changes.

​
Coding Guidelines

Before submitting code, make sure you follow the TensorRT-LLM Coding Guidelines.

​
Pre-commit Hooks

We use pre-commit for automatic code formatting and validation. Install the pre-commit package in your local Python environment: 
pip install pre-commit
pre-commit install
pre-commit will be triggered on every commit: 
git commit -m "fix"

isort....................................................................Passed
CRLF end-lines remover...................................................Passed
yapf.....................................................................Failed
- hook id: yapf
- files were modified by this hook
check for added large files..............................................Passed
check for merge conflicts................................................Passed
check for broken symlinks............................(no files to check)Skipped
detect private key.......................................................Passed
fix end of files.........................................................Passed
check yaml...............................................................Passed
trim trailing whitespace.................................................Passed
check toml...............................................................Passed
mixed line ending........................................................Passed
debug statements (python)................................................Passed
check json...........................................(no files to check)Skipped
autoflake................................................................Passed
clang-format.............................................................Passed
cmake-format.............................................................Passed
codespell................................................................Passed
ruff.....................................................................Passed
ruff-format..............................................................Passed
mdformat.................................................................Passed
If any files were modified by the pre-commit hooks, you will need to stage and commit them again.

​
Tests and Code Review for Protected APIs

Some APIs are committed to be stable; any breaking changes to these APIs require careful design and review. This repo contains an API stability testsuite to protect committed APIs (currently including the core components of LLM API). If your PR brings breaking changes to the protected APIs, the API stability tests will fail, reporting errors like: 
def test_signature(self):
        snapshot = ClassSnapshot.from_inspect(self.TEST_CLASS)
        try:
            snapshot.assert_equal(self.reference)
        except AssertionError as e:
>           raise AssertionError(self.error_msg) from e
E           AssertionError: API stability validation failed. This is probably because you changed LLM's APIs, please ask for reviews from the code owners.

tests/api_stability/test_api_stability.py:241: AssertionError
As the error message suggests, please ask for reviews from the code owners of the corresponding APIs.

​
Signing Your Work (DCO)

We require that all contributors “sign-off” on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
Any contribution which contains commits that are not Signed-Off will not be accepted.
Signing off your commit means you accept the terms of the Developer Certificate of Origin (DCO).

​
How to Sign-Off

To sign off on a commit, simply use the --signoff (or -s) option when committing your changes: 
git commit -s -m "Add cool feature."
This will append the following to your commit message: 
Signed-off-by: Your Name <[email protected]>

​
Developer Certificate of Origin (DCO)

Full text of the DCO: 
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this 
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I have 
    the right to submit it under the open source license indicated in 
    the file; or

(b) The contribution is based upon previous work that, to the best of 
    my knowledge, is covered under an appropriate open source license 
    and I have the right under that license to submit that work with 
    modifications, whether created in whole or in part by me, under 
    the same open source license (unless I am permitted to submit 
    under a different license), as indicated in the file; or

(c) The contribution was provided directly to me by some other person 
    who certified (a), (b) or (c) and I have not modified it.

(d) I understand and agree that this project and the contribution are 
    public and that a record of the contribution (including all 
    personal information I submit with it, including my sign-off) is 
    maintained indefinitely and may be redistributed consistent with 
    this project or the open source license(s) involved.

​
Next Steps

Once you’re familiar with the contribution process:
  1. Review the Coding Guidelines to ensure your code follows project standards
  2. If you need to build from source, see Building from Source
  3. Set up your development environment and start contributing!
Welcome to the TensorRT-LLM community!

Build docs developers (and LLMs) love

Get started for freeTalk to us