Description
Compare the metrics and parameters between two experiments or revisions. This command provides a focused comparison showing what changed between two specific runs, making it easy to understand the impact of parameter changes on model performance.
Use dvc exp diff after running an experiment to quickly see how it compares to your baseline.
Usage
dvc exp diff [a_rev] [b_rev] [options]
Arguments
Old experiment or Git revision to compare. Defaults to HEAD if not specified.Can be:
- Experiment name (e.g.,
exp-a1b2c)
- Git commit SHA
- Git branch name
- Git tag
dvc exp diff main
dvc exp diff exp-baseline
b_rev
string
default:"workspace"
New experiment or Git revision to compare. Defaults to the current workspace.dvc exp diff main feature-branch
dvc exp diff exp-a1b2c exp-d3e4f
Options
Show unchanged metrics and parameters as well. By default, only changed values are displayed. Show only parameters that are stage dependencies (defined in dvc.yaml).dvc exp diff --param-deps
Show output in JSON format for programmatic processing.dvc exp diff --json > diff.json
Show output in Markdown table format (GitHub Flavored Markdown). Don’t show the metric/parameter file path in output. Round metrics to N digits after the decimal point.dvc exp diff --precision 3
Examples
Compare workspace with HEAD
Output:
Metric
──────────────────────────────────────────────────
Path Metric HEAD workspace diff
──────────────────────────────────────────────────
metrics.json accuracy 0.89 0.92 +0.03
metrics.json loss 0.287 0.234 -0.053
Param
──────────────────────────────────────────────────
Path Param HEAD workspace
──────────────────────────────────────────────────
params.yaml train.lr 0.01 0.001
params.yaml train.epochs 50 80
Positive diff values indicate improvement in metrics (when higher is better).
Compare two specific experiments
dvc exp diff exp-baseline exp-new-model
Metric
────────────────────────────────────────────────────────────────
Path Metric exp-baseline exp-new-model diff
────────────────────────────────────────────────────────────────
metrics.json accuracy 0.85 0.92 +0.07
metrics.json f1_score 0.83 0.90 +0.07
metrics.json loss 0.312 0.234 -0.078
Param
────────────────────────────────────────────────────────────────
Path Param exp-baseline exp-new-model
────────────────────────────────────────────────────────────────
params.yaml model.architecture resnet18 resnet50
params.yaml train.lr 0.1 0.001
params.yaml train.batch_size 32 64
Experiment names support tab completion for easier command-line usage.
Show all metrics including unchanged
Output:
Metric
──────────────────────────────────────────────────
Path Metric HEAD workspace diff
──────────────────────────────────────────────────
metrics.json accuracy 0.89 0.92 +0.03
metrics.json loss 0.287 0.234 -0.053
metrics.json val_loss 0.312 0.312 0
Param
──────────────────────────────────────────────────
Path Param HEAD workspace
──────────────────────────────────────────────────
params.yaml train.lr 0.01 0.001
params.yaml train.epochs 50 80
params.yaml model.dropout 0.5 0.5
params.yaml random_seed 42 42
Unchanged values have a diff of 0 or show the same value in both columns.
Hide file paths
Output:
Metric
────────────────────────────────────────
Metric HEAD workspace diff
────────────────────────────────────────
accuracy 0.89 0.92 +0.03
loss 0.287 0.234 -0.053
Param
────────────────────────────────────────
Param HEAD workspace
────────────────────────────────────────
train.lr 0.01 0.001
train.epochs 50 80
Export to JSON
Output:
{
"metrics": {
"metrics.json": {
"accuracy": {
"old": 0.89,
"new": 0.92,
"diff": 0.03
},
"loss": {
"old": 0.287,
"new": 0.234,
"diff": -0.053
}
}
},
"params": {
"params.yaml": {
"train.lr": {
"old": 0.01,
"new": 0.001
},
"train.epochs": {
"old": 50,
"new": 80
}
}
}
}
JSON format is perfect for integrating with CI/CD pipelines or custom analysis scripts.
## Metric
| Path | Metric | HEAD | workspace | diff |
|--------------|----------|-------|-----------|--------|
| metrics.json | accuracy | 0.89 | 0.92 | +0.03 |
| metrics.json | loss | 0.287 | 0.234 | -0.053 |
## Param
| Path | Param | HEAD | workspace |
|-------------|--------------|------|----------|
| params.yaml | train.lr | 0.01 | 0.001 |
| params.yaml | train.epochs | 50 | 80 |
Compare with custom precision
dvc exp diff --precision 2
Metric
──────────────────────────────────────────────────
Path Metric HEAD workspace diff
──────────────────────────────────────────────────
metrics.json accuracy 0.89 0.92 +0.03
metrics.json loss 0.29 0.23 -0.05
Compare branches
dvc exp diff main feature-branch
Both revisions must exist in your Git history or be valid experiment names.
Understanding the Output
Metrics Table
The metrics table shows:
- Path: File containing the metric (e.g.,
metrics.json)
- Metric: Metric name within the file
- Old value: Metric value in first revision
- New value: Metric value in second revision
- Diff: Numerical difference (new - old)
Parameters Table
The parameters table shows:
- Path: File containing the parameter (e.g.,
params.yaml)
- Param: Parameter name within the file
- Old value: Parameter value in first revision
- New value: Parameter value in second revision
Parameters don’t show a diff because they’re often non-numeric values.
Common Workflows
Quick experiment validation
# Run experiment with new parameters
dvc exp run -S train.lr=0.001
# Compare with baseline
dvc exp diff
A/B testing
# Compare two competing approaches
dvc exp diff exp-approach-a exp-approach-b --md > comparison.md
CI/CD integration
# Export diff as JSON for automated checks
dvc exp diff --json > diff.json
# Check if accuracy improved
accuracy_diff=$(jq '.metrics."metrics.json".accuracy.diff' diff.json)
if (( $(echo "$accuracy_diff > 0" | bc -l) )); then
echo "Accuracy improved!"
fi
Documentation generation
# Generate markdown report of improvements
dvc exp diff v1.0 v2.0 --md --all > CHANGELOG_metrics.md
dvc exp show - View all experiments in a tabledvc exp run - Create new experiments to comparedvc metrics diff - Compare metrics without experiment contextdvc params diff - Compare parameters without experiment context
