Skip to main content

​
Overview

The cmp command verifies the correctness of your optimized solution by comparing its output against a slower but proven-correct brute-force implementation across multiple randomly generated test cases.
This is the most common stress testing mode for competitive programming, allowing you to catch edge cases that your optimized solution might handle incorrectly.

​
Basic Usage

quicktest cmp --target-file=main.cpp --correct-file=correct.cpp --gen-file=gen.cpp
Or using the shorter alias: 
qt cmp -t main.cpp -c correct.cpp -g gen.cpp

​
Required Parameters

​
target-file
string
required
Your optimized solution file to test. Can be any supported language (C++, Java, Python, Rust, Go, C, Kotlin).Short form: -t
--target-file=main.cpp
-t main.cpp
​
correct-file
string
required
The brute-force or known-correct reference implementation. This solution should always produce correct output, even if it’s slower.Short form: -c
--correct-file=correct.cpp
-c correct.cpp
​
gen-file
string
required
Random test case generator file. Generates input data for testing.Short form: -gDefault: "" (empty - can use --prefix instead)
The generator and prefix options are mutually exclusive. Use either --gen-file for random generation or --prefix to run existing test cases.
--gen-file=gen.cpp
-g gen.cpp

​
Optional Parameters

​
test-cases
number
default:"1000"
Number of test cases to generate and run.Alias: --tc
--test-cases=500
--tc 500
​
timeout
number
default:"2000"
Maximum execution time per test case in milliseconds.Alias: --tout
--timeout=1000
--tout 1000
​
memory-limit
number
default:"1000000000"
Memory limit in bytes (default is 1GB).Alias: --ml
--memory-limit=512000000
--ml 512000000
​
prefix
string
default:""
Path prefix for existing test case files. Use this to run against pre-existing test cases instead of generating random ones.Short form: -pConflicts with: --gen-file
--prefix=test_cases/testcase_ac
-p test_cases/testcase_ac
​
diff
boolean
default:"false"
Show detailed differences between expected output and actual output when tests fail.Short form: -d
--diff
-d

​
Control Flags

​
break-bad
boolean
default:"false"
Stop execution immediately when a WA (Wrong Answer), TLE (Time Limit Exceeded), or RTE (Runtime Error) occurs.Aliases: --break, -b
--break-bad
--break
-b

​
Save Options

The --save-all and --save-bad flags are mutually exclusive.
​
save-bad
boolean
default:"false"
Save only test cases that fail (WA, TLE, RTE, MLE states).
--save-bad
​
save-all
boolean
default:"false"
Save all test cases regardless of their result.
--save-all

​
Run Filters

Use these flags to run specific subsets of previously saved test cases:
​
run-all
boolean
default:"false"
Run all saved test cases.
​
run-ac
boolean
default:"false"
Run only test cases that previously passed (Accepted status).
​
run-wa
boolean
default:"false"
Run only test cases that previously failed with Wrong Answer.
​
run-tle
boolean
default:"false"
Run only test cases that previously exceeded time limit.
​
run-mle
boolean
default:"false"
Run only test cases that previously exceeded memory limit.
​
run-rte
boolean
default:"false"
Run only test cases that previously had runtime errors.

​
Examples

​
Basic Comparison Test

qt cmp -t main.cpp -c correct.cpp -g gen.cpp --tc 500 --tout 1000

​
Save and Debug Failed Cases

1

Run with save-bad flag

First, run the test and save only failing cases:
qt cmp -t main.cpp -c correct.cpp -g gen.cpp --tc 500 --tout 1000 --save-bad
2

Review failures with diff

Re-run the failing cases with diff output to see exact differences:
qt cmp -t main.cpp -c correct.cpp -g gen.cpp --run-wa --diff
3

Test the fix

After fixing your code, run only the previously failing cases:
qt cmp -t main.cpp -c correct.cpp -g gen.cpp --run-wa

​
Maximum Subarray Problem Example

Here’s a complete example testing an optimized Maximum Subarray solution: 
#include <bits/stdc++.h>
using namespace std;

int main() {
    int n; cin >> n;
    vector<int> values(n);
    for(int &a: values)
        cin >> a;
    
    int best = 0, sum = 0;
    for (int i = 0; i < n; i++) {
        sum = max(values[i], sum + values[i]);
        best = max(best, sum);
    }
    cout << best << "\n";
    return 0;
}
Run the test: 
qt cmp -t main.cpp -c correct.cpp -g gen.cpp --tc 1000 --tout 2000 --break-bad

​
Using Existing Test Cases

Run against pre-existing test files: 
qt cmp -t main.cpp -c correct.cpp -p test_cases/testcase --tc 10
When using --prefix, QuickTest looks for files named like testcase_01.txt, testcase_02.txt, etc.

​
Understanding Test Results

QuickTest displays real-time results with status indicators:
  • AC - Accepted (correct answer)
  • WA - Wrong Answer
  • TLE - Time Limit Exceeded
  • MLE - Memory Limit Exceeded
  • RTE - Runtime Error

​
Tips

Start small, then scale up: Begin with a small number of test cases (e.g., --tc 100) to verify your setup works, then increase to 1000+ for thorough testing.
Use —break-bad during development: When fixing bugs, use the --break-bad flag to stop at the first failure for faster debugging cycles.
Combine with —diff for debugging: The --diff flag shows you exactly where outputs diverge, making it easier to spot the issue.

​
See Also

  • stress - Test for time/memory limits without comparing outputs
  • check - Use custom checker for problems with multiple valid answers
  • output - Generate output files from test cases

Build docs developers (and LLMs) love

Get started for freeTalk to us