Skip to main content
This guide walks you through running your first stress test using Quick Test CLI. You’ll learn the basic workflow and test modes.
Make sure you have installed Quick Test CLI before proceeding.

​
Your first stress test

Let’s run a stress test to validate a solution by comparing it against a brute-force approach. This is the most common use case in competitive programming.
1

Create your solution file

Create a file named main.cpp with your optimized solution. For this example, we’ll solve a simple problem: find the sum of two numbers.
main.cpp
#include <iostream>
using namespace std;

int main() {
    int a, b;
    cin >> a >> b;
    cout << a + b << endl;
    return 0;
}
2

Create a correct solution

Create a file named correct.cpp with a brute-force solution that you’re confident is correct. This can be slower but must always produce the right answer.
correct.cpp
#include <iostream>
using namespace std;

int main() {
    int a, b;
    cin >> a >> b;
    // Same simple solution for this example
    cout << a + b << endl;
    return 0;
}
For more complex problems, your brute-force solution might use nested loops or simpler algorithms, while your main solution uses optimizations.
3

Create a test generator

Create a file named gen.cpp that generates random test cases:
gen.cpp
#include <iostream>
#include <cstdlib>
#include <ctime>
using namespace std;

int main() {
    srand(time(0));
    int a = rand() % 1000;
    int b = rand() % 1000;
    cout << a << " " << b << endl;
    return 0;
}
The generator should output test cases to stdout. Quick Test CLI will feed this output to both your solution and the correct solution.
4

Run the stress test

Now run Quick Test CLI to compare your solutions:
quicktest cmp --target-file=main.cpp --correct-file=correct.cpp --gen-file=gen.cpp
Or use the shorter alias and options:
qt cmp -t main.cpp -c correct.cpp -g gen.cpp --tc 100 --tout 1000
This command will:
  • Compile all three files
  • Run 100 test cases (--tc 100)
  • Set a timeout of 1000ms per test (--tout 1000)
  • Compare outputs from main.cpp and correct.cpp
5

Review the results

Quick Test CLI will display results for each test case:
  • ✅ AC (Accepted): Output matches
  • ❌ WA (Wrong Answer): Output differs
  • ⏱️ TLE (Time Limit Exceeded): Execution timeout
  • 💥 RTE (Runtime Error): Program crashed
If a test case fails, the tool will show you the input and both outputs so you can debug.

​
Understanding test modes

Quick Test CLI supports four different testing modes. Here’s when to use each:

cmp

Compare two solutionsUse when you have a correct but slow solution and want to verify your optimized version.
qt cmp -t main.cpp -c correct.cpp -g gen.cpp

stress

Stress test performanceUse when you only want to verify your solution runs within time limits (no comparison).
qt stress -t main.cpp -g gen.cpp --tout 1000

check

Custom validationUse for problems with multiple valid answers. Requires a checker program that validates output.
qt check -t main.cpp -c checker.cpp -g gen.cpp

output

Batch processingRun your solution on existing test files and save outputs.
qt output -t main.cpp -p testcase_prefix

​
Common options

All test modes support these commonly used options:
OptionShortDefaultDescription
--test-cases--tc1000Number of test cases to generate
--timeout--tout2000Time limit per test in milliseconds
--memory-limit--ml1GBMemory limit in bytes
--break-bad--breakfalseStop on first WA/TLE/RTE
--save-bad-falseSave failing test cases to disk
--save-all-falseSave all test cases to disk

​
Examples with options

qt cmp -t main.cpp -c correct.cpp -g gen.cpp --tc 500 --tout 1000

​
Multi-language support

Quick Test CLI supports multiple programming languages. Simply use the appropriate file extension: 
qt cmp -t main.cpp -c correct.cpp -g gen.cpp
Compiles with: g++ -std=c++17 -Wall -DONLINE_JUDGE=1 -o .qt/main main.cpp
See the Supported Languages page for compilation details and language-specific configuration.

​
Working with saved test cases

When you find a bug, you’ll want to save failing test cases and re-run them after fixing your code.
1

Save failing test cases

Run your stress test with the --save-bad flag:
qt cmp -t main.cpp -c correct.cpp -g gen.cpp --save-bad
Failing test cases will be saved in the .qt/ directory.
2

Fix your solution

Review the saved test cases and fix your code.
3

Re-run failing tests

Use the --run-wa flag to only re-run previously failing tests:
qt cmp -t main.cpp -c correct.cpp -g gen.cpp --run-wa
You can also use:
  • --run-tle: Re-run timeout cases
  • --run-rte: Re-run runtime error cases
  • --run-ac: Re-run accepted cases
  • --run-all: Re-run all saved cases

​
Example workflow

Here’s a typical workflow for competitive programming:
1

Write a brute-force solution

Start with a simple, obviously correct solution in correct.cpp.
2

Write your optimized solution

Implement your faster algorithm in main.cpp.
3

Create a test generator

Write gen.cpp to generate random test cases within problem constraints.
4

Run stress tests

qt cmp -t main.cpp -c correct.cpp -g gen.cpp --tc 1000 --save-bad --break
5

Debug if needed

If a test fails:
  1. Check the failing test case in .qt/ directory
  2. Fix your solution
  3. Re-run: qt cmp -t main.cpp -c correct.cpp -g gen.cpp --run-wa
6

Submit with confidence

Once all 1000 tests pass, submit your solution!

​
Next steps

Now that you’ve run your first stress test, explore more features:

Compare mode

Deep dive into comparison testing with all available options

Stress mode

Learn about performance stress testing without comparisons

Check mode

Use custom checkers for problems with multiple valid answers

Supported languages

See compilation commands and configuration for all languages

​
Getting help

Need assistance?
  • View examples: qt example --cmp
  • Check all options: qt cmp --help
  • Report bugs: GitHub Issues
  • Read full documentation: Browse the sidebar for detailed guides

Build docs developers (and LLMs) love

Get started for freeTalk to us