Skip to main content

​
Overview

xAnalyzer automatically detects loop structures within functions during analysis, providing visual indicators that help you understand code flow and iteration patterns. This feature identifies backward jumps that create loops, marking them clearly in the disassembly.
Loops are only detected within function boundaries (between prolog and epilog/return instructions) to ensure accuracy.

​
How Loop Detection Works

The loop detection algorithm operates during the analysis phase:
1

Function Boundary Detection

When xAnalyzer encounters a function prolog (e.g., push ebp; mov ebp, esp), it records the start address:
if (prolog || (selectionAnal && (CurrentAddress == dwEntry))) {
    addressFunctionStart = CurrentAddress;
    IsPrologCall = true;
}
2

Backward Jump Analysis

For each jump instruction, xAnalyzer checks if it targets an earlier address within the same function:
void IsLoopJump(BASIC_INSTRUCTION_INFO *bii, duint CurrentAddress) {
    if (addressFunctionStart != 0 && 
        bii->addr >= addressFunctionStart && 
        bii->addr < CurrentAddress) {
        LOOPSTACK *loop = new LOOPSTACK;
        loop->dwStartAddress = bii->addr;
        loop->dwEndAddress = CurrentAddress;
        stackLoops.push(loop);
    }
}
3

Loop Stack Management

Detected loops are stored in a stack structure until the function ends:
typedef struct stLOOPSTACK {
    duint dwStartAddress;
    duint dwEndAddress;
} LOOPSTACK;
4

Function End Processing

When the function epilog is reached, all loops are applied to the disassembly:
if (epilog || (selectionAnal && (CurrentAddress == dwExit))) {
    addressFunctionStart = 0;
    SetFunctionLoops();
    ClearLoopStack(stackLoops);
}

​
Detected Loop Types

xAnalyzer identifies various loop constructs: 
; Standard while loop pattern
loc_401000:                     ; ← Loop start
    cmp ecx, 10
    jge short loc_401020        ; Exit condition
    inc ecx
    ; ... loop body ...
    jmp short loc_401000        ; ← Backward jump (loop detected)
loc_401020:                     ; Loop exit

​
Visual Indicators

Once detected, loops are marked in the x64dbg disassembly window with visual indicators:
  • Loop brackets appear in the margin showing the loop extent
  • Start address marks where the loop begins
  • End address marks the backward jump location
  • Depth levels indicate nested loop structures
Loop detection visualization

​
Nested Loop Support

xAnalyzer correctly handles nested loops: 
; Outer loop
loc_outer:                          ; ← Outer loop start
    push ecx
    xor edx, edx
    
    ; Inner loop
    loc_inner:                      ; ← Inner loop start
        inc edx
        cmp edx, 5
        jl short loc_inner          ; ← Inner loop detected
    
    pop ecx
    inc ecx
    cmp ecx, 10
    jl short loc_outer              ; ← Outer loop detected
The loop stack manages multiple levels: 
stack <LOOPSTACK*> stackLoops;      // Global loops stack

​
Function Boundary Constraints

Loop detection only works within function boundaries. If a function contains a RET instruction in the middle of its code, it will be detected as a function end and the loops stack is cleared.

​
Why Function Boundaries Matter

xAnalyzer uses function prologues and epilogues as reference points:

Prolog Detection

push ebp
mov ebp, esp
sub esp, 40h
Marks function start (addressFunctionStart)

Epilog Detection

mov esp, ebp
pop ebp
ret
Triggers loop stack processing

​
Edge Cases

Early Return Handling: 
func_start:
    push ebp
    mov ebp, esp
    test eax, eax
    jz short early_exit
    
    loc_loop:                       ; Loop here
        ; ... body ...
        jmp short loc_loop          ; ← Detected
    
    early_exit:                     ; First RET
        pop ebp
        ret                         ; ← Clears loop stack
    
    ; Any loops after this point won't be detected

​
Analysis Modes

Loop detection works across all analysis modes:
Detects loops within the selected instruction range:
xanal selection
Loops are only marked if both start and end addresses fall within the selection.

​
Implementation Details

Loop detection is implemented in xanalyzer.cpp:

​
Core Functions

IsLoopJump (Line 2567-2577) 
void IsLoopJump(BASIC_INSTRUCTION_INFO *bii, duint CurrentAddress)
{
    if (addressFunctionStart != 0 && 
        bii->addr >= addressFunctionStart && 
        bii->addr < CurrentAddress)
    {
        LOOPSTACK *loop = new LOOPSTACK;
        loop->dwStartAddress = bii->addr;
        loop->dwEndAddress = CurrentAddress;
        stackLoops.push(loop);
    }
}
SetFunctionLoops (Line 2583-2592) 
void SetFunctionLoops()
{
    while (!stackLoops.empty())
    {
        LOOPSTACK *loop = stackLoops.top();
        DbgLoopAdd(loop->dwStartAddress, loop->dwEndAddress);
        procSummary.loopsDetected++;  // Statistics
        stackLoops.pop();
    }
}

​
Data Structures

typedef struct stLOOPSTACK {
    duint dwStartAddress;   // Where the loop begins
    duint dwEndAddress;     // Where the backward jump occurs
} LOOPSTACK;

stack <LOOPSTACK*> stackLoops;  // Global stack
duint addressFunctionStart;     // Current function start reference

​
Analysis Summary

After analysis completes, loop statistics appear in the execution summary: 
[xAnalyzer]: Analysis completed in 0.234000 secs
================================================================================
xAnalyzer Execution Summary
================================================================================
- Defined API Calls detected: 156
- Undefined API Calls detected: 23
- Loops detected: 12
- Comments set: 534
- Labels set: 89
From xanalyzer.cpp (PROCSUMMARY structure): 
typedef struct stPROCSUMMARY {
    duint defCallsDetected;
    duint undefCallsDetected;
    duint loopsDetected;            // ← Loop count
    duint totalCommentsSet;
    duint totalLabelsSet;
} PROCSUMMARY;

​
Benefits for Reverse Engineering

Code Flow Understanding

Quickly identify iterative logic and repetitive operations

Algorithm Recognition

Recognize common loop patterns (search, sort, encrypt)

Malware Analysis

Spot obfuscation loops, decryption loops, and anti-analysis tricks

Performance Hotspots

Identify tight loops that may be performance-critical

​
Known Limitations

According to issue #7, incorrect loop detection may occur for sections with non-conditional jumps inside them.
Current limitations:
  • Loops are only detected within function boundaries (prolog to RET)
  • Functions with multiple RET statements may have incomplete loop detection
  • Non-conditional jumps within loops may cause false positives
  • Loop detection is cleared when function boundaries are unclear

​
Source Code References

  • Loop detection: xanalyzer.cpp:522, 2567-2577
  • Loop application: xanalyzer.cpp:541, 2583-2592
  • Data structures: xanalyzer.h:43-46
  • Function boundary handling: xanalyzer.cpp:528-543

Build docs developers (and LLMs) love

Get started for freeTalk to us