OSED study materials
OSED study resources

OSED: Reverse engineering binaries to locate vulnerabilities

OSED · Work through your exam, one topic at a time.

All guide sections

Reverse engineering binaries to locate vulnerabilities

Reverse engineering binaries to locate vulnerabilities

This domain tests whether you can take an unfamiliar 32-bit Windows binary, map its attack surface, and pin down the exact code path where attacker-controlled data reaches a memory-unsafe operation - then prove it with a controlled crash.

Core rules

  • Triage imports before reading disassembly: strcpy, strcat, sprintf, gets, lstrcpyA/W, wsprintfA, and memcpy/RtlCopyMemory with a length taken from input are all candidates.
  • Follow data forward from the source (socket recv, ReadFile, registry value, GetCommandLineA) and follow the length backward to whatever constant, header field, or caller argument controls it.
  • Read the frame layout literally: sub esp, 0x100 plus lea eax, [ebp-0x100] tells you both the buffer size and the direction of the copy.
  • Confirm reachability from an unprivileged context before investing in an exploit; an unreachable bug is worth nothing on the clock.

Worked example

A service routine calls recv(sock, &buf, 0x400, 0). Its prologue is push ebp; mov ebp, esp; sub esp, 0x104, and buf is referenced as [ebp-0x100]. Reasoning: only 0x100 bytes are reserved, but the length argument passed to recv is the constant 0x400, so a peer that sends 0x200 bytes writes past the saved EBP, over the return address, and roughly 0x1FC bytes beyond it. That is a confirmed stack overflow with a known boundary; no further input gymnastics required.

Traps that cost marks

  • Treating a function as safe because it takes a size argument - always trace where that size came from.
  • Confusing a heap copy with a stack copy; the exploitation strategy and offsets differ completely.
  • Missing a second copy path (logging, wide-char conversion) that truncates the payload at a null before it reaches the return address.
  • Spending the budget on the crashing path when a different, quieter path actually reaches the vulnerable copy.