OSCE3 study materials
OSCE3 study resources

OSCE3: Binary Exploitation and Reverse Engineering

OSCE3 · Work through your exam, one topic at a time.

All guide sections

Binary Exploitation and Reverse Engineering

This domain tests Windows userland exploit development and reverse engineering: finding a memory-corruption bug, understanding mitigations, and producing a reliable exploit. You need to read disassembly, identify control over memory, and handle ASLR, DEP, stack cookies, SafeSEH, and character restrictions.

Core rules and concepts

  • Reverse engineer before exploiting. Identify input paths, buffer sizes, copy routines, and error handlers. Confirm the crash is attacker-controlled, not a null dereference.
  • Measure the overwrite offset precisely with a cyclic pattern, then verify control of EIP and the surrounding stack.
  • Check mitigations per module: ASLR, DEP/NX, stack cookies, SafeSEH, CFG. A non-ASLR module with a useful gadget can change the exploit strategy.
  • For DEP bypass, build a ROP chain or use an API-resolution stub. For SEH overwrites, account for exception registration and handler validation.
  • Bad characters come from the parser and the transport. Encode shellcode and re-test after every change.
  • Reliability includes repeatability across restarts, not a single successful run.

Worked example

A 32-bit Windows service copies a user-supplied field with strcpy into a 256-byte stack buffer. A cyclic pattern shows EIP control at offset 268. DEP is enabled and ASLR is enabled for the main module, but one loaded DLL is non-ASLR and contains a jmp esp instruction. The reasoning is: overwrite the saved return address with the jmp esp address, place a ROP chain or encoded shellcode after it, and avoid bad characters such as null, newline, and carriage return. After the first crash-controlled run, the exploit is adjusted for the exact offset and retested from a clean service restart.

Traps that cost marks

  • Reporting a crash as exploitation without proving EIP or pointer control.
  • Ignoring module rebasing and hardcoding an address that changes after restart.
  • Forgetting that shellcode placement may be before or after the overwrite depending on stack layout.
  • Missing a bad character that truncates the payload only in the real transport.