OSED study materials
OSED study resources

OSED key terms

OSED · Get comfortable with the vocabulary.

Stack frame

The per-call region of the stack holding a function's saved return address, saved registers, local variables, and arguments. Its exact layout depends on compiler, calling convention, and optimization level.

Related: Saved return address, Calling convention, Stack pivot
Saved return address

The value pushed by a call instruction that tells the CPU where to resume after the function returns. Overwriting it with a controlled value is the classic goal of a stack overflow exploit.

Related: Stack frame, EIP control, Offset calculation
Offset calculation

Determining the exact number of bytes between the start of an input buffer and a target field such as the saved return address. Usually found with a cyclic pattern and confirmed in a debugger.

Related: Cyclic pattern, Saved return address, Crash triage
Cyclic pattern

A non-repeating byte sequence, such as a de Bruijn sequence, whose four-byte substrings are unique. The value observed in the instruction pointer after a crash reveals the exact overwrite offset.

Related: Offset calculation, Crash triage, EIP control
EIP control

The condition where an attacker-supplied value lands in the instruction pointer, letting them redirect execution. It is the pivotal milestone that turns a crash into a potentially exploitable condition.

Related: Saved return address, Offset calculation, Stack pivot
Calling convention

The agreed rules for how arguments are passed, who cleans the stack, and which registers are preserved across a call. Common examples are cdecl, stdcall, and fastcall on x86 Windows.

Related: Stack frame, Function prologue, Stack pivot
Function prologue

The opening instructions of a compiled function that set up its stack frame, typically saving the frame pointer and subtracting space for locals. Recognizing it speeds up locating buffers in disassembly.

Related: Stack frame, Disassembler, Calling convention
Disassembler

A tool that converts machine code bytes back into assembly mnemonics for analysis. Static review of suspicious functions often starts here before dynamic confirmation in a debugger.

Related: Debugger, Decompiler, Control flow graph
Decompiler

A tool that attempts to reconstruct high-level source-like code from compiled binaries. It accelerates vulnerability hunting but can misrepresent types and inlined code, so results need verification.

Related: Disassembler, Control flow graph, Static analysis
Control flow graph

A directed graph of basic blocks connected by possible branches, used to reason about reachability of dangerous code. Attackers use it to find paths from untrusted input to an unsafe copy operation.

Related: Static analysis, Taint analysis, Disassembler
Static analysis

Examining a binary without running it, using disassembly, decompilation, strings, and import inspection. It reveals dangerous API usage and hardcoded data without risking execution of hostile code.

Related: Dynamic analysis, Decompiler, Dangerous API
Dynamic analysis

Observing a program while it runs, using debuggers, monitors, and instrumentation to watch memory and register state. It confirms exploitability that static review only suggests.

Related: Static analysis, Debugger, Crash triage
Dangerous API

A library function that copies data without bounds checking, such as strcpy, strcat, sprintf, or gets. Its presence near untrusted input is a strong signal of a buffer overflow.

Related: Static analysis, Stack buffer overflow, Input source
Input source

The point where attacker-influenced data enters a program, such as a network socket, file, command-line argument, or environment variable. Mapping it to a sink defines the vulnerability path.

Related: Sink, Taint analysis, Dangerous API
Sink

A sensitive operation that consumes data, such as a memory copy, format string call, or system command. Exploits exist where tainted input reaches a sink without adequate validation.

Related: Input source, Taint analysis, Format string
Taint analysis

Tracking whether data from an untrusted source can flow into a sensitive sink without sanitization. It can be performed manually or with tooling to prioritize which code paths to examine.

Related: Input source, Sink, Control flow graph
Crash triage

Reproducing a fault and classifying its cause from register and memory state. Distinguishing a read fault from a write fault with attacker data tells you whether exploitation is plausible.

Related: Cyclic pattern, Debugger, Exception handler
Stack buffer overflow

Writing more bytes into a stack-allocated buffer than it can hold, corrupting adjacent locals, saved registers, and the return address. It can redirect execution when the overwritten data is attacker-controlled.

Related: Saved return address, Offset calculation, Stack canary
Stack canary

A random sentinel value placed between local buffers and the saved return address, checked before returning. A corrupted canary triggers a fatal error, blocking naive overflow exploitation.

Related: Stack buffer overflow, Exception handler, SafeSEH
SafeSEH

A Windows linker mitigation that records valid exception handler addresses so the loader can reject handlers pointing outside the image. It limits overwriting a handler pointer on the stack.

Related: Exception handler, SEH overwrite, Stack canary
Exception handler

A routine registered to run when an exception occurs, chained on the stack in older Windows x86 builds. Because the chain lives on the stack, overflows can redirect it before the return address is reached.

Related: SEH overwrite, SafeSEH, Stack canary
SEH overwrite

Corrupting a stack-based exception registration record so that raising an exception transfers control to an attacker-chosen address. It bypasses return-address protections when the handler chain is writable.

Related: Exception handler, SafeSEH, Stack pivot
Stack pivot

Redirecting the stack pointer to attacker-controlled memory so that a return or call consumes a crafted chain. It is often needed when the overwrite space is too small for a full payload.

Related: ROP chain, EIP control, SEH overwrite
Shellcode

Position-independent machine code that an exploit injects and executes to perform an action, such as spawning a process or connecting back. It must avoid null bytes and self-locating assumptions when constraints demand.

Related: Encoder, Egg hunter, Null-free shellcode
Null-free shellcode

Payload bytes containing no zero bytes, required when the vulnerable input is handled by string functions that stop at a null terminator. Achieved by rewriting instructions or encoding the payload.

Related: Shellcode, Encoder, Bad character
Bad character

Any byte value that cannot appear in the payload because the vulnerable input path transforms or truncates it, such as newline, carriage return, or null. Identifying them constrains shellcode and address choices.

Related: Null-free shellcode, Encoder, Shellcode
Encoder

A routine that transforms shellcode into an allowed byte set plus a small decoder stub that reverses the transformation at runtime. It solves bad-character constraints at the cost of extra payload size.

Related: Bad character, Null-free shellcode, Egg hunter
Egg hunter

A tiny search routine that scans memory for a tagged payload and jumps to it, used when the overflow space is smaller than the shellcode. It trades reliability and search time for a compact first stage.

Related: Shellcode, Staged payload, Stack pivot
Staged payload

A small first stage that obtains and executes a larger second stage from a socket, file, or memory location. It keeps the initial overwrite compact while allowing a full-featured final payload.

Related: Egg hunter, Shellcode, Reverse shell
Reverse shell

A payload in which the compromised process connects back to an attacker-controlled listener and provides a command interface. It is often chosen because inbound connections are commonly blocked.

Related: Bind shell, Staged payload, Shellcode
Bind shell

A payload that opens a listening port on the compromised host and serves a command interface to whoever connects. It requires the target to accept inbound connections and the port to be reachable.

Related: Reverse shell, Shellcode, Staged payload
DEP

Data Execution Prevention marks memory pages non-executable, so injected stack or heap data cannot run as code. Exploits respond by reusing existing executable code instead of jumping to their own bytes.

Related: ROP chain, Return-to-libc, VirtualProtect
ASLR

Address Space Layout Randomization randomizes the base addresses of modules, stack, and heap at load time. It forces exploits to first leak a known address or find a non-randomized mapping.

Related: Information leak, Partial overwrite, ROP chain
ROP chain

A sequence of short instruction sequences ending in a return, chained through a corrupted stack to perform arbitrary operations. It defeats DEP because every gadget executes from already-executable module memory.

Related: Gadget, DEP, Stack pivot
Gadget

A short instruction sequence ending in a return or indirect jump, found inside existing executable code. Chaining gadgets lets an exploit perform computation and calls without injecting code.

Related: ROP chain, DEP, Return-to-libc
Return-to-libc

An exploitation technique that overwrites a return address with the address of a library function, arranging arguments on the stack. It predates ROP and bypasses non-executable stack protections.

Related: ROP chain, DEP, VirtualProtect
VirtualProtect

A Windows API that changes the protection flags of a memory region, commonly used to mark shellcode pages executable before jumping to them. Its address must be known or leaked when ASLR is active.

Related: DEP, Return-to-libc, ROP chain
Information leak

Any bug that discloses memory contents, such as a format string read or an uninitialized buffer, revealing module bases or stack pointers. Leaks are the standard precursor to defeating ASLR.

Related: Format string, ASLR, Partial overwrite
Partial overwrite

Overwriting only the low bytes of a saved pointer so it redirects within the same randomized region. Because high address bits stay unchanged, it can bypass ASLR without a separate leak.

Related: ASLR, Information leak, Offset calculation
Format string

A string containing conversion specifiers such as percent-s and percent-n that a printing function interprets. If user input is passed directly as the format, the attacker controls how arguments are consumed.

Related: Format string read, Format string write, Information leak
Format string read

Using conversion specifiers to make a printing function disclose stack or memory contents as formatted output. It yields a read primitive that can leak pointers and defeat ASLR.

Related: Format string, Information leak, Format string write
Format string write

Abusing the percent-n conversion to store the number of characters printed so far into an attacker-chosen address. Careful width specifiers control the written value, enabling targeted memory corruption.

Related: Format string, Format string read, Write primitive
Write primitive

A capability to store a controlled value at a chosen address, derived from a bug such as a format string write or an arbitrary free. It is typically used to overwrite a function pointer or return address.

Related: Format string write, Format string read, Gadget