OSED study materials
OSED study resources

OSED: Custom shellcode development

OSED ยท Work through your exam, one topic at a time.

All guide sections

Custom shellcode development

Custom shellcode development

This domain tests writing your own position-independent, null-free payload rather than relying on a prebuilt blob - resolving APIs at runtime and surviving the character restrictions of the vulnerable input path.

Core rules

  • Never hardcode a module base. Walk the PEB (fs:[0x30] โ†’ PEB_LDR_DATA โ†’ InMemoryOrderModuleList) to find kernel32, then parse its export directory.
  • Hash export names with ROR13 and compare against a precomputed constant, so no API name string appears in the payload.
  • Keep the payload null-free; for string-based paths also avoid 0x0a, 0x0d, and 0x20.
  • Build strings on the stack with push of 4-byte chunks in reverse order, terminated by a push 0 or an equivalent null.
  • Respect calling conventions: Win32 APIs are stdcall and clean their own arguments, but some CRT functions are cdecl and need add esp, N afterward.

Worked example

To call WinExec, walk kernel32's export table, ROR13-hash each exported name, and compare against the hash for WinExec. On a match, take the function RVA, add the module base, and call it with a pointer to a command string pushed as immediates (push 0x00636c61, push 0x632e6c6c, ...). Reasoning: kernel32's base changes between boots and versions, so the PEB walk is the only portable resolution; the string is pushed in reverse so the bytes land contiguously ascending in memory, with the zero dword providing termination.

Traps that cost marks

  • Hardcoded kernel32 addresses that die the moment ASLR is enabled.
  • Assuming kernel32 is always the third InMemoryOrderModuleList entry - that ordering varies by OS version.
  • Forgetting stack cleanup for cdecl calls, corrupting the return path after the payload returns.
  • Testing the payload in a harness that differs from the target's character restrictions.