Exploit development and reverse engineering: concepts

Every memory-safety CVE you have ever read — the buffer overflow in the CVE record, the use-after-free in the Project Zero write-up — is the same story at the bottom: a program written in an unsafe language mishandled memory, and someone turned that mishandling into control of the machine. Exploit development is the discipline of that turning. Reverse engineering is its twin: reading a program you didn't write (or can't read — no source) and recovering what it does. You meet both as a defender constantly: the mitigation survey in this lesson is how you answer "is this system hardened?"; the assembly-reading skill is how you confirm what a suspicious binary actually does; and the whole field of vulnerability research is exploit developers who chose to report instead of weaponize.

Level 3 · Apply — do it unaided on an unfamiliar authorized target, and explain why it worked and what would have stopped it

Walkthrough run 2026-10-09 · mitigation mechanics demonstrated live (ASLR, checksec survey); exploitation technique is reader-side on pwn.college

1. Where this shows up

Every memory-safety CVE you have ever read — the buffer overflow in the CVE record, the use-after-free in the Project Zero write-up — is the same story at the bottom: a program written in an unsafe language mishandled memory, and someone turned that mishandling into control of the machine. Exploit development is the discipline of that turning. Reverse engineering is its twin: reading a program you didn't write (or can't read — no source) and recovering what it does. You meet both as a defender constantly: the mitigation survey in this lesson is how you answer "is this system hardened?"; the assembly-reading skill is how you confirm what a suspicious binary actually does; and the whole field of vulnerability research is exploit developers who chose to report instead of weaponize.

This lesson teaches the concepts — how a flaw becomes control of execution, and the mitigation landscape that makes it hard — and points your hands at the one place built for them: pwn.college, the ASU-run training environment with belt progression (their ranked challenge tracks — each belt is a series of challenges in one topic area, in increasing difficulty). What it does not do, ever: working exploits for real products, weaponizable technique, or anything aimed at a system you don't own.

2. Before you start

• The hard safety line, stated once and held everywhere: every hands-on exploitation step in this track happens on training targets only — pwn.college, exploit.education, ROP Emporium, crackmes. No working exploits for real products appear in this guidebook, in any lesson, at any stage. If a step ever feels like it could be pointed at a real system, you have misunderstood the step — re-read b-authorization-scope.
• pwn.college's own rule: they ask that challenge solutions not be published. Link pwn.college itself; never solution material, never flags, never write-ups of their challenges. Your public artifact from that platform is belt progress and mechanics write-ups in your own words — never the flag.
• You can read unfamiliar code systematically — find where untrusted input enters, follow it to where it acts (a-reading-code); you can script (a-python-security).
• You know what written authorization covers before any hands-on step (b-authorization-scope) — this track's safety line is that concept, applied.
• You know binary structure basics — sections, imports, entropy (d-malware-analysis).
• No prior C required: the C you need for the stack section is taught in §3, from zero.
• Tooling: gcc, readelf, a shell. The walkthrough uses nothing else.

3. How it works

The C you need (no prior C assumed)

Three ideas, nothing more. A pointer is a variable whose value is a memory address — it says where something is, not what it is. A buffer is a fixed-size run of bytes set aside for data, like a row of 64 pigeonholes. And C's classic string-copy functions (strcpy and its cousins) copy bytes from one place to another until they see a terminator, with no check that the destination is big enough — the language trusts the programmer to have measured. When the source is longer than the buffer, the copy keeps writing past the buffer's end into whatever sits next to it in memory. That overrun is the flaw the next section turns into control. You don't need to write C; you need to see why "copy without measuring" is the entire vulnerability class.

From flaw to control: the one idea

A running program is memory with a story: code regions, data regions, and the stack — the region that tracks function calls. When function A calls function B, the machine pushes a return address (where to resume in A) onto the stack, then B's local variables sit above it. A stack buffer overflow is what happens when B copies more input into its local buffer than fits: the extra bytes overwrite whatever sits above the buffer — and above the buffer sits the return address. Overwrite the return address with an address you choose, and when B returns, the processor jumps where you said, not where the program said. That is the entire idea: a memory-safety flaw becomes control of the instruction pointer, and control of the instruction pointer becomes arbitrary code execution. Every stack-smashing exploit ever written is this sentence with details.

The details are where the decades went — shellcode (machine code that fits in the overwritten region), return-to-libc (jumping to existing library code when you can't inject your own), ROP (chaining tiny existing code fragments, "gadgets," into a program of your choosing). You will learn those on pwn.college, against binaries built to be broken. The concept to carry from here: exploitation is programming with someone else's bug — the bug gives you a primitive (write X bytes past the buffer, read past the end), and the exploit is the program you write on top of that primitive.

The mitigation landscape: what each one stops

Modern systems don't rely on programmers never making mistakes. Four mitigations (plus two compile-time helpers) each break one link in the chain above — and the walkthrough surveys them on real binaries:

• NX / DEP (non-executable data): marks data regions (stack, heap) as non-executable at the hardware level. Breaks the classic move: even if you inject shellcode onto the stack, the processor refuses to run it. Doesn't stop: ROP — the attacker reuses code that is already executable.
• ASLR / PIE (address randomization): randomizes where the stack, heap, libraries, and (with PIE, position-independent executables) the program itself load — differently on every run, as the walkthrough shows. Breaks the attacker's need to know addresses: you can't jump to your shellcode if you don't know where it is. Doesn't stop: information leaks — one leaked address defeats randomization for that region, which is why info-leak bugs are prized.
• Stack canaries: a secret value placed between the local buffer and the return address, checked before the function returns. Overflow the buffer and you overwrite the canary first; the check fails and the program aborts instead of jumping to your address. Doesn't stop: overwrites that don't cross the canary, or canary leaks.
• RELRO (read-only relocations): makes the GOT (Global Offset Table — where the program looks up library function addresses) read-only after loading. Breaks GOT-overwrite techniques. Doesn't stop: anything that doesn't target the GOT.
• FORTIFY_SOURCE (compile-time): replaces risky library calls with checked variants that abort on detectable overflows. CFI (control-flow integrity): restricts indirect jumps to valid targets — the modern answer to ROP, still deploying.

The defender's read: these are layers, and each layer converts an exploit primitive into a harder problem — the attacker now needs an info leak plus a write primitive plus a ROP chain. checksec (and the readelf survey in the walkthrough) reads which layers a binary has. A binary with everything on is not "unexploitable" — it is expensive to exploit, and expense is what defense buys.

What the defender sees when a mitigation fires. Mitigations don't just block — they announce. A tripped canary aborts the process and the log shows *** stack smashing detected *** (verified locally on this machine 2026-10-10 — transcript runs/dea-canary-abort.log; the process dies with exit code 134, SIGABRT). An NX violation dies as a segmentation fault with the fault address sitting in a non-executable data region. Either one in production logs is an active-exploitation signal — or a real bug — and both deserve the timeline treatment from d-dfir-deep, not a shrug. The crash that "just happens sometimes" is how missed mitigations-fire events read when nobody is looking.

Reverse engineering: reading without source

Disassembly translates machine code back to assembly (what the processor does); decompilation guesses the C that produced it (what the programmer meant). The professional workflow is static-first: load the binary in Ghidra (free, NSA-built), navigate functions and cross-references (every place a function or variable is used — the map of "who calls this"), rename the meaningless FUN_004012a0 labels as understanding grows, retype data structures — and form hypotheses. Static comes first because it gives the whole map cheaply — every function, every path — while dynamic execution only shows the paths that ran; you form the hypothesis from the complete picture, then spend expensive debugger time confirming rather than exploring. Then dynamic confirmation: run it in a debugger (gdb with GEF/pwndbg — gdb plugins that make registers, stack, and disassembly readable — on Linux, x64dbg on Windows), set breakpoints at the hypothesis points, and watch registers and memory prove or kill each theory. The crackme tradition (crackmes.one, beginner-rated) is the training ground: small programs that ask you to recover an algorithm or a key, where the only rule is don't run what you don't understand on a machine you care about.

4. Walkthrough

Run by us on this machine — transcript: runs/d-exploit-concepts.log. Clean environment: gcc, readelf, coreutils, 2026-10-09. Nothing exploited, nothing injected — this walkthrough demonstrates the mitigations, which is the defender's half and the safe half.

Steps: (1) compile a tiny C program that prints the addresses of a stack variable and of main; (2) run it three times and record both addresses — observe them change on every run (ASLR randomizing stack and PIE randomizing code); (3) re-run twice under setarch -R (ASLR disabled) and observe the addresses freeze — the contrast is the lesson: randomization is the defense, and disabling it is what makes training exploits deterministic; (4) run the readelf-based survey script against eight system binaries and record NX/PIE/canary/RELRO for each; (5) compile the same program deliberately unhardened (-z execstack -no-pie -fno-stack-protector -z norelro) and survey it — all four off, the visible shape of a 1990s binary; (6) write the one-paragraph read: what each mitigation would block in the flaw-to-control chain from section 3. Actual exploitation technique is reader-side on pwn.college (their memory-error modules), per the safety line.

5. On your own

Survey ten more binaries on your own system — include at least one you compiled yourself with default flags and one with hardening explicitly disabled, as the walkthrough did. For each, write which mitigations are present and which link in the flaw-to-control chain each one breaks. Success test: pick your least-hardened and most-hardened binaries and write the attacker's shopping list for each — the primitives needed, in order, to turn a hypothetical stack overflow into code execution despite the mitigations present. The most-hardened list must be strictly longer than the least-hardened one, and every item must name the mitigation that forces it. If you can't say which mitigation forces an item, the list isn't done. Then begin pwn.college's program-misuse and memory-error modules — belt progress is the artifact; mechanics write-ups in your own words (never flags, never solutions).

6. Where it goes wrong

• Disabling mitigations to "make it work" and leaving them off. setarch -R, -z execstack, -fno-stack-protector are training scaffolding. They get re-enabled when the lesson ends. A training flag that migrates into a build system is how real vulnerabilities ship.
• Aiming at real software. The techniques in this track are dual-use by nature — that is exactly why the boundary is absolute: training targets only, never a product, never a system you don't own, never "just to see."
• Publishing pwn.college solutions. Their rule is explicit and the community enforces it. Mechanics in your own words: yes. Flags, solution scripts, challenge write-ups: no.
• Confusing "ran someone's exploit" with exploit development. Running a tool is not the skill; the skill is the primitive-to-program reasoning from section 3. The write-up proves which one you did.
• Reading the survey as a verdict. "All mitigations on" means expensive to exploit, not unexploitable — and "one mitigation off" means a specific attack shape just got cheaper. The survey informs priorities; it doesn't certify anything.

7. Read the source

Read the primary material in this order:

1. pwn.college (pwn.college) — start with the program-misuse modules, then assembly, then memory errors. Read the module concepts pages fully; do the dojos in order — the belt progression is the curriculum. (Verified live 2026-10-09: ASU-run, free.)
2. how2heap (github.com/shellphish/how2heap) — read as a catalog of heap techniques after the stack modules; each technique page names the primitive and the glibc version it targets. Read the technique explanations; skip version-specific offsets until you need them.
3. Open Security Training 2 (opensecuritytraining.info) — the introductory reversing and assembly courses, free. Read alongside the Ghidra work: theory here, hands on the crackme.

Then at most three free resources, each genuinely free: Ghidra (free, NSA — the reverser's workbench), exploit.education (free — Protostar/Phoenix, the classic vulnerable-VM series for the stretch after pwn.college), ROP Emporium (free — the x86-64 ROP series, training binaries only).

8. Check yourself

Answer in your own words — no peeking at section 3:

1. A program has a stack overflow that overwrites a local variable but stops short of the return address. Explain what the attacker can do with this primitive and what they cannot — then name the second flaw (or second step) they would need to reach control of the instruction pointer, and why.
2. The walkthrough showed identical addresses across runs under setarch -R and changing addresses without it. Explain what ASLR randomizes, what PIE adds on top of it, and why an information leak defeats both.
3. A binary has NX on, PIE on, canary on, RELRO on. An attacker has a stack buffer overflow primitive. Name what each mitigation forces the attacker to do in addition — and which single additional bug class would collapse the most of that extra work.
4. Explain the difference between disassembly and decompilation, and why a reverser forms hypotheses statically before confirming them dynamically.
5. Why does this guidebook teach the mitigation survey hands-on but send exploitation technique to pwn.college — and what does that choice say about where the safety boundary sits?

9. Prove it

Artifact: the mitigation survey (ten binaries, four mitigations each, the chain-link each one breaks) plus your pwn.college belt progress and one mechanics write-up in your own words.

Honest claim: "I can explain how a memory-safety flaw becomes execution control, survey a binary's mitigations, and state what each mitigation costs an attacker." What you cannot claim: exploit developer. That title is earned the long way — advanced dojo series completed, original research, findings reported — and this lesson is the map of the territory, not the summit. The capstone the brief demands is public and original: an advanced pwn.college series plus a mechanics-focused write-up series that respects their no-solutions rule.

10. Leads to

• d-redteam-concepts — the operator's view: what these primitives look like from the C2 side, and why defenders built the mitigations you just surveyed.
• d-malware-analysis — the binary on the other side of the survey: reading what it does, statically and dynamically.
• c-public-track-record — when the survey becomes research: writing the finding so the field can use it.

Walkthrough transcript: runs/d-exploit-concepts.log · Training targets only — pwn.college, exploit.education, ROP Emporium, crackmes. Never real products.