Byte JMP
Reverse Engineering Fundamentals: Reading x86 Assembly

Reverse Engineering Fundamentals: Reading x86 Assembly

How compiled code really executes on Intel processors: registers, memory operands, flags, conditional branches, the stack and calling conventions, verified by building a small x86 program and tracing it instruction by instruction in GDB.

·37 min read

TL;DR

Assembly is a readable notation for the machine code a specific processor executes. On x86, almost every function you meet in a disassembler reduces to four activities: moving values between registers and memory, computing with them, setting flags and branching on those flags, and transferring control through call and ret using the stack. Learn roughly twenty mnemonics (mov, lea, add, sub, xor, and, test, cmp, the jcc family, push, pop, call, ret), the register widths, the six status flags and the calling convention of the target platform, and most disassembly becomes readable. The most important security consequence is mechanical: call stores a return address on writable stack memory and ret trusts whatever it finds there, which is the foundation of stack based memory corruption and of the mitigations built against it.

  • Platform: Intel x86 (32 bit); hands on lab on Linux
  • Syntax: Intel syntax, NASM assembler
  • Tools: NASM, GNU ld, objdump, GDB
  • Relevant to: Reverse engineering, binary analysis, debugging, exploit development fundamentals

Introduction

Open any binary in a disassembler and functions appear as sequences like this:

mov eax, 7
add eax, 5
cmp eax, 12
je  .equal

Without the rules, it is a list of cryptic commands. With them, it is a precise statement: place 7 in EAX, add 5, compare the result with 12, and continue at .equal if they are equal. Reverse engineering, malware analysis, crash triage and exploit development all depend on making that translation quickly and correctly.

The CPU never sees C, Python or assembly text. It fetches and decodes bytes. An assembler turns mnemonics into those bytes; a disassembler decodes bytes back into mnemonics. Because the encoding belongs to the processor architecture, x86, ARM and MIPS each have their own registers, instructions and conventions. This article stays with Intel x86 and never mixes architectures.

An instruction is a mnemonic followed by zero or more operands. In Intel syntax the destination comes first:

mov eax, 42        ; EAX = 42
add eax, 8         ; EAX = EAX + 8

The goal here is not to memorize hundreds of opcodes. It is to understand exactly what each common instruction changes (which register, which memory bytes, which flags) because that is the question you ask at every line of a disassembly. Every claim below was verified by assembling the code and inspecting the result in GDB.


From source text to executing bytes

Before looking at instructions, it helps to see what survives each stage of the build. This explains why a disassembly never looks exactly like the source that produced it.

sum-lab.asm        text: mnemonics, labels, comments, equ constants
     │
     │  nasm -f elf32          assembler: mnemonics become opcode bytes
     ▼
sum-lab.o          relocatable ELF object, addresses not final yet
     │
     │  ld                     linker: assigns final addresses, resolves symbols
     ▼
sum-lab            ELF32 executable, entry point 0x08049000
     │
     │  execve()               kernel maps segments, sets EIP to the entry point
     ▼
CPU                fetches and decodes: 31 c0 31 c9 be 00 a0 04 08 ...
     │
     │  objdump / GDB          disassembler: bytes become mnemonics again
     ▼
xor eax,eax / xor ecx,ecx / mov esi,0x804a000 ...
                   comments and equ names are gone;
                   labels survive only as symbols, and only if not stripped

The same bytes can also be printed in two syntaxes. NASM and most Windows tools use Intel syntax; GNU tools default to AT&T syntax, which reverses operand order and decorates registers. This is real objdump output for the first two instructions of the lab program, first with -M intel, then with the default:

  8049000:  31 c0     xor    eax,eax
  8049002:  31 c9     xor    ecx,ecx

  8049000:  31 c0     xor    %eax,%eax
  8049002:  31 c9     xor    %ecx,%ecx

Identical bytes, identical semantics, different text. When a disassembly looks "backwards", check the syntax before anything else.


Lab environment

The examples were assembled and traced on the following setup. Any Linux machine with these packages reproduces them. Most x86-64 kernels support running 32 bit binaries natively.

Host      Linux x86_64 VM (Ubuntu 24.04)
NASM      2.16.01
GNU ld    2.42 (binutils)
objdump   2.42 (binutils)
GDB       15.1

On Debian or Ubuntu:

sudo apt update
sudo apt install -y nasm binutils gdb

The instruction set is the same on Windows. What changes between Windows and Linux is the executable format, the calling convention and the operating system interface, and those differences are covered where they matter.


Registers: the CPU's working storage

Registers are small storage locations inside the processor. Instructions typically load values into registers, transform them there, and write results back to memory.

General purpose registers

RegisterTypical role (convention, not a hardware rule)
EAXArithmetic results, function return value
EBXGeneral purpose, preserved across calls in common ABIs
ECXLoop counter, first argument in Microsoft fastcall
EDXHigh half of multiply and divide, second argument in Microsoft fastcall
ESISource pointer in string instructions
EDIDestination pointer in string instructions
ESPStack pointer: address of the top of the stack
EBPFrame base pointer when the compiler keeps one

A register has no permanent meaning. A few instructions use specific registers implicitly (div uses EDX:EAX, push uses ESP, string instructions use ESI and EDI), and calling conventions assign roles at function boundaries. Everywhere else, the code decides what a register holds.

One register, three names

The smaller names are views into the low part of the same physical register:

bit 31             16 15      8 7       0
┌────────────────────┬─────────┬─────────┐
│                    │   AH    │   AL    │
└────────────────────┴─────────┴─────────┘
                     └─────── AX ────────┘
└──────────────────── EAX ───────────────┘

AH (bits 8 to 15) exists only for EAX, EBX, ECX and EDX. Writing a sub-register leaves the other bits untouched:

mov eax, 0x11223344
mov al,  0xAA            ; EAX = 0x112233AA  (other bits kept)
mov ax,  0xBBBB          ; EAX = 0x1122BBBB  (other bits kept)

Values observed in GDB after each instruction:

$1 = 0x112233aa
$2 = 0x1122bbbb

EIP and EFLAGS

EIP holds the address of the next instruction. Execution normally advances sequentially; jumps, calls, returns and exceptions change EIP. You almost never write EIP directly, but every control flow instruction exists to change it.

EFLAGS holds status and control bits. Six status flags matter for reading code, and four of them do almost all the work:

FlagNameSet when
ZFZeroThe result is zero (for cmp: the operands were equal)
CFCarryAn unsigned operation carried out of, or borrowed into, the top bit
SFSignThe top bit of the result is 1 (negative if read as signed)
OFOverflowA signed result did not fit in the destination
PFParityThe low byte of the result has an even number of set bits
AFAuxiliary carryA carry occurred out of bit 3 (used for BCD)

Not every instruction writes flags. mov, lea, push, pop and jmp leave them alone; add, sub, cmp, and, xor and test update them according to the result. This is why a conditional jump can appear several instructions after the comparison that feeds it, as long as nothing in between writes the flags.


Moving data: MOV, LEA and memory operands

MOV copies

mov eax, 10        ; EAX = 10
mov ebx, eax       ; EBX = 10, EAX unchanged
mov eax, 20        ; EAX = 20, EBX still 10

mov copies; the source keeps its value. Operands come in three kinds: an immediate (a constant encoded inside the instruction, such as 10), a register (such as EAX), and a memory operand written with brackets (such as [eax]). Ordinary mov accepts at most one memory operand, so memory to memory copies go through a register:

mov eax, dword [value1]   ; memory to register: valid
mov dword [value2], eax   ; register to memory: valid
; mov [value2], [value1]  ; memory to memory: rejected by the assembler

Brackets mean dereference

section .data
    number dd 0x12345678

section .text
    mov eax, dword [number]       ; EAX = the 4 bytes stored at number
    lea ebx, [number]             ; EBX = the address of number

The C equivalent makes the distinction obvious:

uint32_t number = 0x12345678;
uint32_t  value = number;    // mov eax, [number]
uint32_t *ptr   = &number;   // lea ebx, [number]

In 32 bit code, memory operands typically use absolute addresses. The assembler stores the full address of number in the instruction encoding. This is visible in the disassembly as an immediate address like mov eax, [0x804a000].

LEA computes an address without reading memory

x86 memory operands follow one general formula:

effective address = base + index × scale + displacement
                    EBX  + ECX   × 4     + 8

base          any general purpose register
index         any general purpose register except ESP
scale         1, 2, 4 or 8 (no other values can be encoded)
displacement  constant offset

lea evaluates the formula and stores the address; mov with the same brackets reads the bytes at that address:

; EBX = 0x1000, ECX = 3
lea eax, [ebx + ecx*4 + 8]   ; EAX = 0x1014, no memory access at all
mov edx, dword [eax]         ; EDX = 4 bytes read from 0x1014

Verified in GDB: after the lea, p/x $eax prints 0x1014. Because lea never touches memory and never writes flags, compilers also abuse it as a fast arithmetic instruction: lea eax, [edi + esi] is simply eax = edi + esi, and lea eax, [edi + edi*4] multiplies by five. When you see lea in a disassembly, ask whether the result is later dereferenced (an address) or only used as a number (arithmetic).

Operand size is part of the instruction

SizeNASM size keywordDeclarationExample
8 bitsbytedbdb 0x41
16 bitsworddwdw 0x1234
32 bitsdworddddd 0x12345678

When a register operand is present, it fixes the size (mov eax, [ebx] reads four bytes). When it is not, the assembler cannot guess, and the size keyword is mandatory:

inc byte  [ebx]          ; modify 1 byte
mov dword [ebx], 100     ; write 4 bytes
mov ax, word [ebx]       ; read 2 bytes

Width is a correctness and security property, not a formatting detail. A dword read from a buffer holding two valid bytes reads two bytes that belong to something else, and many out of bounds bugs are precisely a mismatch between the width the code uses and the size of the object in memory.

Little endian byte order

x86 stores multibyte integers least significant byte first. After the CPU stores the 32 bit value 0x12345678 at address 0x1000, memory contains:

address   0x1000  0x1001  0x1002  0x1003
byte        78      56      34      12
            │                       │
            least significant       most significant

Inside a register the value is simply 0x12345678; endianness only describes how bytes are laid out in memory. It matters whenever you read a hex dump, recognize a pointer inside a buffer, or write an address into memory byte by byte.

Arrays use the same addressing formula, with the scale matching the element size:

section .data
    values dd 10, 20, 30, 40

section .text
    mov ebx, values                ; EBX = address of values[0]
    mov ecx, 2
    mov eax, dword [ebx + ecx*4]   ; EAX = values[2] = 30

Index 2 means byte offset 8 because each dd element is four bytes. A byte array uses scale 1. Recognizing [base + index*4] in a disassembly is how you recognize an array of 32 bit integers or a table of 32 bit offsets.


Arithmetic and bitwise instructions

ADD, SUB, INC and DEC

mov eax, 10     ; EAX = 10
add eax, 5      ; EAX = 15
sub eax, 3      ; EAX = 12
inc eax         ; EAX = 13
dec eax         ; EAX = 12

add and sub update all six status flags. inc and dec update the same flags except CF, which they leave exactly as it was. The difference is easy to demonstrate:

mov eax, -1
stc              ; force CF = 1
inc eax          ; EAX = 0
eflags         0x257               [ CF PF AF ZF IF ]

ZF was set by the result, but CF is still 1 from stc. An add eax, 1 in the same place would have recomputed CF from the addition.

Multiplication and division

mov  eax, 6
imul eax, eax, 7         ; EAX = 42 (three operand form, signed)

imul (signed) has one, two and three operand forms. The one operand form, like mul (unsigned), multiplies by EAX and writes a double width result to EDX:EAX, which is why EDX is often overwritten unexpectedly around multiplications.

Division always works on a double width dividend:

           EDX : EAX            64 bit dividend
div ecx   ───────────  ──►   EAX = quotient
              ECX            EDX = remainder
mov eax, 20
xor edx, edx             ; EDX:EAX = 20 (high half must be prepared)
mov ecx, 6
div ecx                  ; EAX = 3, EDX = 2

For signed division, idiv needs the high half filled with copies of the sign bit, which is what cdq (sign extends EAX into EDX:EAX) does:

mov  eax, -7
cdq                      ; EDX = 0xFFFFFFFF
mov  ecx, 2
idiv ecx                 ; EAX = -3, EDX = -1 (truncates toward zero)

Dividing by zero, or producing a quotient that does not fit in the destination, raises a divide error exception (#DE). On Linux the process receives SIGFPE; on Windows it surfaces as EXCEPTION_INT_DIVIDE_BY_ZERO or EXCEPTION_INT_OVERFLOW. A forgotten xor edx, edx before div is a classic source of that crash.

Bits have no type

mov eax, 0FFFFFFFFh      ; unsigned 4294967295, or signed -1
add eax, 1               ; EAX = 0
eflags         0x257               [ CF PF AF ZF IF ]

The register holds 32 bits and nothing else. As an unsigned addition, the result wrapped around, so CF = 1. As a signed addition (-1 + 1), nothing overflowed, so OF = 0. Both interpretations are recorded at once; the instruction that consumes the flags decides which one the program meant. This is the single most important idea for reading comparisons correctly.

Logic, masks and shifts

mov al, 11001100b
and al, 00001111b        ; AL = 00001100b   clear the high four bits

mov al, 11000000b
or  al, 00000011b        ; AL = 11000011b   set the low two bits

mov al, 10101010b
xor al, 11111111b        ; AL = 01010101b   toggle every bit

not al                   ; AL = 10101010b   invert every bit

and, or and xor clear CF and OF and set ZF, SF and PF from the result. not changes no flags at all.

One idiom appears in nearly every function:

xor eax, eax             ; EAX = 0

It is not identical to mov eax, 0. The encodings differ in size, and xor writes flags while mov does not, as the assembled bytes and the flags confirm:

31 c0                xor    eax,eax      ; 2 bytes, ZF = 1, CF = 0, OF = 0
b8 00 00 00 00       mov    eax,0x0      ; 5 bytes, flags untouched

Compilers prefer xor reg, reg because it is shorter and processors recognize it as a dependency breaking zeroing idiom. In shellcode it has a second motivation: it zeroes a register without placing any 00 byte in the encoding.

Shifts move bits and drop the ones that fall off the edge:

mov eax, 5               ; 0101b
shl eax, 1               ; EAX = 10  (left shift, multiply by 2)
shr eax, 1               ; EAX = 5   (logical right shift, fills with 0)

mov eax, -8
sar eax, 1               ; EAX = -4  (arithmetic right shift, copies the sign bit)

shr treats the value as unsigned and sar as signed, so the choice between them in a disassembly tells you the type of the original variable. rol and ror rotate: the bits leaving one end re enter at the other, which is why they show up in hashing and simple obfuscation routines.


Flags and control flow

Every if, while and for in a compiled program becomes an instruction that sets flags followed by an instruction that reads them.

CMP and TEST

mov eax, 10
cmp eax, 10        ; computes EAX - 10, keeps only the flags
je  .equal         ; taken: ZF = 1

cmp a, b performs the subtraction a - b, updates all six status flags, and throws the result away. Neither operand changes.

test eax, eax      ; computes EAX AND EAX, keeps only the flags
jz   .is_zero      ; taken when EAX == 0

test is the same idea with and: it sets ZF, SF and PF from the result and always clears CF and OF. test eax, eax is the standard zero or null pointer check, and test al, 4 checks whether bit 2 is set.

je and jz are two names for the same encoding, as are jne and jnz. A disassembler picks one name; the CPU only checks ZF.

Conditional jumps

InstructionJumps whenMeaning after cmp a, b
jmpalwaysunconditional
je / jzZF = 1a == b
jne / jnzZF = 0a != b
jl / jngeSF ≠ OFa < b, signed
jle / jngZF = 1 or SF ≠ OFa <= b, signed
jg / jnleZF = 0 and SF = OFa > b, signed
jge / jnlSF = OFa >= b, signed
jb / jnae / jcCF = 1a < b, unsigned
jbe / jnaCF = 1 or ZF = 1a <= b, unsigned
ja / jnbeCF = 0 and ZF = 0a > b, unsigned
jae / jnb / jncCF = 0a >= b, unsigned

The vocabulary is consistent: less and greater are signed, below and above are unsigned.

One comparison, two answers

The difference between signed and unsigned jumps is where many real bugs hide, so it is worth tracing once with real flags:

mov eax, -1        ; bit pattern 0xFFFFFFFF
cmp eax, 1

GDB after the cmp:

eflags         0x282               [ SF IF ]
cmp eax, 1   computes 0xFFFFFFFF - 1 = 0xFFFFFFFE  (discarded)
     │
     ▼
EFLAGS:   ZF = 0   SF = 1   OF = 0   CF = 0
     │
     ├─────────────────────────────┬──────────────────────────────┐
     ▼                             ▼                              ▼
jl   (SF ≠ OF)               ja   (CF = 0 and ZF = 0)        jb   (CF = 1)
signed: -1 < 1               unsigned: 4294967295 > 1        unsigned: below?
TAKEN                        TAKEN                           NOT TAKEN

The same flags say "less" to a signed jump and "above" to an unsigned one. Conditional jumps never modify flags, so any of these branches would see identical state; in a real program only the first taken branch matters, because it moves EIP elsewhere.

This is how a length check can be bypassed: if the source code compares a signed int len against a buffer size, the compiler emits jl/jg, and a negative length passes a "less than size" test. When that same value later reaches memcpy as an unsigned size_t, it becomes enormous. In disassembly, the bug is visible as a signed jump guarding a value that is later used as an unsigned size.

Loops are a compare and a branch

int sum = 0;
for (int i = 1; i <= 5; i++)
    sum += i;
// sum == 15
    xor eax, eax          ; sum = 0
    mov ecx, 1            ; i = 1
.loop:
    add eax, ecx          ; sum += i
    inc ecx               ; i++
    cmp ecx, 6
    jl  .loop             ; repeat while i < 6 (signed, because i is int)
; EAX = 15

There is no loop construct in the CPU. A loop is a label, some work, and a conditional jump backwards. When a disassembler draws an arrow pointing upward, you are looking at a loop.


The stack: PUSH, POP, CALL and RET

The stack is ordinary memory that ESP points into. It holds return addresses, saved registers, local variables and function arguments. On x86 it grows toward lower addresses: pushing decreases ESP.

PUSH and POP

mov  eax, 0x11223344
push eax               ; ESP = ESP - 4, then [ESP] = EAX
xor  eax, eax          ; EAX = 0
pop  ebx               ; EBX = [ESP], then ESP = ESP + 4

Traced in GDB (stack addresses depend on the environment):

before push    esp = 0xffffd4e0
after push     esp = 0xffffd4dc   [esp] = 0x11223344
after pop      esp = 0xffffd4e0   ebx   = 0x11223344
        before push eax                           after push eax

higher addresses                         higher addresses
┌────────────────────────┐               ┌────────────────────────┐
│ previous contents      │ ◄── ESP       │ previous contents      │  0x...d4e0
├────────────────────────┤  0x...d4e0    ├────────────────────────┤
│ (free)                 │               │ 0x11223344             │ ◄── ESP
└────────────────────────┘               └────────────────────────┘  0x...d4dc
lower addresses                          lower addresses

In 32 bit mode push and pop of a general purpose register move four bytes. pop does not erase anything: the old value stays in memory below ESP until something overwrites it, which is why uninitialized locals often contain leftovers from previous calls.

CALL saves a return address, RET trusts it

_start:
    push 5               ; 2nd argument (cdecl: right to left)
    push 7               ; 1st argument
    call add_two         ; push return address, jump to add_two
    add  esp, 8          ; caller cleanup
    ; EAX = 12
    ...

add_two:
    push ebp
    mov  ebp, esp
    mov  eax, dword [ebp+8]    ; 1st argument
    add  eax, dword [ebp+12]   ; 2nd argument
    pop  ebp
    ret                         ; pop the return address into EIP

A near call performs two actions: it pushes the address of the next instruction, then sets EIP to the target. ret performs the reverse: it pops four bytes from the stack into EIP. Stopping at the first instruction of add_two in GDB shows the saved address:

(gdb) x/wx $esp
0xffffd4dc:  0x0804900f
(gdb) info symbol *(int*)$esp
_start + 15 in section .text
caller (_start)                       stack                     callee (add_two)

0x8049000  push 5
0x8049005  push 7
0x804900a  call add_two ──── push ───► [ESP] = 0x804900f
                          └── EIP = add_two ──────────────────► push ebp
                                                                mov ebp, esp
                                                                mov eax,[ebp+8]
                                                                add eax,[ebp+12]
                                                                pop ebp
                                                                ret
                                       EIP = [ESP] ◄─── pop ────────┘
0x804900f  add esp, 8  ◄─────────────┘

Note what ret does not do. It does not check that the value at ESP was written by a matching call, and it does not check that the value points to the instruction after one. It loads four bytes from writable memory into EIP. That single design property is why overwriting a saved return address redirects execution, and it is the reason every stack protection discussed later exists.

The stack frame

32 bit code typically passes arguments on the stack and builds a frame around EBP. Save as frame32.asm:

; frame32.asm  (NASM, 32 bit Linux)
global _start
section .text

add_two:
    push ebp                    ; save the caller's frame pointer
    mov  ebp, esp               ; EBP = base of this frame
    mov  eax, dword [ebp+8]     ; 1st argument
    add  eax, dword [ebp+12]    ; 2nd argument
    mov  esp, ebp               ; discard locals
    pop  ebp                    ; restore the caller's frame pointer
    ret                         ; cdecl: the caller removes the arguments

_start:
    push 5                      ; arguments pushed right to left
    push 7
    call add_two
after_call:
    add  esp, 8                 ; caller cleanup (2 arguments × 4 bytes)
    mov  ebx, eax               ; exit status = 12
    mov  eax, 1                 ; 32 bit Linux syscall number for exit
    int  0x80

Build and run:

nasm -f elf32 -g -F dwarf frame32.asm -o frame32.o
ld -m elf_i386 -o frame32 frame32.o
./frame32; echo $?        # 12
gdb -q ./frame32

Break right after mov ebp, esp (three bytes into the function: push ebp is one byte, mov ebp, esp is two) and dump four dwords from EBP:

The stack address will differ on your machine; the values will not.

(gdb) break *(add_two+3)
(gdb) run
(gdb) x/4wx $ebp
0xffff5960:  0x00000000  0x08049016  0x00000007  0x00000005
(gdb) info symbol *(int*)($ebp+4)
after_call in section .text
higher addresses
┌─────────────────────────────┐
│ 0x00000005  2nd argument    │  [EBP+12]
├─────────────────────────────┤
│ 0x00000007  1st argument    │  [EBP+8]
├─────────────────────────────┤
│ 0x08049016  return address  │  [EBP+4]   pushed by call, points at after_call
├─────────────────────────────┤
│ 0x00000000  saved EBP       │  [EBP]     ◄── EBP
├─────────────────────────────┤
│ local variables             │  [EBP-4], [EBP-8], ...
└─────────────────────────────┘
lower addresses                              ◄── ESP grows down from here

This picture explains a lot of disassembly at once: positive offsets from EBP are arguments, negative offsets are locals, and a local buffer sits below the saved EBP and return address. Writing past the end of that buffer moves upward in memory, toward exactly those two values.

leave is the compact form of the epilogue: it performs mov esp, ebp then pop ebp. Optimized builds frequently omit the frame pointer entirely and address locals relative to ESP, so do not assume every function begins with push ebp.

Calling conventions

The instruction set does not say where arguments go; the calling convention does. Applying the wrong one is the most common way to misread a function.

ConventionUsed byInteger and pointer argumentsReturnStack cleanup
cdecl32 bit C code (Windows and Linux)Stack, pushed right to leftEAXCaller (add esp, n)
stdcall32 bit Win32 API (WINAPI)Stack, pushed right to leftEAXCallee (ret n)
fastcall (Microsoft)32 bit, opt inECX, EDX, then stackEAXCallee

Some details matter constantly when reading code:

  • cdecl vs stdcall. The only difference is who cleans the stack. In cdecl the caller does add esp, 8 after a two argument call. In stdcall the callee does ret 8. A stdcall function ends with ret 8 (encoded c2 08 00) instead of a plain ret (c3); the immediate is the number of argument bytes the callee removes. This is visible in the bytes.
  • fastcall passes the first two arguments in ECX and EDX. The rest go on the stack. This convention is common in Windows kernel code and COM interfaces.
  • thiscall (Microsoft C++). The this pointer goes in ECX; the rest follows cdecl. Recognizing ECX as this before a virtual call is how you reconstruct C++ object layouts.

Practical lab: building and tracing a real program

The lab combines most of the instructions above in one executable. It sums the array 2, 4, 6, 8, writes a message to standard output, and returns the sum (20) as its exit status so the result can be checked from the shell.

Source

Save as sum-lab.asm:

; sum-lab.asm  (NASM, Linux x86 32 bit)

global _start                        ; export the entry point for the linker

section .data
    numbers  dd 2, 4, 6, 8           ; four 32 bit integers
    count    equ 4                   ; assembler constant, no storage
    message  db "Sum calculated. Check the exit status.", 10
    msg_len  equ $ - message         ; length computed at assembly time

section .text
_start:
    xor eax, eax                     ; sum = 0
    xor ecx, ecx                     ; index = 0
    mov esi, numbers                 ; ESI = address of numbers[0]

.loop:
    cmp ecx, count                   ; index compared with 4
    jge .done                        ; leave when index >= 4
    add eax, dword [esi + ecx*4]     ; sum += numbers[index]
    inc ecx                          ; index++
    jmp .loop

.done:
    mov edi, eax                     ; save the sum in EDI

    ; write(1, message, msg_len)
    mov eax, 4                       ; syscall number for write
    mov ebx, 1                       ; fd 1 = stdout
    mov ecx, message                 ; buffer address
    mov edx, msg_len                 ; byte count
    int 0x80

    ; exit(sum)
    mov eax, 1                       ; syscall number for exit
    mov ebx, edi                     ; exit status = 20
    int 0x80

The control flow of _start is small enough to draw completely:

┌───────────────────────────────────┐
│ _start                            │
│ xor eax,eax        sum   = 0      │
│ xor ecx,ecx        index = 0      │
│ mov esi, numbers                  │
└─────────────────┬─────────────────┘
                  ▼
┌───────────────────────────────────┐
│ .loop                             │ ◄─────────┐
│ cmp ecx, 4                        │           │
│ jge .done ────────────────────────┼──┐        │
└─────────────────┬─────────────────┘  │        │
                  │ not taken          │ taken  │
                  ▼                    │        │
┌───────────────────────────────────┐  │        │
│ add eax,[esi+ecx*4]               │  │        │
│ inc ecx                           │  │        │
│ jmp .loop ────────────────────────┼──┼────────┘
└───────────────────────────────────┘  │
                  ┌────────────────────┘
                  ▼
┌───────────────────────────────────┐
│ .done                             │
│ mov edi, eax                      │
│ write(1, message, 39)             │
│ exit(edi)                         │
└───────────────────────────────────┘

Build and run

nasm -f elf32 -g -F dwarf sum-lab.asm -o sum-lab.o
ld -m elf_i386 -o sum-lab sum-lab.o
./sum-lab
echo $?

-f elf32 selects the 32 bit ELF object format, and -g -F dwarf adds DWARF debug information so GDB can show source lines. The program has no C runtime: the linker uses _start as the entry point and the kernel jumps straight to it.

Sum calculated. Check the exit status.
20

Static view: objdump

objdump -d -M intel ./sum-lab
08049000 <_start>:
 8049000:  31 c0                  xor    eax,eax
 8049002:  31 c9                  xor    ecx,ecx
 8049004:  be 00 a0 04 08         mov    esi,0x804a000

08049009 <_start.loop>:
 8049009:  83 f9 04               cmp    ecx,0x4
 804900c:  7d 07                  jge    8049015 <_start.done>
 804900e:  03 04 8e               add    eax,DWORD PTR [esi+ecx*4]
 8049011:  ff c1                  inc    ecx
 8049013:  eb f4                  jmp    8049009 <_start.loop>

08049015 <_start.done>:
 8049015:  89 c7                  mov    edi,eax
 8049017:  b8 04 00 00 00         mov    eax,0x4
 804901c:  bb 01 00 00 00         mov    ebx,0x1
 8049021:  b9 10 a0 04 08         mov    ecx,0x804a010
 8049026:  ba 27 00 00 00         mov    edx,0x27
 804902b:  cd 80                  int    0x80
 804902d:  b8 01 00 00 00         mov    eax,0x1
 8049032:  89 fb                  mov    ebx,edi
 8049034:  cd 80                  int    0x80

Compare it with the source and several build stages become visible:

  • count and msg_len no longer exist. They were equ constants, so they became the immediates 0x4 and 0x27 (39 bytes).
  • NASM local labels became the symbols _start.loop and _start.done. They are present only because the binary is not stripped. After strip, the same bytes disassemble as one anonymous block under <.text> with raw jump targets such as jge 0x8049015, which is what most real targets look like.
  • numbers became the absolute address 0x804a000, stored directly in the mov instruction. message became 0x804a010.
  • Short jumps store signed offsets, not addresses. eb f4 jumps back 12 bytes (0xf4 read as a signed byte is -12) from 0x8049015 to 0x8049009.

Dynamic view: GDB

Static analysis shows what the code can do; a debugger shows what it does with real values.

gdb -q ./sum-lab
(gdb) set disassembly-flavor intel
(gdb) break _start
(gdb) run
(gdb) display/i $pc
(gdb) stepi 3
(gdb) x/4dw $esi
(gdb) break _start.done
(gdb) continue
(gdb) info registers eax ecx eflags
CommandPurpose
set disassembly-flavor intelShow Intel syntax instead of AT&T
break _start / runStop on the first instruction of the program
display/i $pcPrint the next instruction after every step
stepi 3 (or si)Execute instructions one at a time, entering calls
x/4dw $esiExamine 4 units, decimal format, 32 bit words, at ESI
info registersShow selected registers; GDB accepts names such as eax

Relevant output from the session (trimmed):

Breakpoint 1, _start () at sum-lab.asm:14
1: x/i $pc
=> 0x8049009 <_start.loop>:  cmp    ecx,0x4

(gdb) x/4dw $esi
0x804a000 <numbers>:  2   4   6   8

Breakpoint 2, _start.done () at sum-lab.asm:24
eax            0x14                20
ecx            0x4                 4
eflags         0x246               [ PF ZF IF ]

After three steps the mov esi, numbers has executed and ESI points at the array, confirmed by x/4dw. At _start.done the flags explain why the loop ended: the last cmp ecx, 4 with ECX = 4 produced zero, so ZF = 1, SF = 0 and OF = 0. jge jumps when SF = OF, so the branch was taken. IF is the interrupt enable flag, which is normally set for user mode code and can be ignored here.

One more GDB command connects the lab to the security discussion that follows (columns trimmed, annotations added):

(gdb) info proc mappings
  Start Addr   End Addr   Perms  objfile
  0x08048000 0x08049000   r--p   sum-lab
  0x08049000 0x0804a000   r-xp   sum-lab      code: readable, executable
  0x0804a000 0x0804b000   rw-p   sum-lab      data: writable, not executable
  0xfffdd000 0xfffff000   rw-p   [stack]      stack: writable, not executable

The code lives in a readable and executable page, the data and stack in writable but not executable pages. This separation is the basis of NX/DEP protection.

What the loop did

IterationECX before cmpBytes readEAX after add
10[0x804a000] = 22
21[0x804a004] = 46
32[0x804a008] = 612
43[0x804a00c] = 820
exit4none, jge taken20

The address advances four bytes per iteration because the scale is 4. The loop uses the signed jge because the index is conceptually an int; a compiler working with an unsigned index would emit jae instead.

System calls via int 0x80

int 0x80 is how a 32 bit Linux process asks the kernel for a service. Its register convention is:

EAX   syscall number            (1 = exit, 4 = write)
EBX   1st argument
ECX   2nd argument
EDX   3rd argument
ESI   4th argument
EDI   5th argument
EBP   6th argument
EAX   return value

The exit status only exposes the low eight bits to the parent process, so exit(20) is a convenient demonstration and not a general way to print numbers. The numbers themselves are Linux specific. Windows does not document its system call numbers, and they change between builds; programs call APIs in kernel32.dll and other system DLLs, which eventually reach ntdll.dll. A Linux syscall snippet cannot be pasted into a Windows program and expected to work.


Why this matters for security work

Everything above is ordinary, intended processor behavior. None of it is a vulnerability. It becomes a security topic when a program bug lets an attacker write memory the code did not intend to expose, because the mechanisms in this article decide what that write can achieve.

local buffer on the stack (lower address)
     │
     │  write longer than the buffer, moving toward higher addresses
     ▼
saved EBP
     │
     ▼
saved return address  ◄── written by call, trusted by ret
     │
     │  ret loads it into EIP
     ▼
attacker chosen EIP

Each mitigation in modern toolchains and operating systems targets one specific link in that chain:

Mechanism from this articleWhat a memory corruption bug can do with itMitigation tied to the mechanism
Locals sit below the saved return addressA linear overflow reaches the return addressStack canary (/GS in MSVC, -fstack-protector in GCC and Clang): a random value placed between locals and saved data, checked before ret
ret loads EIP from writable memory without validationRedirect execution, or chain ret terminated code fragments (ROP)Shadow stack (Intel CET): call also stores the return address on a protected stack, and ret raises a control protection exception (#CP) if the two copies differ
The stack and data are just memoryPlace machine code in a buffer and jump to itNX / DEP: pages without execute permission, visible as rw-p for the lab stack and data
Addresses fixed at link timeJump to a known addressASLR with position independent code. The lab binary is a non-PIE executable fixed at 0x08049000, and GDB disables randomization by default (show disable-randomization), which is why addresses repeat between runs

Reading assembly is what lets you verify these protections in a real binary rather than assuming them. A function protected by a canary loads the cookie in its prologue (from gs:0x14 on Linux x86, or from __security_cookie on Windows) and calls a failure routine such as __stack_chk_fail or __security_check_cookie before returning. If those instructions are missing, the function is unprotected, whatever the build documentation says.


Common mistakes when reading disassembly

Confusing an address with its contents. lea eax, [value] produces an address; mov eax, [value] reads memory. The brackets look identical; the mnemonic decides.

Giving registers permanent meanings. EAX often holds a return value, but only at a function boundary. Inside a function, it is whatever the compiler put there.

Ignoring operand width. A byte, word and dword access at the same address read different amounts of memory and can produce completely different values.

Mixing signed and unsigned jumps. jl and jb answer different questions from the same flags. The jump, not the register, tells you the type of the original variable.

Assuming cmp modifies its operands. It writes flags only. The conditional jump that follows consumes them.

Treating call as a jmp. A near call also pushes a return address, which shifts every ESP relative offset inside the callee by four bytes.

Applying the wrong calling convention. cdecl cleanup is in the caller; stdcall cleanup is in the callee. Mixing them produces a corrupted stack.

Mixing OS interfaces. Instructions are shared between Windows and Linux; system call numbers, executable formats and ABIs are not.

Confusing syntax with semantics. Intel and AT&T text can describe identical bytes. Check the syntax before concluding that operands are reversed.


Instruction reference

A starting set for integer code, grouped by purpose. The Intel manuals document hundreds more, but these cover most of what compilers emit for ordinary logic.

GroupInstructionEffect
Data movementmovCopy a value; no flags changed
Data movementmovzxCopy a smaller value and zero extend it (unsigned)
Data movementmovsxCopy a smaller value and sign extend it (signed)
AddressingleaCompute an effective address; no memory access, no flags
Arithmeticadd / subAdd or subtract; all status flags updated
Arithmeticinc / decAdd or subtract 1; CF preserved
Arithmeticimul / mulSigned or unsigned multiplication
Arithmeticidiv / divSigned or unsigned division of EDX:EAX
ArithmeticnegTwo's complement negation
ArithmeticcdqSign extend EAX into EDX:EAX
Bitwiseand / or / xorBitwise logic; CF and OF cleared
BitwisenotInvert all bits; no flags changed
Bitwiseshl / salShift left (two names, same operation)
Bitwiseshr / sarLogical or arithmetic shift right
Bitwiserol / rorRotate left or right
ComparisoncmpSubtract for flags only
ComparisontestAND for flags only; CF and OF cleared
BranchingjmpUnconditional jump
BranchingjccConditional jump (je, jne, jl, jg, jb, ja, ...)
BranchingcmovccConditional move; compilers use it to avoid a branch
BranchingsetccStore 0 or 1 in a byte according to a condition
Stackpush / popStore to or load from the top of the stack, adjusting ESP
FunctionscallPush the return address and jump
Functionsret / ret nPop the return address into EIP; ret n also removes n bytes of arguments
Functionsleavemov esp, ebp then pop ebp
Systemint nSoftware interrupt; int 0x80 is the 32 bit Linux system call gate
Debugint3One byte breakpoint (0xCC), the instruction debuggers write over code
MiscnopDo nothing except advance EIP

A note on breakpoints: NASM assembles int3 as the single byte cc, but int 3 as the two bytes cd 03. Debuggers use the one byte form because it can replace the first byte of any instruction.

Assembler directives are not instructions. section, global, db, dw, dd and equ tell NASM how to lay out the file. They produce data or symbols, never executed code, which is why count and msg_len disappeared from the disassembly.


Where to go next

Short exercises that build directly on the lab:

  1. Change the numbers array, predict the exit status, then run the program to check the prediction. Try values whose sum exceeds 255 and explain the result.
  2. Rewrite the loop to count down from 4 to 0 using dec and jnz, and confirm in GDB which flag ends the loop.
  3. Move the summation into a function called with call, passing the array address and count on the stack using cdecl, and returning the sum in EAX.
  4. Compile a small C function with gcc -m32 -O0 and with gcc -m32 -O2, disassemble both, and identify arguments, locals, branches and the return value. Then strip the binaries and repeat.
  5. Compare cdecl (add esp, n in the caller) with stdcall (ret n in the callee) by writing two versions of the same function.
  6. Compile a function with a local char array and -fstack-protector-strong -m32, and locate the canary load and check in the disassembly.

With these fundamentals in place, the natural next topics are stack buffer overflows, calling convention abuse, return oriented programming, and how DEP, ASLR, stack canaries and shadow stacks change what an overflow can achieve. Each of them is a direct consequence of what call, ret and a memory write do at the instruction level.


References

  1. Intel 64 and IA-32 Architectures Software Developer's Manuals (Intel). Volume 1 for registers, flags and the execution environment; Volume 2 for the exact behavior and flag effects of every instruction.
  2. __stdcall and __cdecl (Microsoft). Argument order and stack cleanup responsibility for 32 bit conventions.
  3. syscall(2) (Linux man pages). Registers used for the system call number, arguments and return value on each architecture.
  4. NASM documentation (NASM project). Command line options, output formats, local labels and directives.
  5. GDB: Examining Memory (GNU). Syntax and format letters of the x command.
  6. Ed Jorgensen, x86-64 Assembly Language Programming with Ubuntu (Open Textbook Library). A free textbook covering the same foundations in more depth.

The examples were written for this article and verified with the toolchain listed in the lab environment. Floating point, SIMD (SSE and AVX), privileged instructions and interrupt handling are left for separate articles.