Case Study 6.1: Building a NASM Macro Library
Designing reusable macros for console I/O, string operations, and error handling
Overview
A macro library is a collection of %macro definitions that abstract common patterns into named operations. In C, you'd use #define and inline functions. In Python, you'd write functions. In NASM, you write macros for code that repeats structurally (same instructions, different registers/addresses) and functions for code that repeats behaviorally (runtime logic that makes decisions).
This case study builds a production-quality macro library — one actually useful for the rest of the book's projects — covering console I/O, string operations, error handling, and function management.
Design Principles
Before writing a single macro, establish the design principles:
-
Macros should not clobber registers without documentation. If a macro uses RAX and RDI internally, say so.
-
Prefer function calls for complex logic. A 50-instruction macro that's used 3 times is a waste; write a function.
-
Macros should be composable.
PRINT_INTshouldn't also add a newline — have a separatePRINT_NEWLINEso callers control the output format. -
Use local labels (
%%) for all labels inside macros. Neverjmp .doneinside a macro — it'll conflict with the caller's.done. -
Document the ABI. Which registers are arguments, which are clobbered, which are preserved.
The Complete Library: lib.inc
; lib.inc -- NASM macro library for Learning Assembly Language
; Usage: %include "lib.inc"
;
; Requires: section .data and section .bss to be available
; This file DOES NOT define any data/bss itself (caller must provide sections).
; =====================================================================
; SECTION: Constants
; =====================================================================
; Syscall numbers (Linux x86-64)
%define SYS_READ 0
%define SYS_WRITE 1
%define SYS_OPEN 2
%define SYS_CLOSE 3
%define SYS_STAT 4
%define SYS_FSTAT 5
%define SYS_MMAP 9
%define SYS_MUNMAP 11
%define SYS_BRKO 12
%define SYS_EXIT 60
%define SYS_GETPID 39
; File descriptors
%define STDIN 0
%define STDOUT 1
%define STDERR 2
; Open flags
%define O_RDONLY 0
%define O_WRONLY 1
%define O_RDWR 2
%define O_CREAT 0x40
%define O_TRUNC 0x200
; mmap flags
%define PROT_READ 1
%define PROT_WRITE 2
%define PROT_EXEC 4
%define MAP_SHARED 1
%define MAP_PRIVATE 2
%define MAP_ANONYMOUS 0x20
; =====================================================================
; SECTION: Function frame macros
; =====================================================================
; PROLOGUE -- standard function entry
; Effect: saves RBP, sets RBP=RSP
; Clobbers: nothing extra
%macro PROLOGUE 0
push rbp
mov rbp, rsp
%endmacro
; EPILOGUE -- standard function exit
; Effect: restores RSP=RBP, pops RBP, returns
; Note: does NOT restore any other registers -- caller must do that
%macro EPILOGUE 0
mov rsp, rbp
pop rbp
ret
%endmacro
; ALLOC_LOCALS n -- allocate n bytes of local storage
; Effect: sub rsp, n (rounded up to 16-byte multiple)
; Note: call AFTER PROLOGUE, BEFORE PRESERVE
%macro ALLOC_LOCALS 1
%assign %%rounded_n ((%1 + 15) / 16) * 16
sub rsp, %%rounded_n
%endmacro
; PRESERVE reg1, reg2, ... -- push callee-saved registers
; Effect: push each register in order
; Use RESTORE with same arguments to pop in reverse order
%macro PRESERVE 1-*
%rep %0
push %1
%rotate 1
%endrep
%endmacro
; RESTORE reg1, reg2, ... -- pop callee-saved registers (reverse order)
%macro RESTORE 1-*
%rep %0
%rotate -1
pop %1
%endrep
%endmacro
; =====================================================================
; SECTION: Console I/O macros
; =====================================================================
; WRITE fd, buf, len -- write len bytes of buf to fd
; Clobbers: rax, rdi, rsi, rdx, rcx, r11 (syscall convention)
%macro WRITE 3
mov rax, SYS_WRITE
mov rdi, %1
mov rsi, %2
mov rdx, %3
syscall
%endmacro
; WRITE_STDOUT buf, len -- write to stdout
%macro WRITE_STDOUT 2
WRITE STDOUT, %1, %2
%endmacro
; WRITE_STDERR buf, len -- write to stderr
%macro WRITE_STDERR 2
WRITE STDERR, %1, %2
%endmacro
; PRINT_LITERAL "string" -- print a string literal in-line
; Creates data in .data section inline, then prints it
; Clobbers: rax, rdi, rsi, rdx, rcx, r11
%macro PRINT_LITERAL 1
; We need to emit data AND code. We switch to .data, emit the string,
; then switch back to .text for the syscall.
%push print_lit_ctx
section .data
%%string db %1
%%slen equ $ - %%string
section .text
mov rax, SYS_WRITE
mov rdi, STDOUT
lea rsi, [rel %%string]
mov rdx, %%slen
syscall
%pop
%endmacro
; PRINT_NEWLINE -- print a newline to stdout
; Clobbers: rax, rdi, rsi, rdx, rcx, r11
%macro PRINT_NEWLINE 0
section .data
%%nl db 10
section .text
mov rax, SYS_WRITE
mov rdi, STDOUT
lea rsi, [rel %%nl]
mov rdx, 1
syscall
%endmacro
; READ fd, buf, maxlen -- read up to maxlen bytes from fd into buf
; Returns: rax = bytes read (or negative error)
; Clobbers: rax, rdi, rsi, rdx, rcx, r11
%macro READ 3
mov rax, SYS_READ
mov rdi, %1
mov rsi, %2
mov rdx, %3
syscall
%endmacro
; =====================================================================
; SECTION: Control flow macros
; =====================================================================
; EXIT status -- call sys_exit with given status code
; Clobbers: rax, rdi (doesn't return)
%macro EXIT 1
mov rax, SYS_EXIT
mov rdi, %1
syscall
%endmacro
; EXIT_ERROR -- check rax for syscall error (negative = error)
; If rax < 0, calls sys_exit(1)
; Usage: call a syscall, then EXIT_ERROR
%macro EXIT_ON_ERROR 0
test rax, rax
jns %%no_error
EXIT 1
%%no_error:
%endmacro
; ASSERT_EQ reg, value -- abort if reg != value (debug aid)
%macro ASSERT_EQ 2
cmp %1, %2
je %%ok
; Assertion failed: print message and exit(1)
PRINT_LITERAL "ASSERTION FAILED"
PRINT_NEWLINE
EXIT 1
%%ok:
%endmacro
; =====================================================================
; SECTION: String operation macros
; =====================================================================
; ZERO_REG reg -- zero a register efficiently
; For 64-bit registers: uses 32-bit XOR (shorter encoding)
%macro ZERO_REG 1
xor e%+1, e%+1 ; 32-bit XOR zeroes upper 32 bits too
%endmacro
; NOTE: The above has a subtle issue with registers that don't have
; an e-prefix version (r8-r15). Use this instead:
%macro ZERO 1
xor %1d, %1d ; This only works for r8d-r15d notation
; For a more portable version:
%endmacro
; Safer zero macro using explicit register families:
%macro ZERO64 1 ; zero a 64-bit register
xor eax, eax ; This doesn't work generically...
; In practice, use 'xor %1, %1' for a function call (function can adapt)
; Or explicitly: xor eax, eax for rax, xor ebx, ebx for rbx, etc.
; Macros cannot easily parameterize register names in NASM
%endmacro
; Better approach: just use the actual instructions with a comment macro:
%macro ZERO_EAX 0
xor eax, eax
%endmacro
%macro ZERO_ECX 0
xor ecx, ecx
%endmacro
; (etc. for each register you frequently zero)
Using the Library: A Complete Example
; example_with_lib.asm -- demonstrates the macro library
%include "lib.inc"
section .data
greeting db "Hello from the macro library!", 10
greet_len equ $ - greeting
prompt db "Enter a number: "
prompt_len equ $ - prompt
section .bss
input_buf resb 32
section .text
global _start
; show_greeting: prints the greeting message
; Args: none
; Clobbers: rax, rdi, rsi, rdx (syscall registers)
show_greeting:
WRITE_STDOUT greeting, greet_len
ret
; get_input: read a line from stdin into input_buf
; Args: none
; Returns: rax = bytes read
; Clobbers: rax, rdi, rsi, rdx
get_input:
WRITE_STDOUT prompt, prompt_len
READ STDIN, input_buf, 32
; rax = bytes read or error
EXIT_ON_ERROR
ret
_start:
call show_greeting
PRINT_LITERAL "Starting program"
PRINT_NEWLINE
call get_input
EXIT 0
Macro Design Pitfalls
Pitfall 1: Switch Statement Sections in Macros
The PRINT_LITERAL macro switches between .data and .text sections. After the macro expands, the assembler is in .text. This is correct for most uses, but if you call PRINT_LITERAL from within a .data section (which you generally shouldn't), the results are unexpected. The rule: always use macros from within .text.
Pitfall 2: Register Clobbering
The WRITE_STDOUT macro destroys RAX, RDI, RSI, RDX, RCX, and R11 (via syscall). If the caller has important values in these registers:
; BAD: rdi is clobbered by WRITE_STDOUT
mov rdi, my_func_arg
WRITE_STDOUT greeting, greet_len ; destroys rdi!
call my_func ; rdi is wrong!
; GOOD: save and restore
push rdi
WRITE_STDOUT greeting, greet_len
pop rdi
call my_func
Document this in the library and let callers handle it.
Pitfall 3: Inline Data and .text Ordering
The PRINT_LITERAL macro emits data to .data in the middle of what appears to be .text code. This works correctly — the linker collects all .data content together and all .text content together, regardless of the order in the source file. But it can confuse readers who aren't expecting data declarations in the middle of code.
Alternative: define all string literals at the top of the file and use WRITE_STDOUT label, len instead. This is cleaner and more readable, at the cost of more verbose code at the top.
What This Library Enables
The lib.inc file created here is what the book's subsequent chapters use. When Chapter 7's first programs use WRITE_STDOUT and EXIT, they're using this library. When Chapter 11's exploit examples print debug messages with PRINT_LITERAL, it's this macro. Building the library now means the code in later chapters can focus on the new concepts rather than repeating the syscall boilerplate.