Chapter 20 Exercises: Calling C from Assembly and Assembly from C

Exercise 1: Calling printf — Argument Setup

For each printf call, write the x86-64 NASM assembly to set up the arguments and call printf. You do not need to write full functions — just the setup + call.

a) printf("Hello, World!\n"); b) printf("Count: %d\n", 42); c) printf("%s = %d\n", "answer", 42); (two non-format args) d) printf("Hex: 0x%016lX\n", 0xDEADBEEFCAFEBABEULL); e) printf("Float: %.4f\n", 3.14159); (floating-point — set AL correctly!) f) printf("%d + %d = %d\n", 10, 20, 30); (three integer args)

For each, state: which registers are used for each argument, and what is the value of AL?

Exercise 2: malloc + free

Write an assembly function alloc_and_init(size_t size, uint8_t init_val) that: 1. Calls malloc(size) 2. If malloc returns NULL, return NULL 3. Fills all size bytes with init_val 4. Returns the pointer

Use callee-saved registers appropriately. Which registers will you use and why?

Exercise 3: ABI Compliance Check

The following assembly function has 3 ABI violations. Identify each one:

; Claimed to implement: int64_t multiply(int64_t a, int64_t b)
multiply_buggy:
    push    rbp
    mov     rbp, rsp
    mov     r12, rdi            ; a
    mov     r13, rsi            ; b (oops — not saving/restoring R12, R13)
    imul    r12, r13            ; product in R12
    ; Violation 1: ???
    ; Violation 2: ???
    ; ... do some other work ...
    mov     rax, r12
    mov     rsp, rbp            ; restore RSP from RBP
    pop     rbp
    ; Violation 3: ???
    ret

Rewrite the function correctly.

Exercise 4: Struct Passing

Given this C struct and function:

struct RGB { uint8_t r, g, b, padding; };   // 4 bytes total
uint32_t rgb_to_packed(struct RGB color);

a) How is struct RGB passed to rgb_to_packed? (Which register, and which bytes?) b) Write NASM assembly to call rgb_to_packed({255, 128, 0, 0}) from assembly. c) Where will the return value be?

Exercise 5: Large Struct Return

struct Matrix4x4 { double m[16]; };   // 16 * 8 = 128 bytes
struct Matrix4x4 identity_matrix(void);

a) Is this struct passed in registers? Why or why not? b) Write assembly to call identity_matrix() and store the result. Show how to allocate the return buffer and pass the hidden pointer. c) After the call, how do you access m[0] and m[15] of the returned matrix?

Exercise 6: Red Zone

a) Write a leaf function fast_max(int64_t a, int64_t b) that uses the red zone to store a temporary value (even if not strictly necessary — as practice). b) What would happen if you called another function from within fast_max while using the red zone? c) Why is the red zone prohibited in kernel code? d) The red zone is 128 bytes. If a leaf function needs 200 bytes of local storage, what must it do?

Exercise 7: Variadic Function Analysis

For each variadic call, determine the value of AL and which registers hold which arguments:

a) printf("%d %d %d", 1, 2, 3) — 3 integers b) printf("%f %f", 1.0, 2.0) — 2 doubles c) printf("%d %f %d %f", 1, 2.0, 3, 4.0) — mixed int and double d) printf("%s %d %f %s", "a", 42, 3.14, "b") — mixed types

Exercise 8: Calling puts and fgets

a) Write assembly to call puts("Hello, puts!"); (puts adds a newline automatically) b) Write assembly to call fgets(buffer, 100, stdin) where buffer is a 100-byte stack-allocated array. Show how to allocate the buffer and pass stdin (the C global FILE*).

For (b), to access stdin, use:

extern stdin
; ...
mov rdi, buffer_ptr
mov rsi, 100
mov rdx, [rel stdin]  ; load the FILE* pointer
call fgets

Exercise 9: C++ extern "C"

a) Why does C++ mangle function names? Give an example of what void foo(int x, double y) might mangle to. b) What does extern "C" { void foo(int x, double y); } do in a C++ header? c) If you have an assembly function fast_sort declared as global fast_sort in NASM, what declaration do you need in a C++ file to call it? d) If a C++ class method void Widget::update(int x) is mangled to _ZN6Widget6updateEi, how would you call it from assembly?

Exercise 10: The Complete Mixed Project

Extend the complete project from the chapter (asm_strlen, asm_checksum, asm_strcmp) by adding:

a) int asm_toupper(char c) — returns uppercase version of ASCII character (capitalize if lowercase, else return unchanged) b) void asm_str_toupper(char *s) — convert entire string to uppercase in place (calls asm_toupper or does it inline) c) int asm_count_char(const char *s, char c) — count occurrences of character c in string s

Write the NASM implementation and C declarations for each function. Include test calls in main.c.

Exercise 11: Calling fopen, fwrite, fclose

Write a complete assembly function write_log_entry(const char *filename, const char *message) that: 1. Opens filename for append ("a" mode) 2. If open fails (returns NULL), returns -1 3. Writes message to the file 4. Closes the file 5. Returns 0 on success

Use: - fopen(filename, "a") → FILE* - fwrite(message, 1, strlen(message), fp) → bytes written - fclose(fp) → 0 on success

Exercise 12: ABI in Practice — Callee-Saved Register Audit

The following function uses several registers. Identify which ones must be saved and restored in a proper function prologue/epilogue per System V AMD64:

complex_function:
    ; Uses: RAX, RCX, RDX, RSI, RDI, R8, R9, R10, R11, R12, R13, RBX, RBP
    ; (all of the above are used at some point in this function)
    ; This function also calls another function.

a) Which registers are caller-saved (C can trash them)? b) Which registers must this function save and restore? c) What is the minimal save/restore set if we use only RAX, RBX, and R12 in the body?

Exercise 13: Stack Frame Diagram

Draw the complete stack frame (showing all stored values and their offsets from RBP) for a function with: - Standard prologue (push rbp; mov rbp, rsp) - Local variables: 2 × int64_t (16 bytes) - Saves RBX, R12, R13 (3 × 8 = 24 bytes) - Stack alignment padding (if needed)

Show the final RSP value and the stack layout from RSP to RBP + 8.

Exercise 14: GDB Debugging Mixed Code

Given the functions from the chapter, describe the GDB commands needed to: a) Set a breakpoint at asm_strlen b) After hitting the breakpoint, print the string argument c) Step through the assembly one instruction at a time d) Check the return value after RET e) Verify RBX is unchanged after a function that saves/restores it

Exercise 15: ARM64 Interface (Bonus)

The AAPCS64 (ARM64 calling convention) differs from System V AMD64. Port the asm_strlen function from this chapter to ARM64 NASM/GNU AS format callable from C:

a) What register holds the string pointer argument in AAPCS64? b) What register holds the return value? c) Write the ARM64 version of asm_strlen d) Is extern "C" still needed for ARM64 C++ interoperability?