Case Study 12.1: memcpy() — Four Implementations, Four Performance Stories

The Challenge

memcpy copies N bytes from source to destination. It is one of the most fundamental and most performance-critical operations in systems programming — called by everything from string handling to graphics rendering to network packet processing. This case study implements it four ways and measures the difference.

Implementation 1: Naive Byte Copy

; memcpy_v1: byte-at-a-time
; RDI = dst, RSI = src, RDX = n
; Returns dst in RAX

memcpy_v1:
    mov  rax, rdi          ; save dst for return
    mov  rcx, rdx          ; rcx = n (byte count)
    test rcx, rcx
    jz   .done
.loop:
    mov  al, [rsi]         ; load one byte from source
    mov  [rdi], al         ; store to destination
    inc  rsi
    inc  rdi
    dec  rcx
    jnz  .loop
.done:
    mov  rax, rdi          ; OOPS: return value is wrong — rdi was modified
    ret
    ; Bug: should save original rdi, not return modified rdi

Wait — let us fix that:

memcpy_v1:
    push rbp
    mov  rbp, rsp
    mov  r8, rdi           ; save original dst
    mov  rcx, rdx
    test rcx, rcx
    jz   .done
.loop:
    movzx eax, byte [rsi]
    mov  [rdi], al
    inc  rsi
    inc  rdi
    dec  rcx
    jnz  .loop
.done:
    mov  rax, r8           ; return original dst
    pop  rbp
    ret

Performance: ~1 byte per clock cycle (at L1 cache hit rates). The loop itself has 6 instructions and a dependency chain through RCX.

Implementation 2: Word-Aligned Copy (Quadword at a Time)

For large copies, processing 8 bytes at a time is 8× faster than byte-at-a-time:

; memcpy_v2: 8-bytes-at-a-time where possible
memcpy_v2:
    mov  r8, rdi           ; save dst for return
    mov  rcx, rdx          ; byte count

    ; Handle large portion in 8-byte chunks
    mov  rax, rcx
    shr  rax, 3            ; rax = n / 8 (number of quadwords)
    jz   .byte_tail        ; no full qwords

.qword_loop:
    mov  r9, [rsi]         ; load 8 bytes
    mov  [rdi], r9         ; store 8 bytes
    add  rsi, 8
    add  rdi, 8
    dec  rax
    jnz  .qword_loop

.byte_tail:
    and  rcx, 7            ; rcx = n % 8 (remaining bytes)
    jz   .done
.byte_loop:
    movzx eax, byte [rsi]
    mov  [rdi], al
    inc  rsi
    inc  rdi
    dec  rcx
    jnz  .byte_loop

.done:
    mov  rax, r8
    ret

Performance: ~8 bytes per clock cycle (main loop). The tail loop handles the remainder. For aligned data, this is close to L1 cache bandwidth limit.

Caveat: This assumes the source and destination are both qword-aligned. Unaligned loads/stores are supported on x86-64 (unlike RISC architectures) but may have a small performance penalty.

Implementation 3: REP MOVSQ

; memcpy_v3: using REP MOVSQ
memcpy_v3:
    mov  r8, rdi           ; save dst

    ; Copy quadwords
    mov  rcx, rdx
    shr  rcx, 3            ; rcx = n / 8
    cld
    rep  movsq             ; copy rcx*8 bytes as quadwords

    ; Handle remainder
    mov  rcx, rdx
    and  rcx, 7            ; rcx = n % 8
    rep  movsb             ; copy remaining bytes

    mov  rax, r8
    ret

Performance: rep movsq on modern Intel (Ivy Bridge+) is hardware-accelerated via the "fast string" unit. It achieves throughput close to memory bandwidth for large copies, significantly outperforming the loop-based versions.

For small copies (< 16 bytes), rep movsq has startup overhead that makes it slower than explicit MOVs. For copies above ~64 bytes, it is typically the fastest purely scalar approach.

Implementation 4: SSE2 movdqu (16 Bytes at a Time)

; memcpy_v4: SSE2 version, 16 bytes per iteration
; Requires: n >= 16, or handle small copies separately

memcpy_v4:
    mov  r8, rdi           ; save dst

    mov  rcx, rdx
    shr  rcx, 4            ; rcx = n / 16 (number of 16-byte chunks)
    jz   .byte_tail

.xmm_loop:
    movdqu xmm0, [rsi]     ; load 16 bytes (unaligned)
    movdqu [rdi], xmm0     ; store 16 bytes (unaligned)
    add  rsi, 16
    add  rdi, 16
    dec  rcx
    jnz  .xmm_loop

.byte_tail:
    ; Handle remaining 0-15 bytes
    mov  rcx, rdx
    and  rcx, 15           ; rcx = n % 16
    rep  movsb

    mov  rax, r8
    ret

MOVDQU (Move Double Quadword Unaligned) loads/stores 16 bytes from an unaligned address. MOVDQA (aligned version) faults if the address is not 16-byte aligned but may be faster.

Performance: 16 bytes per clock in the XMM loop. For aligned data, switch to MOVDQA.

Performance Comparison

Measured on Intel Ice Lake, copying from L1-cached source to L1-cached destination:

(cy = CPU cycles, approximate)

Key Observations

For small copies (< 64 bytes): The overhead of the loop, REP startup, or XMM load dominates. Compilers often inline small memcpy calls as a sequence of explicit MOV instructions:

; GCC inline for memcpy of exactly 8 bytes:
mov rax, [rsi]
mov [rdi], rax

; For 16 bytes:
mov rax, [rsi]
mov [rdi], rax
mov rax, [rsi + 8]
mov [rdi + 8], rax

No loop at all. Just 2-4 MOV instructions. This is the fastest possible approach for small fixed sizes.

For medium copies (64B - 4KB): rep movsq and SSE2 perform similarly, both near the L1 cache bandwidth limit.

For large copies (> 4KB): The main bottleneck becomes DRAM bandwidth. Any implementation that processes 1 byte vs. 16 bytes per instruction makes no difference if the bottleneck is memory latency. Software prefetching (PREFETCHNTA) or non-temporal stores (MOVNTQ) can improve large-copy performance by bypassing the cache.

The glibc Production Implementation

The glibc memcpy (in sysdeps/x86_64/multiarch/memcpy-avx2-unaligned.S) uses: - AVX2 vmovdqu (32 bytes per instruction) for the main loop - Different strategies for different size thresholds (0-16, 16-32, 32-128, 128+) - Runtime CPUID dispatch to pick the best version for the current CPU - Non-temporal stores for very large copies to avoid polluting the cache

This 200-line assembly function handles every case optimally. Our Version 4 SSE2 implementation is a simplified prototype that handles the main case but lacks the threshold dispatch and alignment optimization.

What to Use in Practice

1. For fixed small sizes (≤ 32 bytes): let the compiler inline as explicit MOVs.
2. For variable sizes in non-critical code: use memcpy from libc (it is well-optimized).
3. For performance-critical custom copy: start with REP MOVSQ (good baseline), then profile to determine if SSE2/AVX2 intrinsics or handwritten assembly give meaningful improvement.
4. For security-critical copies (avoid leaving sensitive data in cache): use non-temporal stores with rep movsb and appropriate barriers.