Autoregressive text decoder with learned positional embeddings - Cross-attention to encoder outputs - Predicts tokens from a byte-level BPE vocabulary