Further Reading: Large Language Models

The following resources offer deeper engagement with the topics covered in this chapter, organized from most accessible to most technical.

Accessible Introductions

"ChatGPT Is a Blurry JPEG of the Web" by Ted Chiang (The New Yorker, 2023) Science fiction author Ted Chiang offers one of the best non-technical explanations of what LLMs actually do, using the analogy of lossy compression. This essay is widely cited and genuinely illuminating. Available online at newyorker.com.

"You Are Not a Parrot" by Elizabeth Weil (New York Magazine, 2023) A profile of linguist Emily Bender, one of the authors of the "stochastic parrots" paper, exploring her concerns about LLM hype and the risks of anthropomorphizing AI systems. Accessible and thought-provoking.

"A Jargon-Free Explanation of How AI Large Language Models Work" by Timothy B. Lee and Sean Trott (Ars Technica, 2023) A clear, step-by-step walkthrough of how LLMs work, written for a general audience. Covers tokenization, attention, and training in plain language with helpful diagrams.

Deeper Dives

"On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?" by Emily Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell (FAccT 2021) The original "stochastic parrots" paper discussed in Section 5.6. Examines environmental costs, training data risks, and the social implications of large language models. Available through the ACM Digital Library.

"Sparks of Artificial General Intelligence: Early Experiments with GPT-4" by Sebastien Bubeck et al. (Microsoft Research, 2023) A counterpoint to the stochastic parrot view, this paper argues that GPT-4 displays "sparks" of general intelligence. Reading it alongside the Bender et al. paper gives you both sides of the debate. Available on arXiv.

"Language Models are Few-Shot Learners" by Tom Brown et al. (NeurIPS 2020) The original GPT-3 paper that demonstrated the power of scaling up language models. Technical in places but includes accessible sections on capabilities and limitations. Available on arXiv.

The Human Side

"OpenAI Used Kenyan Workers on Less Than $2 Per Hour to Make ChatGPT Less Toxic" by Billy Perrigo (TIME, 2023) Investigative reporting on the labor behind RLHF — the human workers who reviewed toxic content to train the model's safety features. Essential reading for understanding the human cost of AI safety.

"AI's Language Problem" by Douglas Heaven (MIT Technology Review, 2023) Explores the gap between what LLMs appear to understand and what they actually process, with insights from linguists and cognitive scientists.

Books for Extended Study

"The Alignment Problem: Machine Learning and Human Values" by Brian Christian (W.W. Norton, 2020) A comprehensive, readable exploration of the challenge of making AI systems behave in ways consistent with human values. Covers RLHF, reward modeling, and the philosophical puzzles of alignment.

"Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence" by Kate Crawford (Yale University Press, 2021) A broader examination of AI's material infrastructure — the data, labor, and natural resources behind the technology. Provides essential context for understanding who builds these systems and at what cost.

Interactive Resources

"The Illustrated Transformer" by Jay Alammar (jalammar.github.io) A visual, step-by-step walkthrough of the transformer architecture. Uses animations and diagrams to make the attention mechanism intuitive. Requires no math background to appreciate the visuals, though some sections go into technical detail.

OpenAI Tokenizer Tool (platform.openai.com/tokenizer) An interactive tool that lets you type text and see how it gets broken into tokens. Hands-on experimentation with tokenization helps make the concept concrete.