Chapter 24: Further Reading

Foundational NLP Textbooks

  • Jurafsky, D., & Martin, J. H. (2024). Speech and Language Processing (3rd ed. draft). Pearson. The definitive NLP textbook, covering everything from tokenization and n-grams through transformers and large language models. Freely available at https://web.stanford.edu/~jurafsky/slp3/. Chapters on sentiment analysis, information extraction, and language models are directly relevant to this chapter.

  • Eisenstein, J. (2019). Introduction to Natural Language Processing. MIT Press. A modern, mathematically rigorous introduction to NLP with excellent coverage of text classification, sequence models, and representation learning. Particularly strong on the transition from classical to neural methods.

Sentiment Analysis

  • Hutto, C. J., & Gilbert, E. (2014). "VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text." Proceedings of the 8th International AAAI Conference on Weblogs and Social Media (ICWSM). The original VADER paper. Describes the lexicon construction, heuristic rules for handling punctuation/capitalization/degree modifiers, and validation on social media text. Essential reading for understanding the strengths and limitations of rule-based sentiment.

  • Loughran, T., & McDonald, B. (2011). "When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10-Ks." Journal of Finance, 66(1), 35-65. Demonstrates that generic sentiment dictionaries (like Harvard's General Inquirer) misclassify financial text. Words like "liability," "tax," and "capital" are negative in general English but neutral in finance. The paper develops a finance-specific lexicon and establishes the importance of domain-adapted sentiment tools -- a lesson that applies directly to prediction market text.

  • Tetlock, P. C. (2007). "Giving Content to Investor Sentiment: The Role of Media in the Stock Market." Journal of Finance, 62(3), 1139-1168. Landmark paper showing that the pessimism in Wall Street Journal "Abreast of the Market" columns predicts downward pressure on market prices and increased trading volume. Establishes the empirical case that text sentiment leads market movements.

Transformer Models and BERT

  • Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). "Attention Is All You Need." Advances in Neural Information Processing Systems, 30. The paper that introduced the Transformer architecture. Understanding the self-attention mechanism is prerequisite knowledge for working with BERT, GPT, and all modern NLP models. The core equation: $\text{Attention}(Q,K,V) = \text{softmax}(QK^T / \sqrt{d_k})V$.

  • Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). "BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding." Proceedings of NAACL-HLT, 4171-4186. The BERT paper that revolutionized NLP by introducing bidirectional pre-training with masked language modeling. Understanding BERT is essential for practical transformer-based sentiment analysis.

  • Liu, Y., Ott, M., Goyal, N., et al. (2019). "RoBERTa: A Robustly Optimized BERT Pretraining Approach." arXiv:1907.11692. Shows that BERT was significantly undertrained and that careful optimization of pre-training (more data, longer training, larger batches, dynamic masking) produces substantially better results. RoBERTa is often the recommended starting point for fine-tuning.

  • Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019). "DistilBERT, a Distilled Version of BERT: Smaller, Faster, Cheaper and Lighter." arXiv:1910.01108. Introduces knowledge distillation for compressing BERT into a model that is 60% smaller and 60% faster while retaining 97% of BERT's language understanding. The go-to model for latency-sensitive applications like real-time trading signal generation.

Text in Financial and Political Markets

  • Bollen, J., Mao, H., & Zeng, X. (2011). "Twitter Mood Predicts the Stock Market." Journal of Computational Science, 2(1), 1-8. Demonstrates that aggregate Twitter mood dimensions (calm, alert, sure, vital, kind, happy) can predict DJIA movements with 87.6% accuracy in direction. While the specific claim has been debated, the paper established a research program on social media sentiment and market prediction.

  • Beauchamp, N. (2017). "Predicting and Interpolating State-Level Polls Using Twitter Sentiment." American Journal of Political Science, 61(2), 490-503. Shows that Twitter sentiment about political candidates predicts state-level polling averages and election outcomes. Directly relevant to building sentiment features for political prediction markets.

  • Gentzkow, M., Kelly, B., & Taddy, M. (2019). "Text as Data." Journal of Economic Literature, 57(3), 535-574. A comprehensive survey of how economists use text data, covering dictionary-based methods, supervised learning, topic models, and embeddings. Provides a rigorous framework for thinking about text as a quantitative variable in economic models.

Topic Modeling and Text Representation

  • Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). "Latent Dirichlet Allocation." Journal of Machine Learning Research, 3, 993-1022. The foundational paper on LDA topic modeling. Topic models can decompose prediction market news into interpretable themes, enabling topic-conditioned sentiment analysis.

  • Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013). "Distributed Representations of Words and Phrases and Their Compositionality." Advances in Neural Information Processing Systems, 26. Introduces Word2Vec skip-gram and negative sampling. Word embeddings are the conceptual foundation for all modern text representation, including transformer contextual embeddings.

LLMs as Forecasters

  • Halawi, D., Chernoff, F., Willner, M., & Steinhardt, J. (2024). "Approaching Human-Level Forecasting with Language Models." arXiv:2402.18563. A rigorous evaluation of LLM forecasting capability, showing that properly prompted retrieval-augmented LLMs can approach competitive human forecaster accuracy on diverse questions. Essential context for understanding the potential and limits of LLMs in prediction market analysis.

  • Schoenegger, P., Park, P., Karger, E., & Tetlock, P. (2024). "AI-Augmented Predictions: LLM Assistants Improve Human Forecasting Accuracy." arXiv:2402.07862. Evaluates LLMs as assistants to human forecasters rather than standalone predictors. Finds that LLM assistance improves human forecasting accuracy, with implications for how prediction market traders might integrate LLM tools into their workflows.

Software Documentation

  • HuggingFace Transformers Documentation. https://huggingface.co/docs/transformers/ The essential reference for loading pre-trained models, tokenizers, and pipelines. Includes tutorials on fine-tuning for text classification.

  • NLTK Documentation. https://www.nltk.org/ Reference for classical NLP tools: tokenization, stemming, lemmatization, POS tagging, and the VADER sentiment analyzer.

  • spaCy Documentation. https://spacy.io/ Industrial-strength NLP library for tokenization, named entity recognition, dependency parsing, and text classification. Faster than NLTK for pipeline applications.

  • scikit-learn Text Feature Extraction. https://scikit-learn.org/stable/modules/feature_extraction.html#text-feature-extraction Documentation for TfidfVectorizer, CountVectorizer, and HashingVectorizer with practical examples and parameter tuning guidance.

  • Chapter 20: Data Collection and APIs -- Collecting the raw text data (news articles, social media posts, market data) that feeds into the NLP pipeline.

  • Chapter 23: Machine Learning for Prediction Markets -- The supervised learning framework (train/test splits, evaluation metrics, calibration) that NLP features plug into.

  • Chapter 25: Ensemble Methods and Model Stacking -- Combining NLP-derived features with polling-based and market-based models in ensemble forecasts.

  • Chapter 26: Backtesting Prediction Market Strategies -- Rigorously evaluating whether NLP-derived trading signals produce genuine alpha after accounting for transaction costs and biases.