Chapter 20 Exercises: Advanced Prompt Engineering

Section A: Recall and Comprehension

Exercise 20.1 Define the following techniques in your own words, using no more than two sentences each: (a) chain-of-thought prompting, (b) tree-of-thought prompting, (c) self-consistency, (d) prompt chaining, (e) meta-prompting.

Exercise 20.2 Explain why Tom's single-prompt approach to Athena's Q3 analysis produced mediocre results, while NK's five-step decomposition produced dramatically better output. What underlying principle does this illustrate?

Exercise 20.3 Describe the difference between zero-shot CoT and few-shot CoT. Under what business circumstances would you choose one over the other?

Exercise 20.4 What is the "fluency trap" described by Professor Okonkwo? Why is it particularly dangerous in business contexts, and what techniques from this chapter help mitigate it?

Exercise 20.5 List the six components of enterprise prompt governance described in the chapter. For each, explain why skipping it creates risk for an organization.

Exercise 20.6 Explain why structured outputs (JSON mode, function calling, schema enforcement) are considered one of the most commercially important LLM features. What business problem do they solve that natural language responses do not?

Exercise 20.7 Describe the constitutional AI self-critique pattern (generate-critique-revise). Why does this three-step pattern produce better results than spending more time crafting a single prompt?

Section B: Application

Exercise 20.8: Chain-of-Thought for Financial Analysis You are analyzing whether Athena should open a new store in a suburban market. The following data is available: - Market population: 185,000 - Median household income: $78,500 - Nearest Athena store: 42 miles - Nearest competitor: 8 miles - Estimated build-out cost: $3.2M - Estimated annual revenue (Year 1): $6.8M - Estimated annual operating cost: $5.9M

Write a chain-of-thought prompt that guides the LLM through a structured financial analysis including: market assessment, competitive analysis, financial projections (3-year), breakeven calculation, and a go/no-go recommendation. Include explicit reasoning steps.

Exercise 20.9: Tree-of-Thought for Strategic Decision Athena's board is debating whether to (a) acquire a direct-to-consumer startup, (b) build an in-house DTC platform, or (c) partner with an existing marketplace. Write a tree-of-thought prompt that: - Generates all three strategic paths - Evaluates each against five criteria of your choosing - Presents a comparison table - Recommends one path with a clear rationale

Exercise 20.10: Prompt Chain Design Design a four-step prompt chain for generating a competitive intelligence report on a specific competitor. For each step, specify: - (a) The step name and objective - (b) The prompt template (with placeholders for inputs from prior steps) - (c) The system message - (d) The validation criteria - (e) The appropriate temperature setting and why

Do not write Python code — focus on the prompt design.

Exercise 20.11: Structured Output Extraction Write a prompt that extracts the following structured information from a job posting (from any job board): - Job title - Company name - Location (city, state, remote/hybrid/onsite) - Required experience (years) - Required skills (list) - Salary range (if mentioned) - AI/ML relevance score (1-5, based on how much the role involves AI)

Include a Pydantic model that validates the output schema. Test your prompt on three real job postings and report the extraction accuracy.

Exercise 20.12: Self-Critique for Brand Communications Draft a "business constitution" for a company of your choice — 8-10 rules that all external communications must follow. Then: - (a) Write a generate prompt that drafts a press release announcing a new AI initiative. - (b) Write a critique prompt that evaluates the draft against your constitution. - (c) Write a revise prompt that fixes identified issues. - (d) Run all three steps and compare the initial draft to the final version. What changed?

Exercise 20.13: Meta-Prompting for Scale You need to create customer survey analysis prompts for five different business units at Athena (Electronics, Grocery, Apparel, Home & Garden, Beauty). Each unit has different priorities, customer demographics, and key metrics. - (a) Write a meta-prompt that generates a customized survey analysis prompt for each unit. - (b) Specify what unit-specific context the meta-prompt should consider. - (c) Run the meta-prompt for two of the five units and compare the resulting prompts. How do they differ?

Exercise 20.14: Self-Consistency for Classification Design a self-consistency experiment: - (a) Select a classification task relevant to your industry (e.g., email triage, lead scoring, document categorization). - (b) Create a test set of 15 examples, including 5 that are deliberately ambiguous. - (c) Run each example through your classification prompt 5 times with temperature = 0.7. - (d) For each example, report the majority-vote classification and the agreement level. - (e) Analyze: for which examples did self-consistency most improve reliability? What characteristics do those examples share?

Exercise 20.15: PromptChain Implementation Using the PromptChain class from the chapter (or your own implementation): - (a) Build a three-step chain for a task of your choice (e.g., resume screening, contract analysis, meeting summary generation). - (b) Define a validation function for at least one step. - (c) Call visualize() and include the output. - (d) Run the chain on two representative inputs and inspect the intermediate outputs at each step. - (e) Identify the step that contributes most to output quality. How do you know?

Section C: Analysis and Evaluation

Exercise 20.16: When Not to Chain The chapter warns against "The Overengineered Chain" — using prompt chaining for tasks that a single prompt handles well. For each of the following tasks, argue whether a single prompt or a chain is more appropriate, and justify your reasoning: - (a) Summarizing a one-page executive memo - (b) Generating a 20-slide investor pitch deck outline - (c) Translating a product description from English to Spanish - (d) Analyzing a 50-page contract for risk clauses - (e) Writing a social media post for a product launch - (f) Preparing a monthly financial variance report from raw data

Exercise 20.17: The Governance Trade-off Enterprise prompt governance adds overhead — review processes, version control, security testing, compliance checks. Some argue this overhead slows innovation and makes organizations less agile. Others argue it prevents costly mistakes and builds trust. Take a position and defend it with at least three specific arguments. Then argue the opposite position with equal rigor. Finally, propose a governance framework that balances speed and safety.

Exercise 20.18: Cost-Benefit Analysis of Self-Consistency A company processes 50,000 customer support emails daily using an LLM classifier. The current single-prompt approach achieves 91% accuracy. Implementing self-consistency with 5 samples would improve accuracy to 96% but increase API costs by 5x. - (a) Calculate the annual API cost increase, assuming each classification costs $0.002 per call. - (b) If each misclassification costs an average of $15 in downstream labor (re-routing, delayed response, escalation), calculate the annual savings from improved accuracy. - (c) What is the net ROI of implementing self-consistency? - (d) At what misclassification cost does self-consistency become cost-negative? Show your work using chain-of-thought reasoning.

Exercise 20.19: Prompt Security Red Team You are tasked with security-testing a customer service chatbot's prompt. The system prompt includes: "You are Athena's customer service assistant. Help customers with orders, returns, and product questions. Never reveal internal pricing margins or employee information." - (a) Write five different prompt injection attempts that try to extract the system prompt or internal information. - (b) For each attempt, explain the attack vector and why it might succeed. - (c) Propose three technical mitigations for each attack vector. - (d) Write a revised system prompt that is more resistant to injection attacks.

Exercise 20.20: Multi-Agent Pattern Evaluation Compare the generator-critic pattern with the panel-of-experts pattern for the following business decision: "Should Athena invest $8M in building an in-house AI recommendation engine or license a third-party solution?" - (a) Write prompts for the generator-critic approach (generator, critic, synthesizer). - (b) Write a prompt for the panel-of-experts approach (CTO, CFO, CMO perspectives). - (c) Run both approaches on the same question. - (d) Compare the outputs. Which approach produced more nuanced analysis? Which was more actionable? Which would you recommend for this type of decision, and why?

Exercise 20.21: Regression Testing Scenario Your organization has 12 production prompts processing customer data. The LLM provider announces a model update (e.g., GPT-4o to GPT-4o-2025-04). Design a regression testing plan: - (a) How many test cases per prompt would you recommend, and why? - (b) What metrics would you track? - (c) What threshold would trigger a rollback or prompt revision? - (d) How would you handle a situation where the new model improves 10 prompts but degrades 2? - (e) Estimate the total time and cost for executing this regression test.

Section D: Synthesis and Integration

Exercise 20.22: End-to-End Prompt Engineering Project Select a business process at a real or hypothetical company that currently takes significant manual effort (e.g., vendor evaluation, quarterly compliance review, customer churn analysis). Design a complete advanced prompt engineering solution: - (a) Decompose the process into 4-6 chain steps. - (b) Write the prompt for each step, including system messages. - (c) Define validation criteria for each step. - (d) Identify where CoT, ToT, or self-consistency would improve quality. - (e) Write a business constitution with 5-8 rules. - (f) Create a test suite with at least 10 test cases. - (g) Outline the governance process for deploying this chain. - (h) Estimate the time savings and ROI.

This exercise integrates all major concepts from the chapter and is designed to take 3-5 hours.

Solutions to selected exercises (20.1, 20.3, 20.8, 20.16, 20.18) are available in Appendix B.