Chapter 30 Exercises: Verifying AI Output — Fact-Checking Workflows

Instructions

These exercises build your practical verification workflow skills. The exercises progress from conceptual understanding through hands-on practice. Some exercises require you to use actual verification tools — have a browser open with your verification toolkit available.

Part 1: Triage Practice

Exercise 1.1: Marking the Claims

Read the following AI-generated paragraph and mark every element that you would flag for verification. Categorize each as Tier 1 (verify thoroughly), Tier 2 (verify key claims), or Tier 3 (spot-check or proceed):

"The global remote work technology market was valued at $38.7 billion in 2023 and is projected to grow at a compound annual growth rate of 17.3% through 2028, according to a Grand View Research report. This growth is driven by the shift toward hybrid work models, with 58% of American workers now working in a hybrid arrangement at least part of the time (Gallup, 2024). Academic research has reinforced the productivity case: a landmark study by Stanford economist Nicholas Bloom (2015) found a 13% productivity increase among remote workers in a randomized controlled trial, findings that have been replicated in multiple subsequent studies. Importantly, the benefits appear asymmetric — knowledge workers and those in roles with clear deliverables show the strongest gains, while workers in highly collaborative creative roles show more mixed results."

List each claim you flagged, its category, and how you would verify it.

Exercise 1.2: Triage Calibration

For each of the following tasks, describe what a triage pass would look like: what would you be looking for, and what risk tier would the claims likely fall into?

a) AI-drafted email inviting colleagues to a team meeting b) AI-generated market research summary with competitor revenue figures c) AI-assisted blog post explaining the concept of compound interest d) AI-produced regulatory compliance checklist for GDPR data processing e) AI-generated list of potential project names for an internal initiative

Part 2: Verification Method Practice

Exercise 2.1: Citation Verification Drill

Perform full verification on the following citations using the four-step method from Section 4. Record the result of each step.

a) Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.

b) Ariely, D., & Jones, S. (2009). Predictably Irrational: The Hidden Forces That Shape Our Decisions. Journal of Behavioral Economics, 12(3), 44-67.

c) Duhigg, C. (2012). The Power of Habit: Why We Do What We Do in Life and Business. Random House.

For each: Is this real? If not, what elements are real and what are fabricated?

Exercise 2.2: Statistical Verification

For each statistic below, find the original source and verify whether the AI-provided figure is accurate:

a) "The United States has approximately 330 million people as of 2023."

b) "Remote work increased by over 150% between 2019 and 2020 according to the U.S. Census Bureau."

c) "The average cost of a data breach in 2023 was $4.45 million, according to IBM's annual Cost of a Data Breach report."

For each: What source did you check? Is the figure accurate? If not, what is the correct figure?

Exercise 2.3: Regulatory Claim Verification

Verify each of the following regulatory claims against official sources:

a) "GDPR requires organizations to report personal data breaches to the relevant supervisory authority within 72 hours of becoming aware of the breach."

b) "Under HIPAA, covered entities must provide patients access to their medical records within 30 days of request."

c) "The FTC requires that material connections between endorsers and brands be clearly disclosed."

For each: What official source did you use? Is the claim accurate? Are there important caveats the AI omission might be missing?

Part 3: Building Your Personal Toolkit

Exercise 3.1: Domain Toolkit Construction

For your specific professional domain, build a personal verification toolkit. Create a document with the following structure:

My Verification Toolkit

Domain: [Your field]

Complete this table with actual sources you will use. Bookmark all of them.

Exercise 3.2: Triage Threshold Setting

Write a personal "Verification Policy" document — one page or less — that specifies: - What types of AI-assisted content you regularly produce - For each type, what claims qualify as Tier 1 (must verify), Tier 2 (verify central claims), and Tier 3 (spot-check) - What your time budget for verification is for each content type - What you do when a claim fails verification

This document should be specific to your actual work, not generic. The act of writing it forces the calibration thinking this chapter is asking for.

Part 4: The TVD Framework in Practice

Exercise 4.1: Full TVD Walkthrough

Choose a recent AI-assisted work product you have produced or will produce — something with specific factual content (not purely creative work). Apply the full Triage-Verify-Document framework:

Triage: Mark every specific factual claim and assign tier ratings. Verify: Work through all Tier 1 claims and a sample of Tier 2 claims using your toolkit. Document: Create a verification log entry for each claim you checked.

Write a reflection: What did you find? How much time did the process take? What would have gone wrong without it?

Exercise 4.2: The 15-Minute Verification Audit

Set a timer for 15 minutes. Take a 500-word AI-generated piece (generate one if needed, on a topic in your field). Perform:

• Full triage pass: mark all claims
• Tier 1 verification for any high-stakes claims
• Brief documentation

How much can you verify in 15 minutes? What is your verification throughput? What strategies (pre-loaded toolkit, efficient search habits) would increase your speed?

Part 5: Workflow Design

Exercise 5.1: Workflow Mapping

Draw a workflow diagram (on paper or digitally) for your current AI-assisted work process. Now redesign it to incorporate the TVD framework as a distinct step. Where does it fit? What needs to change in your scheduling, project planning, or deliverable timelines to accommodate it?

Exercise 5.2: The Verification Budget

For the types of projects you typically do with AI assistance, calculate the verification time budget using the heuristics from Section 6: - High-stakes: 20-30% of project time - Medium-stakes: 10-15% - Low-stakes: 5%

Does this change how you estimate project time? What would need to change in your scheduling, pricing (if you're freelance), or team communication to make this time real?

Exercise 5.3: Team Verification Standards

If you work with a team, draft a one-page "AI Output Verification Standards" document that could be shared with your team. It should specify: - What types of AI-assisted content require what level of verification - Who is responsible for verification in different workflow contexts - What a verification log should include - How to handle content that fails verification

Part 6: AI-Assisted Verification (With Caveats)

Exercise 6.1: Circularity Testing

Ask an AI model whether a specific claim it generated is accurate. Note its response. Then independently verify the claim. Did the AI's self-assessment correlate with accuracy? What does this tell you about using AI for self-verification?

Exercise 6.2: Cross-Model Comparison

Take a specific factual claim generated by one AI model and ask a different AI model the same question. Note whether the answers differ. If they differ on specific facts, what does the disagreement tell you? If they agree, does agreement provide any reliability assurance?

Exercise 6.3: AI as Verification Support

Identify two legitimate uses of AI assistance in your verification workflow — tasks where AI genuinely helps without the circularity problem. (Examples: summarizing a lengthy primary source document you've found; translating a foreign-language source; helping you formulate search queries.) Document these as a "where AI helps verification" note for your personal toolkit.

Part 7: Reflection

Exercise 7.1: Pre/Post Assessment

Before reading this chapter, how would you have described your verification practice? After completing these exercises, how has it changed? Be specific about what you now do differently.

Exercise 7.2: The Cost of Not Verifying

Think about one scenario in your professional context where an AI error would have significant consequences if not caught. Walk through: what is the error, what would have happened if it wasn't caught, what is the cost (professional, financial, relational, reputational)? How does that cost compare to the time cost of the verification workflow?

Exercise 7.3: Building the Habit

Research on habit formation suggests that new workflows take 4-6 weeks of consistent practice to become automatic. Design a 30-day verification habit plan: what specifically will you do each week to build and reinforce the TVD practice in your actual work context?