Chapter 14: Key Takeaways

When AI Gets It Wrong — Summary Card

1. AI coding failures span a wide spectrum — from immediately obvious (hallucinated imports that crash on import) to deeply hidden (subtle logic errors that silently corrupt data for weeks). Prioritize verification based on impact, not just likelihood.

2. Hallucinated APIs are AI's most distinctive failure mode. AI invents functions, classes, and entire libraries that do not exist. Verify every unfamiliar import against official documentation or PyPI before using it. Never pip install a package you have not confirmed is legitimate.

3. Off-by-one errors are AI's most common logic bug. Always verify comparison operators (> vs >=, < vs <=) in any code involving ranges, pagination, loop bounds, or array indexing. Use Python's chained comparisons (start <= x <= end) when possible.

4. AI reproduces security anti-patterns confidently. SQL injection via string formatting, XSS from unescaped output, hardcoded secrets, path traversal, insecure deserialization, and weak cryptography all appear regularly in AI-generated code. Cross-reference security-critical code against the OWASP Top 10.

5. Performance anti-patterns hide behind clean code. N+1 queries, quadratic algorithms using .count() in loops, blocking I/O in async functions, and unnecessary data copying are common in AI output. Mentally trace time complexity for any code that handles non-trivial data volumes.

6. AI never says "I don't know." The model presents incorrect code with the same confidence as correct code. There are no disclaimers, hedges, or uncertainty signals. Develop a "that's too easy" instinct — when a hard problem gets a suspiciously clean solution, verify extra carefully.

7. AI-generated tests share the same blind spots as AI-generated code. When AI writes both code and tests, the tests often fail to cover the exact edge cases where the code has bugs. Specifically check that tests include boundary values, empty inputs, and single-element cases.

8. Use the VERIFY framework for systematic debugging. Validate imports, Examine edge cases, Review security, Inspect logic, Find performance issues, Yell about types. Work through every step, not just the ones that seem relevant.

9. Know when to abandon a conversation. If three rounds of fixes have not resolved an issue, start fresh. Provide a clear specification, state what went wrong before, and include explicit constraints to prevent the same mistakes.

10. Automate verification with layered tools. Combine type checking (mypy), linting (ruff), security scanning (bandit), import verification, and targeted tests. No single tool catches everything, but together they form an effective safety net.

11. Outdated patterns are a constant risk. AI draws from training data spanning many years and may suggest deprecated APIs, Python 2 remnants, or old framework conventions. Always check that suggested patterns are current for your Python and library versions.

12. The trust-but-verify mindset is the single most important skill. Use AI freely and confidently to accelerate development, but verify every piece of output rigorously. Treat AI code as a strong first draft from a capable but fallible colleague, never as production-ready output.

13. Build organizational resilience. Track AI-generated bugs, share failure patterns across the team, update prompt templates based on recurring errors, and maintain an "AI gotchas" document. Teams that learn from AI mistakes collectively improve faster than individuals working in isolation.

Quick Reference: When to Trust AI More vs. Less

For the full chapter, see index.md. For hands-on practice, see exercises.md.