Key Takeaways: Working with Data: CSV, JSON, and APIs

One-Sentence Summary

Modern programs consume and produce data in standard formats (CSV, JSON) and communicate with other programs over the internet via APIs — transforming isolated scripts into connected, data-driven applications.

The Three Pillars of Data Work

Pillar What It Does Key Python Tool When to Use
CSV Flat, tabular data interchange csv.DictReader / csv.DictWriter Spreadsheet data, database exports, reports
JSON Hierarchical, typed data interchange json.load / json.dump / json.loads / json.dumps API responses, config files, nested structures
APIs Program-to-program communication requests.get() / response.json() Live data, third-party services, web integration

Critical Rules

Rule Why It Matters
Always use newline="" when opening CSV files Prevents extra blank lines on Windows
Never parse CSV with line.split(",") Breaks on commas inside quoted fields
Prefer DictReader/DictWriter over reader/writer Column names are self-documenting and resilient to reordering
Use response.json() not json.loads(response.text) Same result, more concise and idiomatic
Never hard-code API keys in source code Security risk; use environment variables instead
Always set a timeout on API requests Prevents your program from hanging indefinitely
Wrap API calls in try/except Network failures happen; your program should handle them gracefully

The Data Pipeline Pattern

Load → Clean → Enrich → Aggregate → Output
  1. Load: Read from CSV files, JSON files, or API endpoints
  2. Clean: Convert types, remove bad data, handle missing values
  3. Enrich: Add data from other sources (APIs, lookups, calculations)
  4. Aggregate: Sum, count, group, average
  5. Output: Write reports as CSV, JSON, or formatted text

Common HTTP Status Codes

Code Meaning Your Action
200 Success Process the data
401 Unauthorized Check your API key
404 Not Found Check the URL
429 Rate Limited Wait and retry with exponential backoff
500 Server Error Not your fault — retry later

Threshold Concept: APIs as Connective Tissue

Before: "Apps are self-contained programs." After: "Every app talks to dozens of APIs. My Python script can talk to those same APIs. The entire internet is my function library."

JSON Functions Quick Reference

Function Input → Output Mnemonic
json.load(file) JSON file → Python dict/list "load from file"
json.loads(string) JSON string → Python dict/list "load from string"
json.dump(obj, file) Python → JSON file "dump to file"
json.dumps(obj) Python → JSON string "dump to string"

What's Next

Chapter 22: Regular expressions — pattern matching that lets you search, validate, and transform text with surgical precision. TaskFlow v2.1 gets regex-powered search.