Case Study 1: Prompt Makeover — From Vague to Precise

Overview

In this case study, we take five real-world vague prompts that developers commonly use with AI coding assistants and systematically improve them using the five-pillar framework from Chapter 8. For each prompt, we show the original version, analyze its weaknesses, apply the five pillars, present the improved version, and demonstrate how the AI's output changes dramatically as a result.

This exercise mirrors what professional developers do daily as they refine their prompting skills. The transformations are not theoretical — they represent the kind of improvements that turn frustrating AI interactions into productive ones.

Prompt 1: The Data Processor

Original Prompt (Vague)

Write a function to process the data.

Weakness Analysis

Applying the Five Pillars

Clarity: What exactly should the function do? Let us say it needs to clean and aggregate sales data.

Specificity: The function should read CSV data, remove rows with missing values, and calculate totals by category.

Context: The project uses Python with pandas, and the CSV comes from a point-of-sale system with specific columns.

Constraints: Must handle files up to 500MB, must not modify the original file, must handle encoding issues.

Output Format: Return a pandas DataFrame with specific columns, include type hints and a docstring.

Improved Prompt

Write a Python function called `aggregate_sales_data` using pandas that:

Input: A file path (str or Path) to a CSV file with columns:
  - transaction_id (str)
  - date (str, format YYYY-MM-DD)
  - category (str)
  - product_name (str)
  - quantity (int)
  - unit_price (float)

Processing steps:
1. Read the CSV with UTF-8 encoding (fall back to latin-1 if that fails)
2. Drop rows where quantity or unit_price is NaN or negative
3. Add a 'revenue' column (quantity * unit_price)
4. Group by category and calculate: total_revenue, total_quantity,
   average_unit_price, and transaction_count

Output: A pandas DataFrame with columns [category, total_revenue,
total_quantity, avg_unit_price, transaction_count], sorted by
total_revenue descending, with monetary values rounded to 2 decimal places.

Constraints:
- Handle files up to 500MB (use chunked reading if needed)
- Raise FileNotFoundError for missing files
- Raise ValueError if required columns are missing
- Include type hints and Google-style docstring

Impact on AI Output

The vague prompt would likely produce a generic function that reads some unspecified data and does minimal processing — perhaps a function that opens a file and returns its contents, or a placeholder with pass. The developer would need 4-6 follow-up prompts to get the right function.

The improved prompt produces a complete, production-ready function on the first attempt: correct pandas operations, proper error handling, chunked reading for large files, and the exact output format needed. First-attempt success rate goes from near zero to near 100%.

Prompt 2: The API Endpoint

Original Prompt (Vague)

Make an API endpoint for users.

Weakness Analysis

Applying the Five Pillars

We transform this into a precise specification for a user search endpoint in a FastAPI application.

Improved Prompt

I'm building a REST API with FastAPI 0.104+ and SQLAlchemy 2.0 (async).

Existing User model:
class User(Base):
    __tablename__ = "users"
    id: Mapped[int] = mapped_column(primary_key=True)
    email: Mapped[str] = mapped_column(String(120), unique=True)
    full_name: Mapped[str] = mapped_column(String(200))
    role: Mapped[str] = mapped_column(String(20), default="member")
    is_active: Mapped[bool] = mapped_column(default=True)
    created_at: Mapped[datetime] = mapped_column(default=func.now())

Write a GET endpoint at /api/v1/users with the following:

Query parameters:
- search (optional str): Filters users where email or full_name contains the term (case-insensitive)
- role (optional str): Filters by exact role match
- is_active (optional bool, default True): Filters by active status
- page (int, default 1, min 1): Page number
- page_size (int, default 20, min 1, max 100): Items per page

Response (200): JSON with structure:
{
  "users": [{"id": int, "email": str, "full_name": str, "role": str, "is_active": bool}],
  "total": int,
  "page": int,
  "page_size": int,
  "total_pages": int
}

Constraints:
- Use Pydantic models for request validation and response serialization
- Use async SQLAlchemy session (injected via Depends)
- Do NOT return created_at in the user list (privacy consideration)
- Return 422 for invalid query parameters (FastAPI default)
- Add a docstring to the endpoint function that shows up in the OpenAPI docs

Create the Pydantic response models, the endpoint function, and
show an example of the router setup.

Impact on AI Output

The vague prompt might produce a basic CRUD endpoint in Flask, Django, or FastAPI (the AI must guess), possibly with a POST method for creating users rather than the GET endpoint for listing them. The developer gets code they cannot use without significant modification.

The improved prompt produces an endpoint that plugs directly into the existing FastAPI application with the correct models, pagination logic, filtering, and response format. The developer saves 30-45 minutes of iteration.

Prompt 3: The Error Handler

Original Prompt (Vague)

Add error handling to my code.

Weakness Analysis

Improved Prompt

Add error handling to the following Python function that downloads files
from URLs:

def download_file(url: str, dest_path: str) -> None:
    response = requests.get(url)
    with open(dest_path, 'wb') as f:
        f.write(response.content)

Error handling requirements:

1. Network errors (requests.ConnectionError, requests.Timeout):
   - Retry up to 3 times with exponential backoff (1s, 2s, 4s)
   - Log each retry attempt with the retry number and wait time
   - Raise a custom DownloadError after all retries are exhausted

2. HTTP errors (4xx, 5xx status codes):
   - For 404: raise FileNotFoundError with the URL in the message
   - For 429 (rate limited): wait for Retry-After header value, then retry once
   - For 5xx: include in the retry logic above
   - For other 4xx: raise DownloadError immediately (no retry)

3. File system errors:
   - Create parent directories if they don't exist
   - If dest_path is not writable, raise PermissionError with a helpful message
   - If disk is full (OSError), clean up any partial file before re-raising

4. Data integrity:
   - If the response includes a Content-Length header, verify the downloaded
     file size matches; raise DownloadError if not

Constraints:
- Use the logging module (not print) for all log messages
- Define the custom DownloadError exception class
- Include type hints and a docstring
- Keep the function signature the same (add optional retry/timeout params if needed)

Return the improved function along with the DownloadError class definition.

Impact on AI Output

The vague prompt would likely wrap the entire function body in a single try: ... except Exception: ... block — the most generic and least useful form of error handling. It would not include retries, specific exception types, data integrity checks, or cleanup logic.

The improved prompt produces a robust, production-quality download function with comprehensive error handling that addresses every failure mode the developer cares about. The difference is not incremental — it is the difference between toy code and production code.

Prompt 4: The Test Suite

Original Prompt (Vague)

Write tests for the user service.

Weakness Analysis

Improved Prompt

Write pytest tests for the following UserService class:

class UserService:
    def __init__(self, db_session: Session):
        self.db = db_session

    def create_user(self, email: str, password: str, name: str) -> User:
        """Creates a user. Raises ValueError for duplicate email,
        invalid email format, or password shorter than 8 chars."""
        ...

    def get_user_by_id(self, user_id: int) -> User | None:
        """Returns user or None if not found."""
        ...

    def update_user(self, user_id: int, **kwargs) -> User:
        """Updates allowed fields (name, email). Raises ValueError
        for unknown fields. Raises LookupError if user not found."""
        ...

    def deactivate_user(self, user_id: int) -> None:
        """Sets is_active=False. Raises LookupError if not found.
        No-op if already deactivated."""
        ...

Test requirements:

For create_user:
- Happy path: valid inputs create a user with hashed password
- Duplicate email raises ValueError with message containing "duplicate"
- Invalid email format (no @) raises ValueError
- Short password (< 8 chars) raises ValueError
- Empty name raises ValueError

For get_user_by_id:
- Returns user object for existing user
- Returns None for non-existent ID
- Returns None for negative ID

For update_user:
- Successfully updates name
- Successfully updates email
- Raises ValueError for unknown field like "role"
- Raises LookupError for non-existent user

For deactivate_user:
- Deactivates an active user (is_active becomes False)
- No-op for already deactivated user (no error)
- Raises LookupError for non-existent user

Conventions:
- Use pytest fixtures for the db session (mock with MagicMock)
- Use a fixture for creating a test user
- Test names: test_{method}_{scenario}_{expected_result}
- Use pytest.raises for exception tests
- Each test should have a single assert (or pytest.raises)
- Group tests by method using classes: TestCreateUser, TestGetUserById, etc.
- Include a brief docstring for each test class

Do NOT test the database layer itself — mock the session.

Impact on AI Output

The vague prompt might produce 3-4 basic tests that check the happy path for one or two methods, likely with incomplete mocking and inconsistent naming. The developer would need to write or request the remaining 15+ tests individually.

The improved prompt produces a complete test suite with 17+ well-organized tests covering happy paths, edge cases, and error conditions, all following the team's conventions. The test file is immediately usable in CI/CD.

Prompt 5: The Refactoring Task

Original Prompt (Vague)

Refactor this code to be better.

Weakness Analysis

Improved Prompt

Refactor the following Python function to improve readability and
maintainability. The function works correctly — do NOT change its behavior.

def proc(d):
    r = []
    for i in d:
        if i['t'] == 'A' and i['v'] > 100 and i['s'] != 'inactive':
            x = i['v'] * 0.15
            if i.get('p', False):
                x = x * 0.9
            r.append({'id': i['id'], 'tax': round(x, 2), 'net': round(i['v'] - x, 2)})
        elif i['t'] == 'B' and i['v'] > 50 and i['s'] != 'inactive':
            x = i['v'] * 0.10
            if i.get('p', False):
                x = x * 0.9
            r.append({'id': i['id'], 'tax': round(x, 2), 'net': round(i['v'] - x, 2)})
    return r

Context: This processes financial transactions. 't' is transaction type,
'v' is value, 's' is status, 'p' is premium discount eligibility.

Specific improvements needed:
1. Rename all variables to be descriptive (not single letters)
2. Extract the tax rate lookup into a dictionary or config
3. Extract the tax calculation into a helper function to eliminate duplication
4. Add type hints (use TypedDict for the transaction and result dictionaries)
5. Add a Google-style docstring explaining the business logic
6. Replace magic numbers (0.15, 0.10, 0.9, 100, 50) with named constants

Constraints:
- The refactored function must produce identical output for identical input
- Keep it as a single module (no class needed for this)
- Use Python 3.10+ syntax (match/case is OK but not required)

For each change, add a brief comment explaining why the change improves
the code.

Impact on AI Output

The vague prompt might rename a few variables or add a comment, with no structural improvements. Worse, it might "improve" the code by changing its behavior, introducing bugs in the name of optimization.

The improved prompt produces a well-structured refactoring with named constants, a helper function for tax calculation, descriptive names, type hints, and a docstring — all while preserving the exact behavior. The developer receives clean, documented, maintainable code with explanations of each change.

Key Lessons from the Makeover Process

Lesson 1: Vague Prompts Shift Work, They Don't Eliminate It

A vague prompt does not save time — it shifts the work from writing a good prompt to debugging, correcting, and iterating on bad output. The developer spends the same (or more) total time; they just spend it in a more frustrating way.

Lesson 2: The Five Pillars Are a Diagnostic Tool

When the AI produces disappointing output, do not blame the AI. Instead, score your prompt on each pillar. The lowest-scoring pillar is almost certainly the root cause. This transforms a frustrating experience into a learning opportunity.

Lesson 3: Investment in Prompt Quality Has Diminishing Returns

There is a point of diminishing returns. Moving from a Level 1 to Level 3 prompt is transformative. Moving from Level 3 to Level 4 is helpful. Moving from Level 4 to Level 5 is only worthwhile for high-risk, complex tasks. Calibrate your investment to the task's importance and complexity.

Lesson 4: Structure Beats Length

Notice that the improved prompts are not just longer — they are structured. Bullet points, numbered lists, labeled sections, and clear formatting make the prompt easy for both the AI and the developer to parse. A structured 200-word prompt outperforms an unstructured 500-word paragraph.

Lesson 5: Specificity About "What" Frees the AI on "How"

The best improved prompts specify what the code should do without prescribing how it should do it at the implementation level. They define the interface, behavior, and constraints, then let the AI choose the best implementation approach. This leverages the AI's strengths while maintaining developer control over requirements.

Try It Yourself

Take one of your recent prompts that produced unsatisfying results. Score it on the five pillars (1-5 each). Identify the weakest pillar. Rewrite the prompt to strengthen that pillar, keeping the others at least as strong. Submit the improved prompt and compare the results. Repeat this process until you consistently score 3+ on all pillars.