Chapter 15: Building Command-Line Tools and Scripts

"The command line is the most powerful interface ever invented. With AI, it's now the most accessible too."

Learning Objectives

By the end of this chapter, you will be able to:

1. Analyze the architectural components of a production-quality CLI application and explain how they interact (Bloom's: Analyze)
2. Compare argparse and click for argument parsing and select the appropriate library for a given project (Bloom's: Evaluate)
3. Implement configuration management using YAML, TOML, and .env files with proper precedence hierarchies (Bloom's: Apply)
4. Design logging strategies that support both developer debugging and end-user verbose output (Bloom's: Create)
5. Build file processing pipelines that handle streaming, batch, and glob-based I/O patterns (Bloom's: Apply)
6. Integrate progress bars, spinners, and rich formatting into CLI tools using the rich library (Bloom's: Apply)
7. Construct robust error handling systems with meaningful exit codes and user-friendly error messages (Bloom's: Create)
8. Package CLI tools for distribution using setuptools and pyproject.toml with proper entry points (Bloom's: Apply)
9. Create interactive CLI features including prompts, confirmations, and menus (Bloom's: Create)
10. Synthesize all CLI components into a complete, production-ready tool using AI-assisted development (Bloom's: Create)

Introduction

Command-line tools are the backbone of software development. From git to docker to pip, the tools developers rely on every day are CLI applications. Despite the rise of graphical interfaces and web applications, the command line remains the most efficient way to automate tasks, process data, and build developer tooling.

In Chapter 6, we built a simple task manager CLI as our first vibe coding project. That application covered the basics -- argument parsing, file I/O, and user feedback. Now we are going to take those foundations and elevate them to production quality. We will learn how professional CLI tools handle configuration, logging, error reporting, packaging, and distribution. More importantly, we will see how AI coding assistants can accelerate every step of this process.

This chapter covers the complete lifecycle of a CLI tool: from initial architecture through argument parsing, configuration management, logging, file processing, user feedback, error handling, packaging, and interactive features. Each section includes the prompts you would use to generate each component with an AI assistant, the code the AI produces, and the review process to ensure quality.

Prerequisite Note: This chapter assumes familiarity with Python basics (Chapter 5), the vibe coding workflow (Chapter 6), and prompt engineering techniques (Chapters 8-12). You should have Python 3.10+ installed along with an AI coding assistant.

15.1 The Anatomy of a CLI Application

Before writing any code, let us understand what separates a quick script from a production CLI tool. A well-built command-line application has a layered architecture where each layer has a clear responsibility.

The Five Layers of a CLI Application

+------------------------------------------+
|          Entry Point & Routing            |  <- main(), CLI framework
+------------------------------------------+
|         Argument Parsing & Validation     |  <- argparse, click, typer
+------------------------------------------+
|        Configuration & Environment        |  <- config files, env vars
+------------------------------------------+
|           Business Logic                  |  <- core functionality
+------------------------------------------+
|         I/O, Logging, & Feedback          |  <- file ops, progress, output
+------------------------------------------+

Layer 1: Entry Point and Routing. This is the main() function and the mechanism that routes user commands to the appropriate handler. In our Chapter 6 task manager, this was the if args.command == "add" chain.

Layer 2: Argument Parsing and Validation. This layer defines what arguments and options the tool accepts, validates them, and produces helpful error messages when inputs are incorrect. Libraries like argparse and click handle the heavy lifting.

Layer 3: Configuration and Environment. Production tools need to read settings from multiple sources -- config files, environment variables, and command-line flags -- with a clear precedence order. A user might set a default output directory in a config file but override it with a flag for a single invocation.

Layer 4: Business Logic. This is the core of what the tool actually does. In a file organizer, this is the code that categorizes and moves files. This layer should be independent of the CLI framework so it can be tested in isolation and potentially reused in other contexts.

Layer 5: I/O, Logging, and Feedback. This layer handles reading and writing files, logging diagnostic information, displaying progress bars, and formatting output for the terminal. It is the bridge between the tool's internal state and the user.

Key Insight: The most common mistake in CLI development is mixing these layers together. When your argument parsing code directly manipulates files, or your business logic prints to the console, the tool becomes difficult to test, maintain, and extend. AI assistants tend to generate tightly coupled code by default, so you will need to prompt for separation explicitly.

From Script to Tool: The Evolution

Consider the lifecycle of a typical CLI tool:

In this chapter, we will walk through building a production-quality tool, using AI to accelerate each stage of development.

Revisiting the Chapter 6 Task Manager

To ground this discussion, recall the task manager we built in Chapter 6. That tool had:

• A build_parser() function that created an ArgumentParser with subcommands (add, list, complete, delete, search)
• A main() function that routed commands to handler functions
• JSON file persistence with a load_tasks() / save_tasks() storage layer
• Basic error handling (checking for missing tasks, empty descriptions)

What it lacked tells us exactly what this chapter needs to cover:

• No configuration files. The data file path was hardcoded (with a single environment variable override).
• No logging. Diagnostic messages used print(), mixing them with output.
• No progress feedback. Processing was fast enough that this did not matter, but a production tool handling thousands of items would need it.
• No packaging. You could not pip install it. Users had to run python example-01-task-manager.py.
• No structured error handling. Errors produced ad-hoc messages with no consistent exit codes.

By the end of this chapter, you will know how to add every one of these missing layers. More importantly, you will know how to prompt an AI to generate them efficiently.

Why the Command Line Still Matters

Some readers may wonder why we dedicate an entire chapter to CLI tools in an era of web applications and mobile apps. The answer is simple: the command line is where developers live. Consider the daily workflow of a professional software engineer:

$ git pull
$ python -m pytest tests/
$ docker compose up -d
$ curl -s https://api.example.com/status | jq .health
$ ssh production-server "tail -f /var/log/app.log"

Every one of those commands is a CLI tool. When you need to automate a task, process data, or integrate systems, a CLI tool is almost always the fastest path from idea to working solution. They compose through pipes, run in CI/CD pipelines, execute via cron jobs, and work over SSH connections where no graphical interface exists.

AI coding assistants make CLI development even more compelling. Tasks that once required memorizing arcane API details -- argument parsing syntax, logging handler configuration, packaging metadata -- can now be generated from a description. This chapter teaches you to leverage that capability while understanding the generated code well enough to review it critically.

15.2 Argument Parsing with argparse and click

Argument parsing is the front door of your CLI application. It defines how users interact with your tool, what inputs it accepts, and what help information it provides. Python offers two dominant approaches: the standard library's argparse and the third-party click library.

argparse: The Standard Library Approach

The argparse module comes with Python and requires no additional installation. It uses a declarative style where you define a parser object, add arguments to it, and then parse the command-line input.

Here is the prompt we would use to generate a CLI tool with argparse:

Prompt to AI: "Build a Python CLI tool called 'fileutil' using argparse with the following subcommands: (1) 'count' -- counts lines, words, and characters in files, accepts a list of file paths and a --format flag for output format (text or json); (2) 'find' -- searches for files matching a pattern, accepts a directory path and a --pattern glob argument; (3) 'hash' -- computes file checksums, accepts file paths and a --algorithm flag (md5, sha1, sha256). Include a --verbose global flag. Use type hints, docstrings, and follow PEP 8."

The AI will generate something similar to the code in code/example-01-cli-argparse.py. Let us examine the key patterns.

Creating the Parser and Subparsers:

def build_parser() -> argparse.ArgumentParser:
    """Build the top-level argument parser with subcommands."""
    parser = argparse.ArgumentParser(
        prog="fileutil",
        description="A collection of file utility commands.",
        epilog="Use '%(prog)s <command> --help' for command-specific help.",
    )
    parser.add_argument(
        "-v", "--verbose",
        action="store_true",
        help="Enable verbose output",
    )
    parser.add_argument(
        "--version",
        action="version",
        version="%(prog)s 1.0.0",
    )

    subparsers = parser.add_subparsers(dest="command", help="Available commands")
    # ... add subcommands
    return parser

Key argparse patterns to understand:

• action="store_true" creates a boolean flag
• action="version" handles --version automatically
• nargs="+" accepts one or more positional arguments
• choices=[...] restricts values to a predefined set
• default= provides fallback values
• type=int handles type conversion and validation

Handling Subcommands:

# Route to the appropriate handler
handlers = {
    "count": handle_count,
    "find": handle_find,
    "hash": handle_hash,
}
handler = handlers.get(args.command)
if handler is None:
    parser.print_help()
    return 0
return handler(args)

Best Practice: Use a dictionary dispatch pattern instead of a long if/elif chain. This is cleaner, easier to extend, and plays well with type checkers.

click: The Decorator-Based Approach

The click library takes a fundamentally different approach. Instead of building a parser object imperatively, you decorate Python functions with argument and option decorators. This produces cleaner, more readable code at the cost of an additional dependency.

Here is the prompt to regenerate the same tool with click:

Prompt to AI: "Reimplement the 'fileutil' CLI tool using click instead of argparse. Keep the same three subcommands (count, find, hash) with identical functionality. Use click groups for subcommands, proper decorators for arguments and options, and click.echo for output. Show how click handles the same patterns differently from argparse."

The resulting code (see code/example-02-cli-click.py) demonstrates click's decorator-based approach:

@click.group()
@click.option("-v", "--verbose", is_flag=True, help="Enable verbose output")
@click.version_option(version="1.0.0")
@click.pass_context
def cli(ctx: click.Context, verbose: bool) -> None:
    """A collection of file utility commands."""
    ctx.ensure_object(dict)
    ctx.obj["verbose"] = verbose


@cli.command()
@click.argument("files", nargs=-1, required=True, type=click.Path(exists=True))
@click.option("--format", "output_format", type=click.Choice(["text", "json"]),
              default="text", help="Output format")
@click.pass_context
def count(ctx: click.Context, files: tuple[str, ...], output_format: str) -> None:
    """Count lines, words, and characters in files."""
    verbose = ctx.obj["verbose"]
    # ... implementation

argparse vs. click: When to Use Which

Recommendation: Use argparse when you need zero dependencies (system scripts, restricted environments) or when the CLI is simple. Use click for anything with complex subcommand hierarchies, when you value testability, or when you are building a tool that will be maintained long-term.

AI-Assisted Argument Parsing Tips

When prompting an AI to generate argument parsing code, include these details:

1. Specify every flag and argument explicitly. Do not say "add appropriate flags." Instead, list each flag with its short form, long form, type, and default value.
2. Describe the help text you want. AIs generate better help strings when you provide context about the target audience.
3. Request validation. Ask for custom validation functions for arguments that need more than simple type checking.
4. Ask for mutual exclusivity. If certain flags conflict, state this explicitly (e.g., "--json and --csv should be mutually exclusive").

Testing CLI Argument Parsing

One of click's strongest advantages over argparse is its built-in testing support through CliRunner. This allows you to test your CLI without spawning a subprocess:

from click.testing import CliRunner


def test_count_command():
    """Test the count command with a real file."""
    runner = CliRunner()
    with runner.isolated_filesystem():
        # Create a test file
        with open("test.txt", "w") as f:
            f.write("hello world\nfoo bar baz\n")

        result = runner.invoke(cli, ["count", "test.txt"])
        assert result.exit_code == 0
        assert "hello" in result.output or "2" in result.output


def test_count_missing_file():
    """Test that count handles missing files gracefully."""
    runner = CliRunner()
    result = runner.invoke(cli, ["count", "nonexistent.txt"])
    assert result.exit_code != 0

With argparse, testing typically requires calling the main() function with a list of arguments:

def test_count_argparse():
    """Test the argparse-based count command."""
    exit_code = main(["count", "test.txt"])
    assert exit_code == 0

Callout: The Importance of Testable CLIs A CLI tool that can only be tested by running it as a subprocess in a shell is fragile and slow to test. Both argparse and click support testing through function calls, but click's CliRunner goes further by capturing stdout, stderr, and exit codes in a single result object. If testability is a priority (and it should be), click has a clear advantage here.

Typer: A Brief Mention

A third option worth knowing about is typer, built on top of click. Typer uses Python type hints to automatically generate the CLI interface:

import typer

app = typer.Typer()

@app.command()
def count(
    files: list[str],
    format: str = typer.Option("text", help="Output format"),
    verbose: bool = typer.Option(False, "--verbose", "-v"),
) -> None:
    """Count lines, words, and characters in files."""
    ...

Typer is an excellent choice when you want click's power with even less boilerplate. We focus on argparse and click in this chapter because they represent the two fundamental paradigms (imperative and declarative), and understanding them makes typer easy to pick up.

15.3 Configuration Management

Production CLI tools rarely rely solely on command-line arguments. Users expect to set default values in configuration files, override them with environment variables, and further override them with command-line flags. This creates a configuration hierarchy:

Command-line flags          (highest priority)
    |
Environment variables
    |
User config file (~/.toolname.toml)
    |
Project config file (./toolname.toml)
    |
Built-in defaults            (lowest priority)

Configuration File Formats

Python supports several configuration file formats. Here are the three most common:

TOML (Recommended for Modern Python):

# config.toml
[general]
verbose = false
output_format = "text"

[paths]
default_output = "~/output"
temp_directory = "/tmp/fileutil"

[logging]
level = "INFO"
file = "fileutil.log"

TOML is the format used by pyproject.toml and is Python's recommended configuration format since Python 3.11 added tomllib to the standard library.

YAML (Common in DevOps Tools):

# config.yaml
general:
  verbose: false
  output_format: text

paths:
  default_output: ~/output
  temp_directory: /tmp/fileutil

YAML is more flexible than TOML but requires the pyyaml package and has well-known security concerns with yaml.load() (always use yaml.safe_load()).

Environment Variables with .env Files:

# .env
FILEUTIL_VERBOSE=false
FILEUTIL_OUTPUT_FORMAT=text
FILEUTIL_DEFAULT_OUTPUT=~/output

The python-dotenv package loads .env files into the environment. This pattern is especially useful for secrets and deployment-specific settings.

Building a Configuration Loader

Here is the prompt to generate a configuration management system:

Prompt to AI: "Create a Python configuration manager class that loads settings from these sources in order of increasing priority: built-in defaults, a TOML config file, environment variables with a FILEUTIL_ prefix, and command-line arguments. The class should provide a get() method, support nested keys with dot notation (e.g., 'paths.output'), and handle missing config files gracefully. Use tomllib for Python 3.11+ with a fallback to tomli. Include type hints and docstrings."

The resulting configuration class follows this pattern:

import os
import sys
from pathlib import Path
from typing import Any

if sys.version_info >= (3, 11):
    import tomllib
else:
    try:
        import tomli as tomllib
    except ImportError:
        tomllib = None  # type: ignore[assignment]


class Config:
    """Hierarchical configuration manager for CLI tools.

    Loads configuration from multiple sources with clear precedence:
    defaults < config file < environment variables < CLI arguments.
    """

    def __init__(
        self,
        defaults: dict[str, Any] | None = None,
        config_path: Path | None = None,
        env_prefix: str = "FILEUTIL",
    ) -> None:
        self._defaults = defaults or {}
        self._file_config: dict[str, Any] = {}
        self._env_prefix = env_prefix
        self._cli_overrides: dict[str, Any] = {}

        if config_path and config_path.exists():
            self._file_config = self._load_toml(config_path)

    def _load_toml(self, path: Path) -> dict[str, Any]:
        """Load a TOML configuration file."""
        if tomllib is None:
            return {}
        with open(path, "rb") as f:
            return tomllib.load(f)

    def get(self, key: str, default: Any = None) -> Any:
        """Retrieve a configuration value using dot notation.

        Checks sources in priority order: CLI > env > file > defaults.
        """
        # Check CLI overrides first
        if key in self._cli_overrides:
            return self._cli_overrides[key]

        # Check environment variables
        env_key = f"{self._env_prefix}_{key.upper().replace('.', '_')}"
        env_value = os.environ.get(env_key)
        if env_value is not None:
            return env_value

        # Check file configuration (with dot notation support)
        value = self._resolve_dotted_key(self._file_config, key)
        if value is not None:
            return value

        # Check defaults
        value = self._resolve_dotted_key(self._defaults, key)
        if value is not None:
            return value

        return default

    def set_cli_override(self, key: str, value: Any) -> None:
        """Set a CLI-provided override value."""
        if value is not None:
            self._cli_overrides[key] = value

    @staticmethod
    def _resolve_dotted_key(data: dict[str, Any], key: str) -> Any:
        """Resolve a dotted key like 'paths.output' in nested dicts."""
        keys = key.split(".")
        current = data
        for k in keys:
            if isinstance(current, dict) and k in current:
                current = current[k]
            else:
                return None
        return current

Common Pitfall: When AI generates configuration code, it often forgets to handle the case where the config file does not exist. Always verify that your configuration loader fails gracefully with sensible defaults when no config file is present. A tool should work out of the box with zero configuration.

XDG Base Directory Compliance

On Linux and macOS, well-behaved tools store configuration in standardized locations:

def get_config_dir(app_name: str) -> Path:
    """Get the platform-appropriate configuration directory."""
    if sys.platform == "win32":
        base = Path(os.environ.get("APPDATA", Path.home() / "AppData" / "Roaming"))
    elif sys.platform == "darwin":
        base = Path.home() / "Library" / "Application Support"
    else:
        base = Path(os.environ.get("XDG_CONFIG_HOME", Path.home() / ".config"))
    return base / app_name

15.4 Logging and Verbose Output

Logging serves two distinct audiences: the end user who wants to know what the tool is doing, and the developer who needs diagnostic information for debugging. A well-designed CLI tool supports both.

Setting Up Python's Logging Module

Prompt to AI: "Create a logging setup function for a CLI tool that: (1) configures a console handler that shows INFO and above by default, DEBUG when --verbose is passed; (2) adds a file handler that always logs at DEBUG level to a rotating log file; (3) uses a clean format for console output (no timestamps for INFO, full format for DEBUG) and a detailed format for the file. Use Python's standard logging module with type hints."

import logging
import sys
from logging.handlers import RotatingFileHandler
from pathlib import Path


def setup_logging(
    verbose: bool = False,
    log_file: Path | None = None,
    name: str = "fileutil",
) -> logging.Logger:
    """Configure logging for both console and file output.

    Args:
        verbose: If True, show DEBUG messages on the console.
        log_file: Optional path for a rotating log file.
        name: Logger name.

    Returns:
        Configured logger instance.
    """
    logger = logging.getLogger(name)
    logger.setLevel(logging.DEBUG)

    # Console handler
    console_handler = logging.StreamHandler(sys.stderr)
    console_level = logging.DEBUG if verbose else logging.INFO
    console_handler.setLevel(console_level)

    if verbose:
        console_format = logging.Formatter(
            "%(asctime)s [%(levelname)-8s] %(name)s: %(message)s",
            datefmt="%H:%M:%S",
        )
    else:
        console_format = logging.Formatter("%(message)s")

    console_handler.setFormatter(console_format)
    logger.addHandler(console_handler)

    # File handler (optional)
    if log_file:
        log_file.parent.mkdir(parents=True, exist_ok=True)
        file_handler = RotatingFileHandler(
            log_file,
            maxBytes=5 * 1024 * 1024,  # 5 MB
            backupCount=3,
        )
        file_handler.setLevel(logging.DEBUG)
        file_format = logging.Formatter(
            "%(asctime)s [%(levelname)-8s] %(name)s:%(funcName)s:%(lineno)d - %(message)s"
        )
        file_handler.setFormatter(file_format)
        logger.addHandler(file_handler)

    return logger

Logging vs. Printing

A critical distinction in CLI tools:

Key Insight: Normal output goes to stdout; logging goes to stderr. This separation allows users to pipe the tool's output to another command or file without logging messages contaminating the data. Always configure logging handlers to write to sys.stderr.

The --verbose and --quiet Pattern

Professional tools often support both verbose and quiet modes:

parser.add_argument("-v", "--verbose", action="count", default=0,
                    help="Increase verbosity (-v for info, -vv for debug)")
parser.add_argument("-q", "--quiet", action="store_true",
                    help="Suppress all output except errors")

Using action="count" allows stacking: -v for info, -vv for debug, -vvv for trace. This is a convention used by tools like ssh, curl, and ansible.

Structured Logging with Extra Context

For more complex tools, structured logging adds valuable context to log messages without cluttering the format:

# Add contextual information to log records
logger.info(
    "Processing file",
    extra={
        "file_path": str(file_path),
        "file_size": file_path.stat().st_size,
        "encoding": encoding,
    },
)

For production tools that emit logs consumed by monitoring systems (ELK stack, Datadog, etc.), consider python-json-logger which formats log records as JSON:

from pythonjsonlogger import jsonlogger

handler = logging.StreamHandler()
formatter = jsonlogger.JsonFormatter(
    "%(asctime)s %(name)s %(levelname)s %(message)s"
)
handler.setFormatter(formatter)

Callout: Logging Anti-Patterns to Watch For in AI-Generated Code AI-generated code frequently commits these logging mistakes:

1. Using print() for everything. The AI does not distinguish between output and diagnostics unless you ask.
2. Creating a new logger in every function. The correct pattern is to create one module-level logger with logging.getLogger(__name__) and reuse it.
3. Configuring the root logger. This affects all libraries your tool imports, potentially flooding the console with messages from urllib3, requests, etc. Always configure a named logger specific to your application.
4. Logging sensitive information. AI may log file contents, passwords from config, or API keys. Review all log statements for sensitive data.

15.5 File Processing and I/O Patterns

Many CLI tools exist to process files. Whether you are converting formats, analyzing data, or organizing directories, understanding file I/O patterns is essential.

Pattern 1: Streaming (Line-by-Line) Processing

For large files, loading everything into memory is impractical. Stream processing handles files one line (or chunk) at a time:

def process_file_streaming(
    input_path: Path,
    output_path: Path,
    transform: Callable[[str], str],
) -> int:
    """Process a file line by line, applying a transformation.

    Args:
        input_path: Source file path.
        output_path: Destination file path.
        transform: A function that transforms each line.

    Returns:
        The number of lines processed.
    """
    line_count = 0
    with open(input_path, "r", encoding="utf-8") as infile, \
         open(output_path, "w", encoding="utf-8") as outfile:
        for line in infile:
            outfile.write(transform(line))
            line_count += 1
    return line_count

Pattern 2: Batch Processing with Glob Patterns

When processing multiple files, the pathlib glob system and the glob module are your primary tools:

from pathlib import Path


def find_files(
    directory: Path,
    pattern: str = "*",
    recursive: bool = True,
) -> list[Path]:
    """Find files matching a glob pattern.

    Args:
        directory: The directory to search.
        pattern: Glob pattern (e.g., '*.txt', '**/*.py').
        recursive: Whether to search subdirectories.

    Returns:
        A sorted list of matching file paths.
    """
    if recursive and not pattern.startswith("**"):
        pattern = f"**/{pattern}"
    return sorted(directory.glob(pattern))

Pattern 3: Safe File Writing

Writing files safely requires handling several edge cases: existing files, permissions, partial writes, and encoding:

import tempfile
import shutil


def safe_write(
    path: Path,
    content: str,
    backup: bool = True,
    encoding: str = "utf-8",
) -> None:
    """Write content to a file safely using atomic write.

    Writes to a temporary file first, then renames to avoid corruption
    if the process is interrupted.

    Args:
        path: Target file path.
        content: String content to write.
        backup: If True and path exists, create a .bak backup first.
        encoding: File encoding (default utf-8).
    """
    path.parent.mkdir(parents=True, exist_ok=True)

    if backup and path.exists():
        shutil.copy2(path, path.with_suffix(path.suffix + ".bak"))

    # Write to temp file in the same directory, then rename
    fd, tmp_path_str = tempfile.mkstemp(
        dir=path.parent,
        prefix=f".{path.name}.",
        suffix=".tmp",
    )
    tmp_path = Path(tmp_path_str)
    try:
        with open(fd, "w", encoding=encoding) as f:
            f.write(content)
        tmp_path.replace(path)
    except BaseException:
        tmp_path.unlink(missing_ok=True)
        raise

Pattern 4: stdin/stdout Pipelines

Unix philosophy dictates that tools should work with pipes. Supporting stdin/stdout allows your tool to chain with others:

import sys
from pathlib import Path


def get_input(file_arg: str | None) -> str:
    """Read input from a file argument or stdin.

    Args:
        file_arg: File path or '-' for stdin, or None for stdin.

    Returns:
        The full text content.
    """
    if file_arg is None or file_arg == "-":
        return sys.stdin.read()
    return Path(file_arg).read_text(encoding="utf-8")

Prompt Tip: When asking AI to generate file processing code, always specify: (1) the expected file sizes (affects memory strategy), (2) the encoding requirements, (3) whether the tool should support stdin/stdout piping, and (4) how errors with individual files should be handled (skip, abort, or collect).

15.6 Progress Bars and User Feedback

For operations that take more than a second or two, users need feedback. The rich library provides beautiful progress bars, spinners, tables, and formatted output for the terminal.

Installing and Using rich

pip install rich

Progress Bars for File Processing

Prompt to AI: "Add a rich progress bar to a file processing function that iterates over a list of file paths. Show the current filename, a progress bar, percentage complete, elapsed time, and estimated time remaining. Handle the case where the file list might be empty. Use rich.progress."

from pathlib import Path

from rich.progress import (
    BarColumn,
    MofNCompleteColumn,
    Progress,
    SpinnerColumn,
    TextColumn,
    TimeElapsedColumn,
    TimeRemainingColumn,
)


def process_files_with_progress(
    files: list[Path],
    processor: Callable[[Path], None],
) -> dict[str, int]:
    """Process files with a rich progress bar.

    Args:
        files: List of file paths to process.
        processor: Function to call on each file.

    Returns:
        Dictionary with counts of succeeded and failed files.
    """
    results = {"succeeded": 0, "failed": 0}

    if not files:
        return results

    progress = Progress(
        SpinnerColumn(),
        TextColumn("[bold blue]{task.description}"),
        BarColumn(),
        MofNCompleteColumn(),
        TimeElapsedColumn(),
        TimeRemainingColumn(),
    )

    with progress:
        task = progress.add_task("Processing files", total=len(files))
        for file_path in files:
            progress.update(task, description=f"Processing {file_path.name}")
            try:
                processor(file_path)
                results["succeeded"] += 1
            except Exception as exc:
                logging.error("Failed to process %s: %s", file_path, exc)
                results["failed"] += 1
            progress.advance(task)

    return results

Rich Tables for Structured Output

from rich.console import Console
from rich.table import Table


def display_file_stats(stats: list[dict[str, Any]]) -> None:
    """Display file statistics in a formatted table.

    Args:
        stats: List of dictionaries with file information.
    """
    console = Console()
    table = Table(title="File Statistics")

    table.add_column("Filename", style="cyan", no_wrap=True)
    table.add_column("Size", justify="right", style="green")
    table.add_column("Lines", justify="right", style="magenta")
    table.add_column("Modified", style="yellow")

    for entry in stats:
        table.add_row(
            entry["name"],
            entry["size"],
            str(entry["lines"]),
            entry["modified"],
        )

    console.print(table)

Spinners for Indeterminate Operations

For operations where you cannot predict the duration (network requests, database queries), use a spinner:

from rich.console import Console


def fetch_with_spinner(url: str) -> str:
    """Fetch content from a URL with a spinner.

    Args:
        url: The URL to fetch.

    Returns:
        The response text.
    """
    console = Console()
    with console.status("[bold green]Fetching data...") as status:
        import urllib.request
        response = urllib.request.urlopen(url)
        data = response.read().decode("utf-8")
        status.update("[bold green]Processing response...")
        # ... process data
    return data

Callout: Conditional Rich Output Not all environments support rich formatting. When output is piped to a file or another command, you should disable formatting. The rich library handles this automatically -- Console() detects whether stdout is a terminal and adjusts accordingly. However, for explicit control:

```python console = Console(force_terminal=False) # Auto-detect

or check manually:

if sys.stdout.isatty(): # Use rich formatting else: # Use plain text ```

15.7 Error Handling and Exit Codes

Error handling in CLI tools serves two masters: the human user who reads error messages and the script or pipeline that checks exit codes.

Exit Code Conventions

Exit codes are integers returned by a program when it finishes. By convention:

For your own tools, define a clear set of exit codes:

from enum import IntEnum


class ExitCode(IntEnum):
    """Exit codes for the CLI tool."""
    SUCCESS = 0
    GENERAL_ERROR = 1
    USAGE_ERROR = 2
    INPUT_ERROR = 3       # Invalid input file or data
    OUTPUT_ERROR = 4      # Cannot write output
    CONFIG_ERROR = 5      # Configuration problem
    PERMISSION_ERROR = 6  # Insufficient permissions
    DEPENDENCY_ERROR = 7  # Missing required dependency

Structured Error Handling

Prompt to AI: "Create an error handling system for a CLI tool with: (1) a custom exception hierarchy with CLIError as base, InputError, OutputError, and ConfigError as subclasses; (2) a decorator that catches these exceptions and converts them to appropriate error messages and exit codes; (3) user-friendly error messages that do not show tracebacks unless --debug is active. Include type hints and docstrings."

class CLIError(Exception):
    """Base exception for CLI tool errors.

    Attributes:
        message: Human-readable error description.
        exit_code: The exit code to return when this error occurs.
    """

    def __init__(self, message: str, exit_code: int = 1) -> None:
        super().__init__(message)
        self.message = message
        self.exit_code = exit_code


class InputError(CLIError):
    """Raised when input files or data are invalid."""

    def __init__(self, message: str) -> None:
        super().__init__(message, exit_code=ExitCode.INPUT_ERROR)


class OutputError(CLIError):
    """Raised when output cannot be written."""

    def __init__(self, message: str) -> None:
        super().__init__(message, exit_code=ExitCode.OUTPUT_ERROR)


class ConfigError(CLIError):
    """Raised when configuration is invalid or missing."""

    def __init__(self, message: str) -> None:
        super().__init__(message, exit_code=ExitCode.CONFIG_ERROR)

The main entry point catches these exceptions:

def main() -> int:
    """Main entry point with structured error handling."""
    try:
        args = parse_args()
        setup_logging(verbose=args.verbose, debug=args.debug)
        return run_command(args)
    except CLIError as exc:
        logger.error("Error: %s", exc.message)
        return exc.exit_code
    except KeyboardInterrupt:
        logger.info("\nOperation cancelled by user.")
        return 130  # 128 + SIGINT(2)
    except Exception as exc:
        logger.error("Unexpected error: %s", exc)
        logger.debug("Traceback:", exc_info=True)
        return ExitCode.GENERAL_ERROR

Key Insight: The KeyboardInterrupt handler is often overlooked. Without it, pressing Ctrl+C produces an ugly traceback instead of a clean exit. The exit code 130 (128 + 2) follows Unix convention for processes terminated by SIGINT.

User-Friendly Error Messages

Good error messages follow a formula: What went wrong + Why + What to do about it.

# Bad: cryptic technical message
raise FileNotFoundError(f"No such file: {path}")

# Good: actionable message with context
raise InputError(
    f"Cannot read input file '{path}': file not found.\n"
    f"  Check that the path is correct and the file exists.\n"
    f"  Current directory: {Path.cwd()}"
)

15.8 Packaging and Distribution

Once your CLI tool works, you want to make it easy to install. Python's packaging ecosystem has standardized on pyproject.toml as the single source of project metadata and build configuration.

Project Structure

A well-organized CLI project looks like this:

fileutil/
    pyproject.toml
    README.md
    LICENSE
    src/
        fileutil/
            __init__.py
            __main__.py      # Allows python -m fileutil
            cli.py           # CLI entry point
            config.py        # Configuration management
            commands/
                __init__.py
                count.py
                find.py
                hash.py
            utils/
                __init__.py
                logging.py
                progress.py
    tests/
        __init__.py
        test_cli.py
        test_count.py
        test_config.py

pyproject.toml Configuration

Prompt to AI: "Generate a pyproject.toml for a CLI tool called 'fileutil' that: uses setuptools as the build backend; has a console_scripts entry point mapping 'fileutil' to 'fileutil.cli:main'; requires Python 3.10+; depends on click, rich, and tomli (for Python < 3.11); includes dev dependencies for pytest, mypy, and ruff; has proper metadata with author, description, license, and classifiers."

[build-system]
requires = ["setuptools>=68.0", "setuptools-scm>=8.0"]
build-backend = "setuptools.backends._legacy:_Backend"

[project]
name = "fileutil"
version = "1.0.0"
description = "A collection of file utility commands for the command line"
readme = "README.md"
license = {text = "MIT"}
requires-python = ">=3.10"
authors = [
    {name = "Your Name", email = "you@example.com"},
]
classifiers = [
    "Development Status :: 4 - Beta",
    "Environment :: Console",
    "Intended Audience :: Developers",
    "License :: OSI Approved :: MIT License",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Topic :: Utilities",
]
dependencies = [
    "click>=8.0",
    "rich>=13.0",
    "tomli>=2.0; python_version < '3.11'",
]

[project.optional-dependencies]
dev = [
    "pytest>=7.0",
    "pytest-cov>=4.0",
    "mypy>=1.0",
    "ruff>=0.1.0",
]

[project.scripts]
fileutil = "fileutil.cli:main"

[project.urls]
Homepage = "https://github.com/yourname/fileutil"
Repository = "https://github.com/yourname/fileutil"
Issues = "https://github.com/yourname/fileutil/issues"

[tool.setuptools.packages.find]
where = ["src"]

[tool.ruff]
line-length = 88
target-version = "py310"

[tool.mypy]
python_version = "3.10"
strict = true

The __main__.py Pattern

Adding a __main__.py allows running the package directly with python -m fileutil:

"""Allow running the package with python -m fileutil."""
from fileutil.cli import main

if __name__ == "__main__":
    raise SystemExit(main())

Console Scripts Entry Points

The line fileutil = "fileutil.cli:main" in [project.scripts] tells pip to create an executable wrapper that calls the main() function from fileutil.cli. After pip install ., users can simply type fileutil at the command line.

Building and Installing

# Install in development mode (editable)
pip install -e ".[dev]"

# Build a distributable package
pip install build
python -m build

# This creates:
# dist/fileutil-1.0.0.tar.gz      (source distribution)
# dist/fileutil-1.0.0-py3-none-any.whl  (wheel)

# Upload to PyPI (when ready)
pip install twine
twine upload dist/*

Callout: Development Mode During development, always install with pip install -e . (editable mode). This creates a symlink so that changes to your source code are immediately reflected without reinstalling. The -e flag is short for --editable.

15.9 Interactive CLI Features

Some CLI tools need to interact with users beyond simple arguments. Password prompts, confirmation dialogs, multi-select menus, and interactive search are all common patterns.

Basic Prompts and Confirmations

With click:

import click


def delete_files(files: list[Path], force: bool = False) -> None:
    """Delete files with optional confirmation.

    Args:
        files: List of file paths to delete.
        force: If True, skip confirmation prompt.
    """
    if not force:
        file_list = "\n".join(f"  - {f}" for f in files)
        click.echo(f"The following files will be deleted:\n{file_list}")
        if not click.confirm("Do you want to continue?"):
            click.echo("Aborted.")
            return

    for f in files:
        f.unlink()
        click.echo(f"Deleted: {f}")

Password and Sensitive Input

# click handles hiding input automatically
password = click.prompt("Database password", hide_input=True)

# With confirmation
password = click.prompt(
    "Set new password",
    hide_input=True,
    confirmation_prompt=True,
)

Choice Menus

For simple selection from a list of options:

def select_profile(profiles: list[str]) -> str:
    """Present a numbered menu and return the selected profile.

    Args:
        profiles: List of profile names to choose from.

    Returns:
        The selected profile name.
    """
    click.echo("Available profiles:")
    for i, profile in enumerate(profiles, 1):
        click.echo(f"  {i}. {profile}")

    while True:
        choice = click.prompt(
            "Select a profile",
            type=click.IntRange(1, len(profiles)),
        )
        return profiles[choice - 1]

Interactive Filtering with Fuzzy Search

For more sophisticated interaction, the questionary library provides fuzzy search, checkboxes, and other interactive widgets:

# pip install questionary
import questionary


def interactive_file_select(files: list[Path]) -> list[Path]:
    """Present an interactive checkbox list for file selection.

    Args:
        files: List of file paths to choose from.

    Returns:
        List of selected file paths.
    """
    choices = [
        questionary.Choice(title=str(f), value=f)
        for f in files
    ]
    selected = questionary.checkbox(
        "Select files to process:",
        choices=choices,
    ).ask()

    return selected or []

When to Use Interactive Features: Interactive features are powerful but break automation. A tool that requires user input cannot be used in scripts, cron jobs, or CI pipelines. Always provide non-interactive alternatives (flags like --yes, --force, or --no-input) for every interactive prompt.

15.10 Building a Complete CLI Tool with AI

Now let us bring everything together. We will build a complete CLI tool called logparse -- a log file analyzer that demonstrates every concept from this chapter. We will show the prompts used at each step and the AI-generated code that results.

Step 1: Define the Tool's Purpose and Interface

Prompt to AI: "I want to build a Python CLI tool called 'logparse' that analyzes log files. It should support these commands: - 'summary' -- show a summary of log levels (INFO, WARNING, ERROR, etc.) with counts - 'search' -- search log entries by pattern, level, or time range - 'export' -- export filtered log entries to CSV or JSON

Global options: --verbose, --config (path to config file), --format (text/json output). Use click for the CLI framework, rich for output formatting, and support both single files and directories with glob patterns as input. Start by generating just the CLI skeleton with click commands, arguments, options, and help text. Do not implement the business logic yet -- just print placeholder messages. Include type hints and docstrings throughout."

This prompt follows the specification-driven approach from Chapter 10. We start with the interface and add implementation incrementally.

Step 2: Add Configuration and Logging

Prompt to AI: "Now add configuration management to the logparse tool. Create a config.py module that loads settings from: (1) built-in defaults, (2) ~/.config/logparse/config.toml, (3) environment variables with LOGPARSE_ prefix, (4) CLI flags. Also add logging setup with --verbose support. Wire the config and logging into the CLI entry point."

Step 3: Implement the Core Log Parser

Prompt to AI: "Implement the core log parsing logic for logparse. Create a parser.py module that can: (1) parse common log formats (Apache, nginx, Python logging, syslog), (2) auto-detect the log format from the first few lines, (3) extract timestamp, level, source, and message from each line, (4) return structured LogEntry dataclass objects. Handle malformed lines gracefully by logging a warning and skipping them."

Step 4: Add Progress and Rich Output

Prompt to AI: "Add rich formatting to logparse: (1) use a progress bar when scanning multiple files, (2) display the summary command output as a rich table, (3) color-code log levels in search results (red for ERROR, yellow for WARNING, green for INFO), (4) use a spinner when auto-detecting log format."

Step 5: Add Error Handling and Exit Codes

Prompt to AI: "Add proper error handling to logparse: (1) create a custom exception hierarchy (LogParseError, FileAccessError, FormatError, ExportError), (2) catch all exceptions at the top level and convert them to user-friendly messages with exit codes, (3) handle Ctrl+C gracefully, (4) validate all inputs early and fail fast with clear messages."

Step 6: Package for Distribution

Prompt to AI: "Create the pyproject.toml and package structure for logparse so it can be installed with pip. Include a console_scripts entry point, proper dependencies, and development dependencies. Also create a main.py for python -m logparse support."

The Iterative Review Process

After each step, review the AI's output:

1. Run the code. Does it execute without errors?
2. Test edge cases. Empty files, missing files, permission errors, extremely large files.
3. Check the architecture. Is the business logic separate from the CLI layer? Could you reuse the parser module in a web API?
4. Review error messages. Are they helpful? Do they tell the user what to do?
5. Verify the logging. Does --verbose show useful information without overwhelming the user?

Callout: The "One More Thing" Anti-Pattern When building with AI, it is tempting to keep adding features in a single conversation. "Now add color coding. Now add export to PDF. Now add email notifications." Each addition makes the context longer and the AI's output less reliable. Instead, commit working code after each step, start a new conversation for the next feature, and reference the existing code explicitly. This mirrors the iterative refinement techniques from Chapter 11.

Putting It All Together

The complete file processor example in code/example-03-file-processor.py demonstrates many of these patterns working together in a single tool. It includes:

• Click-based CLI with subcommands
• Configuration management with TOML
• Python logging with verbose mode
• File processing with glob patterns
• Rich progress bars and tables
• Structured error handling with exit codes
• Support for stdin/stdout piping

This is the template you should use as a starting point for your own CLI tools. Study the code, understand how the layers interact, and use it as a reference when prompting AI to build your own tools.

Chapter Summary

Building production-quality CLI tools requires attention to many details beyond the core functionality. In this chapter, we covered the complete lifecycle:

1. Architecture: CLI applications have five distinct layers -- entry point, argument parsing, configuration, business logic, and I/O. Keeping these layers separate makes tools testable and maintainable.

2. Argument Parsing: argparse provides a zero-dependency solution; click offers a more elegant decorator-based approach. Choose based on your project's constraints.

3. Configuration: Production tools load settings from multiple sources with clear precedence. TOML is the recommended format for modern Python.

4. Logging: Separate logging (stderr) from output (stdout). Support both verbose and quiet modes. Use rotating file handlers for persistent logs.

5. File Processing: Match your I/O pattern to the task -- streaming for large files, batch processing with globs for multiple files, atomic writes for safety, and stdin/stdout for pipeline compatibility.

6. User Feedback: Rich progress bars and formatted tables make tools pleasant to use. Always provide plain-text fallbacks for non-terminal environments.

7. Error Handling: Define custom exceptions with specific exit codes. Write error messages that explain what went wrong, why, and what to do about it. Always handle Ctrl+C gracefully.

8. Packaging: Use pyproject.toml with setuptools for modern Python packaging. Console script entry points make tools installable and accessible.

9. Interactive Features: Prompts, confirmations, and menus add flexibility but always provide non-interactive alternatives for automation.

10. AI-Assisted Development: Build incrementally -- define the interface first, add implementation layer by layer, and review at each step. Start new conversations for new features.

In the next chapter, we will take these CLI skills to the web, building frontend interfaces with AI assistance. Many of the patterns you have learned here -- argument validation, configuration management, error handling, and user feedback -- translate directly to web development.

Glossary

argparse: Python's standard library module for command-line argument parsing.

click: A third-party Python library for creating command-line interfaces using decorators.

console_scripts: An entry point type in Python packaging that creates executable commands when a package is installed.

entry point: In packaging, a named reference to a Python function that can be discovered and called by tools like pip.

exit code: An integer returned by a program to indicate success (0) or the type of failure (non-zero).

glob pattern: A string with wildcards (* and ?) used to match file paths.

pyproject.toml: The standard Python configuration file for project metadata and build system settings.

rich: A Python library for rich text formatting, progress bars, tables, and other terminal output.

rotating log file: A log file that is automatically archived and replaced when it reaches a certain size.

setuptools: The standard Python build system for packaging and distributing projects.

stdin/stdout/stderr: Standard input, output, and error streams. CLI tools conventionally write results to stdout and diagnostics to stderr.

subcommand: A secondary command under a main command (e.g., git commit where commit is a subcommand of git).

TOML: Tom's Obvious, Minimal Language -- a configuration file format used by pyproject.toml.

XDG Base Directory: A specification for where to store configuration, data, and cache files on Unix systems.