Chapter 26: Business Forecasting and Trend Analysis

"All models are wrong, but some are useful." — George E. P. Box, statistician

"A forecast is not a promise. It is an informed estimate. Present it as one." — Sandra Chen, Acme Corp VP of Sales, to Priya Okonkwo

Opening Story: The Budget Meeting

Sandra Chen set her coffee down and looked at the whiteboard. "Q3 projections," she said. "We need numbers by Friday for the board deck. Last year we walked in with gut feelings and Marcus and I spent two hours defending guesses. Not this year."

Priya Okonkwo had been waiting for this conversation. She had been reading about moving averages for three weeks. "I can build that," she said. "Give me the sales data and two days."

"What can you actually tell me?" Sandra asked.

"I can tell you where the trend line has been going, how much variance there is week to week, and what the most likely range for Q3 looks like based on the last eight quarters. I can't tell you what will happen — but I can give you a defensible range with the data behind it."

Sandra nodded. "That's what I need. Not a number. A range I can defend."

This chapter is about building that capability. Forecasting, done honestly, is one of the most valuable things a business analyst can offer — and one of the most abused. The difference between useful and misleading forecasting comes down to understanding what your methods actually say, communicating uncertainty honestly, and resisting pressure to present a false precision.

By the end of this chapter, you will be able to build the actual analysis Priya delivered to Sandra.

26.1 Why Forecasting Matters for Business Decisions

Forecasting is not an academic exercise. Businesses make irreversible decisions based on expected future conditions:

• Inventory purchasing: Order too little and you miss sales. Order too much and you are sitting on capital that cannot move.
• Staffing: Hire ahead of a growth period and you are ready. Hire late and you miss the window. Lay off early and you lose institutional knowledge.
• Budgeting: Capital allocation decisions depend on expected revenue. If the forecast is wrong, the budget is wrong before the year starts.
• Pricing: Understanding seasonal demand patterns lets you make intelligent pricing decisions instead of guessing.

For Acme Corp, Sandra's question was: "Should we build inventory reserves for Q3, or is Q2's slowdown a real trend?" For Maya Reyes, the question was: "Do I have enough pipeline to justify signing a year-long office lease?"

Both questions have the same structure: given what happened in the past, what is the best estimate of what comes next, and how confident should we be?

What Forecasting Can and Cannot Do

Before writing any code, this distinction is essential:

Forecasting can: - Extrapolate existing trends using principled mathematical methods - Quantify the typical uncertainty in a prediction based on historical variance - Identify seasonal patterns and cyclical behavior - Provide a range of likely outcomes rather than a single point estimate

Forecasting cannot: - Predict unprecedented events (recessions, pandemics, competitor actions) - Be more accurate than the data it is built on - Replace business judgment — it informs judgment - Give you certainty — it gives you probabilities and ranges

The statistician who said "all models are wrong, but some are useful" was not being nihilistic. He was being precise. Every forecast model makes simplifying assumptions about the world. Those assumptions are wrong in some ways. The question is whether the model is useful enough despite those limitations.

26.2 Time Series Concepts: The Language of Forecasting

A time series is a sequence of data points recorded at successive, evenly spaced points in time. Acme's monthly revenue figures are a time series. Maya's weekly billable hours are a time series. Most business data that has a date column is at least potentially a time series.

Time series behavior has four components that analysts look for:

Trend

A trend is the long-term direction of the series — up, down, or flat. If Acme's revenue has been growing roughly 8% per year for five years, that is a positive trend. If it has been declining for three quarters, that is a negative trend. Trend is usually modeled as a line (linear trend) or a curve (polynomial trend).

Seasonality

Seasonality is a pattern that repeats at regular intervals, typically tied to the calendar. Office supply sales peak in September (back to school and back to office), decline in December (holidays and fiscal year-end purchasing freezes), and pick up again in January. This pattern repeats every year — it is predictable, and ignoring it leads to bad forecasts.

In Python:

# Seasonality analysis — average revenue by month
monthly_avg = (
    df.groupby(df["sale_date"].dt.month)["revenue"]
    .mean()
    .reset_index()
)
monthly_avg.columns = ["month", "avg_revenue"]

Cyclicality

Cyclicality is a pattern that repeats over longer, irregular periods — typically economic cycles. A business might see growth during economic expansions and contractions during recessions. Cyclical patterns are harder to model than seasonal ones because they do not repeat on a fixed schedule.

For most business forecasting at the scale covered in this book, cyclicality is acknowledged as a risk factor rather than modeled explicitly.

Noise (Irregular Component)

Noise is the random variation that cannot be explained by trend, seasonality, or cyclicality. Every data point has some noise — a sale happened or did not happen for reasons that were random from the business's perspective. Good forecasting models the systematic components and quantifies how much of the remaining variation is noise.

Understanding the noise level is what gives you confidence intervals. If the data has low noise, your forecast band is narrow. If it has high noise, your forecast band is wide — and that wide band is honest information, not a failure.

26.3 Moving Averages: Smoothing the Signal

Moving averages are among the oldest and most practical tools in time series analysis. They work by averaging a window of recent observations to smooth out short-term noise and reveal the underlying trend.

Simple Moving Average (SMA)

A simple moving average replaces each data point with the average of the surrounding n points (the "window"). pandas makes this straightforward:

import pandas as pd

# Using a 4-week rolling average to smooth weekly revenue data
df["sma_4week"] = df["weekly_revenue"].rolling(window=4).mean()

# Using a 12-month rolling average to smooth monthly data
df["sma_12month"] = df["monthly_revenue"].rolling(window=12).mean()

The rolling(window=4) call creates a rolling window object. .mean() calculates the average of each window. For the first n-1 observations, the result is NaN because there are not enough previous data points to fill the window.

When to use SMA: When you want a simple, interpretable smoothing that weights all observations in the window equally. Good for visualizing trends.

SMA limitation: It reacts slowly to changes. A sharp upward move in recent data takes several periods to register fully in a 12-period SMA.

Weighted Moving Average (WMA)

A weighted moving average gives more weight to recent observations than older ones within the window:

def weighted_moving_average(
    series: pd.Series,
    window: int,
) -> pd.Series:
    """
    Compute a linearly weighted moving average.

    More recent observations receive higher weights within each window.
    The most recent observation has weight n, the oldest has weight 1.

    Args:
        series: Numeric pandas Series (e.g., monthly revenue).
        window: Number of periods in each averaging window.

    Returns:
        pd.Series: WMA values, NaN for the first window-1 periods.
    """
    weights = range(1, window + 1)  # [1, 2, 3, ..., window]
    wma = series.rolling(window=window).apply(
        lambda x: sum(x[i] * weights[i] for i in range(window)) / sum(weights),
        raw=True,
    )
    return wma

When to use WMA: When recent observations are more informative than older ones — for example, in a business that is changing rapidly, last month's revenue tells you more than revenue from eight months ago.

Exponential Moving Average (EMA)

The exponential moving average takes the weighted approach further: it weights all past observations, with weights decaying exponentially as you go back in time. A smoothing factor alpha controls how fast the weights decay.

def exponential_moving_average(
    series: pd.Series,
    span: int = 12,
) -> pd.Series:
    """
    Compute an exponential moving average using pandas ewm().

    EMA gives the most weight to the most recent observation and
    decreasing weight to older observations, with no hard cutoff.

    Args:
        series: Numeric pandas Series.
        span: The equivalent number of periods for the decay. Higher span
              = smoother result but slower to react to changes.

    Returns:
        pd.Series: EMA values for all periods (no leading NaN).

    Example:
        df["ema_12"] = exponential_moving_average(df["monthly_revenue"], span=12)
    """
    return series.ewm(span=span, adjust=False).mean()

pandas' ewm() (exponentially weighted moving) method handles the math. The span parameter is the most intuitive way to control smoothing: span=12 gives roughly the same decay as a 12-period SMA.

When to use EMA: EMA is preferred when you need a smooth trend line that still responds quickly to recent changes. It is the basis of many financial technical indicators and is commonly used in business dashboards.

Comparing the Three Moving Averages

import pandas as pd
import matplotlib.pyplot as plt

def plot_moving_averages(
    df: pd.DataFrame,
    date_col: str,
    value_col: str,
    window: int = 6,
    title: str = "Moving Average Comparison",
) -> None:
    """
    Plot raw data alongside SMA, WMA, and EMA for comparison.

    Args:
        df: DataFrame with date and value columns.
        date_col: Name of the date column.
        value_col: Name of the numeric column to smooth.
        window: Window size for SMA and WMA (also used as EMA span).
        title: Plot title.
    """
    fig, ax = plt.subplots(figsize=(12, 6))

    ax.plot(
        df[date_col], df[value_col],
        color="#cccccc", linewidth=1, label="Raw Data", alpha=0.7,
    )
    ax.plot(
        df[date_col],
        df[value_col].rolling(window=window).mean(),
        color="#0066cc", linewidth=2, label=f"SMA ({window})",
    )
    ax.plot(
        df[date_col],
        weighted_moving_average(df[value_col], window),
        color="#cc6600", linewidth=2, label=f"WMA ({window})", linestyle="--",
    )
    ax.plot(
        df[date_col],
        df[value_col].ewm(span=window, adjust=False).mean(),
        color="#006600", linewidth=2, label=f"EMA (span={window})", linestyle=":",
    )

    ax.set_title(title, fontsize=14, fontweight="bold")
    ax.set_xlabel("Date")
    ax.set_ylabel(value_col.replace("_", " ").title())
    ax.legend()
    ax.grid(True, alpha=0.3)
    plt.tight_layout()
    plt.show()

26.4 Linear Trend Extrapolation

Moving averages smooth the past. To project into the future, you need extrapolation — extending a trend line beyond the observed data.

The simplest approach is linear trend extrapolation: fit a straight line through the historical data and extend it forward. This assumes the trend that existed in the past will continue at the same rate.

Using numpy polyfit

numpy.polyfit() fits a polynomial to data. For a linear trend (degree 1), it finds the line y = mx + b that best fits your data in the least-squares sense.

import numpy as np
import pandas as pd


def linear_trend_forecast(
    df: pd.DataFrame,
    date_col: str,
    value_col: str,
    periods_ahead: int = 3,
) -> pd.DataFrame:
    """
    Fit a linear trend to historical data and project it forward.

    Converts dates to ordinal integers for the regression, then converts
    the forecast dates back to readable strings.

    Args:
        df: DataFrame with a date column and a numeric value column.
        date_col: Name of the date column (must be parseable as datetime).
        value_col: Name of the numeric column to forecast.
        periods_ahead: How many periods to project beyond the last data point.

    Returns:
        pd.DataFrame: Combined DataFrame with historical data plus forecasted
                      rows, flagged with 'is_forecast' boolean column.

    Example:
        forecast_df = linear_trend_forecast(
            monthly_df, "month", "revenue", periods_ahead=3
        )
    """
    df = df.copy()
    df[date_col] = pd.to_datetime(df[date_col])

    # Convert dates to numeric values for regression
    date_ordinals = df[date_col].map(pd.Timestamp.toordinal).values
    values = df[value_col].values

    # Fit linear trend: coefficients[0] = slope, coefficients[1] = intercept
    coefficients = np.polyfit(date_ordinals, values, deg=1)
    slope, intercept = coefficients

    # Generate trend line for historical period
    df["trend_line"] = slope * date_ordinals + intercept
    df["is_forecast"] = False

    # Project forward
    last_date = df[date_col].max()
    date_freq = _infer_frequency(df[date_col])
    forecast_rows = []

    for period in range(1, periods_ahead + 1):
        future_date = last_date + period * date_freq
        future_ordinal = future_date.toordinal()
        forecasted_value = slope * future_ordinal + intercept

        forecast_rows.append({
            date_col: future_date,
            value_col: max(0.0, round(forecasted_value, 2)),  # No negative revenue
            "trend_line": round(slope * future_ordinal + intercept, 2),
            "is_forecast": True,
        })

    forecast_df = pd.DataFrame(forecast_rows)
    combined = pd.concat([df, forecast_df], ignore_index=True)

    return combined


def _infer_frequency(date_series: pd.Series) -> pd.Timedelta:
    """
    Infer the typical interval between dates in a series.

    Returns the median difference between consecutive dates.
    Works for monthly, weekly, and quarterly data.
    """
    sorted_dates = date_series.sort_values()
    diffs = sorted_dates.diff().dropna()
    return diffs.median()

Using scipy.stats.linregress

scipy.stats.linregress() provides more statistical information than polyfit, including the R-squared value (how well the line fits) and the p-value (whether the trend is statistically significant):

from scipy import stats


def linear_regression_with_stats(
    df: pd.DataFrame,
    date_col: str,
    value_col: str,
) -> dict:
    """
    Perform linear regression on time series data with full statistical output.

    Args:
        df: DataFrame with date and value columns.
        date_col: Name of the date column.
        value_col: Name of the numeric column.

    Returns:
        dict with keys: slope, intercept, r_squared, p_value, std_err,
        monthly_growth_rate (if monthly data), trend_direction.

    Example:
        stats_result = linear_regression_with_stats(
            monthly_df, "month", "revenue"
        )
        print(f"R-squared: {stats_result['r_squared']:.3f}")
        print(f"Monthly growth: {stats_result['monthly_growth_rate']:+.1f}%")
    """
    df = df.copy()
    df[date_col] = pd.to_datetime(df[date_col])
    date_ordinals = df[date_col].map(pd.Timestamp.toordinal).values
    values = df[value_col].values

    slope, intercept, r_value, p_value, std_err = stats.linregress(
        date_ordinals, values
    )

    # Convert slope to monthly growth rate for interpretability
    # slope is in units per day (ordinal difference), so multiply by ~30
    avg_value = values.mean()
    daily_growth = slope / avg_value if avg_value != 0 else 0
    monthly_growth_rate = daily_growth * 30 * 100  # As percentage

    return {
        "slope": slope,
        "intercept": intercept,
        "r_squared": r_value ** 2,
        "p_value": p_value,
        "std_err": std_err,
        "monthly_growth_rate": monthly_growth_rate,
        "trend_direction": "increasing" if slope > 0 else "decreasing",
    }

What R-squared Means in Plain English

R-squared ranges from 0 to 1. It tells you what proportion of the variation in your data is explained by the trend line:

• R² = 0.95: 95% of the revenue variation is explained by the linear trend. The line is a very good fit.
• R² = 0.60: 60% of variation is explained by trend. A moderate fit — there is substantial variation that the trend does not capture.
• R² = 0.15: The trend explains only 15% of variation. The data is mostly noise around a weak trend.

For Priya's sales forecast, Sandra asked: "How confident should we be?" Priya's answer included the R-squared value: "The linear trend explains about 82% of the quarterly variation. The remaining 18% is the noise band you see in the confidence interval."

26.5 Percent Change Analysis

Absolute trend lines tell you direction. Percent change tells you rate of growth and lets you compare across different scales.

Month-Over-Month Change

def calculate_period_over_period_change(
    df: pd.DataFrame,
    value_col: str,
    periods: int = 1,
    label: str = "mom_change",
) -> pd.DataFrame:
    """
    Calculate period-over-period percent change.

    Args:
        df: DataFrame sorted by date (ascending).
        value_col: Numeric column to compute changes for.
        periods: Number of periods to look back (1 = month-over-month,
                 12 = year-over-year for monthly data).
        label: Column name for the change values.

    Returns:
        pd.DataFrame: Original DataFrame with added percent change column.

    Example:
        # Month-over-month
        df = calculate_period_over_period_change(df, "revenue", periods=1, label="mom_pct")
        # Year-over-year (monthly data)
        df = calculate_period_over_period_change(df, "revenue", periods=12, label="yoy_pct")
    """
    df = df.copy()
    df[label] = df[value_col].pct_change(periods=periods) * 100
    return df

Year-Over-Year Comparison

Year-over-year (YoY) comparison is often more meaningful than month-over-month because it removes seasonal effects:

def year_over_year_comparison(
    df: pd.DataFrame,
    date_col: str,
    value_col: str,
) -> pd.DataFrame:
    """
    Compute year-over-year growth rates for monthly or quarterly data.

    For monthly data: compares each month to the same month in the prior year.
    This removes seasonality from the growth calculation.

    Args:
        df: DataFrame with date and value columns.
        date_col: Date column name (will be parsed as datetime).
        value_col: Numeric column to compare.

    Returns:
        pd.DataFrame: DataFrame with 'year', 'period', 'value', and 'yoy_growth_pct'.
    """
    df = df.copy()
    df[date_col] = pd.to_datetime(df[date_col])
    df["year"] = df[date_col].dt.year
    df["month"] = df[date_col].dt.month

    # Pivot to wide format for easy same-month comparison
    pivot = df.pivot_table(
        index="month", columns="year", values=value_col, aggfunc="sum"
    )

    years = sorted(df["year"].unique())
    results = []

    for i in range(1, len(years)):
        current_year = years[i]
        prior_year = years[i - 1]

        if prior_year in pivot.columns and current_year in pivot.columns:
            for month in range(1, 13):
                if month in pivot.index:
                    current_val = pivot.loc[month, current_year]
                    prior_val = pivot.loc[month, prior_year]

                    if pd.notna(current_val) and pd.notna(prior_val) and prior_val != 0:
                        yoy = ((current_val - prior_val) / prior_val) * 100
                        results.append({
                            "year": current_year,
                            "month": month,
                            value_col: current_val,
                            "yoy_growth_pct": round(yoy, 2),
                        })

    return pd.DataFrame(results)

26.6 Seasonality Detection

Seasonality analysis answers: "Does this business have predictable patterns tied to the calendar?"

Groupby Analysis for Monthly Seasonality

def monthly_seasonality_profile(
    df: pd.DataFrame,
    date_col: str,
    value_col: str,
) -> pd.DataFrame:
    """
    Build a monthly seasonality profile by averaging across all years.

    Compute the average value for each calendar month and express it
    as a percentage of the overall monthly average. Values above 100%
    are above-average months; below 100% are below-average months.

    Args:
        df: DataFrame with date and numeric columns.
        date_col: Date column name.
        value_col: Numeric column to analyze.

    Returns:
        pd.DataFrame: One row per month with avg_value and seasonality_index.

    Example:
        profile = monthly_seasonality_profile(df, "sale_date", "revenue")
        # September might show seasonality_index=128, meaning 28% above average
    """
    df = df.copy()
    df[date_col] = pd.to_datetime(df[date_col])
    df["month"] = df[date_col].dt.month
    df["month_name"] = df[date_col].dt.strftime("%b")

    monthly_avg = (
        df.groupby(["month", "month_name"])[value_col]
        .mean()
        .reset_index()
        .rename(columns={value_col: "avg_value"})
        .sort_values("month")
    )

    overall_avg = monthly_avg["avg_value"].mean()
    monthly_avg["seasonality_index"] = (
        monthly_avg["avg_value"] / overall_avg * 100
    ).round(1)

    return monthly_avg


def quarterly_seasonality_profile(
    df: pd.DataFrame,
    date_col: str,
    value_col: str,
) -> pd.DataFrame:
    """
    Build a quarterly seasonality profile.

    Args:
        df: DataFrame with date and value columns.
        date_col: Date column name.
        value_col: Numeric column to analyze.

    Returns:
        pd.DataFrame: One row per quarter (Q1-Q4) with avg_value
                      and seasonality_index.
    """
    df = df.copy()
    df[date_col] = pd.to_datetime(df[date_col])
    df["quarter"] = df[date_col].dt.quarter

    quarterly_avg = (
        df.groupby("quarter")[value_col]
        .mean()
        .reset_index()
        .rename(columns={value_col: "avg_value"})
    )

    overall_avg = quarterly_avg["avg_value"].mean()
    quarterly_avg["seasonality_index"] = (
        quarterly_avg["avg_value"] / overall_avg * 100
    ).round(1)
    quarterly_avg["quarter_label"] = quarterly_avg["quarter"].map(
        {1: "Q1 (Jan-Mar)", 2: "Q2 (Apr-Jun)", 3: "Q3 (Jul-Sep)", 4: "Q4 (Oct-Dec)"}
    )

    return quarterly_avg

Interpreting seasonality indices for Acme Corp: If Q3 shows a seasonality index of 87, it means Q3 revenue is typically 13% below the annual average. Sandra can use this to adjust her linear trend forecast: if the trend says $1.2M for Q3, the seasonality-adjusted forecast is $1.2M × 0.87 = $1.04M.

26.7 Introduction to statsmodels: SimpleExpSmoothing and Holt's Method

statsmodels is a Python library for statistical modeling. For business forecasting, two methods are particularly accessible: Simple Exponential Smoothing and Holt's Linear Trend method.

pip install statsmodels

Simple Exponential Smoothing

Simple Exponential Smoothing (SES) is the formalized version of EMA for forecasting. It works well for data with no clear trend — essentially a sophisticated weighted average of past values:

from statsmodels.tsa.holtwinters import SimpleExpSmoothing
import pandas as pd
import numpy as np


def ses_forecast(
    values: pd.Series,
    forecast_periods: int = 3,
    smoothing_level: float | None = None,
) -> tuple[pd.Series, pd.Series]:
    """
    Apply Simple Exponential Smoothing and generate a forecast.

    SES is appropriate when data has no consistent trend or seasonality.
    If your data has a clear upward or downward trend, use Holt's method instead.

    Args:
        values: Numeric time series (pd.Series), evenly spaced.
        forecast_periods: Number of future periods to forecast.
        smoothing_level: Alpha parameter (0 to 1). Higher values put more
                         weight on recent observations. If None, statsmodels
                         optimizes it automatically.

    Returns:
        Tuple of (fitted_values, forecast_values) as pd.Series.

    Example:
        fitted, forecast = ses_forecast(monthly_revenue, forecast_periods=3)
    """
    model = SimpleExpSmoothing(values, initialization_method="estimated")

    if smoothing_level is not None:
        result = model.fit(smoothing_level=smoothing_level, optimized=False)
    else:
        result = model.fit(optimized=True)

    fitted = result.fittedvalues
    forecast = result.forecast(forecast_periods)

    return fitted, forecast

Holt's Linear Trend Method

Holt's method extends SES to handle data with a trend. It maintains two smoothed equations: one for the level (where the series is) and one for the trend (how fast it is changing):

from statsmodels.tsa.holtwinters import Holt


def holts_trend_forecast(
    values: pd.Series,
    forecast_periods: int = 3,
    smoothing_level: float | None = None,
    smoothing_trend: float | None = None,
) -> tuple[pd.Series, pd.Series]:
    """
    Apply Holt's Linear Trend method and generate a forecast.

    Appropriate when data has a consistent upward or downward trend
    but no strong seasonal pattern. For seasonal data, use
    statsmodels.tsa.holtwinters.ExponentialSmoothing with seasonal='add'.

    Args:
        values: Numeric time series with a consistent trend.
        forecast_periods: Number of periods to forecast.
        smoothing_level: Alpha (level smoothing). If None, auto-optimized.
        smoothing_trend: Beta (trend smoothing). If None, auto-optimized.

    Returns:
        Tuple of (fitted_values, forecast_values).

    Example:
        fitted, forecast = holts_trend_forecast(quarterly_revenue, forecast_periods=4)
    """
    model = Holt(values, initialization_method="estimated")

    fit_kwargs: dict = {}
    if smoothing_level is not None:
        fit_kwargs["smoothing_level"] = smoothing_level
    if smoothing_trend is not None:
        fit_kwargs["smoothing_trend"] = smoothing_trend

    result = model.fit(**fit_kwargs, optimized=(not fit_kwargs))

    fitted = result.fittedvalues
    forecast = result.forecast(forecast_periods)

    return fitted, forecast

When to Use Which Method

26.8 Confidence Intervals: Communicating Uncertainty

A point forecast ("Q3 revenue will be $1.2M") is almost certainly wrong. The real value will be higher or lower. A confidence interval tells you the range within which the actual value is likely to fall, given your model's uncertainty.

What Confidence Intervals Mean in Plain English

A 95% confidence interval does NOT mean "I am 95% sure the answer is in this range." It means: if we repeated this forecasting process many times with different data from the same process, 95% of those intervals would contain the true value.

For business communication, it is most useful to say: "Our best estimate for Q3 revenue is $1.15M to $1.28M, with the center of that range at $1.21M. We have modeled the historical variability in our revenue, and this range reflects that uncertainty."

Computing a Simple Forecast Band

A practical approach for business use: use the residuals (the differences between actual and fitted values) to estimate a standard error, then build a band around the forecast:

import numpy as np
import pandas as pd


def compute_forecast_with_confidence_band(
    historical_values: pd.Series,
    fitted_values: pd.Series,
    forecast_values: pd.Series,
    confidence_level: float = 0.95,
) -> pd.DataFrame:
    """
    Compute confidence bands around a point forecast using historical residuals.

    The band width is based on the standard deviation of historical residuals,
    which represents how much the model has typically been off in the past.
    Bands widen for further-ahead forecasts to reflect increasing uncertainty.

    Args:
        historical_values: Actual historical time series values.
        fitted_values: Model's fitted (in-sample) values.
        forecast_values: Model's out-of-sample forecast.
        confidence_level: Confidence level for the bands (default 0.95 = 95%).

    Returns:
        pd.DataFrame with columns: 'forecast', 'lower_bound', 'upper_bound',
        'is_forecast' for each forecast period.
    """
    from scipy import stats

    # Compute residuals from historical fit
    residuals = historical_values.values - fitted_values.values
    residual_std = np.std(residuals, ddof=1)  # Sample standard deviation

    # Z-score for given confidence level (1.96 for 95%)
    z_score = stats.norm.ppf((1 + confidence_level) / 2)

    forecast_rows = []
    for i, forecast_value in enumerate(forecast_values):
        # Uncertainty grows with forecast horizon (sqrt of horizon)
        horizon_multiplier = np.sqrt(i + 1)
        margin_of_error = z_score * residual_std * horizon_multiplier

        forecast_rows.append({
            "period_ahead": i + 1,
            "forecast": round(forecast_value, 2),
            "lower_bound": round(max(0, forecast_value - margin_of_error), 2),
            "upper_bound": round(forecast_value + margin_of_error, 2),
            "margin_of_error": round(margin_of_error, 2),
        })

    return pd.DataFrame(forecast_rows)

26.9 Visualizing Forecasts with matplotlib

Numbers alone rarely persuade. The forecast chart — showing historical data, the trend line, the point forecast, and the confidence band — is what Sandra needs for the board deck.

import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
import pandas as pd
import numpy as np


def plot_forecast(
    historical_df: pd.DataFrame,
    forecast_df: pd.DataFrame,
    date_col: str,
    value_col: str,
    forecast_date_col: str = "forecast_date",
    title: str = "Sales Forecast",
    value_label: str = "Revenue ($)",
    source_note: str = "",
) -> plt.Figure:
    """
    Create a professional forecast chart with confidence bands.

    Shows: actual historical data (solid line), trend line (dashed),
    point forecast (open circles), and shaded confidence band.

    Args:
        historical_df: DataFrame with historical dates and values.
        forecast_df: DataFrame from compute_forecast_with_confidence_band(),
                     must have 'forecast', 'lower_bound', 'upper_bound' columns.
        date_col: Date column in historical_df.
        value_col: Value column in historical_df.
        forecast_date_col: Date column in forecast_df.
        title: Chart title.
        value_label: Y-axis label.
        source_note: Optional footnote (e.g., data source and date range).

    Returns:
        matplotlib.figure.Figure: The completed chart figure.
    """
    fig, ax = plt.subplots(figsize=(12, 6))
    fig.patch.set_facecolor("#fafafa")
    ax.set_facecolor("#fafafa")

    # Plot historical data
    ax.plot(
        historical_df[date_col],
        historical_df[value_col],
        color="#0066cc",
        linewidth=2,
        label="Actual",
        zorder=3,
    )

    # Plot trend line if available
    if "trend_line" in historical_df.columns:
        ax.plot(
            historical_df[date_col],
            historical_df["trend_line"],
            color="#ff6600",
            linewidth=1.5,
            linestyle="--",
            label="Trend",
            zorder=2,
            alpha=0.8,
        )

    # Plot forecast point estimates
    if forecast_date_col in forecast_df.columns:
        ax.plot(
            forecast_df[forecast_date_col],
            forecast_df["forecast"],
            color="#0066cc",
            linewidth=2,
            linestyle="--",
            marker="o",
            markerfacecolor="white",
            markeredgecolor="#0066cc",
            markeredgewidth=2,
            markersize=8,
            label="Forecast",
            zorder=3,
        )

        # Shade confidence band
        if "lower_bound" in forecast_df.columns:
            ax.fill_between(
                forecast_df[forecast_date_col],
                forecast_df["lower_bound"],
                forecast_df["upper_bound"],
                alpha=0.2,
                color="#0066cc",
                label="95% Confidence Band",
                zorder=1,
            )

    ax.set_title(title, fontsize=14, fontweight="bold", pad=16)
    ax.set_xlabel("Date", fontsize=11)
    ax.set_ylabel(value_label, fontsize=11)
    ax.legend(loc="upper left", fontsize=10)
    ax.grid(True, alpha=0.3, linestyle=":")

    # Format y-axis with commas for large numbers
    ax.yaxis.set_major_formatter(
        plt.FuncFormatter(lambda x, _: f"${x:,.0f}" if x >= 0 else "")
    )

    if source_note:
        fig.text(
            0.01, 0.01,
            source_note,
            fontsize=8,
            color="#888888",
            transform=fig.transFigure,
        )

    plt.tight_layout()
    return fig

26.10 Acme Corp Scenario: Priya's Quarterly Sales Forecast

Priya's deliverable for Sandra is a forecast of next quarter's revenue by region. Here is her workflow, simplified:

"""
acme_sales_forecast_workflow.py — Abbreviated version showing the approach.
See acme_sales_forecast.py in the code/ directory for the full implementation.
"""

import pandas as pd
import numpy as np
from scipy import stats
from datetime import datetime


def build_quarterly_forecast(
    df: pd.DataFrame,
    region: str,
    quarters_ahead: int = 2,
) -> dict:
    """
    Build a quarterly sales forecast for one Acme Corp region.

    Args:
        df: Sales DataFrame with 'sale_date', 'region', and 'revenue'.
        region: Region name to forecast.
        quarters_ahead: Number of quarters to forecast ahead.

    Returns:
        dict with 'region', 'forecast_quarters', 'regression_stats',
        'seasonality_note'.
    """
    # Filter to region and aggregate to quarterly
    region_df = df[df["region"] == region].copy()
    region_df["sale_date"] = pd.to_datetime(region_df["sale_date"])
    region_df["quarter"] = region_df["sale_date"].dt.to_period("Q")

    quarterly = (
        region_df.groupby("quarter")["revenue"]
        .sum()
        .reset_index()
        .sort_values("quarter")
    )
    quarterly["quarter_ordinal"] = range(len(quarterly))

    # Linear regression
    slope, intercept, r_value, p_value, std_err = stats.linregress(
        quarterly["quarter_ordinal"],
        quarterly["revenue"],
    )

    # Confidence band from residuals
    fitted = slope * quarterly["quarter_ordinal"] + intercept
    residuals = quarterly["revenue"] - fitted
    residual_std = np.std(residuals, ddof=1)

    # Forecast next N quarters
    last_ordinal = quarterly["quarter_ordinal"].max()
    last_period = quarterly["quarter"].max()
    z_95 = 1.96

    forecast_quarters = []
    for i in range(1, quarters_ahead + 1):
        future_ordinal = last_ordinal + i
        point_forecast = slope * future_ordinal + intercept
        margin = z_95 * residual_std * np.sqrt(i)
        future_quarter = last_period + i

        forecast_quarters.append({
            "quarter": str(future_quarter),
            "point_forecast": round(max(0, point_forecast), 2),
            "lower_bound": round(max(0, point_forecast - margin), 2),
            "upper_bound": round(point_forecast + margin, 2),
        })

    return {
        "region": region,
        "r_squared": round(r_value ** 2, 3),
        "quarterly_growth_rate": round((slope / quarterly["revenue"].mean()) * 100, 2),
        "forecast_quarters": forecast_quarters,
    }

When Priya presented to Sandra, she showed the forecast with two explicit caveats:

1. "The R-squared is 0.78, meaning the linear trend explains about 78% of the quarterly variation. The remaining 22% is captured in the confidence band."

2. "This model assumes the trend that held over the past eight quarters continues at the same rate. If there is a macro change — new competitor, supply disruption — the model will not capture it. These numbers should be treated as a baseline, not a guarantee."

Sandra nodded. "That's the kind of analysis I can use. Now show me the chart."

26.11 Maya Reyes: Forecasting Consulting Revenue

Maya uses a different approach. Her revenue comes from projects with known durations and rates, so she can forecast from her pipeline as well as from historical patterns.

Her approach combines two signals: 1. Pipeline-based forecast: Known projects × expected hours × rate = committed revenue 2. Historical-pattern forecast: Based on seasonal patterns in her past billing

"""
maya_revenue_forecast.py — Abbreviated workflow.
See case-study-02.md for full scenario details.
"""

import pandas as pd
from datetime import date, timedelta


def pipeline_based_forecast(
    projects_df: pd.DataFrame,
    months_ahead: int = 6,
    hourly_rate: float = 175.0,
) -> pd.DataFrame:
    """
    Forecast Maya's revenue based on her current active project pipeline.

    Args:
        projects_df: DataFrame from maya_projects.csv.
        months_ahead: How many months to forecast.
        hourly_rate: Maya's standard hourly rate.

    Returns:
        pd.DataFrame: Monthly forecast with committed and probabilistic revenue.
    """
    today = date.today()
    months = []

    for month_offset in range(months_ahead):
        month_start = (
            date(today.year, today.month, 1) +
            pd.DateOffset(months=month_offset)
        ).date()
        month_end = (month_start + pd.DateOffset(months=1) - pd.DateOffset(days=1)).date()

        # Projects active during this month
        active = projects_df[
            (pd.to_datetime(projects_df["start_date"]).dt.date <= month_end)
            & (pd.to_datetime(projects_df["deadline"]).dt.date >= month_start)
            & (projects_df["completion_percent"] < 100)
            & (~projects_df["on_hold"].astype(bool))
        ]

        # Estimate billable hours: completion pace suggests remaining hours
        estimated_monthly_hours = active.shape[0] * 15  # Rough: 15h/project/month
        committed_revenue = estimated_monthly_hours * hourly_rate

        months.append({
            "month": month_start.strftime("%Y-%m"),
            "active_projects": active.shape[0],
            "estimated_hours": estimated_monthly_hours,
            "committed_revenue": committed_revenue,
        })

    return pd.DataFrame(months)

26.12 Business Interpretation: Presenting Forecasts Responsibly

The most technically correct forecast, poorly communicated, is useless. The most useful forecasts balance accuracy with interpretability.

What to Say, and What Not to Say

The Limitations That Must Be Stated

Every forecast presentation should include explicit limitations:

1. Model assumptions: Linear trend assumes the same rate of growth continues. If the business environment changes significantly, the model will be wrong.
2. Data quality: Forecasts are only as good as the historical data. If the data has errors or gaps, the forecast inherits them.
3. Event risk: Major unexpected events (economic shock, competitive disruption, operational crisis) are not captured in any trend-based model.
4. Horizon confidence: Forecasts one period ahead are more reliable than forecasts four periods ahead. Confidence bands widen with horizon for this reason.

Sandra's reaction when Priya added these caveats to the board deck: "This is exactly right. I can defend this. I can't defend a number with no context."

26.13 Chapter Summary

You have built a complete business forecasting toolkit. Here is what you can now do:

Understand time series components: trend, seasonality, cyclicality, and noise are the fundamental vocabulary for describing how data changes over time.

Apply moving averages: SMA, WMA, and EMA smooth historical data to reveal trends, with each offering different trade-offs between responsiveness and smoothness.

Extrapolate trends linearly: numpy.polyfit and scipy.stats.linregress fit trend lines to historical data and project them forward. linregress additionally provides R-squared and p-values for assessing fit quality.

Analyze seasonality: groupby month and quarter reveals predictable calendar-based patterns that should adjust point forecasts.

Apply statsmodels methods: Simple Exponential Smoothing for flat-trend data and Holt's method for trending data provide more statistically rigorous forecasts than simple extrapolation.

Compute and communicate confidence intervals: forecast bands based on historical residuals give honest ranges rather than false-precision point estimates.

Visualize forecasts professionally: the forecast chart with confidence band is the standard presentation format for business forecasting.

Communicate uncertainty honestly: every forecast should include its model assumptions, data quality context, and explicit limitations.

Priya delivered the quarterly forecast to Sandra with a two-page summary: one page of methodology and limitations, one page of charts. Sandra presented it to the board as "our data-driven baseline with a defensible range." The board asked intelligent questions. Nobody demanded impossible certainty. That is the best outcome a business forecast can achieve.

What Is Next

Chapter 27 explores customer analytics — segmentation, lifetime value, and churn analysis. Chapter 28 applies the forecasting tools from this chapter to sales pipeline analysis: which deals are most likely to close, and what does that mean for next quarter's revenue?

End of Chapter 26