Case Study 2 — Lighthouse Adds an Outcome Evaluation (and an Evaluator)
Composite, for teaching. Lighthouse Community Services and the funder are illustrative.
The Situation
Lighthouse Community Services is applying to a large federal workforce program for \$600,000 to expand its reentry job-training program. Its draft evaluation says: "We will report the number of participants enrolled and the number who complete the program." For a small foundation grant this might pass; for a high-stakes federal grant scored against a rubric, this chapter's lessons say it is dangerously thin.
Applying the Chapter
She matches rigor to the stakes. Lighthouse's evaluation director recognizes that a \$600,000 federal grant carries high stakes and that the funder requires specific performance measures and likely expects an independent evaluator. Output counts alone won't survive the rubric. The evaluation must measure real outcomes and do so credibly.
She maps to the funder's required measures. Reading the NOFO (Chapter 3), she finds the program's required performance measures: credential attainment, job placement, and six-month retention. She writes SMART objectives for each, in the funder's own metric language, and adds a column to her evaluation matrix mapping her indicators to the program's required measures — signaling she'll report exactly what the funder is accountable for (Chapter 7).
She designs a real outcome evaluation with comparison. Rather than just counting completers, she specifies measuring placement and retention against targets justified by Lighthouse's track record, and — because the stakes justify the cost — a comparison against baseline and, where feasible, a comparison group, so the results are interpretable as impact rather than mere counts.
She engages an external evaluator early. Given the scale and the funder's expectation of independence, Lighthouse contracts a qualified external evaluator — and brings them in during proposal development, so the evaluation design is sound and the evaluator's letter of commitment reflects a real plan, not a borrowed name. The evaluation becomes a budget line (Chapter 11), which Lighthouse justifies as the cost of the credible results the funder requires.
She adds process evaluation. Implementation fidelity, reach, and dosage are tracked so that outcome results can be interpreted — essential for a program whose success depends on participants actually completing intensive training.
The Trap She Avoids
Lighthouse's draft — enrollment and completion counts — would have lost points mechanically against a rubric that weights outcomes and evaluation heavily, and would have signaled to a sophisticated federal reviewer that Lighthouse didn't grasp what the program is accountable for. By building a real outcome evaluation mapped to the required measures, with an external evaluator and a comparison basis, Lighthouse meets the rubric and the stakes.
The Payoff
Lighthouse's evaluation now speaks the funder's metric language, measures genuine outcomes against a comparison, carries the credibility of independence, and includes the process data to interpret results. It is sized correctly for a high-stakes grant — neither under-rigorous nor over-engineered — and it positions Lighthouse to report exactly what the funder needs, strengthening the relationship for future funding.
Discussion Questions
- Lighthouse used an external evaluator while RYCC (Case Study 1) used internal evaluation. Explain why both choices were correct for their respective situations.
- Mapping indicators to the funder's "required performance measures" did double duty. What were the two benefits?
- Why does bringing the external evaluator in during proposal development matter, beyond just having a letter?