Case Study 1 — RYCC Builds an Evaluation Funders Trust
Composite, for teaching. RYCC, the data, and the funder are illustrative.
The Situation
Denise Okafor's first draft evaluation reads: "We will track the number of students served and survey participants on their satisfaction with the program." This chapter tells her that this is a process measure plus a vanity measure — and that a funder increasingly wants to know whether the program actually works.
Applying the Chapter
She builds the logic model first. Inputs (funding, instructors, space, curriculum) → activities (the classes) → outputs (90 students served, 5,400 contact-hours) → outcomes (proficiency gains → tech-interest → HS CS enrollment) → impact (narrowed digital-skills gap). Building it, Denise sees clearly that her first draft measured only the outputs (students served) and a vanity indicator (satisfaction), and never touched the outcomes the program exists to produce.
She writes SMART objectives for each outcome. "By program end, ≥70% of students will show a measurable proficiency gain on a validated pre/post assessment." "Within one year, ≥40% of completers will enroll in a high-school CS course." Each has an indicator, a justified target (grounded in her existing site's track record), a data source, and a method/timing — laid out as a clean matrix a reviewer can scan.
She includes process and outcome measures. Process: were the classes delivered at all three sites, was attendance maintained, was the curriculum followed? Outcome: did proficiency, interest, and enrollment change? She explains why both: if outcomes disappoint, the process data will tell her whether the model needs strengthening or the implementation faltered.
She builds in formative evaluation. A midpoint review of attendance and early proficiency data lets RYCC course-correct a struggling site while there's still time — signaling to the funder that RYCC will catch and fix problems during the grant, not just report them at the end.
She matches the evaluator to the scale. For a \$50,000 program, an expensive external evaluator would crowd out the program. Denise plans rigorous internal evaluation with validated instruments and honest methods, which is appropriate to the stakes and budget — and she says so explicitly, showing she's thought about the question.
The Trap She Avoids
Denise's first draft — outputs plus satisfaction — would have read to a modern funder as a relic: an applicant who counts activity and measures whether people liked the program, not whether it worked. By building a logic model, writing SMART outcome objectives, and including process measures, she converts a vague report-card promise into a credible accountability plan.
The Payoff
RYCC's evaluation now shows a reviewer exactly what change the program will produce, how RYCC will measure it, what counts as success, and how RYCC will course-correct along the way. In a competitive round, this is often the section that separates RYCC from an otherwise-comparable applicant who promised only to "track progress and report success." And the outcome data RYCC collects will become the track record that makes its next grant easier to win.
Discussion Questions
- Denise's first draft measured "satisfaction." Why is satisfaction a vanity indicator here, and what should it be replaced with?
- RYCC chose internal evaluation for a \$50,000 grant. Was that right? When would the answer change?
- How does building the logic model first prevent the output/outcome confusion that plagued Denise's first draft?