Capstone 3 Grading Rubric: The Campaign Analytics Plan

Total points: 100 Deliverable weights: D1: 25 pts | D2: 25 pts | D3: 20 pts | D4: 20 pts | D5: 10 pts

Deliverable 1: Voter Universe Analysis (25 points)

Component 1A: Universe Construction Rationale — Written Document (10 points)

Sub-component 1A-i: Threshold justification for all three universes (4 points)

Score Criteria
4 Each universe's entry criteria are explained with reasoning that goes beyond circular definition. Persuasion threshold (support score 40-60) is justified with reference to what voters at the boundary actually look like — not just "they're uncertain." GOTV threshold is explained in terms of what makes high-support/medium-turnout voters more valuable than high-support/high-turnout voters. Fundraising universe logic is clearly distinct from the persuasion/GOTV logic, drawing on donor profile analysis.
3 All three universes have threshold explanations. One or two explanations are somewhat circular or generic.
2 Universe thresholds are stated but only one receives substantive justification.
1 Thresholds stated without meaningful justification.
0 Not present.

Sub-component 1A-ii: Turnout propensity proxy documentation (2 points)

Score Criteria
2 Proxy construction is documented with specific weights for each vote history cycle and age adjustment, with reasoning for each weighting choice. Student explains why recency weighting (giving 2022 more weight than 2018) is methodologically appropriate. Uncertainty in the proxy is acknowledged.
1 Proxy is documented but weighting choices are not justified, or recency weighting logic is absent.
0 Not documented.

Sub-component 1A-iii: County-level strategic analysis (4 points)

Score Criteria
4 Three counties identified with specific strategic rationale for each. Strategic reasoning is grounded in the universe analysis data (universe size, demographic composition, expected margins) rather than generic characterizations. Each county's primary program assignment (persuasion vs. GOTV) is appropriate for its universe composition.
3 Three counties identified with reasonable rationale. One or two selections rely on characterizations not clearly grounded in the data.
2 Fewer than three counties analyzed substantively, or all three rely primarily on characterization rather than data.
1 County-level analysis is superficial or absent.
0 Not present.

Component 1B: Python Universe Construction (15 points)

Sub-component 1B-i: Turnout propensity calculation (3 points)

Score Criteria
3 Turnout propensity is calculated for all voters using a function that: (a) incorporates all three vote history variables, (b) applies differential weighting with reasoning, (c) includes some form of age adjustment, and (d) produces reasonable output distribution (most voters between 20-80, with distinct patterns for high/low vote history).
2 Turnout propensity calculated but missing one of the four components or weighting is undifferentiated (all three cycles equal weight).
1 Turnout propensity attempted but incorrect — e.g., only using one cycle, or calculating something unrelated to voting behavior.
0 Not implemented.

Sub-component 1B-ii: Persuasion universe with priority tiers (4 points)

Score Criteria
4 Persuasion universe applies all three required filters (support score range, persuadability threshold, turnout propensity minimum). Priority tiers are implemented with clear, defensible criteria creating meaningfully different groups. Universe output is reported (size, tier breakdown) and is consistent with the written rationale.
3 Persuasion universe implemented correctly. Tiers present but criteria for tier assignment are not clearly differentiated (e.g., Tier 1 and Tier 2 look nearly identical).
2 Persuasion universe implemented but missing one filter, or tiers not implemented.
1 Persuasion universe attempted but substantially incorrect.
0 Not implemented.

Sub-component 1B-iii: GOTV universe (3 points)

Score Criteria
3 GOTV universe correctly applies support score minimum AND turnout propensity window (not just minimum). Priority designation distinguishes high-value GOTV targets from standard. Universe size is reported and is distinct from the persuasion universe (not simply the mirror image).
2 GOTV universe implemented but uses only support score filter (missing turnout propensity window), or priority designation is absent.
1 GOTV universe attempted but substantially incorrect.
0 Not implemented.

Sub-component 1B-iv: County summary table and visualizations (3 points)

Score Criteria
3 County summary table includes all required columns (universe size, priority tier counts, relevant demographic breakdowns) for all counties. Both visualizations are appropriate for the data being shown, clearly labeled, and add analytical value (they reveal something useful, not just repeat the table).
2 County summary table present but incomplete. One visualization present or both are appropriate.
1 County summary or visualizations present but minimal.
0 Not present.

Sub-component 1B-v: Code quality and reproducibility (2 points)

Score Criteria
2 Code is clearly commented, logically organized, and reproducible from the provided dataset without modification. Variable names are meaningful. Results are interpretable without running the code (printed outputs have clear labels).
1 Code runs but is poorly commented or organized, or requires modification to run.
0 Code does not run or is entirely absent.

Deliverable 2: Targeting Strategy and Contact Program (25 points)

Component 2A: Demographic Targeting (6 points)

Score Criteria
6 All four demographic priority segments are addressed with: current support level (appropriately sourced from polling data), specific strategic rationale (not just "important demographic"), and message track assignments that follow logically from the segment's priorities. The reasoning explains why these four segments and not others — what makes them the highest-leverage targets.
5 Four segments addressed with good rationale. One segment's message assignment is poorly justified or support level is not sourced from the data.
4 Four segments addressed but rationale is largely generic ("Latino voters are important in this state") without demographic specifics.
3 Fewer than four segments addressed substantively.
2 Targeting strategy is present but not connected to segment-specific data or rationale.
1 Superficial targeting discussion.
0 Not present.

Component 2B: Geographic Resource Allocation (6 points)

Score Criteria
6 Expected-value framework is applied (even if informally) to justify geographic allocation. The three priority counties are identified with specific quantitative reasoning: universe sizes, expected contact rates, expected lift per contact. The student shows, not just asserts, why these counties receive the most investment.
5 Priority counties identified with good reasoning. Expected-value framework referenced but not fully quantified — reasoning is primarily qualitative.
4 Priority counties identified. Reasoning is intuitive (large diverse county = high investment) without quantitative grounding.
3 Geographic allocation present but not connected to universe analysis from Deliverable 1.
2 Generic geographic allocation without county-specific reasoning.
1 Geographic component superficial or absent.
0 Not present.

Component 2C: Contact Program (per-program detail) (10 points, 2 pts per program)

For each of the five programs (canvassing, phone, mail, text, digital):

Score Criteria per program
2 Universe/timeline/projected contacts/KPIs/resource requirements all present. Projected contacts are realistic given stated contact rates and universe sizes. KPIs are specific and measurable. Timeline is specific (not just "throughout the campaign"). Resource requirements are concrete.
1 Most elements present but one or two are missing or imprecise. Projected contacts may not be internally consistent with stated rates.
0 Program is missing or substantially incomplete.

Component 2D: Message Matrix and Integration Note (3 points)

Score Criteria
3 Message matrix is complete (all segments × primary/secondary tracks × channel priority). Integration note explains specifically how digital targeting lists and VAN field data will be coordinated (not just "they will be integrated").
2 Message matrix complete. Integration note is present but vague ("we will coordinate digital and field").
1 Message matrix incomplete or integration note absent.
0 Not present.

Deliverable 3: Polling and Measurement Framework (20 points)

Component 3A: Internal Polling Plan (8 points)

Sub-component 3A-i: Four survey specifications (4 points, 1 pt each)

Each of the four surveys (benchmark + three tracking) receives 1 point if it specifies: timing, sample size, key questions beyond horse race, and what would be done differently depending on results. Award 0.5 for partial specification.

Sub-component 3A-ii: Split-sample message test design (2 points)

Score Criteria
2 Two specific framings are identified for testing. The question used to measure effect is appropriate for detecting the expected difference between framings. The student describes what they would do differently depending on whether Framing A or Framing B performs better — not just "we'd use the winning one" but how it would change the contact program.
1 Test is designed but one element is missing or underdeveloped.
0 Not present.

Sub-component 3A-iii: Internal vs. public polling reconciliation (2 points)

Score Criteria
2 Student correctly identifies that the public aggregate is generally a better estimate of reality than a single internal poll, explains the reasons an internal poll might diverge upward from the aggregate (sampling optimism, question order, favorable field timing), and specifies what internal polls are actually useful for (diagnostics, message testing) when they conflict with the aggregate. Demonstrates Nadia's hedged, evidence-based perspective on internal polling.
1 Correct answer (trust the aggregate) without adequate explanation of why.
0 Incorrect or not addressed.

Component 3B: Measurement Framework (12 points)

Sub-component 3B-i: Consolidated KPI table (4 points)

Score Criteria
4 KPI table is complete (all five programs), specific (quantitative targets where appropriate), and internally consistent with the contact program from Deliverable 2. KPIs distinguish between leading indicators (contact rate, volunteer retention) and lagging indicators (early vote rate, universe penetration).
3 KPI table complete and mostly specific. One or two KPIs are vague or inconsistent with Deliverable 2.
2 KPI table covers most programs but missing specificity.
1 KPI table present but substantially incomplete.
0 Not present.

Sub-component 3B-ii: Sample weekly brief (5 points)

Score Criteria
5 Brief follows the four-section format exactly. Numbers are internally consistent (percentages match universe sizes from Deliverable 1). Section 2 (What the numbers mean) demonstrates genuine analytical interpretation — not just summarizing Section 1 in different words. Section 3 (Decisions needed) presents a real decision with options, expected outcomes, and a recommendation. Section 4 (What to watch) identifies genuine leading indicators. Tone is professional and appropriately hedged — not falsely optimistic or falsely pessimistic.
4 Brief follows format. Four sections present. One section is thin (usually Section 4) or numbers are not fully consistent with Deliverable 1.
3 Format followed. Content is present but Section 2 or 3 is underdeveloped.
2 Brief is present but does not follow the four-section format, or two or more sections are superficial.
1 Brief is substantially incomplete.
0 Not present.

Sub-component 3B-iii: A/B test designs (3 points, 1 pt each)

Each test receives 1 point if it specifies: what is being tested, randomization method, outcome measure, and whether the sample size is adequate for the expected effect size (reasoning required, not a power calculation).

Deliverable 4: Budget and Ethics/Equity Review (20 points)

Component 4A: Budget Memo (8 points)

Sub-component 4A-i: Budget table with rationale (4 points)

Score Criteria
4 All four major categories are present with line items. Dollar amounts are internally consistent across the plan (mail budget consistent with mail volume from Deliverable 2, polling budget consistent with the survey plan from Deliverable 3). One-sentence justifications are specific to the line item — not just "this is important" but what the money buys and why that's worth the cost. Total is credible for a competitive mid-sized Senate race.
3 All four categories present. One or two line items are implausible or inconsistent with the rest of the plan. Justifications present.
2 Budget table present but not internally consistent with the contact program, or missing a major category.
1 Budget is present but cursory.
0 Not present.

Sub-component 4A-ii: Case for analytics investment (2 points)

Score Criteria
2 Argument makes at least two distinct claims, each grounded in a specific mechanism (not just "data helps"). Arguments address Renata's perspective as a campaign manager — specifically what she gets for the investment in terms she can take to the finance team.
1 Argument is present but makes only one distinct claim or is entirely generic.
0 Not present.

Sub-component 4A-iii: Contractor vs. staff decision (2 points)

Score Criteria
2 One specific function is identified for contracting rather than staffing, with reasoning that engages with the specific tradeoffs (cost, control, expertise, availability). Reasoning is connected to the principle in Section 7 ("defined deliverable vs. strategic judgment").
1 Decision is made but reasoning is generic or doesn't engage with the tradeoffs.
0 Not present.

Component 4B: Ethics and Equity Review (12 points)

Sub-component 4B-i: Nadia's test applied to three components (4 points)

Score Criteria
4 Three specific program components are assessed. At least one genuinely fails the test or requires modification — if all three pass easily, the student has not applied the test seriously. For the component(s) that require attention, the modification is specific and would actually address the concern (not just a policy statement).
3 Three components assessed. One receives genuine scrutiny. Two are assessed superficially.
2 Three components assessed but all pass too easily. No genuine tensions identified.
1 Test is applied to fewer than three components, or applied entirely superficially.
0 Not present.

Sub-component 4B-ii: Equity audit (5 points)

Score Criteria
5 Three equity checklist questions applied with specific answers. At least one identifies a genuine equity gap in the student's own plan (not just a generic concern). For each identified gap, a concrete remediation is proposed that would actually change something in the plan. The student demonstrates understanding that equity review is about methodology, not just intentions.
4 Three questions applied. Two receive specific answers. One genuine gap identified with a specific remediation.
3 Three questions applied but answers are primarily generic ("we will ensure equitable representation") without specific methodological implications.
2 Fewer than three questions applied substantively.
1 Equity review is present but entirely formal — no specific issues or remediations.
0 Not present.

Sub-component 4B-iii: Microtargeting ethics constraint (3 points)

Score Criteria
3 Constraint is specific: names a particular data type or targeting practice that is being excluded, explains why (with reference to the asymmetric information argument from Section 3 — that voters have not consented to this use of their data), and acknowledges the cost (this constraint may reduce targeting efficiency).
2 Constraint is present. Reference to asymmetric information or consent is implicit rather than explicit.
1 Constraint stated without substantive ethical reasoning.
0 Not present.

Deliverable 5: Executive Summary (10 points)

Component 5A: One-Page Overview (4 points)

Score Criteria
4 All six sections present and within appropriate length. Universe summary table is legible and complete. Contact program table shows projections that are consistent with Deliverable 2. Key decisions are genuine (not trivial or already decided). Risk flags are specific and campaign-relevant (not generic). Bottom line sentence is specific, appropriately uncertain, and honest.
3 All sections present. One or two sections are thin or slightly inconsistent with the detailed plan.
2 Most sections present but the overview feels like a generic template rather than a specific plan for this campaign.
1 Overview present but substantially incomplete or inconsistent with the underlying plan.
0 Not present.

Component 5B: Executive Summary (6 points)

Score Criteria
6 Eight-paragraph structure followed. Written for a non-technical audience — no jargon, no code references, no unexplained acronyms. The strategic situation is clearly and honestly conveyed (not just "we're in a good position"). Universe logic is explained in plain language without losing the key insight. Contact program rationale is accessible. Measurement and ethics sections are substantive, not pro forma. Budget section demonstrates the analytics team's value without being self-promotional. The ask is clear. Tone is professional, confident, and appropriately hedged. Would pass Nadia's "hand to Renata" test.
5 All eight paragraphs present and substantive. Minor jargon issues or one paragraph that reads as generic.
4 Structure followed. Content is present but some paragraphs feel like summaries rather than analytical communication.
3 Executive summary is present but does not follow the specified structure, or two or more paragraphs are superficial.
2 Executive summary is substantially incomplete.
1 Minimal effort.
0 Not present.

Grade Thresholds

Score Range Grade
90-100 A
80-89 B
70-79 C
60-69 D
Below 60 F

Notes for Graders

On universe construction: There is legitimate variation in threshold choices. A student who sets the persuasion support score range at 38-62 rather than 40-60 is not wrong — if they justify the choice. Grade the reasoning, not just the number. The most common errors are: (1) not applying a minimum turnout threshold to the persuasion universe, (2) creating a GOTV universe without an upper turnout bound (which leads to wasting GOTV resources on certain voters), and (3) defining the fundraising universe as simply the top of the support score distribution.

On contact program projections: Students frequently overestimate canvassing contact rates (claiming 50%+ for door-to-door contact, when 25-35% is realistic) and underestimate phone banking abandonment (forgetting that many phone numbers are bad). If a student's projected contacts are inconsistent with stated contact rates and universe sizes, deduct points from the relevant sub-component.

On the budget: Dollar amounts need to be internally consistent and credible, not exactly matching the textbook examples. A student who builds a plan around $500K (a leaner operation) and maintains internal consistency should not be penalized for differing from the textbook's $1.12M example. A student who claims to run a full statewide canvassing and mail program for $50K should be flagged for unrealistic budgeting.

On the executive summary: This is the deliverable where students most commonly either over-write (including methodology details that don't belong in an executive summary) or under-write (producing a bullet-point summary instead of a narrative document). The best executive summaries tell a coherent story about the campaign's strategic situation and the analytics plan's role in responding to it. They do not simply summarize each deliverable.

On internal consistency: The most important cross-deliverable check is whether the contact program from D2, the measurement framework from D3, and the budget from D4 are all internally consistent with the universe analysis from D1. A persuasion universe of 3,000 voters combined with a canvassing plan that projects 50,000 canvassing contacts is incoherent — the canvassing program is larger than the universe. When you find these inconsistencies, note them as a specific deduction from the relevant later deliverable.