Moving the objective one step downstream, for Grand Canyon University

The problem

A large university split a flat media budget across four channels every quarter. TV, third-party internet leads, branded paid search, unbranded paid search. Every quarter the same argument: the school had one set of numbers, the vendors had another, and neither side could prove the other wrong.

Underneath the argument sat a worse problem. The school optimised on cost per inquiry, because an inquiry shows up in days. What a school actually pays for is a student who enrols, and that takes weeks to see. The channels producing the cheapest inquiries produced the worst start rates. The plan that looked best on the visible number performed worst on the real one.

What we did

We moved the objective one step downstream, from cost per inquiry to cost per enrolled start, and built one weekly document where both numbers sat next to each other. The Director of Paid Media kept their number. The VP of Enrolment kept theirs. The allocation arbitrated between them.

Then we asked the vendors for a number nobody had asked for before. Not a forecast, which a vendor hedges to protect their reputation. A boast: the most you could credibly deliver if we asked. Vendors inflate a boast, and an optimiser knows how to dial back from a ceiling. It does not know what to do with a hedged midpoint.

The word stuck. "What is your boast for Q3" turned out to be a better question than "how many leads can you commit to," and it gave both sides one thing to argue about instead of four.

What happened

We backtested the method against four quarters of held-out actuals. Cost per enrolled start came down in every quarter, with a rise in enrolled starts alongside it. No quarter got worse.

Then it ran in production, on real money, for a real quarter. The result landed inside the projected band.

The consistency mattered more than the size. A planner defending a rebalanced budget to a CFO needs four bars pointing the same direction, not one good quarter.

The related work that followed, across three institutions running more than $2.5M a month in media spend, improved return on that spend by about 10% within a quarter and took roughly half a full-time role out of the monthly cycle.

What it cost them to find out

Nothing on the pilot itself, which is the point of running one. What it had been costing for years was the gap between the two numbers. Every quarter of buying the cheapest inquiries was a quarter of paying more per enrolled student than necessary, and nobody could see it, because the two metrics lived in two reporting lines that never met in the same spreadsheet.

What we got wrong, and where this does not apply

Two things we got wrong. We applied a flat ten percent upside ceiling to every vendor because we ran out of time to negotiate individual boasts before the quarter started. Some vendors had far more headroom than that. We left money on the table.

The optimiser also recommended cutting deeply into one brand channel, and the school did not act on it. They were probably right. Brand effects on branded-search performance were real and a single-objective model could not see them. That is an attribution problem, and it is the part of this work that has genuinely changed since 2012.

The full methodology, the five steps, and the scorecard weighting, on Jeff’s personal site.