If the NCAA really wants to replace RPI, it should go with something like WAB

The NCAA announced today that it has invited several folks—most notably Jeff Sagarin and Ken Pomeroy—to Indianapolis next Friday (Jan. 20th) for a meeting "designed to speed the inclusion of more metrics in the selection process for the basketball tournament." Specifically, at the behest of the coaches, the NCAA is looking into using "advanced metrics" to help with the selection process: "an even more powerful microscope to go with the time-honored RPI."

Leaving aside the hilarious image of the RPI as a "powerful microscope," this is a potentially positive development. I certainly trust Ken Pomeroy to advocate effectively for a reasonable replacement for the deeply flawed RPI. But skepticism is warranted. The RPI persists for a few main reasons: (1) massive institutional inertia; (2) it's the devil they (the coaches) know; (3) it can be gamed, but only consistently by teams with scheduling power—i.e., the teams with money and power within college basketball. None of these things are going away or even fading in importance.

Based on the comments of Dan Gavitt, NCAA vice president for basketball, the push for something to at least augment the RPI seems at least partly about PR:

[Advanced metrics are] the way so many people engage with following sports these days, and in particular in this case, college basketball. In some ways, maybe young people are right at the top of that list. 

You need to stay relevant in the age that you’re operating in. Certainly relevant today is embracing analytics and technology to the appropriate level.

So I'm a little worried that the result of this will be essentially a sop to the college basketball nerds—just enough to keep the wolves at bay. But it's mean to assume bad faith, and I won't do it. It's more like home-court officiating: there are understandable biases at play here, and that will make meaningful change difficult.

But assuming that can be overcome, what kind of replacement should we be advocating for? It's clear that the NCAA will not endorse anything that significantly emphasizes margin of victory at the expense of pure wins, losses, and strength of schedule. (Wins, losses, and strength of schedule is what the RPI tries to do, but it does it in a poor way.) They primarily want to reward good resumés over good teams, and that is fine.

Given that, I'd propose using a composite power rating, say a straight average of Kenpom, BPI, and Sagarin—or perhaps a new but similar separate power rating that has a hard cap on MOV—as the seed for creating a strength of record ranking. Strength of record basically boils down to measuring how impressive it is for a team to amass a given record against its schedule; all you need to do it is some underlying measure of team quality. ESPN already does this with BPI as the seed (at least when the website is working) with its "Strength of Record." I do it in realtime at T-Rank with the "Wins Above Bubble" or "WAB" ranking. Seth Burn does WAB periodically at his website using Kenpom as the source.

The great thing about this way of doing things is that wins and losses are ultimately king and debatable differences in underlying power rating mostly come out in the wash. Kenpom, T-Rank, and BPI are obviously pretty similar rating systems, but they do have their differences. If you look at a strength of record or WAB using each of them, though, they end up extremely similar. So all you need is a power rating that passes the smell test, and you can create a credible—and useful—resume rating that will pretty closely track the general perception of which teams really deserve to be in the tournament.

So: Death to the RPI, long live WAB or SOR or whatever you want to call it.