One common refrain about the Pomeroy data is that it consistently overrates the Badgers. The Badgers are almost always ranked higher in Pomeroy's ratings than they are in the human AP poll. For instance, the Badgers have finished ranked in the AP top-25 nine times in the since 2003. (That's every year except 2006 and 2009—amazing!) In eight of those nine years, they finished with a higher rank in the final Pomeroy ratings. The 2007 team is the only exception: it finished 6th in the final AP poll but 8th in the final Pomeroy ratings. But even that may be explained by the fact that the Pomeroy ratings reflect that team's rather poor NCAA tournament showing while the final AP poll does not.
Still, this difference is easy to overstate. Although the Badgers are consistently rated higher in the Pomeroy ratings, the difference is not usually that big. They've never finished more than 10 spots higher in the Pomeroy ratings than the AP, and the average difference is about 5.5. In other words, about one seed-line.
Also, one other pattern you see every year is that the Badgers start out highly ranked in Pomeroy but unranked or low-ranked in the AP, and over the course of the year the AP ranking inches toward the Pomeroy ranking. The Badgers have started the season in the AP top-20 just three times under Bo ('04, '07, and '12) but have finished there eight times. And only once—in 2009, when they were initially ranked 25th—have the Badgers finished lower in the AP than they started. So it's clear that the humans underrate the Badgers far more than the Pomeroy ratings overrate them.
Also, one other pattern you see every year is that the Badgers start out highly ranked in Pomeroy but unranked or low-ranked in the AP, and over the course of the year the AP ranking inches toward the Pomeroy ranking. The Badgers have started the season in the AP top-20 just three times under Bo ('04, '07, and '12) but have finished there eight times. And only once—in 2009, when they were initially ranked 25th—have the Badgers finished lower in the AP than they started. So it's clear that the humans underrate the Badgers far more than the Pomeroy ratings overrate them.
Next, let's look at tourney performance, where there are some stark indications that Pomeroy's system has overrated the Badgers. Here is my data on the biggest tournament underacheivers—in terms of expected wins versus actual wins—since 2003:
There's Wisconsin, checking in at 8th worst. To explain this a little, what this means is that my simulations have Wisconsin winning an average of 18 games over the past 11 seasons, and they've actually won only 15. The total difference is 3.2 games below Pomeroy par.
There's Wisconsin, checking in at 8th worst. To explain this a little, what this means is that my simulations have Wisconsin winning an average of 18 games over the past 11 seasons, and they've actually won only 15. The total difference is 3.2 games below Pomeroy par.
The discrepancy is why some tempo-free adherents are frustrated by the Badgers. They don't perform as well in the tournament as Pomeroy's tempo-free stats indicate they should.
But, again, let's not overstate this. Before last season's rare first-round exit, Wisconsin was at just 1.7 games below par. That would still put them in the lower 25, but that's a far cry from holding Wisconsin up as proof that tempo-free stats are bogus. Another point: among the teams even worse than the Badgers in this metric is Duke. Although you'll often hear people making fun of Duke's occasional tournament flops, there's no outcry that they are drastically overrated by Pomeroy's system. (In fact, the one year that there were such complaints—when Duke was number 1 in the Pomeroy ratings all year despite being seemingly not-so-great—Duke went ahead and won the national championship.)
What's more, there is another explanation for Wisconsin's less-than-expected tournament success: underseeding. Because humans consistently think Wisconsin isn't as good as Pomeroy's system thinks it is, Wisconsin tends to get a worse seed in the tournament than Pomeroy's system thinks it should. That means that Wisconsin has to play tougher games sooner in the tournament. Of course, this is partly accounted for in my data because the "expected wins" I'm using as a baseline are calculated by anticipating actual match-ups; so if the Badgers are put on a collision course with a great team in the second round, that will hold down their expected wins somewhat. But I think this underseeding hypothesis is at least partly to blame here.
The most egregious example of Wisconsin's underseeding was 2004. That was Devin Harris's junior year, and the Badgers were very good. They finished 10th in the final AP poll. They won the Big Ten tournament. But they received a six-seed and were forced to play a second-round game against a very, very good 3-seed in Pitt—which was ranked 9th in the final AP poll and had been in contention for a 1-seed going into the final weekend of the regular season. That's right: the committee set up a battle of top-ten teams in the second round. The tragedy is that the 2004 team was probably Wisconsin's very best chance to make a run to the Final Four, given that it featured a true superstar in Devin Harris—by far the most talented player ever to play for Bo Ryan.
Alas.
Another way to analyze the Badgers is Performance Against Seed Expectation (PASE). We can look at the numbers and see how many actual wins the average 1-seed, 2-seed, 3-seed, etc. gets. The best way to do this is probably to go all the way back to 1985, when the tournament expanded to 64 teams, but for consistency I will just use the data I have, from 2003-2013 (see chart on right).
Most teams that underachieve in Pomeroy expected wins also underachieve on PASE. But not the Badgers, who are in fact slight overachievers, with an overall PASE of +0.7 (good for 47th out of 210 tourney teams since 2003).
This discrepancy leads to the starkest stat of all. If you subtract PASE from the Expected Wins, you get a differential in which the Badgers are an extreme outlier:
Are these outliers mis-seeded, whether high or low, or are they mis-rated? Or some combination? Or neither? Hard to tell.
The only thing I can tell for sure is that Wisconsin is an extreme outlier—the only team above +/- 2.9 in this differential, and they're a full 35% further from the mean (0) than the second-furthest team (Gonzaga). (In case you're wondering, that number is almost six standard deviations from the mean.
The easiest interpretation of Wisconsin's extreme outlier status in this metric is that Wisconsin has generally been correctly seeded and generally been overrated by Pomeroy's rating. In other words: dog bites man.
--Bart Torvik
I don't buy the seeding argument because UW has lost to lower seeded teams just as much as they have been beaten by them in the last ten years. They are 5-5 in elimination games, i.e. they were eliminated by lower seeded team just as often as they have by higher seeded teams.
ReplyDeleteI don't think that's a very helpful stat. Overall, Bo Ryan is 15-5 against lower seeds in the NCAA tourment, which is not too shabby. And take a look at some of those "lower seeds" the Badgers lost to:
Delete1) Steph Curry's Davidson, a 10-seed, in the Sweet 16. Steph Curry was probably the best player in college basketball that year, and Davidson went on to take eventual national champion Kansas to the wire in a final possession game.
2) Butler, which almost won a national championship the year before and then went to the Final Four again that year. This was a Sweet 16 game.
3) Cornell, which was a second-round defeat against a team that had already beaten a five-seed, so was clearly underseeded.
Well, they've never beaten a higher seeded team during the Bo Ryan era, so it's hard to claim that they are chronically underseeded.
DeleteNot true, actually. They won as a 12 seed in 2009. Of Bo's losses to higher seeds, four were one seeds (Maryland, Kentucky, North Carolina and Syracuse) and if they'd been seeded one line higher those years, maybe they face a more beatable two seed instead—or face the one seed a round later—that's the argument anyhow. Then there was the loss to Pitt in 2004 which should have been a third round game, arguably, instead of a second round game.
Delete