Thursday, March 5, 2015

How good (or bad) is Wisconsin's defense, really?

If you compare the Kenpom ratings to the T-Rank, one difference that may jump out is that Kenpom ranks Wisconsin's defense 48th in adjusted defensive efficiency, and T-Rank has it ranked 24th. That's a fairly large and significant difference, particularly if you're one of those people who filters out national championship contenders by metrics such as "top 25 in both offensive and defensive efficiency" or some such thing.

So what accounts for this difference? Basically, the difference is caused by a rather extreme feature of Kenpom 2.0: it deeply discounts (sometimes all but ignores) the results of mismatched blowouts. So Wisconsin's opening game demolition of Northern Kentucky -- which Wisconsin held to an incredible .526 points per possession -- likely doesn't much figure into Wisconsin's Kenpom rating.

To see how drastic this Kenpom 2.0 adjustment affects the Badgers, compare their defensive numbers to Villanova. If you look at the raw efficiency numbers, Wisconsin and Villanova have essentially identical numbers:

Team Raw DE Raw DE Rank
Wisconsin 93.5 23
Villanova 93.3 21

This is just "total points allowed divided by total possessions" and by this metric the two defenses are very similar.

But Wisconsin's raw defensive efficiency gets adjusted significantly downward, while Villanova's gets adjusted slightly upwards:

Team Adj DE Adj DE Rank
Wisconsin 94.9 48
Villanova 93.1 20

You might think this adjustment is being made based on strength of schedule -- perhaps Villanova has just played better offensive teams, which would naturally lead to an adjustment in their favor. But according to the Kenpom numbers, that isn't so:

Team Opp. Adj OE Rank
Wisconsin 105.0 51
Villanova 104.9 56

So strength of opponent would favor Wisconsin, if anyone.

Other than strength of opponent and adjustment for mismatches, the other adjustment that could affect the ratings is recency. Kenpom weighs more recent games more heavily than older games (as does T-Rank). But this is a minor factor, and cannot explain the large downward adjustment compared to Villanova. Besides, Wisconsin has been playing better on defense recently.

The T-Rank algorithm discounts the effect of mismatches too, but obviously not as aggressively. I'm quite certain that this accounts for the difference in the adjusted defensive ratings. Many of the Badgers' most impressive defensive performances have come in the kinds of games (such as the Northern Kentucky game) that Kenpom 2.0 discounts or ignores.

There's reason to believe the T-Rank is closer to the truth. One clue to this is that Kenpom had Wisconsin's adjusted DE ranked in the 80s just a few games ago. Meanwhile, T-Rank has consistently had Wisconsin's adjusted DE ranked much higher, in the top 35 all along, and it has therefore more accurately predicted the Badgers' performance in recent games.

One of the fun things about doing the T-Rank this year is that it has given me some insights into the vagaries of the Kenpom ratings. They have amply earned their status as "authoritative" but they are not perfect, and they are by no means the simple application of god-given math. Judgments are involved, and sometimes those judgments -- even if they are correct in the macro sense that they will most often produce better predictions for most teams most of the time -- sometimes produce error.

No comments:

Post a Comment