Comparing the Preseason Computer Ratings

Since we're about half-way through the college basketball regular season, I thought it'd be a good time to check in on the performance of the major preseason ratings systems: mine, Pomeroy's, Team Rankings (David Hess), and Dan Hanner's. Behold:

These are the current Top 50 in the Pomeroy ratings, and the first four columns after the team names are the where each system put them in the preseason. Cells shaded green show which teams each system predicted to be in the Top 50. The highest rated team that none of the systems predicted to be in the Top 50 is UMass.

The next four columns (labeled "off") show how much better or worse each system rated each particular team compared to the other systems. This is calculated by comparing how far off a particular system's prediction was, and then comparing it to the average of how far off the other three systems were. For example, No. 39 Florida State comes out as +46 for the T-Rank because the T-Rank was only 3 spots low in its prediction for FSU, but the other three systems averaged 49 spots low. This is by far the biggest "hit" for any system (for teams currently ranked in the top 50). 

The biggest miss was Pomeroy's projection for Arkansas.

One way to measure the relative performance of the systems against each other is to add up the "off" columns (which sum to zero when all four are put together), with higher totals being better. The result:

1.  T-Rank +54

2.  Hanner +38

3.  Hess -23

4.  Kenpom -69

There are other ways of analyzing the relative performances, but none of them are so flattering to T-Rank, so they will not be publicized at this time.

The ultimate test, of course, will be the NCAA tournament's S-Curve.