Advanced Metrics? Conference Records? How to Choose At-Large Teams

This is the second part of a two-part series examining the 2019 NCAA Tournament bubble. Here, I discuss the reasons how to determine at-large bid recipients for the NCAA Tournament. For part 1, where I discuss why this year’s bubble is so weak, click here.
Note: records/rankings are accurate through games played on Feb. 22.

We get it – the bubble is weak. Part of that is the nature of a 68-team tournament, although there are some specific factors in play this year that make it even weaker than normal. But surely, that doesn’t mean an Oklahoma team with a 4-9 record in the Big 12 deserves an at-large spot, right?

From a straight philosophical perspective, it’s arguable, although in reality, the Sooners are in a pretty safe position right now as my top #10 seed in the field. Sure, everyone likes to see teams with better records make the field of 68. It makes us feel a little better that these teams, at least on the surface, appear to deserving of spots. But a quick check of the top 68 overall teams in terms of wins (who all have at least 18+) will quickly show why choosing based on wins alone is not the best idea.

One team well above the cut line is 21-5 Abilene Christian, which boasts an 10-3 record in the Southland Conference. But four of the Wildcats’ wins came against non-Division I opponents, and of the remaining 17, none fall under the category of NET quadrant 1 or 2 wins (something we’ll talk more about later). Out of 353 Division I teams, Abilene Christian ranks 340th in strength of schedule, and in their only game any team in at-large consideration, they lost to Texas Tech by 34. ACU wouldn’t make a field of 128.

Notable teams without 18 wins at the moment include Baylor, Ohio St., TCU, Minnesota (17); Oklahoma and Seton Hall (16); and Texas, Florida, and Alabama (15). All of these teams are in my current field, and if you want to extend to 18 wins, you find a Wisconsin team I have as a 4-seed. Now, obviously, this is because not all schedules and conferences are created equally, but it doesn’t hurt going through the exercise.

The way we determine who gets at-large bids to March Madness is much more complicated than simply looking at records. There are a number of factors to consider. Record is one of them, but so is strength of schedule, home/road/neutral court records, various advanced metrics, and quality wins. While there isn’t an actual number associated, injuries to key players can also factor in. You can actually take a look at the exact team sheets the NCAA uses here.

You’ll notice that conference record isn’t noted anywhere on the team sheet, nor is a record in their last 10/12 games (although the date of each game is listed). This is because the NCAA does not look at conference affiliation. A team’s conference is also nowhere to be found on the team sheet. ESPN’s bracketologist Joe Lunardi argues that to be eligible for an at-large bid, a team must finish .500 or better in conference play plus their conference tournament. However, as long as the NCAA doesn’t consider conferences, that will never come into play.

And for good reason, too. Take a conference like the Big 12, which, in my book, has eight of the top 40 teams in the country. It’s nearly impossible to get eight teams to finish .500 or better in a 10-team league, so by looking at each game individually and not as part of a larger conference, deserving teams can still make the field.

That being said, there is a cut off. No one 5-13 in conference should make the field, but you do reach a point where when losses add up, the overall resume reflects all the losses, so I don’t think a formal rule needs to be introduced.

As far as team evaluation goes, this year’s big change comes in the form of a new primary tool used by the committee, the NCAA Evaluation Tool (NET) which replaces the Rating Percentage Index (RPI), which the NCAA had used since 1981. The RPI was a long outdated metric, a simple combination of record and strength of schedule which was heavily flawed.

The NET is an attempt at creating a predictive metric, similar to the KPI, KenPom, Sagarin, and BPI ratings that have been included on NCAA team sheets in recent years. Having a metric like this also allows the NCAA to better classify quality wins through the use of the quadrant system. While, still imperfect, the quadrant system looks like this, via WarrenNolan.com:

For example, a win on the road against the team ranked #60 in the NET would be a Quadrant 1 victory, while a loss at home against team #200 would be a Quadrant 4 loss. At a very basic level, teams want to pick up Quad-1/2 wins while avoiding Quad 3/4 losses.

The NET still has problems from a transparency and disclosure standpoint. While we know the elements involved, the official formula has yet to be released. The daily NET rankings also come out at inconsistent times (and times not at all), and have occasional not even been fully correct, showing too little movement between teams. All this has led to theories that the NCAA is still modifying the formula or perhaps even making some manual tweaks. We don’t have near enough evidence to be sure of this, but it is the NCAA we’re talking about.

Still, the NET is clearly better than the RPI. The disparity between the two metrics, though, is alarming at times and shows how wildly different this year’s tournament outlook might be if the RPI was still in use. A team like NC State is 32 in the NET but just 91 in the RPI. Right now, the Wolfpack are a clear tournament team. However, if the RPI was still the dominant metric, would they even be in the conversation? Nebraska, at 14-12 overall (vs. D-1) is barely hanging in the bubble watch, and ranks #46 in the NET, but falls all the way to 109 in the RPI.

These huge swings work in the other direction as well. Current Missouri Valley-leading Drake is 60 in the RPI and 137 in the NET, while Princeton falls nearly 100 spots from 83 to 177 when moving between the two metrics. For most teams, the numbers are fairly similar, but these outliers are pretty shocking, and almost seem impossible.

And while the current advanced metrics used by the NCAA are the best tools we have available to us right now, there is still much variability between them. Let’s take Kansas, for example. The Jayhawks rank 15th in the NET, KenPom, and BPI, and are 10th in the Sagarin. But by KPI, Kansas rises all the way to #3, and held the #1 spot just a few weeks ago. This, despite a pretty abysmal 2-6 road record.

The NET and KPI have Houston at #4 and #5, while the KenPom, BPI, and Sagarin have the Cougars at #17, #17, and #19, respectively.

To cherry pick a few more, NET has Liberty at #55, while Sagarin rates the A-Sun leaders 107th. South Dakota State is #84 in KenPom, but #140 in KPI. Finally, Auburn is as high as #12 in the Sagarin and as low as #37 in KPI. Again, usually there is a relative consensus on where a team is ranked, but sometimes there just isn’t.

I try to look at each team’s full resume when making predictions, but find that an average of all these advanced metrics (weighting the NET a little heavier, not because I think it’s the best, but because it’s what the committee will look at first) is a decent place to start. If a team is really high or low in one metric, but about the same in the rest, place more value on the majority. Basic stuff. But you can crunch as many numbers as you want – a big part of bracketology just comes down to personal opinion and what you value most. We’ve still got a long ways to go as far as advanced metrics are concerned, and a perfect formula is impossible to create, and even more impossible to prove.

What we do know is that the committee places a high value on quality wins, especially Quadrant-1 wins. Additionally, in recent years, teams from major conferences have been receiving at-large bids at a much higher rate, although that might change this year.

One last thing I find worth noting about predictive metrics: they don’t place a ton of value on pure wins and losses. In fact, some don’t at all. Instead, metrics like KenPom they focus on offensive and defensive efficiencies while factoring in quality of opponent. That leads to some interesting situations, such as a 10-16 (3-12 Big Ten) Penn State team that entered the day at #48 in the KenPom, just a spot below 17-10 (7-9 Big Ten), and current 11-seed Minnesota.

To Penn State’s credit, they’ve played the fourth-hardest strength of schedule this season, including an almost ludicrous 14 Quadrant-1 games (of which they’ve won just two). Hypothetically, if a team plays a top-10 in the nation opponent every game and loses by just five each time, they’d have no wins, but would be arguably a very strong team. Would they deserve NCAA Tournament consideration?

Bracketology is an ever-changing field, as different and updated metrics come into play. With every different season brings unique resumes and difficult bubble debates. We can do the best we can to project the field, but ultimately it all comes down to a 10-man committee to decide who gets to play for a national championship. The competition to mirror their thinking continues.