March Madness 101
Big Changes In Store For March Madness
The Committee will be embracing big data this year, dumping the RPI and moving to the NET rankings. But how will it affect your team?
Filed under sports | Updated March 13th, 2019 | Pulled together by 1440 staff
It’s That Time of Year Again. Time to enter your work pools, fill out your brackets, and settle in for the most jam-packed sports weekend of the year. But this year, a relatively simple change is likely to have a big impact on how the bracket shakes out.
We’re here to break down what’s new, and which teams are set to get in - or left out.
The NCAA Selection Committee has historically used a system that relied on a seemingly cryptic mix of hard numbers and emotional gut calls, and the closed-door process often left many people scratching their heads. In an era where advanced analytical tools have crept into every aspect of the game, the committee used the equivalent of a caveman’s stone hammer- until now.
What’s the Deal? The excitement of the tournament is almost matched by the announcement of the field itself. This year, the NCAA Selection Committee will reveal the men’s 68-team field on Sunday, March 17th, at 6 pm ET.
As it currently stands, 32 teams gain at-large bids by winning their conference championship. That means the committee must select the next 36 best teams (32 for the women’s tournament) to fill out the remainder of the bracket. But how to determine the most deserving teams?
For years, the committee - made up of 10 representatives, including 8 athletic directors and 2 conference commissioners - relied on the Ratings Percentage Index (RPI) as the main quantitative metric along with qualitative assessments (i.e. The Eye Test). The NCAA loved the RPI because it was a simple tool to rank teams. Analysts hated the RPI because it was a simple tool to rank teams.
Although tweaked in recent years, the RPI is so simple it can (almost) be done in your head. The formula is:
RPI = (Win % x 0.25) + (Opponents’ Win % x 0.5) + (Opponents’ Opponents’ Win % x 0.25)
By weighting a team’s opponents, and the teams their opponents play, much heavier than a team’s own winning percentage, the RPI emphasizes strength of schedule in addition to wins and losses. But many complained that the approach didn’t take into account things like where the game was played, margin of victory, and other variables.
To make matters worse, committee members often incorporated what they feel are important, non-statistical considerations like injuries, when during the season a game was played, and more. Occasionally the process led to odd decisions. Consider the 2004 Utah State Aggies, who became the first-ever ranked team left out of the tournament after losing by one point in their conference semifinals despite a 25-3 record. The Aggies finished the regular season ranked 43 in the RPI.
Another criticism of the old ranking system is that the more strategic teams could “trick” the RPI. with 353 Division I teams across 32 conferences, it was possible to find teams who were likely to win their conference, beating other teams with high winning percentages in the process.
These teams would have a high calculated RPI, and while talented were not objectively as strong as a power conference team that had a lower RPI. Some teams strategically scheduled these games, thinking they could knock off a high-RPI opponent from a lower-tier conference, boosting their RPI in the process (see an in-depth breakdown here).
One more troublesome aspect of the RPI is that the differences between teams become smaller as teams get worse. As of this writing, the difference in raw scores between RPI #1 Kansas (21-7) and RPI #5 North Carolina (23-5) is 0.027. That is roughly the same difference between RPI #30 Louisville (18-11) and RPI #50 Ohio State (18-10). In other words, uncertainty in the RPI as a metric makes it nearly useless in comparing teams near the bottom of the bracket.
Rankings Don’t Matter (Sort of). The biggest misconception around the way the committee uses the ratings, old or new, is that the team’s ranking matters most. It doesn’t.
That’s right, it’s not a team’s ranking that matters. In fact, what matters most is its opponents’ rankings.
When trying to understand what the committee is thinking, don’t look at teams’ rankings - look at who each team has beaten during the regular season and conference tournaments. For years, the committee used a simple approach to evaluating the strength of wins, dividing the 350-ish teams in any given year into four buckets, or quadrants.
The committee would assess beating any team with an RPI ranking between 1-50 as a Quadrant 1 (Q1) win, regardless of where the game was played. Likewise, a loss to such a team would be a Q1 loss. A win against the #75 team would be ranked as a Q2 win. And so on.
Under this system, when it came to tallying Q1-Q4 wins and losses, it didn’t matter whether the opponent was the #1 or #50 team - both counted as Q1 games. That’s not to say it didn’t matter what a team was ranked, or if they actually did beat the #1 team, but those considerations were usually used in a subjective way, such as when trying to decide between two teams for a single spot or evaluating bubble teams.
This old system had obvious drawbacks. But again, many assumed the NCAA enjoyed the simplicity of the approach.
Baby Steps. There were two main issues with the previous system. First, the rankings system didn’t distinguish between home or away games, and didn’t provide a way to compare good wins (e.g., beating a good team), bad wins (barely beating a bad team), good losses (playing well – but losing – to a superior team), and so on. Secondly, and more fundamentally, the RPI, as the main quantitative tool in the committee’s toolbox, was nowhere close to an objective measure of the quality of a team.
The committee began to address the first problem last year, by splitting the RPI buckets into different quadrants based on the location of the game. Starting last year, the committee emphasized playing away from home.
This is the committee’s way of saying that it’s very difficult to win on the road in college basketball. As an example, under this system beating the #10 team at home is basically equivalent to beating the #70 team on the road - both are considered a Quadrant 1 win. Alternatively, a road loss to the #68 team is equal to a home loss against the #3 team - both are Quadrant 1 losses.
And if you lose to the #76 team at home (for the record, the current RPI #76 is Kent State (18-8))? Watch your tournament hopes take a nosedive.
While the move was applauded by programs, it didn’t solve the second problem. The RPI, which only considers win percentage, was a horrible measure of a team’s true strength.
So, what’s new? Beginning in 2019, the committee will no longer use the RPI when evaluating teams to fill out the men’s field (the women’s field will continue to use the metric for the time being).
It’s been replaced by the very-literally-named NCAA Evaluation Tool - or NET - rankings. NET is meant to be the objective measure of team strength that the RPI wasn’t. Where the RPI relies on simple win percentages, the NET relies on five factors:
- The Team Value Index: An algorithm that assesses each game based on opponent strength, location, and other factors.
- Net efficiency: The difference between offensive efficiency (total points divided by number of possessions) and defensive efficiency (your opponent’s total points over their total possessions).
- Win percentage: Exactly what it says it is, your team’s wins divided by the total number of games played.
- Adjusted win percentage: Your winning percentage, with each game weighted to account for home, road, and neutral site wins and losses.
- Scoring margin: The point differential of a game, capped at 10 points (meaning a 10-point win is favored more than a 5-point win, but is favored equally to a 30-point win).
If you forget any of this, check out this infographic. While the NCAA has released these five factors as the components of its NET ratings, they’ve made it difficult to look under the hood further. The formula is closely-guarded and proprietary. The Team Value Index in particular reportedly relies on a machine learning algorithm in an attempt to assess the quality of a team’s game-by-game performance.
The ratings are then used to determine the Quadrant 1-4 wins and losses using the table above. Essentially, the system is the same, but the archaic RPI has been replaced with the power-charged NET. You can see NET ratings updated on a daily basis here.
The NCAA has taken some criticism for the lack of transparency around the new model - especially when the first rankings had Ohio State inexplicably ranked as #1 (the team ended the regular season 19-14). Whereas the RPI was a simple sum, the NCAA also doesn’t reveal whether one of the five components is weighted more than the others.
Still, the new measure aligns with the best predictive models available. And it makes a huge difference in comparison to the RPI. Below are four teams that highlight the changes.
Right away, the difference between the two systems is clear. While Kansas is the top-ranked team in the RPI, the NET system says Kansas is objectively the 20th best team. As we explained above, where a team ranks is actually not that important. The fact that Kansas is #19 (as of this writing) may come into to committee discussions around location, but it’s combined with many other considerations.
But consider the other teams. Under the new system, a home win against Florida is considered a Quadrant 2 win under last year’s RPI, but is now a Quadrant 1 win under the NET. A home loss against NC State would have been a solid, very damaging Quadrant 3 loss under the RPI, but is right on the border of a Quadrant 1 loss under the NET ratings.
And beating Georgia Southern on the road last year would have been considered a Quadrant 1 win, considered the same as beating a team like Duke, Kansas, or Kentucky.
What does all this mean? At the end of the day, the overall process the committee has used for years will stay intact. They’ll use a quantitative metric to look at a team’s wins and losses, add in their subjective observations, and work within constraints like not scheduling teams from the same conference in the first two rounds to fill out the 36 at-large bids.
The big difference is that the quantitative tool they use will be vastly improved, representing a much more objective assessment of a team’s strength than the RPI. Teams will no longer benefit from playing opponents with overrated RPI’s (see Georgia Southern above), and teams will get more credit for playing teams, like NC State, that would would’ve been dramatically underrated under the RPI.
It’s also worth noting that last year’s change, which split the quadrants into different ranges for home, neutral, and away games gives an edge to teams who play on the road more often. For teams from mid-major conferences, who may need to go on the road to play many of the top teams, racking up Quadrant 1 wins (or at least walking away with Quadrant 1 losses), the change offers inherent advantages.
Despite a lack of transparency around the actual formula, particularly the Team Value Index component, the NET ratings will offer the committee a much more grounded basis for many of the tougher decisions that need to be made.
Enough talk. Who are the top seeds? You didn’t think we’d end without making prognostications of our own, did you? Below is our guess at how the committee would bracket the top four teams in each region.
These projections take into account Q1-Q4 wins and losses for each team, along with expected final wins and losses and strength of schedule. That said, like any year, the committee is sure to throw some curveballs into the process.
Make sure to check back in on Sunday, March 17th to see how we did - and remember, tip-off begins with the play-in games on Tuesday, March 19th (6pm ET, TruTV).
Enjoy the madness!
- 1440 Team