ncaa mock selection

Bracketology Trip: NCAA Mock Selection Committee: Part Four

Shawn Siegel — Sat, 18 Feb 2012 11:01:05 -0800

Now that I'm back from the NCAA's Mock Selection Committee, I finally have time to sit down and answer some of the many questions asked by readers and followers on twitter. Sitting in with the likes of Committee chairman Jeff Hathaway and NCAA Senior VP Greg Shaheen (the person who understands the selection process best), I certainly know more about the process than ever before.

While the NCAA does have guidelines on how the process works, ridiculously titled "NCAA Division I Men's Basketball Championship Principles and Procedures for Establishing the Bracket," it does not really give you a clear picture. That document, as I noted in my last piece on the event, cannot explain how much time is really spent deliberating about the merits of each team. Tons of arguments are made, and statistics are mulled over to come to decisions. Perhaps even more information is thrown into the discussions than is really needed. The fact that committee member X watched team Y and thought they looked swell and had great guard-play is nice.. but not necessarily relevant to ranking teams. Regardless, thousands of man-hours are spent in the process.

So while my 15 hours spent in the Mock Committee might not stack up to those thousands, the types of discussions had and the process used was roughly the same.

Taken from twitter, here are some interesting questions and their answers. The language of some questions was changed for grammar (and or expletive) reasons. For reference, check out the final NCAA Mock Selection Bracket.

@stevenwolfjr What are the bracketing guidelines that cause changes in the s-curve?

If the Tournament was held at one venue, then you'd expect teams to play each other in a pure s-curve. However, due to many scheduling issues, the bracket is not set up as a pure s-curved. In fact, teams not only get moved around within a seed, but can change entire seeds. (Seton Hall for example was the 43rd team, in the 11-seed range, but ended up in a 12-seed game). The NCAA does a good job spelling out the scheduling concerns in the Principles and Procedures document noted above. Some of the big ones: Keeping teams close to home, not putting too many teams from the same conference in the same region, not playing on one's home-court, etc.

@DadofTwins2 Ohio U is playing in Nashville & Belmont sent to Louisville. Could they have reversed? OVC is the Nashville host, not Belmont.

This question speaks to the scheduling issues discussed above. Belmont ended up as a 14-seed. Located in Nashville, it seems logical that Belmont would have been sent to play their home games there considering they were not the official host, and thus could play in that venue. They also did not play 3 times in that venue, and so were not precluded by that rule. But, the committee aims to protect teams in the first 5 seeds from playing "road" games. Thus, the committee felt it was unfair to Marquette to have to force them to play Belmont in Nashville.

@charlespeterjr The eye-ball test is garbage. Why is it included?

The "eye-test" is certainly controversial. Besides the pure results, the committee is "encouraged" (Jeff Hathaway's word) to bring their knowledge of teams learned by watching/attending games to the table. This helps ensure teams pass the "eye-test." In other words, the "eye-test" enables the committee members to bring completely subjective feelings to the table. I was blown away when Greg Shaheen mentioned that 50-60% of the discussion that takes place is this sort of discussion.

This brings up one of the major lessons learned from my days in Indianapolis: while the committee is supposed to be selecting the "best" teams (whatever "best" means), there are no real set criteria for how they go about this. Individual members can focus on the RPI (or not), the Sagarin (or not), the Coaches Poll (or not), favor quantity of wins or quality of wins, etc, etc.

@columbusbuck Where did Northwestern and Minnesota finish? First four out?

Neither of these teams made the field, and there was no specific ordering of teams not in the field. Based on discussions and voting, I'd guess they were within the next 8 teams out. Note however that the Mock Selection was based on Mock Conference Tourney results, which included a lot of upsets. I think there were 2 or 3, maybe -4 less at-large teams than would be expected in the real bracket. Thus being 6th out in the Mock might put them 2 out in reality, and thus firmly in the mix.

@jamesgurland What did the committee think of Seton Hall?

Seton Hall ended up being one of the last four at-large teams in the field. Though Seton Hall has a high RPI, their quality wins are looking less impressive as the season wears on. The win over UCONN is no longer considered elite due to UCONN's struggles, and West Virginia has faded as well. Wins over St. Joe's and Dayton, while rsolid, would have been more respected if one of those teams made the at-large field.

@ervinsm What was said about Xavier and Cincy? Which one did the committee like more?

Xavier ended up as an 11th seeded automatic qualifier after winning the nock A-10 tournament. Cincy was left out in the cold. While Cincy does have big wins (at G'Town and at UCONN), the committee was turned off by a couple things: a) The home loss to Presbyterian (fluke or not) is still shocking when compared to other tournament-level teams, b) losses to sub-100 RPI teams St John's and Rutgers don't look great, c) Cincy got blown-out by fellow bubble squad Xavier, d) Cincy's pathetic non-conference schedule including 9 sub-200 teams was almost offensive to the committee. In the end, while Xavier did not make it as an at-large team, they were respected slightly more.

@setty1333 Do they ever consider margin of victory?

One of the reasons the NCAA uses the RPI is that because the RPI does not take into account margin of victory. Committee members can look at other rankings that do use this data, and there was a lot of discussion about how close or not teams played in individual games. Xavier really blew out Cincy, Florida barely topped Arizona, etc, etc. So while committee members seem to care about margin of victory, the RPI does not.

@bearcatsblog What's going to happen to Murray St when they lose in their mock conference tournament?

This was perceptive. The Mock results did in fact have Murray losing to SEMO in the OVC finals. Murray ended up 27th in the seeding order, but before that SEMO loss, they were only a few spots higher so I can't say the committee killed them for that mock result.

@waldenORU What does committee value most when comparing mediocre BCS teams to quality mid-majors. Quantity or quality of wins?

The answer is simply that each committee member can bring their unique perspective to the table. Some people favored teams with 5 quality wins, even if they had 10 tries. I favored teams that might have only won 2 quality games, but only had 3 tries.

Shawn Siegel took part in the NCAA's "Mock Selection Committee" in Indianapolis. Check out the Bracketology Blog for commentary on the event and the selection process. He'll be back with one more article on the event before moving on.

Bracketology Trip: The NCAA Mock Selection Bracket

Shawn Siegel — Sat, 18 Feb 2012 06:34:00 -0800

There is still much to talk about after the whirlwind NCAA Mock Selection Committee. After Thursday night's session, I wrote about the major flaws in the selection process. These include an over-reliance on the "eye-test", a delusion that the RPI is not the most important ranking, yet it is the basis of the entire interface, and a general ambiguity about what it really means to select the "best" teams.

But the process is ultimately rather successful. While there is no perfect way to make a bracket, the committee generally picks 37 darn good teams, and seeds them in a fashion that is generally worthwhile. Part of the reason for this success (despite the flaws) is that the committee members themselves are highly passionate about the process, and the process does afford for endless nuanced discussion over team merits. The actual committee has 7-8x more time to make their decisions, and I can only imagine the endless discussion that takes place.

As the mock committee got down to the bubble, the difference between teams became slim to negligible. In fact, what often develops are philosophical debates. Does a team with four Top 50 wins (but also 9 chances at getting them) deserve more respect than a team with only one Top 50 win (but only 3 chances)? There is really no right or wrong answer to this question. (Though the committee could create a guideline stressing one way or the other, but they keep things open for personal interpretation.)

Below is the Mock Selection Committee's final product: The 2012 Media Mock Bracket. Later today, I will follow up with more discussion of how this bracket came to be, some of the debates that were had, and more answers to reader questions. For now, enjoy the bracket:

2012 NCAA Mock Selection Bracket

Shawn Siegel took part in the NCAA's "Mock Selection Committee" in Indianapolis. Check out the Bracketology Blog for commentary on the event and selection process. Feel free to post question on Twitter @collegehoopsnet.

Bracketology Trip: NCAA Mock Selection Committee: Part Three

Shawn Siegel — Thu, 16 Feb 2012 23:09:09 -0800

Shawn Siegel is taking part in the NCAA's "Mock Selection Committee" in Indianapolis. Each day, he's reporting on the day's events and answering questions on Twitter @collegehoopsnet. Follow the action as the bracket is finalized on Friday.

The NCAA first held the media "Mock Selection" in 2007 as part of an ongoing effort to improve the transparency in what was once a very secretive NCAA Tournament selection process. Of course, with greater transparency comes greater scrutiny. And although the NCAA Selection Committee usually ends up with a pretty darn good field of 68, there is much about the process that deserves to be scrutinized.

What Does "Best" Really Mean?

To me, the largest glaring problem with the selection process is the broad question of: What are they actually selecting? The NCAA's official position is that they select "the 37 best at-large teams." But "best" can mean two very different things. Best can mean a) The 37 teams that had the best results before Selection Sunday, or best can mean b) The 37 teams that the committee thinks will do the best in the days after Selection Sunday. This is a huge distinction, although the committee chair and others from the NCAA didn't seem to understand or care about the distinction.

Let me give you an example: Alabama is under a bit of strife with multiple player suspensions, and it was unknown whether the players would return for remaining games. (In the hypothetical mock selection, the remaining games would only be NCAA Tournament games since the season hypothetically ended today.) Some committee members suggested that the status of those players would hugely alter their seeding, or whether they even made the tournament. This suggests that the seeding/selecting is based on how the team will fare going forward, but yet the committee chair (the real one, not the mock one) Jeff Hathaway said specifically the committee does not "project" the future. Based on the totality of their regular season, Alabama might be considered the 35th best team, but without those players, they might be considered the 100th as of tomorrow. It is unclear how this issue should be resolved because of the NCAA's ambiguity on what the whole purpose of the tournament is. On the one hand, the chairman says its simply about the "totality of the season", but on the other hand, its not.

(This leads to a larger question about the lack of real criteria for the selection process. Although the committee chair spoke of criteria, the committee members can pretty much bring whatever thoughts on a team they feel like to the table, and thus the criteria are not really criteria, but simply vague preferences.)

The RPI Is One of Many Factors.. But Not Really

The second glaring problem is how the NCAA presents their statistics to the committee members. The NCAA says, "The RPI is one of many resources/tools available to the committee." In some sense, this is true, the committee has access to other resources (including but not limited to the human polls, other rankings, and personal perspectives on teams from advisory members). However, the major information interface that the committee members sees is entirely based on the RPI. For example, the "Team Sheet" includes RPI, records vs RPI Top 50, Top 100 etc, Avg RPI win, AVG RPI loss, all the opponents with their RPI next to them, etc. Nowhere on the main interface is any other statistical measure referenced.

Now, I don't have a major bone to pick with the RPI. To me, it is no better or worse than other ratings (Sagarin, Pomeroy, etc), but it is clearly different. It is one perspective amongst many. All one has to do is see the wild computer variations in a team like Cincinnati (buried at 93rd in the RPI, but up at 41st in Pomeroy's rating) to see how different these two computer systems see things.

The committee will say that the RPI is not the "only" or "main" rating they use, but it is by far the most prominent (by a wide, wide, wide margin.) Whether or not committee members solely focus on the RPI, there has to be an unconscious or conscious effect of seeing Cincinnati 93rd over and over (vs 41, 41, 41).

Odd Reasoning

The NCAA's retort for why the RPI is so prominent was two-fold: a) there has to be some underlying statistical analysis to present the information in a clear and easy fashion, and b) we can all agree the top RPI teams are "really" the good ones and the bad ones the bad ones (and the top teams in the RPI are similar to the top one's in other ratings), and since it is only part of the puzzle.. so be it.

To counter the first retort: This is true, one unifying system is preferred. Being presented with 6 team sheets for one team, multiplied by 100 teams, multiplied by the dozens of times each team is looked at, would be unmanageable. So agreed, there should be one set of numbers to organize the information. But the question remains: why the RPI? Or, as some of the media members suggested, why not an aggregate/consensus ranking of the RPI, Sagarin, Pomeroy, etc. As shown by the Cincinnati example (just one of many), there is huge variation amongst the ratings (especially once you get out of the Top 10) and a consensus would help remove the noise.

To counter the second retort: That's just ridiculous.

It's Not All Bad. Though There's Still More Bad.

I don't want to make it seem like the Selection Process is all bad. It's not. (Though I could write a whole article criticizing the shocking focus on the "eye-ball" test and live/tv analysis of teams as opposed to cold, hard numbers. Committee members are encouraged to use subjective feelings about how good teams "look" in their decision-making process. There were echoes of "Moneyball" here. The NCAA Selection Committee is unfortunately the Art Howe of "Moneyball" and not the Peter Brand/Paul DePodesta. If these guys showed up at a fantasy baseball draft, these are the guys who would pick AJ Burnett ahead of CC Sabathia because they saw him throw "nasty" stuff a few times.)

Okay, Finally, The Good

Anyways, back to the good. The good is that the committee cares. A lot. They spend a ton of time dissecting and discussing the pros and cons of each team. They prepare by watching and attending tons of games (whether this should matter or not is besides the point. The point is they put in time.) They are proud of being part of the process. They take precautions to remove personal biases from the process. The shuffle and sort ad nauseum. (Literally, I was a bit nauseous following an ill-chosen dinner break of hot wings.)

And they care a ton about getting things right. The only problem is that what they're getting at is ambiguous at "best".

I'll have more to discuss later tonight and tomorrow once the field is finalized. There are fan questions that remain to be answered, as well as more criticisms to discuss. There are also a few more positives to discuss as well.

Bracketology Trip: NCAA Mock Selection Committee: Part Two

Shawn Siegel — Thu, 16 Feb 2012 06:57:17 -0800

I've been to Indianapolis almost a dozen times, and each time it perplexes me. Outside of the government employees working downtown during the day, the rest of downtown is one giant convention/conference center. Except it is not contained to the actual convention center, but spread out across 8 blocks.

In town for the NCAA's Mock Selection Committee, I guess I am no different from the other conference-goers filling up chain restaurants (I've counted about 120 so far) and crowding my chain-hotel elevator (the Parts Unlimited crew is pretty rowdy).

At 1 o'clock I'll gather with other members of the media for about 24 hours of non-stop basketball discussion. I don't entirely know how we're going to fill the time. Or at least, I don't know how we're going to fill the time productively. I have projected the field every week for the last 6 seasons, and have very strong opinions about how the field should look. If part of the time is to be spent researching the teams, I consider myself researched-out already.

It will be interesting to see how much of the time is spent over nuanced debates over deserving bubble teams, and how much is spent shooting down ridiculous suggestions: Really, you think Texas-Arlington should be a 7 seed?

In their quest to educate about the selection process, the NCAA held a conference call yesterday, led by selection committee chair Jeff Hathaway. Hathaway will be sitting in on the conference this week, but offered some interesting insight on the call.

For example, when discussing how teams take into account injuries, illnesses, and suspensions, he suggested that performance is what really matters and that those issues are not as relevant. One fan question I was asked on Twitter was whether the committee takes into account injuries, but ignores suspensions. This is something I intend to find out later today.

Hathaway also stressed that the committee does not take into account timing in terms of judging wins. "It's the total body of work," he said. To me, this sounds like the right approach, but I question whether the committee members really act on this.

One comment Hathaway mentioned about the "eye-ball test" irked me. My personal take is that all that should matter in judging a team's NCAA resume is results. How a team "looks" on TV or in person, or their style of play, etc, etc should have no bearing. Hathway said, "I watch how good teams are." Washington "looked" pretty good in nationally televised TV games against Marquette and Duke earlier in the year.. but does this matter? They lost those games and lost many more games. How a team "looks" should have no bearing on the selection committee.

I'll be reporting back throughout the day with more thoughts and insight from the NCAA Mock Selection Committee.

Bracketology Trip: NCAA Mock Selection Committee: Part One

Shawn Siegel — Wed, 15 Feb 2012 06:43:13 -0800

In a few minutes I'll be heading out towards the airport to catch a flight to Indianapolis (via a changeover in Philadelphia unfortunately). There, I'll be meeting with members of the media (paper, tv, internet) for the NCAA's yearly Mock Selection Committee. The NCAA started this conference in 2007 to give members of the media a closer look at the selection process, so that they'd have a better understanding of how it works (and thus erase what ever misconceptions existed.)

As someone who spends a lot of time projecting the NCAA Tournament field (in my weekly Bracketology projections), I have a ton of questions. I've also received interesting reader questions on Twitter (@collegehoopsnet) that I'm going to try and answer. These include questions about individual teams (what would the committee do if Murray St lost the OVC tournament, how will the committee treat struggling Big East teams like UCONN and Seton Hall) and questions about the process (can 2 teams from the same conference play in the play-in game, does the committee really not pay attention to the last 10-12 games).

For whatever reason, I've had a knack at nailing the tournament fields better than the major media players (Joe Lunardi, etc.) I wonder if being part of the Mock process might actually make my projections worse. My normal seeding process is quite simple really: rank the teams in the order that seems right based on the RPI, human polls, and intuition.. and then adjust for the committee's historical biases. My guess is that part of the process will aim to show that the committee is not biased towards power-conference teams over mid-majors. I hope this doesn't fool me into changing my picks come March.

Over the last 6 years, the 5 highest RPI teams not to make the NCAA Tournament were from mid-majors. I wrote an article on this last year (called A Brief History of Mid-Major Bias) and tend to update it in a few weeks. This is more than a statistical coincidence. There is something about the selection process that (either wittingly or unwittingly) is predisposed towards favoring power conference teams. I hope to learn more about why this is.

I'll be checking back throughout the next few days with more thoughts on the trip and answers to reader questions. If you have a question about the NCAA Selection Process, drop a note @collegehoopsnet.