Ken pomeroy

Bracketology Trip: NCAA Mock Selection Committee: Part Three

Shawn Siegel — Thu, 16 Feb 2012 23:09:09 -0800

Shawn Siegel is taking part in the NCAA's "Mock Selection Committee" in Indianapolis. Each day, he's reporting on the day's events and answering questions on Twitter @collegehoopsnet. Follow the action as the bracket is finalized on Friday.

The NCAA first held the media "Mock Selection" in 2007 as part of an ongoing effort to improve the transparency in what was once a very secretive NCAA Tournament selection process. Of course, with greater transparency comes greater scrutiny. And although the NCAA Selection Committee usually ends up with a pretty darn good field of 68, there is much about the process that deserves to be scrutinized.

What Does "Best" Really Mean?

To me, the largest glaring problem with the selection process is the broad question of: What are they actually selecting? The NCAA's official position is that they select "the 37 best at-large teams." But "best" can mean two very different things. Best can mean a) The 37 teams that had the best results before Selection Sunday, or best can mean b) The 37 teams that the committee thinks will do the best in the days after Selection Sunday. This is a huge distinction, although the committee chair and others from the NCAA didn't seem to understand or care about the distinction.

Let me give you an example: Alabama is under a bit of strife with multiple player suspensions, and it was unknown whether the players would return for remaining games. (In the hypothetical mock selection, the remaining games would only be NCAA Tournament games since the season hypothetically ended today.) Some committee members suggested that the status of those players would hugely alter their seeding, or whether they even made the tournament. This suggests that the seeding/selecting is based on how the team will fare going forward, but yet the committee chair (the real one, not the mock one) Jeff Hathaway said specifically the committee does not "project" the future. Based on the totality of their regular season, Alabama might be considered the 35th best team, but without those players, they might be considered the 100th as of tomorrow. It is unclear how this issue should be resolved because of the NCAA's ambiguity on what the whole purpose of the tournament is. On the one hand, the chairman says its simply about the "totality of the season", but on the other hand, its not.

(This leads to a larger question about the lack of real criteria for the selection process. Although the committee chair spoke of criteria, the committee members can pretty much bring whatever thoughts on a team they feel like to the table, and thus the criteria are not really criteria, but simply vague preferences.)

The RPI Is One of Many Factors.. But Not Really

The second glaring problem is how the NCAA presents their statistics to the committee members. The NCAA says, "The RPI is one of many resources/tools available to the committee." In some sense, this is true, the committee has access to other resources (including but not limited to the human polls, other rankings, and personal perspectives on teams from advisory members). However, the major information interface that the committee members sees is entirely based on the RPI. For example, the "Team Sheet" includes RPI, records vs RPI Top 50, Top 100 etc, Avg RPI win, AVG RPI loss, all the opponents with their RPI next to them, etc. Nowhere on the main interface is any other statistical measure referenced.

Now, I don't have a major bone to pick with the RPI. To me, it is no better or worse than other ratings (Sagarin, Pomeroy, etc), but it is clearly different. It is one perspective amongst many. All one has to do is see the wild computer variations in a team like Cincinnati (buried at 93rd in the RPI, but up at 41st in Pomeroy's rating) to see how different these two computer systems see things.

The committee will say that the RPI is not the "only" or "main" rating they use, but it is by far the most prominent (by a wide, wide, wide margin.) Whether or not committee members solely focus on the RPI, there has to be an unconscious or conscious effect of seeing Cincinnati 93rd over and over (vs 41, 41, 41).

Odd Reasoning

The NCAA's retort for why the RPI is so prominent was two-fold: a) there has to be some underlying statistical analysis to present the information in a clear and easy fashion, and b) we can all agree the top RPI teams are "really" the good ones and the bad ones the bad ones (and the top teams in the RPI are similar to the top one's in other ratings), and since it is only part of the puzzle.. so be it.

To counter the first retort: This is true, one unifying system is preferred. Being presented with 6 team sheets for one team, multiplied by 100 teams, multiplied by the dozens of times each team is looked at, would be unmanageable. So agreed, there should be one set of numbers to organize the information. But the question remains: why the RPI? Or, as some of the media members suggested, why not an aggregate/consensus ranking of the RPI, Sagarin, Pomeroy, etc. As shown by the Cincinnati example (just one of many), there is huge variation amongst the ratings (especially once you get out of the Top 10) and a consensus would help remove the noise.

To counter the second retort: That's just ridiculous.

It's Not All Bad. Though There's Still More Bad.

I don't want to make it seem like the Selection Process is all bad. It's not. (Though I could write a whole article criticizing the shocking focus on the "eye-ball" test and live/tv analysis of teams as opposed to cold, hard numbers. Committee members are encouraged to use subjective feelings about how good teams "look" in their decision-making process. There were echoes of "Moneyball" here. The NCAA Selection Committee is unfortunately the Art Howe of "Moneyball" and not the Peter Brand/Paul DePodesta. If these guys showed up at a fantasy baseball draft, these are the guys who would pick AJ Burnett ahead of CC Sabathia because they saw him throw "nasty" stuff a few times.)

Okay, Finally, The Good

Anyways, back to the good. The good is that the committee cares. A lot. They spend a ton of time dissecting and discussing the pros and cons of each team. They prepare by watching and attending tons of games (whether this should matter or not is besides the point. The point is they put in time.) They are proud of being part of the process. They take precautions to remove personal biases from the process. The shuffle and sort ad nauseum. (Literally, I was a bit nauseous following an ill-chosen dinner break of hot wings.)

And they care a ton about getting things right. The only problem is that what they're getting at is ambiguous at "best".

I'll have more to discuss later tonight and tomorrow once the field is finalized. There are fan questions that remain to be answered, as well as more criticisms to discuss. There are also a few more positives to discuss as well.

Consensus Ranking: KenPom, RPI, Sagarin

Shawn Siegel — Mon, 06 Feb 2012 11:33:18 -0800

Consensus Ranking

Nothing riles up fans more than thinking their team is under-rated, under-respected, and under-ranked in the national polls. As the voice behind CollegeHoops.net's Top 50 Ranking, I'm well aware of how angry fans get when they feel slighted. No doubt, I occasionally slip up, and no doubt the dozens of voters in the AP poll slip up even more often.

With the advent of computer rankings & metrics, we have many tools that attempt to overcome historical biases to rank teams fairly. Taking a consensus average of these rankings, we get a good sense of the lay of the land and can see where the human voters differ with the computers. More importantly, we can see whether there is a pattern to the human biases and whether certain teams are consistently mis-valued compared to their peers.

Throughout the season, I'll be updating the Consensus Ranking and commenting on notable trends.

What Is It?

72 teams are ranked, consisting of A) any team with AP votes, B) any team in the RPI Top 50, C) any team in CHN Top 50, and D) any team previously included. The teams are ranked by an average of their RPI, Sagarin, and KenPom ratings. At the right of the chart is the team's current AP ranking and their consensus ranking in the last two updates.

Below the ranking, you'll find a list of the most overrated and underrated teams, as well as teams that moved the most since the last update.

This Week's Notes: February 6th

- Ohio St takes over the top spot in the Consensus Ranking thanks to a hard-fought loss by Kansas.

- Saint Louis moves into the Top 20 for the first time at 19th. They've consistently been moving up the Consensus Ranking, although the AP voters haven't noticed. The Bills did not receive a single vote this week.

- While SLU is getting no love, Notre Dame is getting a lot of respect by the voters. The Irish have no doubt been a surprise team of late, but the computers suggest that this run is a bit flukey. The Irish are 50th in the ranking.

- The computers are not on Murray St's bandwagon either. Despite remaining unbeaten, the Racers are dropping down the ranking each week thanks to their weak opponents. Murray has only played 3 Top 100 RPI teams all season. Michigan St has played 12 for comparison.

- Miami, Notre Dame, Akron, and Washington all enter the tracking this week. Akron has snuck up on people, falling in the Top 60 in each ranking. Miami, of course, shot up thanks to beating Duke on the road.

- Despite being .500 in Big Ten play, the computers still like Indiana more than their 23rd position in the AP Poll. The Computers are content that losing games at the likes of Michigan, Wisconsin, and Ohio St is no big deal. The Hoosiers play 5 of their last 7 games at home, and have a good chance to regain the voters respect.

Rank	Team	RPI	KenPom	Sagarin	Avg	Last	2 Weeks	AP
1	Ohio St	4	1	1	2.0	2	2	3
2t	Syracuse	1	6	3	3.3	3	1	2
2t	Kentucky	6	2	2	3.3	4	4	1
4	Michigan St	3	5	6	4.7	5	6	11
5	Kansas	9	3	4	5.3	1	7	7
6t	Baylor	2	11	9	7.3	6	3	6
6t	UNC	7	8	7	7.3	7	11	5
8	Missouri	11	7	5	7.7	8	8	4
9	Duke	5	16	11	10.7	9	5	10
10	Indiana	18	9	10	12.3	13	9	23
11	Georgetown	13	13	13	13.0	12	10	12
12	Florida	14	14	12	13.3	16	30	8
13t	UNLV	10	18	14	14.0	10	12	14
13t	Wisconsin	30	4	8	14.0	11	18	21
15	Marquette	12	22	16	16.7	14	15	18
16	Florida St	17	17	20	18.0	18	38	15
17	Creighton	15	24	17	18.7	17	22	17
18	Wichita St	27	15	15	19.0	15	21	31
19	Saint Louis	33	12	19	21.3	21	34
20	St. Mary's	23	26	21	23.3	23	16	16
21	New Mexico	44	10	18	24.0	28	32	33
22	Michigan	16	28	29	24.3	29	31	22
23	Memphis	20	27	27	24.7	25	26
24	Louisville	29	29	24	27.3	36	37	24
25	Virginia	42	19	22	27.7	27	25	19
26	Alabama	34	25	25	28.0	26	13
27t	Gonzaga	24	35	30	29.7	20	14	30
27t	West Virginia	26	32	31	29.7	24	17
29	UCONN	22	34	34	30.0	30	20
30	Southern Miss	8	44	39	30.3	38	39	28
31	Vanderbilt	28	31	33	30.7	31	36	35t
32t	Kansas St	49	23	23	31.7	19	23	34
32t	California	48	21	26	31.7	22	24
34	Temple	21	39	36	32.0	32	40	29
35t	BYU	41	33	32	35.3	34	28	38t
35t	Iowa St	35	36	35	35.3	48	51	27
37t	Harvard	40	37	37	38.0	41	41	25
37t	Texas	66	20	28	38.0	42	42
39	Long Beach St	36	48	43	42.3	37	55	32
40	MTSU	45	42	41	42.7	40	50
41	San Diego St	19	63	47	43.0	33	49	13
42	Illinois	46	41	44	43.7	43	27
43	Minnesota	50	45	46	47.0	45	43
44t	Purdue	64	40	38	47.3	47	33
44t	Arizona	62	38	42	47.3	54	58
46	Miami	38	53	53	48.0	x	x	38t
47	Murray St	39	61	45	48.3	35	29	9
48	Mississippi St	32	69	49	50.0	44	46	20
49	Seton Hall	31	66	55	50.7	39	19
50	Notre Dame	56	46	52	51.3	x	x	26
51	Iona	57	50	48	51.7	51	44	35t
52	Cleveland St	61	47	50	52.7	60	66	35t
53	Xavier	53	56	59	56.0	49	47
54	Northwestern	37	68	64	56.3	50	35
55	St. Joseph's	43	65	63	57.0	57	56
56	Akron	54	59	60	57.7	x	x
57	Oral Roberts	47	73	62	60.7	53	52
58	Northern Iowa	51	74	68	64.3	56	48
59	Cincinnati	95	49	51	65.0	58	62
60	Washington	76	64	65	68.3	x	x
61	Arkansas	59	75	73	69.0	61	64
62t	Davidson	67	72	71	70.0	52	57
62t	Stanford	96	60	54	70.0	59	53
64	Marshall	60	76	79	71.7	63	54
65	Colorado St	25	102	91	72.7	65	59
66	Virginia Tech	92	67	69	76.0	55	60
67	Wagner	75	82	77	78.0	68	65
68	Ohio	79	85	74	79.3	62	63
69	Dayton	71	88	81	80.0	46	45
70	Ole Miss	52	104	94	83.3	66	68
71	UCF	63	96	93	84.0	64	67
72	Denver	88	90	87	88.3	67	61

Teams the AP voters overrate vs. the computer ranking:

Murray St +38

Mississippi St +28

San Diego St +28

Notre Dame +24

Cleveland St +17

Iona +16

Harvard +12

Iowa St +8

Miami +8

Virginia +6

Teams the AP voters most underrate vs. the computer ranking:

Teams w/ votes

Indiana -13

Wichita St -13

New Mexico -12

Wisconsin -8

Teams w/ no votes

Saint Louis

Memphis

Alabama

West Virginia

UCONN

Biggest Changes Since Last Week:

Rising

Iowa St +13

Louisville +12

Arizona +10

Cleveland St +8

Southern Miss +8

Falling

Dayton -23

Xavier -14

Kansas St -13

Murray St -12

Seton Hall -10

Biggest Changes In Last Two Weeks:

Rising

Florida St +22

Florida +18

Iowa St +16

Long Beach St +16

Saint Louis +15

Falling

Seton Hall -30

Dayton -24

Northwestern -19

Murray St -18

Illinois -15

What do you think? Post comments below, send a tweet to @collegehoopsnet, or follow facebook.com/collegehoopsnet.