The System Worked

Since the CFB Playoff Committee first formed, there has been a long line of people ready to complain about its every misstep. The basic complaint is this: any time the committee substitutes their own judgment for my own, they are clearly out of control. So let's get this out of the way:

The committee did a great job, and probably got this right.

Now, I'll admit that I have a horse in this race, as I was rooting for Baylor to snag that fourth slot. Had I figured out a way to worm my way on to the committee, I would have cast my vote for Baylor. However, disagreement is not a mistake. Ohio State is a perfectly defensible choice, even if it is the safe choice. They win the helmet test, and let's be honest, that didn't hurt them at all.

There have been calls for an objective ranking of the team, and even a return of the BCS formula, as if that was ever objective. A formula designed to take the top two teams in the coaches' poll is not exactly a model of mathematical rigor. Even if the BCS formula was remotely credible, which it is not, there simply is no way to objectively rank the top four teams in the country. Anyone who says differently is selling you something.

There are 128 teams in the FBS division, and 65 teams in the Power 5. Thinking you can rank even those 65 Power 5 teams with accuracy with a mere 12-game schedule is utter folly. There just aren't enough games between the Power 5 conferences to judge similar teams across conferences in an objective, mathematical model.

I love the statistical revolution in sports. I grew up on Bill James and Pete Palmer. I think the world of Bill Connelly, and seriously believe he should be on the playoff committee. But part of analytics is knowing the limits of data. Creating the playoff field is not an objective measurement, it is a holistic process which requires well-intentioned, well-informed people to make value judgments.

There's also been some complaints about the weekly polls. There is a simple solution there: those polls not only didn't matter, they TOLD us they didn't matter. That didn't stop everyone from rushing to judgment and trying to go all Kremlinologist on every release. If you were fooled by the weekly releases, that says more about what you projected on the committee than anything they actually did. Fans and the media overreacted every week to the committee's pronouncements, digging for meaning that wasn't really there, other than that these polls were incredibly fluid.

In the final poll, the committee made its final judgment calls, and they did so in accordance with their stated criteria. That's really all we can ask. Let's take the language straight from the webpage:

When circumstances at the margins indicate that teams are comparable, then the following criteria must be considered:

• Championships won
• Strength of schedule
• Head-to-head competition (if it occurred)
• Comparative outcomes of common opponents (without incenting margin of victory)

The site does not state the order of importance, but I'm going to make a dangerous assumption that the list runs from the most to the least important factors. Actually, the Committee gives us some further guidance on that front.

Under the current construct, polls (although well-intended) have not expressed these values; particularly at the margins where teams that have won head-to-head competition and championships are sometimes ranked behind non-champions and teams that have lost in head-to-head competition. Nuanced mathematical formulas ignore some teams who "deserve" to be selected.

As we expand from two teams to four teams we want to establish a human selection committee that: (1) will be provided a clear set of guidelines; (2) will be expected to take the facts of each case and specifically apply the guidelines; and (3) will be led by a Chairperson who will be expected to explain publicly the committee's decisions.

Some of the guidelines and protocols expected to be established to guide the committee would include, but not be limited to, the following:

• While it is understood that committee members will take into consideration all kinds of data including polls, committee members will be required to discredit polls wherein initial rankings are established before competition has occurred;
• Any polls that are taken into consideration by the selection committee must be completely open and transparent to the public;
• Strength of schedule, head-to-head competition and championships won must be specifically applied as tie-breakers between teams that look similar;

Committee members associated with any team under consideration during the selection process will be required to recuse themselves from any deliberations associated with that team;

(Bolding is mine)

OK, head to head REALLY matters, as do championships. Rankings at the time of the game don't matter, as specifically stated in the rules. Ohio St, TCU, and Baylor are close enough to be considered "tied" in the rankings, so we must apply the tiebreakers they proscribe.

The poll I'm going to use for general quality of competition is Jeff Sagarin's ratings. I feel this meets the "open and transparent" criteria, as Sagarin's ratings predate the BCS even and have long been generally accepted as a quality ranking system that is relatively objective.

Strength of Schedule

Ohio St. ranks 52nd, TCU 42nd, and Baylor 56th according to Sagarin's strength of schedule ratings. No one is covering themselves in glory here, and all of those rankings are pretty similar. We're slicing the bread pretty thin, but TCU gets a small nod here. But remember, the guidelines specifically tell us to look beyond the numbers as "nuanced mathematical formulas ignore some teams who "deserve" to be selected."

How you did against the schedule should matter, as does top tier competition. Of the three TCU, is the only one without a win over a top 10 team, which means their schedule lacks top end quality so perhaps it is not a strong as the formula would have us believe. Sagarin also includes the record against the top 30, and TCU and Baylor both went 3-1 against the top 30. Ohio St went 2-0.

Baylor has gotten some slack for losing to lowly West Virginia, but WVU is 26th in the Sagarin poll, and they are especially stout at home. That's not a terrible loss. It gives Baylor a top 30 loss, and it also stands as TCU's third best win. West Virginia ranks ahead of #34 Minnestoa, TCU's other big win.

It's also worth noting that TCU and Baylor played almost the same exact schedule. They had 9 opponents in common plus they played each other. TCU additionally played Samford and Minnesota while Baylor played Northwestern St. and Buffalo. TCU has a schedule edge, but it is minute.

Head to Head

TCU's slight schedule edge is completely eradicated by head-to-head. Baylor beat TCU on the field, and the committee's rules specifically mention this not only as a factor, but as an important factor that can override strength of schedule. Baylor gets the huge nod here, and Ohio St. is left unaffected. Though I guess this is where we mention TCU crushed Minnesota and Ohio St. only beat them by 7. If anything, that helps Baylor even more, as it would make the priority Baylor then TCU then Ohio St.

Conference Titles

Ohio St. won its conference title without dispute. We can argue the relative quality of the Big 10 and the Big 12, and I'd say the Big 12 is the better conference. There are two ways of looking at the Big 12 conference title, so let's follow both methods.

The first method is actually following the Big 12's stated tiebreakers, and the first tiebreaker is head-to-head. By this method, Baylor is the Big 12 champ, which essentially kills TCU's claim. Looking at our other factors, we have similar strengths of schedule, though Baylor played more top end teams than Ohio St. Baylor also won the better conference and head to head is not factor. I think it would be close, but by the stated criteria, using this method as Baylor as the undisputed Big 12 champion, the Bears make the playoffs.

But that's not the method the committee chose. The Big 12 changed its rules midseason to give itself two bites at the playoff apple, and ignored that whole One True Champion thing they had been trumpeting. So the committee took the Big 12 at its word, the Big 12 had co-champs. Now, the analysis is different.

We still have the similar strengths of schedule, but now Ohio St. has the edge in the conference title tiebreaker. Ohio St. is the undisputed champ, while TCU and Baylor are both merely co-champs. Given the narrow difference in their strengths of schedule, the gap between being an undisputed champion and a disputed co-champ is enough to tilt the scales in Ohio St.'s favor.

It also has, not coincidentally, the added benefit of smacking the Big 12 for clumsily trying to rig the process in their favor.

The process worked, and if the committee followed its procedures, then Ohio St. is the proper selection. It is not an "objective" choice, but it's not objective process. It's making a value judgment, and that's why we have a committee in the first place. But the selection of Ohio St and not either Big 12 team is good for college football fans in general, going forward. After scolding people for reading too much into what the committee does, I'm going to make the same mistake. There are three lessons from the committee, all positive for fans of college football:

ONE. Conference titles matter. And they matter a lot. This is great news for the "regular season matters" crowd. The most important goal of every team is to win their conference. And the committee valued that above all else. It wasn't the only factor, but it's a big one.

TWO. Play somebody. One of the worst things about the BCS era is the erosion of the cross-regional game. It was more important not to lose, so teams scheduled accordingly, stuffing their schedule full of speed bumps. Baylor's out of conference schedule is downright shameful and it's a good thing they are being kept out of the playoffs because of it. Hopefully, this will lead more teams to schedule more quality games amongt Power 5 opponents. We need September to matter again. Now teams are more likely to schedule tough games, which is good for fans.

THREE. Politics will always matter, but transparent attempts to pull a fast one on the committee will be punished. The Big 12 essentially changed its rules to back the horse they felt had a better shot, and the committee didn't buy it. Good. Now, there is the question of whether the committee just selected the team with the better Q-rating in Ohio St. That is a concern, but I'd prefer the helmet test over conferences trying to rig the game.

I was rooting for Baylor, but it is better for college football that they didn't make the playoffs. Now teams might start beefing up their OOC schedules. It's even better that TCU missed the playoffs, as a team is not rewarded for a conference changing its rules midseason. Also, it's important that the committee's guidelines weren't just hot air. Head to head matters. They said so, and they stuck to it.

But the biggest positive is this: if the BCS were still in place, the best team in the country would not have a title shot. Under last year's system, Alabama is playing Florida St. for the title, and we're hearing people trying to justify screwing over Oregon. This year, the Ducks get a chance to win it on the field. That's what progress looks like. The system worked just by moving the "who got screwed" discussion.