An Indictment of the MLS Playoff Structure, Part 1: Letting Too Many Teams in to the Party

Note: This the first of a several part series that I will be writing about MLS's playoff structure and how it penalizes successful teams.

Pay As You Play begins its concluding section with the following observation.

"Contrary to popular wisdom, things don't even out exactly - after all, no higher power exists to bring immaculate balance (and if there are football gods, one assumes that they exist purely to mock us, not provide parity on refereeing decisions) - but a whole campaign is a better barometer for the quality of a football team or a manager than one or two games."

Sadly, that's not how we determine our league champion in the United States' top flight of soccer. Our little revolution in 1776 has led to an odd cultural habit in our country - we refuse to ever adopt something as-is, often insisting on some uniquely-American twist to a perfectly fine product. This even applies to our soccer league. Sure, we've gotten rid of some of the worst examples of such attempts at creating "American soccer" (backwards counting clocks, insisting on eliminating the tie, etc), but the abomination that is a playoff-based championship persists and it penalizes successful clubs. Let me explain...

Some Background on US Professional Sports and Playoffs

A playoff-based championship format makes sense under two conditions:

1. When a league is too large to play a balanced schedule, OR
2. When the time constraints of the competition do not allow for a balanced schedule.

Scenario (1) applies to much of the professional US sports landscape. We're a nation of 300+ million people, we rank third or fourth in land mass (depending on the calculation method used), and most of our leagues are well over 20 teams. The National Football League (NFL) has 32 teams, and the National Basketball Association (NBA), Major League Baseball (MLB), and the National Hockey League (NHL) each have 30 teams. Travel from coast-to-coast can take 4-5 hours on a direct flight, with a three hour time change also involved. Beyond the geography, the each league would require between 58 and 62 games in a balanced schedule.

Given these difficulties, each league has split themselves along conferences and subdivided divisions, as well as adopted a playoff system. The format ensures that each team plays a balanced schedule within their division, and then a rotating batch of teams within their conference and across conferences. This overall unbalanced schedule then demands a playoff to decide the champion, with seeding determined by each team's relative performance during the season. Each of the leagues break a differing percentage of their teams to the playoff rounds:

• NFL: 37.5% (8/32 12/32)
• NBA: 53.3% (16/30)
• MLB: 26.7% (8/30)
• NHL: 53.3% (16/30)

Each round of the leagues' playoffs consist of a best-of-five or a best-of-seven series of games (the NFL is the one league that doesn't, opting for a single elimination playoff). Using the a five or seven game series minimizes the chances of a team pulling off a string of a few upsets to win a championship, and instead forces teams to play a minimum of 12-16 games to win a title.

The MLS Playoff Structure

The top flight of US soccer, Major League Soccer (MLS), also uses a playoff system to determine its champion. For a number of years MLS has been admitting eight teams each season into their championship playoff system (in 2011 they will admit 10 teams), even when they reached their modern era low of ten teams after contraction in the early 2000's. I have focused on the 2005 season onward, as the related financial data I use in other analyses stops being reliable any further back in the league's history. From 2005 to 2011, the league has increased from 12 to 18 teams. This has generated the following percentage of teams making it into the MLS playoffs.

• 2005: 66% (8/12)
• 2006: 66% (8/12)
• 2007: 61.5% (8/13)
• 2008: 57% (8/14)
• 2009: 53.3% (8/15)
• 2010: 50% (8/16)
• 2011: 55.6% (10/18)

The tournament is a bit of a hodge-podge of elimination formats. The initial round is a two match home-and-away format with the team with the highest aggregate number of goals moving on to conference finals (penalty kicks are used as tie breakers). The final two rounds - the conference finals and the MLS Cup - are both decided by single elimination matches. Since 2007, the league has used the following format to determine seeds:

1. The top two finishers in each conference (MLS has two - East and West) are automatically seeded as #1 and #2 in each conference bracket (overall seeds 1 through 4)
2. The next four seeds are populated with the teams with the next four highest point totals, regardless of which conference they are from. These teams can be referred to as wild cards. If a conference's four spots are already filled by their top two finishers and two of the wild cards from their conference, any remaining wild cards from that conference will be placed into the opposite conference's bracket.

This structure provides for some interesting pairings. First, it doesn't seem to reward outstanding play of top finishers in a conference. In all other leagues, the reward of being seeded #1 in the playoffs is getting to play the lowest seed. That is not the case in MLS where situations like the 2010 season arise - the two Eastern Conference qualifiers got to face Colorado (#7) and San Jose (#8). Meanwhile, Los Angeles and Real Salt Lake, who fought to the last week of the season for the top-of-the-table position, were rewarded for all their hard work in facing the hottest team in the league (Seattle at #6) and a team whose home pitch was a fortress all season long (Dallas at #5).

Second, it has resulted in the last three MLS Cups having teams from the same conference facing each other for the league championship. One might question the fairness of a playoff system where a team who has underperformed all season long gets to cross over to the theoretically weaker conference (assuming that the conference with fewer teams is weaker) for their playoff run. No one would make the serious argument that New York (2008), Real Salt Lake (2009), or Colorado (2010) were one of the top four (let alone two) teams in MLS each of the seasons they reached the final. Yet those teams got a shot at winning the league championship after underachieving all season, with Real Salt Lake and the Colorado Rapids each walking away with the title.

If the standard of admittance to the MLS Cup playoffs is so low, what quality of play is required to reach that low standard?

Points and Goal Differential Required to Qualify for the Playoffs

What exactly does it take to qualify for the MLS Cup playoffs? Luckily, data compiled from the 2005 season onward can give us insight into the typical points and goal differential required to finish eighth or higher in the overall league table.

Given the increasing number of teams since 2005 and the varying level of matches played in a season, each of the variables has been normalized. Finish position was translated via the now-familiar Soccernomics natural logarithm transform, which also provided a good way to assure the data set was normally distributed. Each team's points were translated into a percentage of the available points earned in a season. The plots below show the relationship between each metric - finish position versus % of points earned, finish position versus goal differential, and % of points earned versus goal differential. Click on each image below to enlarge them.



















Now that the relationship between points, goal differential, and finish position is understood, the performance required for an 8th place finish and qualification for the MLS Cup playoffs can also be understood. Applying the equations in the regression plots to each season produces the table below.







What one immediately sees is the low quality of teams that will typically qualify for the MLS Cup playoffs. A losing record (< 50% of available points earned) can be expected, and so can a negative goal differential. The requirements of a typical 8th seed were getting more difficult from 2005 through 2010 as the number of playoff spots remained the same while the number of the teams in the league increased, so that by 2010 the typical team would have earned 46% of the available points and had a positive goal differential. Fans were disappointed when the MLS commissioner announced that the playoff field would be expanded to 10 teams in 2011, lowering qualification standards back to 2008 levels.

When examining the actual points accrued and goal differentials of the 8th seeded teams from 2005 to 2010, the picture looks only a little bit better. The table below (click to enlarge) summarizes their results versus the regression equations, with the yellow columns representing the data for each of the teams.

As would be expected, three of the teams earned a percentage of points above the predicted value and three below. San Jose's 46 point haul over thirty games in 2010 was the best performance of an eighth seed over the six years. This should come as no surprise as it was also the most lopsided table over the period with six Western Conference teams qualifying for the playoffs, a result of the Western Conference's domination throughout the season.

The results for goal differential look like a great overachievement, but they mask the teams around the 8th seeds that had substantially worse results. In 2006, Colorado was tied on points with New York and had a -13 goal differential. In 2007, Chicago tied Kansas City on points and had a -5 goal differential, while FC Dallas was ahead of Kansas City with 44 points yet had a -7 goal differential. Kansas City, New England, and Chivas all qualified in 2008 with negative goal differentials. New England qualified again in 2009 with a negative goal differential, this time at -4.

What's even more disturbing is the final column of the table, which indicates the final round in which each team participated. Save for New York in 2006, each of the 8th seeds has made it out of the first round. Three of them have gone on to the MLS Cup, with two of them winning the Cup (LA in 2005 and Real Salt Lake in 2009). It seems that most teams that have qualified eighth have been able to pull off at least two games above their seasonal average poor performance and move on to the next round of the playoffs.

Preliminary Conclusions

It seems as if MLS is not only pretty generous with who they let in to the postseason, but their playoff format almost seems to favor teams that have underperformed. Without a multi-game series that gives the higher seeded team some sort of advantage (home field, first game being away and counting away goals more than home goals, etc.) there seems to be no benefit in the MLS playoffs to performing better during the regular season. We Americans love our postseason tournaments - arguably the biggest sporting event each year is known as "March Madness" for all the upsets it produces. This style of single elimination playoff championship is only found in one of the four major US professional sports leagues, and it is not the way MLS should be determining it's champions after nearly seven months of a balanced regular season schedule.

All discussion to in this post has been based upon point data - single observations or groups of observations but no study of the statistical distributions. What really drives success in the MLS playoffs, especially the critical first round where upsets seem to abound? Luckily, someone else has compiled good summary data of what can be used to predict first round success. The topic of what seems to predict first round success will be the studied in detail in Part 2 of this series.