MLS payrolls: a mixed bag of escalating costs

Not even his $6.5M salary can create

a team payroll disparity within MLS

Much of this past MLS offseason was spent worrying about a potential strike or lockout due to the league and the players union not being able to come to a quick agreement on a new collective bargaining agreement (CBA). Most of the owner's concerns centered around player salaries and policies that would enable them to rise at a faster rate. Along with that concern was one of team competitiveness - we have seen how a lack of any spending cap in the EPL and La Liga has led to a pay-to-win philosophy.

This leads to two fundamental questions that I will answer with statistics.

1. Did the league see an increase in team payroll disparity during the last CBA?
2. Did the league see an overall increase in team payrolls during the last CBA?

Background

MLS and its players union signed a CBA that covered the 2005 through 2009 seasons. It included salary caps, limited player movement, a no strike clause, and a variety of other regulations for the league and its players. Prior to the 2007 season, the league announced the Designated Player rule in a desire to attract talented, visible stars from around the world while providing minimal impact to a team's salary cap. A few teams have taken advantage of that rule, the most notable being David Beckham and his $6.5m salary.

Of note is the salary cap during the 2005-2009 seasons. It has ranged from $2.0M in 2005 up to $2.3M in 2009. The cap has always been a bit of a fungible figure. Seven of the 12 teams ran payroll higher than the cap in 2006, with 12 of the 15 running higher than the cap in 2009. I am sure that MLS has some creative accounting rules that show the teams are under the cap, but it is not clear to me how this is done. The reality is that the clubs are spending a good bit more than the cap, and reality is what I am interested in.

At first blush, there seems to be a good bit of disparity in team expenditures that has grown over the life of the last CBA. Figure 1 shows how much each team has spent as a multiple of the league average for each season.

Figure 1: MLS team payroll, 2005-2009, as a function

of the average team payroll for each year (click to enlarge).

Statistics will tell us whether or not the disparities we see starting in 2006 are real, and whether or not there has been general upward movement in league salaries over the five years of the CBA.

The Prerequisite: Normality

The pre-requisite for many of the frequently used parametric statistical analyses (t-tests, test for equal variances, etc.) is that the data being analyzed be normally distributed. This is because the tests being performed on the data use concepts based upon the Central Limit Theorem to test two sample populations. Thus, normality is a prerequisite and the team payroll data must be checked for this property before beginning any analysis.

An interesting phenomena is observed when each year's distribution of team payrolls is checked for normality. The 2005 and 2006 seasons both test to be normal. That is to say, when a normality test is performed on the data we find a p-value greater that 0.05, which indicates a low risk of accepting the data is normal when it might actually be non-normal. See the Figure 2 as an example of what a "successful" normality test looks like.

Figure 2: Descriptive statistics for 2005 MLS team payroll.

(click to enlarge)

However, once the designated player was introduced in 2007, team payrolls for each season went to a non-normal distribution. That is to say, the p-value for normality tests of the 2007-2009 seasons is much less than 0.05, indicating that we would be at high risk of drawing the incorrect conclusion if a normal distribution were assumed. See the Figure 3 as an example of what such non-normal distributions look like.

Figure 3: Descriptive statistics for 2009 MLS team payroll.

(click to enlarge)

These results have very important implications for the types of tests used to answer the two fundamental questions posed at the beginning of this post. There are now two options:

1. Use a transformation method, like Box-Cox, to transform the 2007-2009 data to make it normal. This makes determining the absolute effects difficult due to the presence of the transformation function.
2. Use a non-parametric statistical test on the original data. Non-parametric tests are far more forgiving of the original data, but end up requiring a far greater spread in two sample data sets to show a statistically significant difference when compared to similar parametric tests.

In this case, I have chosen to pursue course (2) for my study. I am more concerned with testing whether there are differences, as well as easily determining how large a gap there is for tests that show a statistically significant one. Avoiding transforming data via (1) is a huge benefit in this case, especially since the statistics I am interested in don't benefit a whole lot from using parametric tests.

Has team payroll disparity increased?

To answer this first question, I turn to a comparison of the 2009 and 2005 team payroll data. Using this data means I am looking at the bookends of the last CBA. I did this, rather than analyzing each of the five years of the CBA, as it is reasonable to suspect that any change in the distribution of team payrolls will be gradual from year-to-year. Thus, to see any potential difference one must look at the beginning and the end of the CBA, which is what the owners and players really cared about anyways when they were negotiating the new CBA.

To do this, I performed a test for equal variances between the 2005 and 2009 seasons' team payrolls. Performing such an analysis can identify whether the variation in team payrolls between the two seasons is statistically different. If this were so, and if the 2009 variance was higher than 2005, we could conclude that the gap between "rich" and "poor" clubs' expenditures on player payroll was widening. As the 2009 data is non-normal, I used a specific subset of the test for equal variances, Levene's test, to make this evaluation. If the Levene's test comes out with a p-value of 0.05 or lower, we can reject the null hypothesis that variances are equal and safely assume that they are different. Figure 4 displays the results of such a test.

Figure 4: Test for equal variance results,

2005 vs. 2009 MLS team payroll.

(click to enlarge)

The first conclusion that can be drawn from the analysis is that we cannot reject the null hypothesis (p-value from Levene's test = 0.219), thus we cannot declare that there has been a statistically significant increase in the spread in MLS team payrolls throughout the last CBA. We must accept the null hypothesis: there is no difference in the variation in team payroll between 2005 and 2009.

The second conclusion is that this test demonstrates the danger of not performing the initial check for normality. If I hadn't known that the data was non-normal, I might have used the results from the F-test, and it's p-value of 0.00 would have led me to conclude that there was a difference in payroll variation between 2005 and 2009. This would have been the wrong conclusion.

The third conclusion is that not even David Beckham's 6.5M+ salary and the resultant $10M LA Galaxy payroll (the star on the far right of the Levene's test box plots) can generate such a disparity. One might be able to rationalize excluding Beckham's salary from the analysis, or at least cutting it down to the $3M range, as it is more than double the next closest DP salary. This would only close the gap in 2009 salaries, raising the p-value even more and making any test for equal variances more difficult to reject.

Has the average team payroll increased from 2005, and if so by how much?

Now that we know that the variability in team payroll has not increased, it is time to determine whether or not average team payroll has increased from 2005 to 2009 and by how much. To answer these two questions, I used the Mann-Whitney two sample test for non-parametric data. In this case, the null hypothesis is that there is no difference between the two seasons while the alternative hypothesis is that 2009 team expenditures were greater than 2005. Figure 5 shows the results of the test.

Figure 5: Results from Mann-Whitney

test, 2009-2005 team payroll

The results from the test return a p-value of 0.0018, which means we conclude that there is a difference between the 2009 and 2005 season payrolls. The test also gives an estimate for the difference - $746k. Thus, we can conclude that the average team's payroll has increased by $746k over the five year CBA.

General conclusions and new questions

Through statistical analysis, we can see that the gap in team payroll has not grown over the last three years, but the average team payroll expense has increased by almost three quarters of a million dollars. For a league concerned about financial stability and a modest $2.3M salary cap, this had to be part of the reason for the concern in negotiating a new CBA. Teams are spending far more money in 2009 than in 2005, regardless of the good intentions of the CBA.

This analysis leads to a whole new set of questions. Two of which that come to mind are:

1. How much of this increase is due to normal competitive pressures year-over-year in a pro league vs. the unique salary impacts of the DP?
2. Like the EPL and La Liga, can the difference in finishing position year-to-year in the league be partly explained by team expenditures?

I will address these questions in future blog posts.