The Premier League

is not very equal (2016, apart, when Leicester won). The same teams (Liverpool, Man United, Man City, Chelsea) always win. The Bundesliga (Bayern Munich) and Serie A (Juventus) are worse. The NFL is much better.

All statements that seem to make sense. But how accurate are they? Is there real support for them? Well, measurements do exist. Sports economists have long applied measures of competition across industries to sports leagues to quantify this. Below is the “concentration ratio”, a ratio of the market share of a subset of firms in an industry, applied to the English Premier League since 1888. That is, the proportion points earned by the top 3 teams compared to the rest:

That’s all well and good, but it depends on the number of teams in the league – in the early years there were only 12 or 14 teams. Also, we may care about the distance between teams across the division. Maybe we care about relegation battles as well as top battles. With different preferences for what interests us, a plethora of different metrics exist.

Dimitris Karlis from the Athens University of Economics and Business recently presented a fascinating article on this topic. In collaboration with Ioannis Ntzoufras, they ask the question: what is the uncertainty associated with these measurements?

That is, if we observe, say, 0.25 in the graph above for the Premier League in 2016, is that really is this different from the observation of 0.18 in the mid-1990s?

Many measures consider how far competition in a league is from a perfectly balanced ideal: all teams are evenly matched. Such ideas simply assume that in the situation where the teams are evenly matched, all teams robotically win an equal number of games. Similarly, the most unequal league has the best team robotically beating everyone below them, and the team below them all others except the teams above them.

But sport, and football in particular, is not like that. Football has an element of randomness. This element we love. When the underdog succeeds – when Villa hammers Liverpool 7-2. So what can we do about it? Well, we can assume that results, and even goals, occur according to a certain probability distribution – that is, they are randomly generated.

Not totally random. Just pretty random – say, there’s a 30% chance of Liverpool scoring once against Man United.

If we are a bit more specific and assume that each team’s goals are independently Poisson distributed (not the best assumption, but it doesn’t matter too much for our goals), then if the team A should score 2 and team B 1 , there is only a 10% chance that the game will end 2-1 for team A. There is still a 10% chance that it will end 2- 2, or 1-1, or 1-0 for team A. At 1-1 or 2-2 a toss leads to a final result other than 2-1 or 1-0, and therefore to a different final table, and to a different measure of competitive balance.

Good teams on paper can lose a lot of games. The best team doesn’t always win.

A better method would therefore be to consider quantifying the uncertainty surrounding each measure of competitive equilibrium in the same way as we do for statistical or hypothesis tests. Is this effect statistically significant? Is this measure of competitive equilibrium statistically significant? That is, is there enough evidence that the teams are not evenly balanced?

Karlis and Ntzoufras created the graph below:

This is the previously mentioned concentration ratio, but normalized to be between 0 and 1 (0 best, 1 worst), and plotted for England and Germany. The simple dots and lines that connect them are the measurements for both leagues. They suggest England have been less competitive recently. But once the vertical lines, the confidence intervals (or confidence bands) are added, we see that they overlap for each season.

There is too much uncertainty in football for us to say with certainty that England were more or less competitive as a league than Germany between 2000 and 2016.

More than that, Karlis and Ntzoufras calculated the range of values ​​that would be compatible with a perfectly balanced league. These are the horizontal dotted lines. Therefore, if a confidence band for a league falls inside the horizontal dashed lines, it suggests that for that particular season that particular league was indistinguishable, statistically, from a league in which all teams were perfectly equal in talent.

In this regard, England and Germany have moved away from this in recent years, but many observations on the sample are such that there is no difference. These leagues could have been leagues with evenly balanced teams, statistically speaking.

What does that really mean, though? Well, that forces us to question our perceptions a bit more. It’s hard to really think that a Premier League with Man United, Chelsea, Arsenal and Man City dominating as they have over this period can really be considered equally balanced. This either suggests that as a test of balanced leagues they are not particularly powerful measures – they are not able to distinguish between the case of an equal league and an unequal league.

But things often happen very, very unexpectedly – in keeping with a high enough level of uncertainty. Leicester City won the Premier League in 2016. Again, food for thought. Not if they could do it again, but how do we really know how strong the teams are?