Thanks for joining me for my follow up Ballon d’Or piece!
Last time I focused on the attacking metrics of the game to rank each player, which admittedly left little room to praise or highlight the four goalkeepers in contention for the 2016 edition of the award. As a result of that I am now going to provide an in-depth analysis of these goalkeepers, with the aim of showing that there is more to football than the people putting the ball into the back of the net and making chances.
The last (and only) time a goalkeeper won the Ballon d’Or was way back in 1963, when Lev Yashin lifted the coveted trophy. So who are the four goalkeepers vying to do the same this time around? We have Bayern Munich and Germany’s Manuel Neuer (100/1), Juventus and Italy’s Gianluigi Buffon (66/1), Sporting Lisbon and Portugal’s Rui Patricio (500/1), and finally Tottenham and France’s Hugo Lloris (150/1).
The bookmakers aren’t giving any of them much chance at all, but can anything in our metrics show that they are perhaps more deserving of further recognition than the market thinks? Also, just which of the four goalkeepers deserves to have the shortest odds based on his output in 2016? I will uncover this and much, much more, but let’s start with the basics:
Like last time, let’s level the playing field by measuring how many minutes each goalkeeper has played. We’ll use this to normalise each player’s metric set so that we’re making sure we’re being fair to all four nominees.
Handily StrataData covers each player’s domestic league (English Premier League, German Bundesliga 1, Italian Serie A and Portuguese Primeira Liga) as well as the Champions League, Europa League and Euro 2016. So here’s the amount of time each player has spent on the pitch across those competitions in 2016:
Neuer has spent most time on the pitch with 3651 minutes under his belt for Germany and Bayern Munich. Patricio, Lloris and Buffon, with 3629, 3545 and 3376 minutes respectively, follow him.
Now let’s take a look at key goalkeeper metrics in the form of actual saves made versus expected saves, with the ratio of these two metrics indicating overall performance. My expected saves in this instance are simply calculated by totalling the actual number of saves made and the goals conceded before adjusting for minutes played.
Remember that we’re focusing only on saves made here, which basically boils down to shots on target that required goalkeeping intervention to stop a goal (as opposed to shots that were blocked or hit the woodwork.) These metrics are outlined in the graphic below:
The blue bars represent actual saves made per 90 by each goalkeeper, while the green bars represent expected saves made per 90, as a function of saves made and goals conceded. To highlight goalkeeper performance, I’ve also plotted the ratio between these two metrics in yellow. I’ll be using this ratio as the key performance indicator for the goalkeepers. I’ve calculated this by dividing the saves made by the expected number of saves made to give us a ratio from 0-1; with a larger number indicating better performance (i.e. that the goalkeepers are saving as many shots that they are expected to save), whereas a smaller number (closer to 0) indicates that the goalkeeper is not saving as many shots as he is expected to save.
Neuer again tops the rankings for saves made, with 1.73 saves per 90 minutes, followed by Lloris, Buffon and Euro 2016 winner Patricio. Neuer also appears to be performing the closest to what is expected from him, with the largest ratio (0.74) between what he actually saves and what he is expected to save. In this light, Patricio is the worst performer, with a ratio of 0.59 saves per 90 below expectation.
There is a nice spread between all four goalkeepers, clearly showing that at the most basic level Neuer appears to be the most reliable goalkeeper, closely followed by Hugo Lloris. However, to go a bit deeper let’s now consider the type of chance the four goalkeepers are saving.
In the following analyses, a weight is given to each type of save a goalkeeper makes to help us differentiate between routine saves and exceptional, match winning saves. Weighted adjustment is a commonly applied technique and in simple terms it allows us to assign an adjustment weight to the metric of our choice (in this case, saves and expected saves) to get a richer picture of goalkeeper performance.
Saves (Adjusted for Chance Type)
In a similar way to how I evaluated the attacking players in my previous post, I’ve calculated the expected number of saves per 90 as the number of actual saves on the number of chances (i.e. shots) faced (both saved and conceded).
Combining chances saved with those that are not saved allows us to work out what a goalkeeper is expected to stop given the overall number of chances they face. I have worked this out using the average conversion rates of our six chance categories, labelled Superb (~75% conversion), Great (~40% conversion), Very Good (~25% conversion), Good (~15% conversion), Fairly Good (~8% conversion) and Poor (~2% conversion), summing a goalkeeper’s total chances faced per 90 by total conversion rate rather than by individual category.
This has allowed me to weight each goalkeeper’s saves and goals conceded by the quality of chance to provide a firm expected save number. This is an important metric to highlight the ability of our nominees to keep the ball out of their own net, though it is key to note that it assumes that every chance on target a goalkeeper faces can be saved. Once again, for ease of interpretation, I’ve plotted actual saves and expected saves, as well as the ratio of the two as the key performance indicator:
After adjusting for chance type, Neuer still tops the list for saves made (0.34 per 90), but this time has the second best save ratio of the group (0.52). Indeed, it seems that Lloris has been the most efficient goalkeeper in 2016 so far, just edging Neuer with the greatest ratio between number of saves made and the expected number of saves (0.53). Patricio is again the worst performing goalkeeper with a ratio of 0.41, indicating that he is not saving as many shots as he might be expected to and probably should not be in this discussion in the first place.
Saves (Adjusted for Chance Type and Shot Quality)
The depth of StrataData means that I can also take this analysis further and use a multi-element weighting method. To provide a much richer picture of all four goalkeepers, I’ve added the Shot Quality rating as captured by our analysts. This rating ranges from 0-5, where 0 illustrates when a shot was unintentional and 5 indicates that the shot was nigh on unstoppable. Therefore, a chance with a rating of Superb and a shot quality of 5 wouldn’t be expected to be saved, whereas a chance with a rating of Poor and a shot quality of say 2, for instance, would be expected to be saved.
Creating and applying a weighting based on chance rating and shot quality provides a much deeper picture of goalkeeping performance. I calculated the relative weighting for each shot quality and chance rating using all goals and chances conceded and faced, and applied these weights to our four goalkeeper’s saves and expected saves. This adjusted data can be seen in the graphic below:
Once again, Neuer tops the list for saves made, even after adjusting for chance rating and shot quality, with 0.4 saves made per 90, closely followed by Lloris with 0.39, Patricio with 0.25 and Buffon with 0.23. Despite that fact Buffon has the lowest save rate amongst the goalkeepers, what I find most interesting here is that he has the best expected save rate (0.76 saves per 90), giving a better save ratio (0.30) than Patricio (0.29), who again drops down being the worst performer of the group.
In terms of the best performing goalkeeper in this category, we see that Lloris still stands at the top with his save-per-chance ratio of 0.41, though Neuer is again right behind him and is ultimately the “busier” of the two.
At this point in the piece it seems as though if there were a goalkeeper seriously in contention for the 2016 Ballon d’Or then it would be a straight shootout between Germany’s number one and France’s number one, with Patricio not in the same class and Buffon falling away badly when efficiency is brought into the equation.
Still, let’s round things off by looking at something not always capture in mainstream statistics – individual mistakes made leading to chances and/or goals.
You may remember we looked at red and yellow cards for outfield players in the previous blog, so this time I decided to look at mistakes made by our four goalkeepers. Mistakes are an unwanted aspect of any player’s game and can be useful to look at as a final key differentiator when deciding between two players for scouting purposes, or if hypothetically discussing if a goalkeeper might win the Ballon d’Or…
With that in mind I counted the number of mistakes attributed to each player from the 1st of January to the 31st of October 2016 and once again normalised for minutes played:
Although mistakes are not at all common for goalkeepers of this quality, a few still appear and have to be taken as a red mark against them. Lloris undoes some of his prior good work by topping the list, making 0.20 mistakes a game that lead directly to chances or goals. Neuer and Patricio aren’t too far behind, with 0.17 and 0.15 mistakes a game respectively, but Buffon recovers somewhat to show his enduring class here, with a mere 0.05 mistakes per 90.
Ultimately this might be enough to tilt some people in the direction of Neuer, but again the mistakes are so few between he and Lloris that it really becomes a game of fine margins.
I hope that I have shown how calculating expected saves alongside saves made and then adjusting for key performance metrics is important to see the true value of a goalkeeper. We saw in the first analysis that Neuer appeared to be comfortably the “best” goalkeeper based on his number of saves per 90 and the overall save ratio for each goalkeeper.
However, to provide a richer picture of the data we adjusted the goalkeeper data to account for the quality of chance he faces and the quality of the shot he faces. After carrying out the first adjustment Lloris stood out above the rest and was once again at the top when shot quality was taken into account, though Neuer was right behind him in both instances. The only red mark against Tottenham’s captain were the elevated number of mistakes he has made versus Neuer in 2016, though the difference really was negligible once again.
Indeed, if a goalkeeper were to be in the running for the 2016 Ballon d’Or then the evidence would point towards the Frenchman being the “value” bet at 150/1 versus Neuer’s 100/1. Still, Buffon is the surprising favourite amongst this quartet at 66/1, while Patricio rightly does not seem to feature in the discussion at all at 500/1! If it were me choosing then Lloris would be the man and I’m pleased that the numbers I have uncovered have shown him as a peer to Neuer, who is already widely accepted as being a great in this era of the game.
Until next time,
Sagar Jilka (@DrSagarJilka)
PS: To more easily compare the players evaluated in detail in my last two posts I will soon be releasing some radar plots showing their performances in each key metric category, so please keep your eye out for those! Credit for the inspiration behind them must go to www.ramimo.com for their NBA originals in 2013 and a post on www.statsbomb.com from Ted Knutson (@MixedKnuts) uncovering these back in April of this year.