Calculated Reactions: Goalie rankings, 2009-10; or, why goals-against-average tells you nothing

So far we've developed and presented player rankings in basketball, volleyball, and football. These aren't the most sophisticated measures of player value, but I think it's the best, or close to the best, that we can do without the same degree of detail you find in professional-league statistics.

Today we move onto hockey--specifically, goaltenders.

Many factors influence goaltenders' performance: the quality of the defence in front of them, the overall league scoring context, the quality of the shots (and of the players) they face, and so on. We can't account for all of these, but we can try to correct for some of them.

We'll use last year's stats in this post. As an example, take Ottawa's Riley Whitlock (you'll see why soon enough). In '09-10, Whitlock had a .904 save percentage, 10th nationwide among goalies with 1000 minutes or more.

Now we adjust for the quality of his penalty-kill unit, by adding or removing goals from his record depending on whether his team was really good or really bad while shorthanded. (This serves as a proxy for Ottawa's defensive performance.) As it turns out, the Gee-Gees were successful at killing penalties at just a 72.4% rate, lowest in the OUA, so we're going to adjust Whitlock's goals-against down somewhat. Leaguewide, the rate was 81.4%, and the Gee-Gees were shorthanded 196 times so that means they gave up 17.6 "more" shorthanded goals than expected. (Not all powerplays are the same length, so this isn't strictly true, but until I actually figure out the amount of time each team spent at 5-on-4, 5-on-3, and so on, we'll have to assume they are the same length.)

Whitlock was on the ice for 85% of Ottawa's total minutes (shorthanded or otherwise). 17.6 times 85% is 15 goals, then we'll take away one of every five of those--since Whitlock was one of five Ottawa players on the ice in a typical shorthanded situation, we'll assume he was responsible for 20% of each goal--and we're left with 12 goals. So instead of giving up 92 goals on 959 shots, we treat Whitlock as if he gave up 80 on 959, for a .917 save percentage.

And then we make two simple adjustments, one for the leaguewide scoring context and the other for his team's strength of schedule. Whitlock's at .917 and the OUA average is .895 so that's 20 or so goals more than expected given the number of shots he faced. And Ottawa's strength of schedule wasn't great, so we take a little bit of credit away from him, down to 19.8 "extra goals saved."

From there, we just have to apply the goals-to-wins conversion (about one win per eight goals), which tells us that Riley Whitlock was, by this measure, worth 2.5 wins in 2009-10, the highest of any CIS goaltender.

Here are the men's goaltenders from last year who were worth at least one win above average:

WinsNameTeamSV%GAA
+2.5Riley WhitlockOttawa.9043.81
+2.3Jean-Christophe BlanchardUQTR.9242.28
+1.9Anthony GriecoWestern.9541.53
+1.6Steve ChristieManitoba.9172.51
+1.3Maxime JoyalConcordia.9073.75
+1.1Ryan DanielsLaurier.9143.38
+1.0Wayne Savage UPEI.9043.23

I included the last two columns to illustrate how you really can't just look at save percentage or GAA and conclude much of anything. Whitlock numbers in those categories are the worst on this list...but then again Ottawa was the worst non-RMC team last year (second-last in RPI). So not all of that 3.81 is on his shoulders. Put him behind the Western defence and he'll look a lot better.

Steve Christie was named the top goaltender in both Canada West and CIS last year, and while he might be somewhat behind some other non-CW players on this list, he's tops in the West, just like our Evan Daum thought ("without a doubt"). In fact, Evan argued that Christie was the best in his conference, even if his other numbers didn't look so good, and here we are agreeing with him. Works for me.

**

We can do the very same thing for the women's results from last year too:

WinsNameTeamSV%GAA
+2.6Jamie TessierWindsor.9352.32
+2.3Liz KnoxLaurier.9600.89
+2.3Mel Dodd-MoherQueen's.9421.82
+1.8Kathy DesjardinsMoncton.9431.61
+1.7Meghan Corley-ByrneMount Allison.9143.77
+1.3Sydney McMurterYork.9361.98
+1.3Vanessa FrederickSaskatchewan.9271.88
+1.2Kendyl ValentaToronto.9272.41
+1.2Beth ClauseBrock.9242.55
+1.0Audrey Doyon-LessardConcordia.9093.41

Liz Knox was last year's player of the year, goalie or otherwise, so seeing her behind Jamie Tessier (and tied with Mel Dodd-Moher) is a little bit of a surprise. But not much of one, because:
  1. There's virtually no difference between 2.6 and 2.3;
  2. Tessier faced a lot more shots than Knox (930 vs. 550);
  3. While Knox is very good, so is the team in front of her (this is related to point 2); and
  4. Tessier, at one point, had a .950 (!) save percentage against Laurier across a three-year period despite playing for the unremarkable Lancers, so she's definitely got the skills to be put in the same sentence as Knox.

The second-team all-star goaltender last year was Manitoba's Stacey Corfield, who would be on that list if I extended it to include three more names (+0.8 wins).

**

There are two cases above of a goaltender on a great team being ranked behind a goaltender on a not-so-great team: Anthony Greico behind Whitlock and Knox behind Tessier. While some may object, based on the philosophy that the best goaltender in the league can't play for the worst team, I think it's more accurate, and that this is a better way to figure out who really had the biggest impact on his or her team.

Also note that you need to take those numbers above and regress heavily to the mean if you want to predict how they will do once 2010-11 is done. A 28-game season is hardly enough to draw conclusions from. I suppose I could include the partial stats from this season, but I'll wait until the end of the year and include both '09-10 and '10-11 (playoffs as well).

So the full 2009-10 rankings are here, and again, the 2010-11 version will follow when this season is over.
Next PostNewer Post Previous PostOlder Post Home

1 comment:

  1. Great analysis, an interesting read. As a Laurier hockey fan, I look forward to next year's results. Ryan Daniels has been lights out this year, and if he keeps it up, I look forward to seeing how many games he has "stolen" as Laurier's mens team is 9-4-4 despite being outshot almost every game.

    ReplyDelete