This time out, I am going to illustrate the strange inconsistencies in the scorekeeping in CIS basketball. We've all made jokes about it ("Completely Inaccurate Stats"), and I'm by no means the first to notice the curiosities. But I wanted to sit down and go through the issue completely, hopefully going past the jokes to a more removed and calculated reaction (see what I did there?).
But wait a second Rob, why does it matter if the stats are wonky, anyway?
For one, you'll see below that this inconsistency can affect the statistical records of the game. Not enough to change the outcome of any game, but certainly enough to cast doubts on many team and individual statistics, and the methods by which those statistics are kept. I realize that only about seven people in the world care about the number of assists in a CIS game, and five of them are me, but there's no reason for these issues to be present.
I'll also admit that, selfishly, I want my own rankings to be better, which would require consistency in the stats across all conferences and schools.
The idea for this started when I recently looked into the difference between home and away stats, just as a curiosity. You can access home-away splits for each team on the CIS site--here is UBC's men's team from last year--and a quick glance at some teams can reveal some odd results, but I was more interested, at first, in overall trends, so I went down to the boxscore level and rebuilt the home-away stats from the bottom up.
Here's what I mean by overall trends. The categories in this table are sorted by the home-court advantage (HCA), meaning the percentage difference between a team's results at home and that same team's results on the road (so the average CIS men's team shoots 4.4% better, or 1.8 percentage points better, at home). Statistics are either expressed per game (200 player-minutes), or as percentages.
Men's basketball, home and away splits, 2009-10
Before I get into the big numbers (looking at you, assists), notice that the home team does, in fact, enjoy an advantage in nearly every category. Most of these are beyond the interpretation of an official or the scorers' table--either the ball goes in the basket or it doesn't--and are exactly what you'd expect: the home-court advantage, whether due to travel, sleeping in your own bed, familiarity with the arena, or whatever, does exist. It's important to know that there's a true HCA (however small) so that we can greater appreciate how bizarre the extreme results are for, say, blocks and assists.
Assists seem particularly suspicious, but you probably get more opportunities to record assists at home (if home teams take more shots, for example). So let's add a row to that chart and see if that advantage goes away:
So the average team, when at home, is awarded assists on 56% of their made baskets, about 8 percentage points higher than when they're on the road. That's not that bad, though, and mis-allocation of assists is of course a basketball universal. So if it's just part of the game everywhere, then it all balances out and there's no problem.
If there's no problem here, and the schools were generally consistent--meaning, the home team generally has an 8-point bonus over their own results on the road--then the 43 home assist bonuses (one for each men's team) would be centred around 8, with simple random variation causing some to be higher or lower, and a few to be much higher or lower. But only a few.
Rather than overload you with another table, let me overload you with letters and colours and lines instead:
The two axes cross at 48% and 56%--the CIS average for home and away assist rates--but what we're (or I'm) really interested in here is teams above the red line, meaning they receive many more assists at home than on the road, or below the blue line, meaning they receive many more on the road than at home.
If the results were consistent, most everyone would be between the red and blue lines. But instead we have 9 schools--CBU, Memorial, Lakehead, Western, Toronto, Manitoba, Laurentian, Brandon, and Concordia--above the red line, meaning their home bonuses are significantly higher, and another seven below the blue line. So that's 16 out of 43 with extreme results, when statistically we'd expect two or three.
You remember what the bell curve looks like? Well, all of these 16 teams are in the tiny parts at the far right or far left of that curve. In order to even see the curve, MUN and CBU would need to get in a plane and fly for a few hours. And stop to refuel.
But maybe there's something about the Capers and Sea-Hawks men's teams (and the Mustangs and Thunderwolves...) that make them legitimately more likely to receive more assists at home. In other words, it has nothing to do with the scorekeeping (at whatever school or schools), but the players on the floor. I don't know why this would be true, but let's assume it is. If so, then the women's team assist bonus shouldn't be the same, because they're different people playing a different game with a different strategy.
You can see where this is going.
Were the men above the red line? More often than not, the women were too. There's a very strong correlation between the home assist bonus experienced by the men's team and the same bonus for the women's team. For the nine men's teams with the highest home bonuses, seven of the corresponding women's teams saw similar extreme results. The same trend holds for the teams below the blue line as well.
Assists should be recorded in the same way by all CIS scorekeepers, and I am pretty sure that the above analysis shows they are not.
I haven't been to games at enough schools to know whether everyone's following the FIBA interpretations, and even if I had been, I honestly don't know the rules enough to tell you whether School X is over- or under-rewarding assists. So I'm not accusing anyone of pumping up the assist totals at home, or of ignoring visiting players' accomplishments while keeping score.
All I'm pointing out here is the inconsistency.
We haven't talked much about blocks, but those numbers were also pretty high; a 40% increase in blocks at home is worth investigating further. Just to confirm that the visiting team isn't giving the home team more opportunities for blocks: home teams received blocks on 8.2% of missed shots by their opponents. Visiting teams? Just 5.8%. So that 40% difference, or 2.4 percentage point difference, remains.
That difference of 2.4 wasn't consistent, either. Some schools saw a much higher bonus for the men and women (where it was actually 1.5 points on average). Most notably, York's home block bonus was 7.1 points for the men and 6.2 points for the women. Going back to the bell curve, we'd expect there to be two, maybe three out of 86 teams (men's and women's) whose home bonus was more than two standard deviations above the mean (and, of course, two or three more who are 2 SD below). Not a lot of room under that curve for more teams than that.
Of course you have guessed the punchline: there were 10. In order of how ridiculous their home block bonus (or "road block demerit") was: Winnipeg men, SFU women, UBC women, SMU men, Ottawa men, York women, Laval women, York men, Winnipeg women, UQAM women.
These are mostly different teams than we saw on the list of extreme assist bonuses, which shows you how statistical oddities aren't just limited to a handful of schools.
The results between home and road blocks shouldn't differ by this much. Remember, we're comparing every team to themselves, and no matter how dominant the Clan were, they were generally the same players at home as they were elsewhere. (Kate Hole has assured me she was the same height in every province.)
You may be wondering why I haven't looked at home-team stats vs. road-team stats in the same game--say, why I compared the Carleton men's home results to their away results, rather than comparing Carleton's home results to their visiting opponents' results in those games.
I didn't do it that way because different offences move the ball differently and may not actually have the same true assist rates; we'd be assuming things are the same when they might not be. Also, different teams play different opponents at home, meaning we'd be introducing another variable we couldn't really account for. There's far less variation in looking at the same team home and away, meaning any strange results we find are less likely to be caused by styles in play or opponent.
But to satisfy those who are curious, let's do it anyway. Here's what you get if you add everything up by venue:
Again, I don't endorse looking at it this way, at least not in isolation. The problem with looking at this chart and saying, "Aha, I knew it, those Waterloo scorekeepers are biased!" is that Waterloo, or anyone, could move left or right on that graph based solely on the ability of their opponents to receive assists, not based on their home scorekeepers.
You can compare travel partners, though, because their opponents are the same. UW and WLU are about equal on handing out assists to the visitors (they're right above/below each other), as are Western and Windsor, and Toronto and Ryerson. But Carleton's much lower than Ottawa (the Gee-Gees are hidden with Calgary, to the right of WLU on the chart). So there might be something there for any of you Ottawans to decipher.
But for the most part, in this analysis we're keeping the scorekeepers constant and changing the teams. Earlier, we kept the teams constant and changed the scorekeepers. I think the first method was a better way to look at this problem.
Again, why does this all matter?
Well, aside from keeping Kate from officially breaking her PB of seven blocks on the road one time ("when I looked in the stats sheet it said one block...it pissed me off"), and questioning the integrity of all past MUBL champions, it makes you think twice about leaderboards and fair treatment of all players.
Western's Ryan Barbeau was tied for second in the country in assists last year at 120, 17 behind Josh Gibson-Bascombe. Barbeau received 80 of those at home, in 326 minutes (9.8 per 40). He received the other 40 on the road, in 427 minutes (3.7 per 40). Quite a drop-off.
JGB, on the other hand, had 6.4 at home, and 7.5 on the road. If Gibson-Bascombe had Barbeau's road assist rate, he would have nearly 37 fewer assists over the entire season, and therefore finish behind him (nearly out of the top 10, actually). Or, if Barbeau's home assist rate were any lower (it is kind of high), he would have fewer assists than Christian Upshaw...see how it goes?
Unless Barbeau's passing skills deteriorate once he leaves London (it's a nice city, but not that nice), there's something weird going on. I don't know what it is, but the only thing universal about this assist mis-allocation? It's not consistent at all.