Monday, December 06, 2010

Calculated Reactions is a new, hopefully-recurring feature that takes an hopefully-interesting look at CIS statistics, with the goal of taking the numbers and giving them greater meaning.

This time out, I am going to illustrate the strange inconsistencies in the scorekeeping in CIS basketball. We've all made jokes about it ("Completely Inaccurate Stats"), and I'm by no means the first to notice the curiosities. But I wanted to sit down and go through the issue completely, hopefully going past the jokes to a more removed and calculated reaction (see what I did there?).

But wait a second Rob, why does it matter if the stats are wonky, anyway?

For one, you'll see below that this inconsistency can affect the statistical records of the game. Not enough to change the outcome of any game, but certainly enough to cast doubts on many team and individual statistics, and the methods by which those statistics are kept. I realize that only about seven people in the world care about the number of assists in a CIS game, and five of them are me, but there's no reason for these issues to be present.

I'll also admit that, selfishly, I want my own rankings to be better, which would require consistency in the stats across all conferences and schools.

The idea for this started when I recently looked into the difference between home and away stats, just as a curiosity. You can access home-away splits for each team on the CIS site--here is UBC's men's team from last year--and a quick glance at some teams can reveal some odd results, but I was more interested, at first, in overall trends, so I went down to the boxscore level and rebuilt the home-away stats from the bottom up.

Here's what I mean by overall trends. The categories in this table are sorted by the home-court advantage (HCA), meaning the percentage difference between a team's results at home and that same team's results on the road (so the average CIS men's team shoots 4.4% better, or 1.8 percentage points better, at home). Statistics are either expressed per game (200 player-minutes), or as percentages.

Men's basketball, home and away splits, 2009-10
CategoryAwayHomeHCA
BLK2.23.039.9%
AST12.515.524.0%
STL7.68.27.5%
DRB22.924.25.9%
ORB10.511.15.5%
Pts73.576.94.7%
FG%41.042.84.4%
3PT%33.935.03.2%
FT%68.968.1-1.2%
PF19.519.0-2.3%
TO15.514.7-5.4%

Before I get into the big numbers (looking at you, assists), notice that the home team does, in fact, enjoy an advantage in nearly every category. Most of these are beyond the interpretation of an official or the scorers' table--either the ball goes in the basket or it doesn't--and are exactly what you'd expect: the home-court advantage, whether due to travel, sleeping in your own bed, familiarity with the arena, or whatever, does exist. It's important to know that there's a true HCA (however small) so that we can greater appreciate how bizarre the extreme results are for, say, blocks and assists.

Assists seem particularly suspicious, but you probably get more opportunities to record assists at home (if home teams take more shots, for example). So let's add a row to that chart and see if that advantage goes away:

CategoryAwayHomeHCA
AST/FGM0.480.5617.6%

It doesn't.

So the average team, when at home, is awarded assists on 56% of their made baskets, about 8 percentage points higher than when they're on the road. That's not that bad, though, and mis-allocation of assists is of course a basketball universal. So if it's just part of the game everywhere, then it all balances out and there's no problem.

If there's no problem here, and the schools were generally consistent--meaning, the home team generally has an 8-point bonus over their own results on the road--then the 43 home assist bonuses (one for each men's team) would be centred around 8, with simple random variation causing some to be higher or lower, and a few to be much higher or lower. But only a few.

Rather than overload you with another table, let me overload you with letters and colours and lines instead:


The two axes cross at 48% and 56%--the CIS average for home and away assist rates--but what we're (or I'm) really interested in here is teams above the red line, meaning they receive many more assists at home than on the road, or below the blue line, meaning they receive many more on the road than at home.

If the results were consistent, most everyone would be between the red and blue lines. But instead we have 9 schools--CBU, Memorial, Lakehead, Western, Toronto, Manitoba, Laurentian, Brandon, and Concordia--above the red line, meaning their home bonuses are significantly higher, and another seven below the blue line. So that's 16 out of 43 with extreme results, when statistically we'd expect two or three.

You remember what the bell curve looks like? Well, all of these 16 teams are in the tiny parts at the far right or far left of that curve. In order to even see the curve, MUN and CBU would need to get in a plane and fly for a few hours. And stop to refuel.

**

But maybe there's something about the Capers and Sea-Hawks men's teams (and the Mustangs and Thunderwolves...) that make them legitimately more likely to receive more assists at home. In other words, it has nothing to do with the scorekeeping (at whatever school or schools), but the players on the floor. I don't know why this would be true, but let's assume it is. If so, then the women's team assist bonus shouldn't be the same, because they're different people playing a different game with a different strategy.

You can see where this is going.


Were the men above the red line? More often than not, the women were too. There's a very strong correlation between the home assist bonus experienced by the men's team and the same bonus for the women's team. For the nine men's teams with the highest home bonuses, seven of the corresponding women's teams saw similar extreme results. The same trend holds for the teams below the blue line as well.

Assists should be recorded in the same way by all CIS scorekeepers, and I am pretty sure that the above analysis shows they are not.

I haven't been to games at enough schools to know whether everyone's following the FIBA interpretations, and even if I had been, I honestly don't know the rules enough to tell you whether School X is over- or under-rewarding assists. So I'm not accusing anyone of pumping up the assist totals at home, or of ignoring visiting players' accomplishments while keeping score.

All I'm pointing out here is the inconsistency.

**

We haven't talked much about blocks, but those numbers were also pretty high; a 40% increase in blocks at home is worth investigating further. Just to confirm that the visiting team isn't giving the home team more opportunities for blocks: home teams received blocks on 8.2% of missed shots by their opponents. Visiting teams? Just 5.8%. So that 40% difference, or 2.4 percentage point difference, remains.

That difference of 2.4 wasn't consistent, either. Some schools saw a much higher bonus for the men and women (where it was actually 1.5 points on average). Most notably, York's home block bonus was 7.1 points for the men and 6.2 points for the women. Going back to the bell curve, we'd expect there to be two, maybe three out of 86 teams (men's and women's) whose home bonus was more than two standard deviations above the mean (and, of course, two or three more who are 2 SD below). Not a lot of room under that curve for more teams than that.

Of course you have guessed the punchline: there were 10. In order of how ridiculous their home block bonus (or "road block demerit") was: Winnipeg men, SFU women, UBC women, SMU men, Ottawa men, York women, Laval women, York men, Winnipeg women, UQAM women.

These are mostly different teams than we saw on the list of extreme assist bonuses, which shows you how statistical oddities aren't just limited to a handful of schools.

The results between home and road blocks shouldn't differ by this much. Remember, we're comparing every team to themselves, and no matter how dominant the Clan were, they were generally the same players at home as they were elsewhere. (Kate Hole has assured me she was the same height in every province.)

**

You may be wondering why I haven't looked at home-team stats vs. road-team stats in the same game--say, why I compared the Carleton men's home results to their away results, rather than comparing Carleton's home results to their visiting opponents' results in those games.

I didn't do it that way because different offences move the ball differently and may not actually have the same true assist rates; we'd be assuming things are the same when they might not be. Also, different teams play different opponents at home, meaning we'd be introducing another variable we couldn't really account for. There's far less variation in looking at the same team home and away, meaning any strange results we find are less likely to be caused by styles in play or opponent.

But to satisfy those who are curious, let's do it anyway. Here's what you get if you add everything up by venue:


Again, I don't endorse looking at it this way, at least not in isolation. The problem with looking at this chart and saying, "Aha, I knew it, those Waterloo scorekeepers are biased!" is that Waterloo, or anyone, could move left or right on that graph based solely on the ability of their opponents to receive assists, not based on their home scorekeepers.

You can compare travel partners, though, because their opponents are the same. UW and WLU are about equal on handing out assists to the visitors (they're right above/below each other), as are Western and Windsor, and Toronto and Ryerson. But Carleton's much lower than Ottawa (the Gee-Gees are hidden with Calgary, to the right of WLU on the chart). So there might be something there for any of you Ottawans to decipher.

But for the most part, in this analysis we're keeping the scorekeepers constant and changing the teams. Earlier, we kept the teams constant and changed the scorekeepers. I think the first method was a better way to look at this problem.

***

Again, why does this all matter?

Well, aside from keeping Kate from officially breaking her PB of seven blocks on the road one time ("when I looked in the stats sheet it said one block...it pissed me off"), and questioning the integrity of all past MUBL champions, it makes you think twice about leaderboards and fair treatment of all players.

Western's Ryan Barbeau was tied for second in the country in assists last year at 120, 17 behind Josh Gibson-Bascombe. Barbeau received 80 of those at home, in 326 minutes (9.8 per 40). He received the other 40 on the road, in 427 minutes (3.7 per 40). Quite a drop-off.

JGB, on the other hand, had 6.4 at home, and 7.5 on the road. If Gibson-Bascombe had Barbeau's road assist rate, he would have nearly 37 fewer assists over the entire season, and therefore finish behind him (nearly out of the top 10, actually). Or, if Barbeau's home assist rate were any lower (it is kind of high), he would have fewer assists than Christian Upshaw...see how it goes?

Unless Barbeau's passing skills deteriorate once he leaves London (it's a nice city, but not that nice), there's something weird going on. I don't know what it is, but the only thing universal about this assist mis-allocation? It's not consistent at all.

11 comments:

  1. Fascinating study Rob. I wonder if you will get any reaction from the flagrantly biased home courts? Probably not.

    Of course in hockey there are fewer goals and assists, and in the AUS at least each coach gets a final say after the game to adjust the official score sheet to make sure goals and assists are doled out properly. I'm guessing this would be unworkable in basketball.

    ReplyDelete
  2. However, I believe there one corollary to this in hockey - shot counts. In some arenas *cough*Wolfville*cough* the shot counters are notoriously stingy, but I wouldn't necessarily label them as biased (although some coaches might ...).

    ReplyDelete
  3. interesting report...any explanation as to why Laurentian's men are above the read line and the ladies are below the blue line?

    ReplyDelete
  4. @Dave: I was careful not to call anyone biased, flagrantly or otherwise, not that I would know who would qualify as such anyway.

    @prizby: That's the Laval women (LAV) below the blue line. Laurentian (LAU) are mixed in with Bishop's and York above the red. My fault for making the chart unreadable when teams are close together (though it's only the outliers that are important here, really).

    ReplyDelete
  5. This is great. I think that this sort of study is useful for scorekeepers everywhere - it is easy to learn what you may or may not be doing right from such a thing. This isn't to take shots at anyone's professionalism but it's a useful reminder that even more than making friends with the home team, it's important to be fair...

    ReplyDelete
  6. I know you're being careful Rob. I'm not.

    ReplyDelete
  7. The integrity of last year's MUBL champion should be questioned at all times. Good stuff.

    ReplyDelete
  8. These stats are just a capsule of what I suspect is happening across most CIS sports. Unfortunately, the CIS does not have the power to effect much change as that typically falls upon each conference to manage. However, the conference's are short-staffed and usually under-funded. As a result, they tend to hire inexperienced people who quickly become overwhelmed with other tasks, leaving stats to the back-burner. The problem is further compounded at the institutional level of each CIS member school, where they typically hire students who are not trained and/or supervised properly in stats gathering. Finally, there is no consistency from team to team and conference to conference because there are no universally accepted standards for statistical interpretations at the CIS level. In football, for example, a QB sack in one conference could be recorded as a QB run for negative yards in another conference. A backwards pass-and-run play behind the line of scrimmage is judged as a run at one school and as a completed pass at another school. A blocked punt can be judged as a fumble in one school and a punt for 0 yards at another school. The CIS needs to produce a universal "stats interpretation guide" for each sport (like the NCAA does) and all conferences should be required to adopt the guide. This would be a great starting point. Then we would have to work on enforcing annual statistical training workshops for each conference, something which has been done to a minor extent in the past.

    Earl Zukerman, Communications Officer, McGill Athletics & Recreation

    ReplyDelete
  9. Well said Earl. Care to lead the charge ...

    :-)

    ReplyDelete
  10. Trust me... I've been trying to champion this cause for years but there has been so much turnover in sports info officers at both the institutional and conference levels, that the newer people don't really grasp the importance or don't care to create more work. Nonetheless, I am working on this for upcoming conference and CIS meetings. I thought that this idea would be universally adopted but this has not been the case yet. Each conference seems more interested in what is good for them based on their historical records, rather than what is good for the collective on a national level. More articles that discuss or highlight the statistical discrepancies from venue to venue, such as one above, certainly will help the cause to produce and adopt a unified "statistical interpretation manual" for each major sport.

    Earl Zukerman

    ReplyDelete
  11. Very interesting article and not surprised that MUN's scorekeepers are among the most generous. I have been noticing in the box scores for most MUN games that there is an inordinately high number of assists given the number of field goals scored. Very often, a team with 30 field goals will have 25 or 26 assists. While this might be a sign of great ball movement, having watched MUN in recent years (as hard as it is to do that these days), I can safely say this isn't the case.

    However, I have also noticed that the number of assists tends to be high for both the home and visiting teams although the home team gets a slightly bigger bump. I'm not sure that this is necessarily due to a home court bias or advantage or if it's due to an overly liberal interpretation of what constitutes an assist in basketball. It seems to me that the scorekeepers at MUN tend to hand out basketball assists in the same manner as hockey... if a basket is scored, regardless of how it happened, the last teammate to have touched the ball prior to the scorer, gets credited with an assist.

    ReplyDelete