Longtime readers may recall an analysis from about three years ago that looked at "the strange inconsistencies in the scorekeeping in CIS basketball." Mostly, it was the allocation of assists in different venues that raised a few eyebrows.
I didn't speculate on the reasons for those discrepancies then, because I'm not an expert on the FIBA rulebook and also because it was based on only one year of play, the 2009-10 season. However, over the last little while I've expanded and improved that study to include six years of women's play and five years of men's play (2007-08 to 2012-13 for WBB, and 2008-09 to 2012-13 for MBB). This is the new version.
The basic methodology here is lifted from baseball, specifically park adjustments. In baseball, the question is how much does the venue (dimensions, climate, altitude) affect scoring? In CIS basketball, the games are obviously weather-controlled, and no gym can really affect the scoring rate overall or the outcome of the game, but the subjective statistics are affected — undeniably — by the officials and statkeepers at each school. Assists are the most obvious but it happens in other categories as well.
There is a complicated way to calculate these factors for baseball parks but the key ingredient for any particular team's park factor is runs scored and allowed at home divided by runs scored and allowed on the road. For our purposes, it's not runs or points per game, but assists per field goal, or blocks per game, and so on. This allows us to control for the team's tendencies: different teams legitimately have more or fewer assists than others, or allow more or fewer, and that isn't part of the subjectivity or bias or mistakes introduced by the referees or anyone else.
What we get at the end is an adjustment that we can use to "correct" assist totals (again, or blocks or anything else that is systematically over- or under-counted in each venue) for each player on that team. Outside of a few cases it doesn't end up making a big difference in the overall numbers but it is still worthwhile to be aware of these things, especially with some of the examples I'm about to introduce.
So, you remember Memorial. Let's use them as an example. When we add it all up, we find that in their home games (women's only, for simplicity), both teams combined for 2664 assists on 3132 field goals, or a rate of 85%. In Memorial's road games, both teams combined for 1577 on 2860 — 54%. Not quite the same, is it? Dividing the home rate by the road rate and correcting for the other AUS gyms gives us 1.43, which means assists are given out 43% more often at MUN than they "should" be, when compared to the rest of the AUS statistics. Since half of a player's games are played at home, we take half of the difference between 1.43 and 1, so about 1.21, and that is what we'd use as a generic adjustment to a Memorial player's assist totals.
In the most recent year there wasn't a Memorial player near the top of the CIS leaderboards in assists, but in 2011-12 their assists leader was 12th (4th in AUS), and 14th on a per-game basis, with 81 in 20 games. According to what we've just done here, the corrected number of assists should be 81 / 1.21, or 67, which would move her from 12th to a tie for 26th. Those 14 assists don't change wins into losses or anything that severe, but they do change our perception of players, and it wouldn't surprise me if some coaches have decided to disregard assist totals completely given this and similar results.
What are those similar results, you ask? Here are the top 10 gyms for assist over-allocation (broken down into both the men's and women's games even if the scorekeepers are the same), among schools with more than one season of CIS play:
Most extreme assist over-allocation
43% Memorial WBB
39% Memorial MBB
21% Lakehead MBB
20% UBC MBB
17% Lakehead WBB
16% Winnipeg WBB
14% Manitoba WBB
14% UBC WBB
14% Western MBB
13% Brock WBB
Note that MRU also had very high numbers (21% women, 20% men) but it's only one year for them.
Both Memorial and Lakehead being on this list isn't a shock. We have noticed Western's proclivity before but they are not in the same league as MUN.
Because those differences at Memorial are so stark, while I was in the process of researching this, I sent a preliminary version of the above analysis to their sports information department and asked if they could explain, and received a response via email from the athletic director, Michelle Healey. She actually said she hadn't heard about this before (not that she necessarily would have, with much more on her plate than basketball scorekeeping): "We have never had this point raised by any team – home or visiting, so it’s the first time it has come to my attention."
Healey also said that "obviously there must be an incorrect interpretation of an assist being made by our minor officials group that oversee our stats" and that the very high assist rate "definitely indicates that the assist stat is being incorrectly applied."
Actually chasing down the spotter for their basketball games and asking "what's the deal?" would be an unreasonable amount of effort, either for me to do or to expect them to do, not to mention it's the middle of the summer. And while I cannot comment on the reliability or qualifications of their statkeepers, this is an example of what could happen when one has to rely on improperly-trained staff. (Here is another example, which I still cannot believe happened.)
While it's not surprising to see MUN at the top of the list, it is a little unexpected if indeed nobody has ever brought this up to anyone there. This has been a consistent issue for as long as I've been following CIS basketball, and almost certainly longer than that. Coaches devour all statistics they can find on their players and would definitely notice these differences.
It's possible that I care about this more than anyone else (in which case this may represent my last piece on the matter), or that coaches simply don't see the benefit in complaining — or, of course, some other possibility.
What about the flip side? If some schools over-allocate assists, others should under-allocate them, right?
Most extreme assist under-allocation
-20% UPEI MBB
-19% UPEI WBB
-18% McGill WBB
-18% Ryerson MBB
-16% Saskatchewan WBB
-16% Calgary MBB
-15% Ryerson WBB
-13% Guelph WBB
-13% Waterloo WBB
-12% Carleton MBB
(Again a recent Canada West entrant makes an appearance on the "not enough games to qualify for this list" list; this time, it's UNBC who are at -27% for men's games and -24% for women's games.)
I'll leave it up to people who have seen more games than I have at UPEI (0) or Ryerson (2) to comment on what may be going on there. But you can see it's not on the same scale as Memorial, either.
It's not just assists that introduce subjectivity. Blocks and pretty much any foul could, potentially, be called or awarded by one crew but not by another. Here are the most extreme stats from all other categories:
Most extreme over- and under-allocation, everything but assists
60% Manitoba WBB - blocks
54% UBC WBB - blocks
41% SMU WBB - blocks
40% UBC Okanagan MBB - FTA per FGA
32% Winnipeg MBB - blocks
-56% Saskatchewan MBB - blocks
-54% Saskatchewan WBB - blocks
-42% UBC Okanagan MBB - blocks
-39% UFV WBB - blocks
-31% UBC Okanagan WBB - blocks
Blocks are fairly uncommon, so it's not entirely accurate to say these numbers are more notable than a 43% overcounting of assists. Still: 60%, Manitoba?
The only other stat to show up here is related to shooting fouls in Kelowna (albeit based on just two years of data), and the related stat of fouls per game is right up there too (35%) which may indicate some whistle-happy tendencies among certain groups of officials there. If we extended that list past five schools, we'd see the opposite results with respect to fouls in games played at Manitoba and at Winnipeg: around 15% to 20% fewer foul calls compared to the Canada West average. Hard to find a ref in Winnipeg when you need one, I guess.
Other findings: the biggest variation in these variations (if you follow me...) from venue to venue is actually in block rate, followed by assist rate, then FT rate (FTA/FGA, not FT%) and steals, all of which require some sort of judgement call from officials, at the table or elsewhere. Some schools are curiously high or low on offensive rebounds and turnovers as well (Saskatchewan, again, is low — is there an undercounting epidemic in Saskatoon?).
All of the above can be summed up like this: it only makes sense that different humans see things differently, but whether that should happen when recording the results of the game is up for debate.