Well, that's not entirely true. We have team defensive stats; we know how many goals each team allowed. And since Point Allocation uses a top-down approach, beginning with team results and then allocating these results to the players on that team, that's actually quite a good start. But there's no way we'll get anywhere without some judicious fudging. Remember that the system is not designed to identify who the best players are in any era; it largely assumes that those players considered to be the best were the best, and bends to make them so, to a reasonable degree. The system is meant to put players from different eras on the same relative footing, and to evaluate the players from all eras that are not among the very best. In a sense, the very best players in a particular league-season are used to calibrate the Point Allocation system for that particular league-season, via the league quality adjustment. The ratings of other players fall out of that.
Before we allocate defensive points to players, we first divide them between the team's goaltenders and its skaters. On a league-wide basis, this is done in order to make netminders (who receive no offensive points) as valuable, on average, as any other position. And then we have our first fudge factor. The reputation of a team's goaltender, versus that of its skaters, is considered when dividing defensive points among them. We can't assume a constant ratio of goalie to skaters points for every team in a league; that's just not realistic. And since we don't have anything like save percentage to evaluate goaltenders on something other than team goals-against average, we need this adjustment. Defensive Point Allocation is just full of fudgy goodness, and all the better for it.
From there, it's a simple matter to allocate points to goaltenders, if a team had more than one in a particular season. Each goaltender's Marginal Goals saved is calculated based on his GAA, and the points are allocated on that basis.
Now for the skaters, we first use the basis of the very first edition of Point Allocation from 2002 or so. That is, if we know how much a player played, and we know approximately how good he was offensively, we should be able to derive that player's approximate defensive ability, since offence plus defence should equal playing time. So when you have a player who produces a relatively low amount of offence (compared to his position, and his teammates), the system assumes he's probably a relatively good defensive player. It doesn't assume a perfect relationship between the two, of course, and sometimes this will result in some inaccurate results, but this is largely overcome by examining players by career (or at least groups of seasons) rather than by individual seasons.
The same logic is used for the amount of defence assigned to each position as a baseline. The standard defensive contribution of each position is determined based on the league-wide offensive contribution of each position, such that each position is approximately equal in overall value. On top of this, and the adjustment mentioned above, we have the other big 'ole fudge factor. Players with excellent defensive reputations have their defensive value adjusted upwards, and those with poor reputations are adjusted downwards. It's as much art as science, of course, but just doing straight computations on these numbers will produce less accurate results than if we apply what we know about these players, that is not necessarily reflected in their stats.
Oh, one final thing. Players lose points based on the penalties they take, since penalties cost your team goals. That's about it for the system. There are probably some details I've left out, which I'll mention when I think of them. Let's have a look at the complete results for the 1907 Montreal Wanderers, which we've been using for an example so far, normalized to an 80-game schedule, 20 minutes per game for forwards, 25 for defencemen and 60 for goaltenders. OP is Offensive Points, DP is Defensive Points, PP is Penalty Points, TPA is Total Points Allocated and TPAK is the same per 1,000 minutes.
1906/07 Montreal Wanderers
Player | Pos | GP | MIN | OP | DP | PP | TPA | TPAK |
---|---|---|---|---|---|---|---|---|
Russell, Ernie | 5 | 72 | 1440 | 10.0 | -0.3 | -0.5 | 9.2 | 6.39 |
Stuart, Hod | 2 | 64 | 1600 | 0.7 | 9.0 | -0.5 | 9.2 | 5.75 |
Kennedy, Rod | 1 | 24 | 600 | -0.1 | 3.0 | -0.1 | 2.8 | 4.67 |
Marshall, Jack | 7 | 24 | 480 | 1.1 | 1.1 | 0.0 | 2.2 | 4.58 |
Hern, Riley | G | 80 | 4800 | 0.0 | 21.1 | 0.0 | 21.1 | 4.40 |
Patrick, Lester | 14 | 72 | 1720 | 2.7 | 6.2 | -0.2 | 7.1 | 4.13 |
Johnson, Moose | 6 | 80 | 1600 | 4.0 | 3.2 | -0.8 | 6.4 | 4.00 |
Blachford, Cecil | 7 | 56 | 1120 | 3.3 | 1.0 | -0.1 | 4.2 | 3.75 |
Glass, Pud | 4 | 80 | 1600 | 1.7 | 3.9 | -0.4 | 5.2 | 3.25 |
You can see the Wanderers had two absolute superstars in Russell and Stuart, a bunch more playing at an elite level, and two others (Blachford and Glass) producing solid if unspectacular results. Little wonder the team was undefeated in 10 games.
I've always loved the concept behind defensive point allocations. Questions:
ReplyDelete1. How about a minimum level of ice-time to get defensive point allocations? If a player is getting depth-line minutes only and producing no marginal offense, isn't it reasonable to assume he's producing no marginal defense either? In fact, if he's playing less than depth-line minutes (goon) he should get negative defense.
2. If a player takes a lot of penalties and yet gets a lot more ice-time than his offense justifies, doesn't that mean his defense is even better than an equivalent player who doesn't take penalties? You know, that a coach would use him even though he costs them so many power play goals.
3. I'm intrigued at the idea of re-adjusting a player's defensive point allocations based on his career (e.g. Aaron Johnson's time in Chicago). How is that calculated, exactly - obviously some sort of recursive procedure?
4. I'm still intrigued by players like Pavel Datsyuk and Martin St. Louis who could earn their ample ice-time both offensive and defensively. How do you avoid short-changing great two-way players, whose ice-time appears to be explained by offensive contributions?
Rob:
ReplyDelete1. Players earn defensive points based (partly) on their ice time per game, so a goon will get very little. Not necessarily zero, as you suggest, but low enough to be effectively zero in most cases.
2. That's possible. The penalty ajustment I'm using is quite conservative, so I don't think it would make a significant difference except in the most extreme cases.
3. I assume you mean the fudge factor I'm talking about here - I only use that for historical PA calculations, which uses a different system than I use for modern stats. Since the great expansion we have ice time information that we don't have before, and use that to get a better idea of that player's defensive abilities. The fudge is a purely subjective one, based on the player's defensive reputation, and is used to avoid shortchanging players due to there being no such thing as individual defensive stats from their era.
4. The (modern) Point Allocation system does not make a 1:1 adjustment to effective ice time based on offence, so great offensive players are not assumed to be terrible defenders to begin with. It also considers such things as PK time and total ice time in the calculations.
Take a look at Aaron Johnson as an example.
ReplyDeleteStudying his career it's obvious that Aaron Johnson is an adequate 3rd-pairing D-man.
However, in 2008-09 he played on the ridiculously powerful Chicago Blackhawks and was +19 in 38 games - best on the team on a per-game basis.
As a consequence every measurement out there rated him very highly defensively.
There are two ways to avoid falling into the "Aaron Johnson is awesome" trap.
1. The idea of looking at a player's career before assigning defensive points, which is why I'm intrigued by your idea. Even a quick glance tells you that Aaron Johnson wasn't responsible.
2. Looking at his ice-time. He played just 14:09 per game. He was obviously a replacement-level 3rd-liner playing minimal minutes while the real defenders rested.
My proposal: figure out the replacement-level amount of ice-time at each position, and award defensive shares only for all ice-time in excess of that replacement-level that can't be explained by his offensive contributions, and penalize him for every minute he comes up short.
What do you think?
I haven't done PA calcs for 2008/09 yet, but the modern version of the system should avoid the AJIA trap by looking at ice time, as you suggest, and also by regressing a player's individual plus-minus significantly to the mean. I don't have a lot of faith in +/- by itself, especially for a single player-season (or half-season in this case).
ReplyDeleteGiven his lack of PP time, Johnson's scoring rate seems to be pretty decent. The PA system takes that fact, looks at his low ice time, and figures he's not good defensively, which is what you're after, I think.
But why are you polluting my precious history blog with this dirty modern stuff? Around here, Aaron Johnson hasn't even happened yet!
I should say, though, that in some sense defensive replacement level is built into Point Allocation, but at the team level. The Marginal Goal basis of the system means that only above-replacement-level value is allocated to the players.
ReplyDeleteI think your suggestion has merit, though. I think PA takes care of some of this stuff at the team level.
Ok the 1925-26 Ottawa Senators. Hec Kilrea, Ed Gorman, Frank Finnigan, John Duggan and Alex Smith might all look like the best defensive players in history for no other reason than they played non-offensive roles on the most outstanding defensive team in history.
ReplyDeleteIf you had some outstanding defensive players, and a handful of replacement-level players, how do you avoid the replacement-level guys getting credit for the contributions of the outstanding (other than the two methods already discussed).
To avoid my blathering on in the comments section here, can you just show me by touching on the 1925-26 Senators (or a suitable alternative) in an upcoming piece?
P.S. If you want to see what I mean, check out the defensive ratings of the aforementioned Senators either at WhatIfSports.com, or Defensive Point Shares at Hockey-Reference.com.
ReplyDeletehttp://www.hockey-reference.com/teams/OTS/1926.html
Alright, I can do a quick post using the '26 Senators as an example. Any system like this will only be able to do so much - players on good defensive teams will always be subject to overration of defensive values, which is why looking at careers, or at least blocks of years, is important. But it looks like I do a better job than Hockey Reference. It's all about the fudge.
ReplyDelete