Season CSSI Wrap-Up: Goaltender Stats

by J.J. from Kansas on Jul 9, 2011 8:12 AM CDT in CSSI

Give me back the stick, Chris; I'm the starter now!

Finishing off the overview of the last set of adjustments for the Common Sense Scoring Index, we now turn our attention to the goalie stats that came out of this season. These stats had very little to do with the players' points adjustments, but had quite a few impacts on plus/minus hits. In general, when a goaltender gave up a bad goal or a half-bad goal, the rule of thumb was to clear all (or half) of the player minuses. After all, while there should be some attempt by players to prevent a shot, there should be no reason to punish him for a shot taken which should not be expected to beat the goaltender. Plenty of times, players were allowed to keep minuses on bad goals, but those were all tied to adjustable events.

For the goalie analysis, I did not go back through and create additional categories. A bad goal is a bad goal is a bad goal and there are simply too many variables that make up what constituted a "big save" to further reduce statistically. I did add assists to goaltenders as I saw fit, but those happened so rarely that they didn't warrant discussion in the points adjustment post. Four assists over 60 games for Jimmy Howard is always nice to see, but he's no Turco or Brodeur when it comes to helping his offense start the other way.

Player Name	GP	SV%	GAA	SOL	Big	Bad	+/-	Assists	Rating-Per-Game
Howard, Jimmy	59.87	.908	2.79	2	172	27	+145	4	+2.42
Osgood, Chris	10.33	.903	2.77	0	28	7.5	+20.5	1	+1.98
MacDonald, Joey	11.80	.917	2.58	1	39	4.5	+34.5	1	+2.92

One thing to mention before we get deeper into the analysis, the GP line is not official. I did some minor rounding when coming to the figures. You'll notice that Thomas McCollum's 14:37 of 0.625 save % hockey doesn't appear anywhere and that the GP chart does not match up with NHL.com's stats. I will discuss this and more about the chart below the jump.

So, as far as the GP disparity goes, it is very minor, but will be something I do differently next season. Technically, Jimmy Howard got credited for appearing in 63 games, but I'm more interested in his ice time. In total, Howard played 60.5 total increments of 60-minutes (3,615:15 minutes). It's probably more fair to go by that for games played, except that sometimes a game is 60 minutes and sometimes it is 65 and sometimes it is any number between those two. When trying to figure out a per-game rating, I let that time get lost in the shuffle. This does not do a great deal of moving the numbers around, but will be something that I fix for next season. The GP stat next year will match exactly with the official numbers for minutes played divided by 60. It will simply be easier to track that way.

As far as the individual ratings goes, the project evolved slightly as the season went along and we noticed that the plus/minus rating for a game had a very good correlation with the overall feel of a goalie's game. A +2 rating for a game was the baseline standard for acceptable goaltending. Numbers from there stayed in the zero-to-plus-4 range with 0 being absolutely terrible and 4 being great. You can see by the numbers that Howard's season was about halfway between being just acceptable and being pretty good. Meanwhile, In Osgood's few games, he played just under a level that I would consider acceptable goaltending. The context of MacDonald's rating being better than Howard's is that MacDonald was utilized in much different situations. I do not believe MacDonald's ratings would have held up so well under additional pressure. MacDonald came into six games which he did not start. It is generally impossible to quantify (but I might try to do so later anyway), but there is a generally accepted axiom that teams play better in front of relief goalies.

At the end of the season, all we can really compare these numbers to are each other, since there aren't adjusted ratings for other goalies. As such, there's good reason to compare the seasons of Joey MacDonald and Jimmy Howard. I don't think there's any argument that Osgood outplayed either of them. I'm going to break this down into two methods, the short and the long. If you don't want a huge explanation, skip ahead to the short section.

Jimmy Howard vs. Joey Macdonald - The Long Version

If the axiom that relief duties involve drastically different context holds up, it can also help to explain how MacDonald has more big saves and fewer bad goals on a per-game basis:

Player	Big Saves/Game	Bad Goals/Game
Howard, Jimmy	2.87	0.45
Osgood, Chris	2.71	0.73
MacDonald, Joey	3.31	0.38

As it stands,MacDonald gave up 31 goals all season. 2 of those goals came in 76 shots against during relief appearances. The remaining 29 came on the 296 shots he faced while starting. Despite the 37-save shutout against the Blue Jackets he had on March 17th, that puts his saves percentage during starts at a pedestrian .902 while his in-relief saves percentage was an incredible .974. That difference alone is a good indication that a team playing in front of a goalie who has had to come on in relief does a much better job of lowering total scoring chances (or total opportunities for bad goals).

On the other side of the coin though, playing from a trailing position will force a team to take more chances and give a goalie more opportunity for big saves without the risk of him having to worry as much about bad goals. The numbers here bear that out as well, as MacDonald gave up every one of his 4.5 bad goals against in starts, but had 34.6% (13.5 total) of his big saves come on in relief when he saw only 76 of the 372 (20.4%) total shots he faced. The Wings took more offensive chances while playing from behind for MacDonald in relief and the opponents got more odd-man rush attempts on the goaltender as a result.

So let's look at it as though MacDonald were two separate goalies. One goalie is the starter MacDonald and the other is the relief-goalie. To do so, let's normalize some numbers for comparison. If we're going to call 60 minutes a game, let's re-organize the goalie's numbers to show their games played stat as a function of their total ice time divided by 60 and adjust their per-game plus/minus accordingly. Combining this adjustment while splitting Joey Mac's ice time between starts and relief appearances looks like this.:

Player	GP	SV%	GAA	Big	Bad	+/-	Rating-Per-Game
Howard, Jimmy	60.50	.908	2.79	172	27	+145	+2.40
Macdonald, Joey (Relief)	3.19	.974	0.63	13.5	0	+13.5	+4.23
Macdonald, Joey (Starter)	8.82	.902	3.29	25.5	4.5	+21	+2.38

Now we go back to the per-game basis for big saves and bad goals:

Player	Big Saves/Game	Bad Goals/Game
Howard, Jimmy	2.84	0.45
Macdonald, Joey (Relief)	4.23	0.00
MacDonald, Joey (Starter)	2.89	0.51

If anything, the rating-per-game of Joey MacDonald as a starter might have meant I was going a bit easier on him or a bit harder on Howard. Ultimately, that's a small sample size anyway. If we take Starter Joey Mac's numbers and extrapolate out to Howard's playing time, then MacDonald makes three more big saves all season at the cost of four more bad goals. Of course, while we're playing with goofy numbers, that GAA alone equals out to 30 more goals allowed over the course of the season

So what have we proven? Well, I guess it shows what we knew going into this: Jimmy Howard is a better goaltender than Joey MacDonald, but he has room for improvement. I'd like to look at some other backups' stats though to see if MacDonald's numbers in relief are still so amazing when compared to the concept that all goalies who come into games in relief appearances have such good numbers.

_____________________________________________________________

Jimmy Howard vs. Joey MacDonald - The Short Version

For this statistical comparison, I will use significantly fewer numbers to show that Howard is a better goaltender than MacDonald (for all those two or three doubters... maybe)

Player	Games In Which the Coach Felt Comfortable Playing Him	NHL Teams Who Consider Him a Starter	NHL Contracts Held
Howard, Jimmy	63	1	1
Osgood, Chris	11	0	0
MacDonald, Joey	15	0	0

Ta-Daaaa! If you skipped all the junk above, just know that there's a reason Howard is the only goalie with a contract right now.

_________________________________________________________

Going Forward with Goaltender CSSI Analysis

I'm not going to lie, I'm not horribly happy with the way this season's goalie adjustment system worked. It's a nice concept that a goalie needs to have two or more big saves for every bad goal he gives up, but I never really felt comfortable tying that into his overall rating. Basically, I created a system where 2=0 and that seems unnecessarily convoluted.

I did like the concept of counting bad goals. Those helped keep track of skater faults when it came to goals against and I believe they form a good baseline for longitudinal looks at goalie performance. I'm not sure if I'm going to keep counting big saves, though. The problem with the big save is that it's arguable that every save a goaltender makes is a big one. There are spectacular saves that are made every season; most of those involve big, sweeping and impressive movements. The problem is that a goalie who is doing his job perfectly is not often caught in a position where he has to make a big obvious movement to stop a puck from going in. Generally, a goalie who is performing well is also performing somewhat quietly. That's not to say that every big, sweeping movement to make a save is an example of a goalie being in bad position and recovering, just that it's very difficult to break down between a man making an acrobatic save to stop a fantastically generated chance and a man making an acrobatic save to make up for a temporary jaunt outside of his own fundamentals (since a goalie who is consistently fundamentally flawed will not make it in the NHL anyway). The noise on this particular stat is very loud and I'm not sure I have the expertise to filter it.

Either way, I do not intend to factor the difference between those two counts for a ratings anymore. Instead, I'm proposing a rating system that goes from -4 to +4 and which starts at 0. Regardless of the count of big saves and bad goals, a keeper will be given an overall game score based on how well or poorly he played. A 0 rating will be the baseline average. A goaltender who was deemed a non-factor in the game one way or another would get a 0. +4 and -4 will be very rare counts. I can't imagine a goalie getting to a -4 without getting pulled (possibly even shot). Alternately, a +4 rating should result in the team in front of him being hunted for sport after forcing such a difficult task on a goalie. The ratings between will be used to measure performance that was either above average, excellent, below average, or terrible.

I'd really like some reader feedback on what kind of changes can be made to the goalie tracking system to make it better. Do you think I've got it fixed with this or would you like to see a different format used? I don't think last season's system was terrible, but it had extra parts I feel it didn't need. Howard's plus/minus did a decent job of passing the sight test of a season where he was good more than he was bad, but he left us wanting more.

Let me know your thoughts in the comments below. Thanks!

Tweet Comment 25 comments | 0 recs |

Do you like this story?

Comments

(since a goalie who is consistently fundamentally flawed will not make it in the NHL anyway)

Hey, Dan Cloutier fooled an awful lot of people for a pretty decent while, though.

I’m with you on your analysis of your own project, at the bottom of the post, and was thinking the same thing while reading up to that point – systems where negative = good, or 0 is not neutral, they often end up being a lot harder to work with and understand.

That said, for your -4/4 scoring system, I’d propose tightening it down to -2/2. As you mentioned, it would have to be almost impossible for a goalie to be -4, unless he was secretly playing for the other team. Similarly, the 4 number is impossible too – so since they’re not realistic, they shouldn’t be included. -3/3 should also be dropped, because its still to wide of a net. If 0 is a normal, decent, pretty unremarkable game, then you can figure 1 is a real solid game, and 2 is an excellent game. Same for going the other way into negatives. Going further than that, and you’re really splitting hairs; instead of good/great, you have to decide if this game was aboveaverage/good/great/fantastic, and you need to appropriately mirror this across the other side of 0. In fact, it seems you kind of slanted this way when you said you would judge performances as “above average, excellent, below average, or terrible” – 4 types of performances, 2 to either side of 0.

I think keeping it smaller and simpler will help make it more accurate, because you’ve got less leeway in your judgement to be a bit more forgiving or kind (even unconsciously so), which I think is something you have to be vigilant of. With the binary good/bad result of every play a goalie makes, and the fact that they make more saves (including big ones) than they give up (including bad ones), it’s easy to get trapped in “well, he gave up a bad goal, but he made two big saves in overtime” and end up shading more towards the positive. Especially when you consider that a goalie who is consistently bad will not be starting often, you’re already going to end up with positive skewed numbers (or at least, for the Wings sake, I hope you do). You don’t want to exacerbate that effect, subliminally or not.

http://sacrificethebody.blogspot.com/
Sacrifice the Body - Examining the NHL through statistical analysis, reasoned thought, and blind conjecture.

by IAmJoe on Jul 9, 2011 9:36 AM CDT reply actions

Thanks

I bounced around between +/-4 and +/-3, and I definitely see your point about tightening the ratings system to keep important numbers from getting lost in the shuffle.

What I worry about with a 4-rating system is that it forces me to consider every excellent game as the most excellent a goalie can play. If the Wings backup starts a game in March where he puts up 37 saves in a shutout, that’s going to get a +2. But, is that +2 game really as impressive as the 51-save performance that Howard put up on the Kings two seasons ago?

What do you think about lowering the range from 4 down to 3 with the idea that 3s are going to be used INCREDIBLY judiciously? I can’t speak for Tim Thomas or Pekka Rinne, but among Wings goaltenders, there have probably been only three or four games in the past two seasons which would truly be worth of a +3 rating. This would allow for above-average, excellent, and stellar rankings.

The thing there is that I can’t think of ANY games over the last two seasons where a Red Wings goaltender would deserve a -3 rating though. Even Joey MacDonald giving up 7 goals in that 10-3 loss to St. Louis wouldn’t fit into the category, so I still have a number on a rating system that I can’t see ever using.

by J.J. from Kansas on Jul 9, 2011 9:54 AM CDT up reply actions

I can’t think of ANY NHL game I’ve seen where the goaltending performance could be described as “worse than terrible”. Hell, I play beer league, and I’ve only seen a handful of them there (one was my performance as well as my opponent’s, in a game in which I was forced to fill in as goalie, and almost blew an eight goal lead).

On the flip side though, you’ve also got to figure, you’ve almost never seen a 3 performance either, and frankly, the only 3 performances I can remember off the top of my head mostly are in the playoffs in some absolutely insane games/streaks. Tim Thomas this year I think had a couple, JS Giguere had a mess of them in 2003. Basically, 3’s are the kind of performance that earns guys Conn Smythes in losing efforts, and are amplified by the high stakes. With that in mind, I don’t know that you could even possible have a 3 in the regular season. Maybe 3’s are exclusive to playoffs, wherein you have to consider the impact of the situation (every game in the regular season, even in a “win and you’re in” situation, is ultimately 1 out of 82), the length of the game (playing a 3 OT, 120 minute game is bigger than playing in a 60 minute game).

Ultimately though, since it’s so infrequently used, and may potentially be impossible to achieve in the regular season, I don’t see the issue with dropping the +3 score. It’s something you were never going to use anyways, and keep in mind, just because a guy gets a shutout doesn’t necessarily mean he earned a +2 or even a +1. You’re grading based on the performance of the goalie, not the end result, as impacted by the team around him (i.e. stifling another team to only 14 shots on goal and getting a shutout maybe doesn’t even deserve a +1, if – is just doing your basic adequate job).

That’s an important thing to remember too, plenty of games are won by 0 or even -1 performances by goalies. If 0 is just “do your job and turn in an average performance”, that’s still making a couple big saves, which happen in every single game. If 0 is a decent average game by a goalie, you’ve still got plenty of wiggle room with both “good” and “great”.

http://sacrificethebody.blogspot.com/
Sacrifice the Body - Examining the NHL through statistical analysis, reasoned thought, and blind conjecture.

by IAmJoe on Jul 9, 2011 10:11 AM CDT up reply actions

How about Roy's last game in Montreal?

Would that qualify as a -4?

by SweetLouDoubleU on Jul 9, 2011 1:24 PM CDT up reply actions

it definitely brings a smile to my face.

by SlapshotGoal on Jul 9, 2011 1:28 PM CDT up reply actions

Also, its worth considering, Howard (and Wings goalies in general) should have less opportunities for positive games, given the fact that the Wings are, overall, a better team that most, generally carrying play and taking more shots. To me, the idea of these positive performances would be more in the context of a team getting outshot and/or giving up a lot of PP time, and yet they still pull off a 2-1 win. The Wings are usually the team outshooting (and sometimes losing that 2-1 game where they outshot the other team 17-35), so there’s really a lot less chance for Howard or his backups to be a bigger piece of the win – they’re frankly just not asked to do as much to win, as a lot of opposing goalies are.

While this isn’t a league-wide project, in my mind, it ought to work such that it could easily be scaled up, and still work and give results that make sense. If Howard finishes the season at +30, with 60 starts, that’s fine, but what does that say for someone like Tim Thomas or Pekka Rinne, guys whose teams really count more on having that spectacular goaltending. Or even a situation like Tomas Vokoun and Roberto Luongo had in Florida, being among the elite goaltenders, with the hardest jobs on one of the worst teams? It just doesn’t make logical sense if the result of this can turn out to be “JIMMY HOWARD IS AWESOME” because then by comparison, you have to be able to say that Pekka Rinne was better than awesome, and Tim Thomas was even better than better than awesome. Which kind of comes back to the previous point you’d had about 2 = 0, where your baseline is set artificially high, that it detracts from the common sense aspect of the project.

http://sacrificethebody.blogspot.com/
Sacrifice the Body - Examining the NHL through statistical analysis, reasoned thought, and blind conjecture.

by IAmJoe on Jul 9, 2011 9:54 AM CDT up reply actions

Actually, in writing this… I don’t know that you can even do much with this data anyways, even improving it going forward. Your backup goalies are likely going to play so few games that anything you get from them is filled with noise, you’re just playing small sample size theatre. Comparing anything from that will be essentially worthless, if you want to seriously analyze that info, because of the sample size. Therefore, the only useful comparison to Howard would be other starting goaltenders… and I assume you don’t want to watch every single NHL game next year. The only way this isn’t an issue is if something bad happens (Howard is hurt, or Howard sucks) and is forced to split games almost equally with his backup. But even then, in 40 game sets, noise is still significant.

I’m just brainstorming here, but what if you don’t grade Howard’s performance individually, but grade it against the goalie at the other end of the rink? You could track bad goals for the other goalie too (I would actually be curious how many bad goals the Wings score versus how many they give up, just because their outshooting numbers should lead to more goals, including bad goals in general), and maybe make it a zero-sum sort of thing between the two goalies, where one has to outperform the other in order to get the +1. Something like this:

Detroit beats Columbus, 1-2, being outshot 28-16. Neither gave up any bad goals, but Mason had to do more work, and his work was harder – maybe CLB gave up a lot of PP’s or just didn’t get good chances that night, while DET did. Mason gets the decision over Howard, despite actually losing the game.
Detroit beats San Jose 3-2, being narrowly outshot 26-24. The two teams played essentially equal quality hockey, and the goalies played essentially equal quality in goal – point being that neither was particularly outworked, neither “stole” the game, neither gave it up either. Split decision, no goalie gets the win.
Detroit beats Chicago, 1-0, outshooting CHI 31-28. Crawford was solid, while Howard had significantly more work on the penalty kill, and stopped a breakaway or a penalty shot or something. Howard gets the decision.

Now, while you wouldn’t get real full numbers for opposing goalies, you’d get some decent data on other Central division goalies. You’d also get data in terms of how much is Howard outplaying his opponents, versus just doing enough. This could be more useful info, given the team effect DET has, and the fact that DET requires less out of their goaltender than less talented teams do. This would be relevant because if we can see that Howard really isn’t doing much outside of “enough”, well, then we can judge if we really ought to pay him more money when his contract comes up, or go with another, cheaper option in the crease. Keep in mind, this team did pretty well with a post-lockout Ozzy, who was significantly below replacement level, so this is an important question to keep in mind going forward, when we supposedly have a “franchise goalie”, whose SV% this year is noticeably close to replacement level.

http://sacrificethebody.blogspot.com/
Sacrifice the Body - Examining the NHL through statistical analysis, reasoned thought, and blind conjecture.

by IAmJoe on Jul 9, 2011 10:33 AM CDT up reply actions

Now that's a hell of an idea.

I think counting bad goals for other goaltenders against the Red Wings is a definite must going forward. If I’m going to auto-clear minuses for players when their own guy gives up a goal he shouldn’t have, then I should likely auto-clear pluses of the same variety. Of course, I’ll have the ability to still give pluses and minuses back to players who earned them in spite of the goal that shouldn’t have been, but at least those pluses will be better-defined.

I also love the concept of making a -1, 0, +1 system based solely on whether I think the Wings goalie outplays his opponent. If context and sense are important to the rating system (which they are), then a goaltender’s ultimate job is to play better than the guy on the other end of the rink. All stats and metrics we have for goalies are designed to help us predict whether one goaltender can be expected to do that. We’ll still have those metrics, this system would allow us a stat which tracks how well they might correlate. Ultimately, I don’t care if a goalie has 10 shutouts against the worst teams in the league if he loses every game against a contender 4-3. Comparing him directly to the challenge that faces him should bring an interesting and much more comparable number.

I think I’m going with this unless somebody else wows me with a good reason not to. I’ll think more on it myself, but I’m having a hard time finding fault in this method.

by J.J. from Kansas on Jul 9, 2011 11:08 AM CDT up reply actions

Well, allow me to object

I don’t like the idea of comparing Howard to other goalies because it’s always going to be biased. It handicaps Howard’s possible positive ratings against bad teams (with decent goalies). Those games count just as much as games against good competition. If we want to truly rate how well Howard played, we need to judge him as independently as possible. While that’s ultimately an impossible task, the whole point of CSSI is to strive for these individual stats (hence adjusted +/-). You can do this without the “outplaying” concept. If Detroit wins 6-0 against Edmonton, and Edmonton had like 15 shots all game, well, just give Howard a 0 rating then. He didn’t mean anything in the game, despite the shutout. I fear that Howard may be punished for technically being outplayed when he played well enough to win but the team in front of him didn’t.

Just my opinion. It’s a good idea, but it seems to go against the whole point of the exercise in my mind. It would start to blame Howard in the games which his teammates weren’t able to score…ultimately, he can’t do anything about that. And this concept may come back to punish Howard for things that are out of his control. Besides, if you talk to any goalie, I doubt they truly worry about “outplaying” the guy at the other end…they’re only worried about stopping the shots they face.

Instead, I like the idea of a 2/+2 system, but giving you leeway to give a +3 if a goalie was absolutely mindnumbingly spectacular. Kinda like giving a player an extra plus on some plays for doing more than one awesome thing. It seems to somewhat mirror the adjusted +/, which is bascially a 4-point scale anyway because of the ability to hand out half-plusses and half-minuses (and the leeway to give multiple plusses on an individual play)

I do agree that the good save part of goalie CSSI was problematic. It doesn’t take into account context of a game…a good save when you’re winning 4-0 doesn’t mean as much as a good save when it’s 0-0. Plus, it’s extremely subjective. Way moreso than bad goals.

My 2 cents.

by tehGOALIE on Jul 9, 2011 1:02 PM CDT up reply actions

Ignore the strikeouts. Didn't mean for that text to have a line through it.

Oops.

by tehGOALIE on Jul 9, 2011 1:04 PM CDT up reply actions

If Detroit wins 6-0 against Edmonton, and Edmonton had like 15 shots all game, well, just give Howard a 0 rating then. He didn’t mean anything in the game, despite the shutout. I fear that Howard may be punished for technically being outplayed when he played well enough to win but the team in front of him didn’t.

If Edmonton’s goalies get completely hung out to dry in that situation, there’s actually a chance Howard could get a -1 rating on a game that his team won 6-0. There’s also a chance he could get a 0 or a +1. Most likely, I’d say he’d get a +1 because most teams don’t score six goals without at least one of them being soft. It all depends on the context. Yes, being tested more is going to give a goalie more chances to earn or lose the + rating in a given game though.

Maybe simply tracking both methods is the way to go.

by J.J. from Kansas on Jul 9, 2011 1:55 PM CDT up reply actions

If you're willing to put in the extra effort...

Then I’d say go for it! I think there are going to be issues with whatever method is chosen, but the real question is which one is better…which at this point we have no idea and any arguments made either way are just conjecture.

by tehGOALIE on Jul 9, 2011 3:29 PM CDT up reply actions

I fear that Howard may be punished for technically being outplayed when he played well enough to win but the team in front of him didn’t.

No, he wouldn’t, and that was the point. It was all contextual. Maybe Howard plays out of his mind, but the Wings go on the PK 12 times in the game and are consequently outshot and they lose the game, 5-3. Yet Howard could still be awarded the +1 for the game, because he outplayed the other goalie, he just got hung out to dry by his team. As for the opposite issue, when the Wings are generating more shots the other way, Howard still shouldn’t be getting outplayed, if he’s that good. If the other goalie is giving up bad goals himself or making mistakes, even if he’s getting shelled, that’s still a +1 for Howard, because he played better than the other goalie, even though the other goalie’s job may have been harder anyways. It’s all context based.

Because you’ll never be able to judge Howard independently (he’s part of a team, and you can’t swap an alternate goalie in to last week’s game to see how he would’ve played vs Howard), this is the best you can do, I think. If Howard is truly an upper-echelon goalie, then he shouldn’t get many -1’s, and he’ll mostly collect 0’s (when playing against equal competition) or he’ll get +1’s, for being better than most goalies in the league. If Howard is truly an average goalie, playing on a team that only requires him to be average, then the -1’s will creep in for giving up bad goals here and there (even if they may not necessarily have led to a loss – this is how Chris Osgood made his money for a few years after the lockout), and the effects will be visible.

Plus, since this is all tracked on a per-game basis, at the end of the year, you could also do things like Howard’s score vs other Western Playoff teams, or Playoff Teams in general. Or maybe you can do Howard vs Central Division, to see if he is getting outplayed in our division, potentially helping keep us short of a division title. Shit like that.

It’s not necessarily about Howard vs Mason/Rinne/Luongo/whoever. It’s about Howard vs CBJ OFF/NSH OFF/VAN OFF, and being able to compare that against Mason vs DET OFF/ Rinne vs DET OFF/Luongo vs DET OFF. Plus, its really not all that much more tracking because its basically an overall call on the game at its conclusion, not necessarily a bunch of pluses and minuses like tracking how/why pluses, minuses, and points were given out for skaters.

http://sacrificethebody.blogspot.com/
Sacrifice the Body - Examining the NHL through statistical analysis, reasoned thought, and blind conjecture.

by IAmJoe on Jul 9, 2011 3:55 PM CDT up reply actions

If that's the way you meant then it makes more sense

I was thinking along the lines of if Howard plays well but the goalie at the other end plays better and Detroit happens to lose, then Howard still gets a -1, despite being an asset to the team that day. Because in the end, that -1 will have more to do with Detroit’s inability to score rather than its goaltending performance. Especially if the goals Howard lets in he isn’t expected to stop (e.g. fluky deflections). But perhaps I’m thinking of the extreme special case there.

Still, it seems doing it the way you propose makes the stat unnecessarily convoluted (it would be more of a team stat like wins rather than a just goalie stat) and more subjective than the 4-point scale. But maybe I’m still misinterpreting it, or thinking too far into it. In the end I might just be splitting hairs here. Plus you’re right, even doing both of these methods would be simpler than the big save/bad goal method, because it’s a game summary stat rather than a play-by-play stat.

by tehGOALIE on Jul 9, 2011 6:08 PM CDT up reply actions

I don’t think its any more subjective, probably just as subjective. With the -2/+2 scale, you’d have the very difficult task of trying to judge whether a goalies performance was “average” for a 0, or a “above average” for a 1. When you figure that “average” is actually pretty good, and the entire range of goaltending performances is not actually that big (especially if you throw out the obvious observations at -2 and +2, where its pretty apparent the quality of the game), the entire -1, 0, 1 range is going to be awfully tight, and might devolve into a practice of splitting hairs. The main concern with that, especially considering we’re all very biased towards the Red Wings, is that in the absence of a easy “check” to see if the goalie was average or not (we can’t put in a different goalie after the fact for comparison’s sake), that in large part what you would get is a lot of close games that maybe should’ve been a 0 instead get a 1, because as the recorder, you think of say 2 big saves he made in the third to preserve a 1-goal win. Of course, if he’d made those 2 big saves in the first when the game was tied, it meant just as much to the win, it’s just an issue of perception at that point, and perception is a tricky bitch.

In all likelihood, considering that almost no goalie in the NHL is actually bad, what you would end up with is a situation where, if you did the same exercise for every goalie in the NHL, maybe you find out Howard is a +30, Thomas is a +40, and Nik Khabibulin is the lowest goalie in the league at +20 (discounting backups). Essentially, what you would end up with then is another situation where 0 (average) is actually +30. Which could tell us something (the goaltending talent variance in the NHL is way smaller than the goaltending salary variance, so don’t pay for the higher end of the scale), but still becomes somewhat unwieldy. And again, since we won’t be collecting those scores for other goalies, even when you get to the end, you won’t have anything to compare them to, so you finish the season saying “Howard is a +30 goaltender” but you have no context to know exactly what that means.

My idea I think actually deviates significantly enough from team wins (a terrible goaltending stat) because it’s judged independent of who actually won or lost the game. A goalie can get the +1, even if he lost the game, by still playing a real good game in a losing effort, as long as he doesn’t give up bad goals. Consider that the Wings “outplay” their opponent via outshooting, say, 80% of the time. Yet, they don’t win 80% of their games, or even have an 80% points percentage. Why? This stat would help explain why, I think. We can look at these losses to see if Howard gave up the bad goal in a tight game, or maybe that the other guy played lights out, while Howard was average, or even that the two goalies were fairly even, and most of what drove the standings result happened outside of the two creases.

It’s not perfect, there’s still a lot more that we don’t know, but I think this is extremely useable after the season, in terms of being able to look at it and say things like “Detroit won 7 games this year in which Howard was outplayed by the opposing goalie”, which indicates DET non-goaltending strength, or something like “Howard was outplayed in 11 of the teams 24 losses”, giving an indication of why games were lost, or you could flip that to “Howard outplayed his opposition in 6 of the teams 24 losses”, indicating times when Howard did everything he could, but the team in front of him lost those games. If you can’t analyze the data afterwards, whether by lack of context or whatever, then there isn’t much point to collecting it.

http://sacrificethebody.blogspot.com/
Sacrifice the Body - Examining the NHL through statistical analysis, reasoned thought, and blind conjecture.

by IAmJoe on Jul 10, 2011 12:28 AM CDT up reply actions

Just asking a general question

But aren’t none of these methods accurate at all?

Generally speaking, most of these games are watched with Red Wings coloured glasses. So what may be quatified as a “huge save” by us Wings fans may be routine for a team like Florida with Vokoun or Nashville with Rinne. Therefore, our definition of big save (or Ken Daniels definition of a huge save) is definitely going to be different.

For example: Howard is accustomed to…embelishing glove saves. He does the splits and reacts as if he snagged a fly with chopsticks out of the air. Usually, the flourish of his wrist is enough to sell the save as a “big save” but in reality, if you follow the trajectory of the puck, it will most likely go wide of the net.

Therefore, in the above example, the “big save” really wasn’t anything at all. Whereas a “routine” kick save at the goal mouth that leads to an odd-man break is not considered a “big save”, and yet it led to a rapid and instantaneous change in both momentum and game shape.

I think the only true way to measure a goalie’s value would be something like the baseball sabermetric WHIP (Walks + Hits per Inning Pitched). Something with hard, quantifiable data like that would probably be a more useful indicator of a goalie’s true worth to his team.

However, I’m not a math guy, nor would I know how to go about creating a new sabermetric stat. For example, simply taking Goals Against + Shots Against divided by Minutes Played doesn’t really yield any useful data either.

Anyone else have any input? Or am I just talking out of my ass.

by eight_legged_freaks on Jul 9, 2011 2:41 PM CDT reply actions

A good point and part of the reason why I am planning on doing away with the concept of “big saves”. However, I’d like to point out that I have only used CSSI-adjusted numbers to compare Red Wings to other Red Wings. Accusations of bias are minimized unless you want to accuse me of liking Howard that much more than I like Osgood.

Which is fine, if you’re willing to go through the CSSI posts for each of them and find examples of where an accused bias such as that has clouded the numbers…

It’s also fine if you’ve kept up all season and have a good indication that I’ve fallen victim to Howard’s “embellishments” when counting big saves. I didn’t start fully expounding on which saves I counted as “big” until about the midpoint of the season, but there is a sufficient audit trail there for further questioning of my methods.

Hard and quantifiable data is the reason for the CSSI to exist in the first place. The point of the entire project (and the part that I had the most difficulty with as far as goalies go) was to add proper context. Grant Fuhr’s “hard and quantifiable” numbers show that he had only one season in the NHL where his save percentage was as high as .901. The reason he’s considered one of the greats is because he was a goalie that made the saves that mattered. If there were truly a way to take the hard and quantifiable data and plug it into an overall value generator, then I would think people would have already done that.

It’s the beauty of hockey – there are hockey sabremetricians who are no closer to unlocking the great challenge of the “win-shares” debate than anybody because hockey is such a complex sport.

by J.J. from Kansas on Jul 9, 2011 2:55 PM CDT up reply actions

It's really the beauty of all team sports

Hockey especially, of course. They’re so complex that purely objective analysis of a particular facet is nearly impossible…there are so many different ways to quantify the game that we see happening before our eyes, in addition to the already endless list of official stats being kept.

by tehGOALIE on Jul 9, 2011 3:37 PM CDT up reply actions

Grant Fuhr’s "hard and quantifiable" numbers show that he had only one season in the NHL where his save percentage was as high as .901. The reason he’s considered one of the greats is because he was a goalie that made the saves that mattered.

Heh. That’s a kind way to put it.

http://sacrificethebody.blogspot.com/
Sacrifice the Body - Examining the NHL through statistical analysis, reasoned thought, and blind conjecture.

by IAmJoe on Jul 9, 2011 3:58 PM CDT up reply actions

I apologize JJ

I didn’t mean to suggest that I felt your observations were biased. That wasn’t my intention and I feel as I offended you. Your hard work was greatly appreciated and has yielded some truly incredible data and some fascinating food for thought.

Again sorry if I offended you.

I was merely hypothesizing and wondering aloud whether there was a mathematical way to remove any bias for or against and make it a truly quantified analysis.

Again, no harm intended.

by eight_legged_freaks on Jul 9, 2011 4:09 PM CDT via mobile up reply actions

No worries

Re-reading my comment, that did come across as a lot more defensive than I wanted it to. Sorry for the tone.

by J.J. from Kansas on Jul 9, 2011 4:17 PM CDT up reply actions

You’re right in that the definition of a “big save” is entirely subjective, and really, not all that useful, and could actually be more an indication of showmanship than anything else.

Currently, the most useful stat we have for diciphering a goalies true talent is his Even Strength SV%. EV% is pretty consistent for goalies, because they spend the vast majority of their time at Even Strength, so flukes, streaks of good/bad luck, and other kinds of noise become diminished in the vastness of the sample. The primary generator of year-to-year overall SV% fluctuation is the PP SV% and SH SV%, and those situations are considerably smaller sample sizes (particularly SH%, which also often are dangerous chances on the rush, so while they don’t happen often, they’re frequently a better scoring chance than usual).

At first glance, you might look at that, and think that sounds ridiculous – SV% for NHL goalies ranges generally from 0.900 to 0.940, and an individual goalie will generally vary less than that, maybe a total of 0.030 from top to bottom. But when you actually think about what those numbers mean… the difference between a 0.900 goalie and a 0.940 goalie is huge, in terms of stats, but in terms of what it actually means, the 0.900 goalie will give up 40 more goals on 1000 shots. And that’s the larger end of the possible range of values we’re talking about here, so for an individual goalies year-to-year SV% variance, you’re talking 10, 20, 30 goals per 1000 shots. When you think about it like that (and keep in mind that PP SV% is, i believe, more like in the area of 0.750-0.825, as a very rough estimate) and think about how much special teams plays a role in today’s game, and how a team can have a good or a bad run on the PK, you can start to understand how those special teams SV %‘s will make that big of an impact on a number which truly isn’t moving as much as we think it is.

http://sacrificethebody.blogspot.com/
Sacrifice the Body - Examining the NHL through statistical analysis, reasoned thought, and blind conjecture.

by IAmJoe on Jul 9, 2011 4:07 PM CDT up reply actions

That's true

Just look at the numbers for Lidstrom’s plus/minus stats if he had a goalie with Thomas’ or Rinne’s save percentage behind him instead of Howard.

http://www.wingingitinmotown.com/2011/4/12/2103092/a-study-of-lidstroms-plus-minus-rating

If everything else stayed the same, Lidstrom would have been a +11 with Rinne’s save percentage behind him, and that was only a difference of 0.018.

by J.J. from Kansas on Jul 9, 2011 4:21 PM CDT up reply actions

I have no idea how to solve the goalie rating conundrum, but here are a few other thoughts.

I’ve always liked the concept of the Quality Start in baseball (Pitcher has to pitch 6 full innings minimum and give up no more than 3 earned runs, regardless of the W/L result), and I’d like to see a stat like that for goalies. Hockeyprospectus.com attempts to define a Quality Start as a combination of GAA and SV%. A few others have taken stabs at it as well. I’m not statistically inclined enough to decide if that’s a good method or not. Maybe simplified, 60 minutes of 2 goals or less is a quality start? Maybe it needs to be purely SV% based to account for games where the team is just playing poorly, similar to the condition of Earned Runs for baseball.

In addition to the QS, baseball has the concept of a “tough loss” where a pitcher gets the loss despite recording a quality start. This pairs with run support and I think can work with hockey as well. Behindthenet.ca has shots for per goalie over 60 minutes, but I cant seem to find goals for. Perhaps I’m just blind.

It might not be a bad plan to isolate some situational plays as well. Breakaways, odd-man, power play, and shorthanded goals/chances. Especially if you plan to continue tracking “big saves” at all. Some of this can be found on Behindthenet.ca, but they tend to average everything out over 60 minutes and only count the goals. A pure ratio might work out better. This tracking could help inform the overall game ratings, and may provide a measure of “clutchiness.”

At any rate, absolutely fantastic work all around, and major kudos for your dedication.

by Nate A on Jul 10, 2011 12:36 PM CDT reply actions

I know part of the point

was to get away from a system where “0” isn’t average. Having said that, you could do away with “big saves” without getting away from “weak goals”. You could record the ones that should have been stopped (less subjective), and ignore how a save is made. A save is a save is a save; flourish or no.

Now, obviously every goalie would end up with a minus rating, but the further from 0 they were, the worse they played. You can do the “per 60” stat to compare goalies who played different amounts of time.

The plus/minus system just seems to subjective to me. A lot of different looking performances could all be +1 performances in that system. Ditto for 0 or -1.

by Big Z in Orlando on Jul 11, 2011 4:25 PM CDT reply actions