Is the save a misleading statistic? It is certainly governed by a strange set of rules. Most simply stated, a save is awarded to the pitcher who finishes a game (who isn’t in line for the win) who entered the game with a lead of three runs or less unless there are less than three outs remaining when the pitcher entered. Then you have to consider runners on base and outs in the inning to determine whether a save should be awarded. This doesn’t even bring up the elusive three-inning save, but the point is that the save could be considered a somewhat overengineered statistic. Personally, I’m in favor of overengineered statistics (just wait, you’ll see) but given that current bullpen usage is designed to maximize save opportunities for the designated “closer”, the number of saves across the board has become inflated and almost the sole property of designated closers.
There is a growing sentiment which holds that the save, in the current era, doesn’t necessarily measure what it claims to. For example, consider the Twins game of July 29, 2007:
Neshek enters with one out in the 8th inning, Twins lead 3-1, one runner on, one out. He gets Ryan Garko to ground into a double play to get out of the inning.
Lew Ford hits a homer before Nathan enters in the ninth with a three-run lead. He doesn’t allow a run and is awarded a save for his efforts.
Now, both pitchers were effective, but one could argue that the situation that Neshek stepped into was more crucial to the outcome of the game, with the tying run at the plate Neshek ended the threat. That seems deserving of a save as well.
Inspired by this situation, and borrowing a bit from a Baseball Prospectus article, I set out to generate some new criteria for bullpen performance making use of Win Probability Added (WPA, which can be found at fangraphs.com). Going forward, these criteria will be used to award a save (remember, I promised overengineering):
- WPA of 10.0 or higher.
- Highest WPA of relief pitchers not earning a win.
- If starter has WPA greater than or equal to 10.0, the win goes to the starter, otherwise, highest WPA (over 10.0) gets the win.
You may notice that for relief pitchers, there’s not much difference betwixt a win and a save, so I’ve combined the two under the heading Relief Win in the table at the end of this article.
This method makes it a little more difficult to determine save opportunities so its difficult to see if someone is converting opportunities at a high rate, or just receiving a lot of chances. So, let’s check out the opposite of a Relief Win, the Relief Loss:
- Loss goes to the lowest WPA less than or equal to -10.0
- Blown save awarded if WPA less than or equal to -10.0 and lead is lost or goes from tied to behind when the pitcher is in the game.
Using these two stats (Relief Wins and Relief Losses) we can get a rough percentage of how a relief pitcher has affected a teams wins/losses. Collecting all this data for the 2007 Minnesota Twins bullpen gives the data below (+G = WPA over 0.0, ++G = WPA over 9.9, –G = WPA under -9.9). The “Score” column gives more positive weight to higher WPA and Relief Wins, while also punishing lower WPA and Relief Losses.
No surprise that Nathan, Neshek, and Guerrier are the top three, but it’s pretty surprising that there is such a dropoff after that. Nick Blackburn has only pitched in three games, but well enough to earn one Relief Win (a save) and lead the middling pack of relievers (Miller, Perkins, Cali, Ortiz). Rincon and Reyes have been pretty bad, and they’ve gotten lots of chances to prove it (again, not groundbreaking, but it shows that the method holds some water). Recently it seems that Neshek and Guerrier have been getting used a lot, but looking at these numbers, there aren’t many effective options in an important situation.
When the starters’ records are calculated using these guidelines the results are shown below under WPWin-Loss record and compared to their traditional Win-Loss record. The bullpen numbers were discussed above, the bullpen’s record is 14-16, which leaves 33 games that the offense was the largest factor in determining the outcome. Not surprisingly the offense caused more losses than wins, in those games the Twins went 14-19. The last column is the offense’s record in games started by that starter.
After Johan Santana, there’s not much success by the starters. I was somewhat surprised to see Scott Baker as the Twins’ second best starting pitcher by this metric (I thought Silva would be better).
Here’s the bullpens numbers, broken down into Wins, Losses, Saves, and Blown Leads. Joe Nathan still has the most saves, but where he has 32 of the Twins’ 33 traditional saves, here there’s a little bit more distribution.