Coffee is for Closers – Evaluating the 9th Inning Guy

holtzman

The save statistic has an interesting backstory.  Although some teams tracked the pitchers who finished games, little was done to standardize the measurement for a number of years.  That is, until Jerome Holtzman developed the criteria for awarding a save in the early 1960s and tracked it for The Sporting News through the 1960s.  The save entered public consciousness by being regularly written about and finally became an official statistic tracked by Major League Baseball starting in 1969.

Major League Baseball’s rule 10.19 states the official scorer shall credit a save when a pitcher meets all four of the following conditions:

  1. he is the finishing pitcher in a game won by his team
  2. he is not the winning pitcher
  3. he is credited with at least 1/3 of an inning of work
  4. he satisfies one of the following – (he enters the game with a lead of no more than three runs and pitches at least one inning, he enters a game, regardless of the count, with the potential tying run either on base, at bat, or on deck, he pitches at least three innings.

rulebook

Saves are difficult to evaluate. There are baseball players and fans who believe that it takes a player who possesses a special make-up to consistently go out in the 9th inning and record the final three outs of a game.  Other fans and players don’t believe there is a difference between the final three outs or any other inning within a game.  The philosophical question of the save becomes, what is the value of a pitcher who gets the final three outs of a game and does that ability warrant a statistical acknowledgement that has far reaching implications on things like arbitration-driven salary, free agent price, and trade value?

gossage          sutter         eck         rivera         kimbrel

The role of closer has morphed over time which may make the save statistic less meaningful when tracking totals over decades.  In the past, it was not uncommon for a closer to work multiple innings in an appearance (see Gossage or Sutter).  For example, in 1982 there were 439 instances of a player earning a save based on an appearance of 2 or more innings.  As the 1980s progressed, teams like LaRussa’s Oakland A’s began using a specified 9th inning pitcher and workloads moved towards one inning (see Eckersley).  By 1989, there were 325 instances of 2 plus inning saves.  The one inning appearance took hold throughout the 1990s (1997 had 95 instances of 2 or more inning saves) and became even more ingrained during the 2000s (2008 had 35 saves of 2 or more innings).  The one inning closer came to define the role (see Rivera and Kimbrel).  Conceptually speaking, the save statistic has created a single anointed closer and the shift in thinking from a multi-inning fireman to a highly specialized designated game ender appeared to be the natural evolution of the job.  That is, until recently.

The use of sabermetrics has slowly moved the pendulum away from a single 9th inning closer who is responsible for the save opportunity regardless of whether a team is leading by one, two, or three runs.  The use of statistical analysis and game theory have indicated that it may be more efficient to utilize a team’s best arm (for argument’s sake, let’s consider the best arm the closer) in the highest leverage situations (For example, Francona’s use of Andrew Miller after his arrival to Cleveland during 2016, especially the post season).  Similarly, Showalter’s failure to use Britton in this fashion during the 2016 playoffs may have cost Baltimore moving forward in the playoffs and at the very least created a talking point that is still discussed as the league prepares for the 2017 season.

jansen          britton

In statistics, larger sample sizes lend themselves to more meaningful analysis and less chance for statistical noise and error.  In the case of closers, data is restricted due to their typical workload.  Take Kenley Jansen for example.  In 2016, he made 71 appearances, finished 63 games, and earned 47 saves in 68 and 2/3 innings pitched.  Zach Britton is similar.  In 2016, he appeared in 69 games, finished 63, and earned 47 saves in 67 innings pitched.  Limited appearance totals, limited innings pitched, and limited batters faced per appearance make the results of statistical analysis tenuous at best.  By focusing solely on saves, the amount of data being considered shrinks even more.

In 2016, there were 1061 saves that were based on 1 inning of work.  An additional 44 saves were based on 2/3 of an inning of work and 60 saves were based on just 1/3 of an inning of work.  This means that in addition to 1061 instances of one inning saves, there were an additional 104 saves awarded based on an appearance of less than one inning.  Conversely, there were only 29 saves in 2016 of 2 or more innings and almost all of these saves were earned by ‘non-closers’.  Clearly, all saves are not created equal and the data pool for analysis, especially using a pitcher’s performance per appearance (since an appearance can be as short as one batter faced) difficult to assess.

 

It is my contention that the save is a diluted statistic and tells us much less about a pitcher’s performance than it could.  It is akin to a list of runners that completed a marathon.  All finishers are listed but no distinction between someone who finished the race in 4, 5, or 12 hours is made.  Without the crucial finish time, a good deal of information about each individual’s performance is lost.  I have created five types of saves with the goal of more clearly documenting the work of closers.  The five types are the Ultrasave, Powersave, Plainsave, Uglysave, and Disastersave (this type is extremely rare).  While the concept of ‘saves by gradation’ cuts the data into smaller parts, it allows for more meaningful information about pitcher performance to be conveyed.

 

Below are the five types of saves and the criteria for each:

UltraSave = no walks, no hits, no HBP (no baserunners), at least 1 IP, struck out all batters faced, earned the save.

PowerSave = no walks, no hits, no HBP (no baserunners), at least 1 IP, at least 1 K, earned the save.

PlainSave = no walks, no hits, no HBP (no baserunners), at least 1 IP, no Ks, earned the save.

UglySave = at least one hit, at least one run, no Ks, at least 1 IP, earned the save.

DisasterSave = at least two hits, at least two runs, no strikeouts, at least 1 IP, earned the save.  This is the rarest save type (thankfully).  Of the Top 50 Save Leaders of 2016, only Kintzler and Gomez each earned one in 2016.

 

2016 Saves Leaders (Top 50)

Name              Sv/Op        BlnSv          UltraS          Pwrsv             Plnsv      Ugsv

Familia            51/56             5              0                   22                        1              1

Britton            47/47            0               1                    15                       5              0

Jansen             47/53             6              3                   24                        2              1

Melancon       47/51             4               1                    13 (7 Pt 6 Wa) 5              3

Rodriguez       44/49            5               1                    11                        7              1

Ramos             40/43             3              0                   13                         0             1

Dyson              38/43              5              1                    8                          6             2

Colome           37/40              3              1                    11                         2            0

Gomez            37/43               6              0                   5                          9             4

Robertson      37/43              7              2                  12                          2             0

Chapman        36/39              3              2                  13 (8 NY 5 Ch)    1             1

Osuna              36/42              6              0                 11                           5             0

Allen                32/35               3              0                 13                          2              1

Casilla             31/40               9              0                  7                           3              1

Kimbrel           31/33               2              4                  15                         0              0

Madson          30/37               7              0                  10                         5              2

Davis               27/30              3               1                    4                         1               0

Jeffress           27/28              1              0                    4                         1                1

Cishek             25/32             7              0                    8                          1               0

Rodney           25/28              3              1                    7 (1 Mia 6 SD)  2               0

Ziegler            22/28            6              0                   2 (1 Bos 1 AZ)     1               0

Johnson          20/23            3               1                    6                          1               1

Oh                     19/23             4              0                    8                           0              0

Papelbon        19/22            3              1                     4                           2              0

Diaz                 18/21             3              1                     9                           1               0

Rondon           18/23            5              1                     9                           2               0

Cingrani         17/23           6              0                    4                            2               0

Kintzler          17/20          3              0                    3                             2               3

Giles                15/20            5              0                    5                           0              2

Gregerson     15/21            6             1                     7                            2              0

McGee            15/19             4             0                     3                          2               1

Watson           15/20            5             0                    4                           1               2

Rosenthal      14/18           4             1                     4                            1              0

Maurer           13/19           6              0                    4                            2              1

Thornburg    13/21           8             0                     5                             1              0

Betances       12/17            5             1                      4                           0              0

Harris            12/15            3             0                     4                            0              3

Herrera         12/15            3             0                     7                            2              0

A Miller         12/14           2              2                    4 (3 NY 1 Cle)     1               0

Estevez         11/18            7             0                     5                            2               0

Tolleson       11/15            4             0                     2                            1                0

Vizcaino       10/14           4            0                      3                            0                0

Street             9/12              3            0                      3                          1                 0

Jepsen            7/11              4             0                     0                         0                 1

Kelley             7/9               2             0                      1                         0                0

Ottavino        7/12             5              1                      4                        0                0

Uehara           7/9               2             1                       4                        0                0

Bailey             6/7               1             0                       2                        1                 0

Cabrera          6/7               1             0                        1                       2                 0

Iglesias          6/8              2             1                         5                        0                 0

 

As previously stated, the save statistic is diluted and tells us little about a pitcher’s performance.  I have created five types of saves with the goal of more clearly documenting the work of closers.  The five save types are the Ultrasave, Powersave, Plainsave, Uglysave, and Disastersave.  Although ‘saves by gradation’ cuts the already lean relief pitcher data into smaller parts it allows for more meaningful information about pitcher performance than merely using a save total as an approximation for quality of a pitcher/quality of closer performance.

Ask managers, front office employees, or even fans of the game and most (everyone?) will tell you that the key characteristic of a closer is dominance.  In the 9th inning of a 1, 2, or 3 run game, any type of baserunner is a hazard to a team looking to shut the door on an opponent.  Thus, my save model rewards pitchers who do not allow balls in play (valuing the safest out – the strikeout) and avoid baserunners of any kind (HBP, walks, hits).  Greater penalties are given to pitchers based on the level that they violate these two criteria.

There are a number of ways to utilize my data system to achieve a better understanding of the quality of each closer.  The first way is to award a number value to each type of save and then tally the points.  The values I gave each type of save are listed below:

Ultra = 3

Power = 2

Plain = 1

Ugly = -1

Disaster = -2

 

Based on the new point total for Quality of Save, the 2016 leaderboard looks like this:

Name                   Point Total

Jansen                  58

Familia                 44

Kimbrel                42

Britton                 38

Robertson           32

Chapman             32

Melancon            31

Rodriguez            31

Colome                27

Osuna                  27

Allen                    27

Ramos                 25

Dyson                  23

Madson               23

Rondon                23

Diaz                      22

Rodney                19

Gregerson           19

Cishek                  17

Casilla                  16

Oh                         16

Herrera                16

A Miller                15

Johnson               15

Papelbon             13

Gomez                 13

Iglesias                13

Davis                    12

Rosenthal           12

Estevez                12

Thornburg          11

Betances             11

Ottavino              11

Uehara                 11

Cingrani              10

Maurer                 9

Jeffress                 9

Giles                      8

McGee                  7

Watson                 7

Street                    7

Vizcaino               6

Harris                    5

Tolleson                5

Bailey                    5

Cabrera                 4

Ziegler                   3

Kintzler                 3

Kelley                    2

Jepsen                  -1

 

This rating system gives a more accurate picture of player performance than using traditional save totals.  However, there is another adjustment that makes the rankings even better.  When a 1 point penalty for each blown save is applied, the 2016 leader board shifts again:

Name                   Point Total

Jansen                  52

Kimbrel                40

Familia                 39

Britton                 38

Chapman             29

Melancon            27

Rodriguez            26

Robertson           25

Colome                24

Allen                     24

Ramos                  22

Osuna                   21

Diaz                      19

Rondon                18

Dyson                   18

Madson               16

Rodney                16

Herrera                13

A Miller                13

Gregerson           13

Johnson               12

Oh                         12

Iglesias                 11

Cishek                  10

Papelbon             10

Davis                     9

Uehara                  9

Rosenthal             8

Jeffress                 8

Gomez                  7

Casilla                   7

Betances              6

Ottavino               6

Estevez                 5

Street                    4

Bailey                    4

Cingrani               4

Giles                      3

McGee                   3

Maurer                  3

Thornburg            3

Cabrera                 3

Watson                 2

Harris                    2

Vizcaino                2

Tolleson                1

Kelley                    0

Kintzler                 0

Ziegler                  -3

Jepsen                  -5

 

Even within this system, closers that held the job all season and made more appearances still get a positive bump due to their increased usage/opportunities.  However, less dominant compilers take a small hit while the dominators get a positive nudge.  Rank movement for each player from the original 2016 saves leaders list is listed below (basing the present order on my formula).

Place – Name – Total Spots Moved

  1. Jansen +1
  2. Kimbrel +8
  3. Familia -2
  4. Britton -2
  5. Chapman +2
  6. Melancon -4
  7. Rodriguez -4
  8. Robertson -2
  9. Colome -3
  10. Allen -1
  11. Ramos -6
  12. Osuna -4
  13. Diaz +4
  14. Rondon +3
  15. Dyson -8
  16. Madson -4
  17. Rodney -2
  18. Herrera +6
  19. A Miller +6
  20. Gregerson +3
  21. Johnson -2
  22. Oh -1
  23. Iglesias +9
  24. Cishek -6
  25. Papelbon -3
  26. Davis -8
  27. Uehara +6
  28. Rosenthal -1
  29. Jeffress -9
  30. Gomez -15
  31. Casilla -12
  32. Betances -1
  33. Ottavino +3
  34. Estevez -1
  35. Street 0
  36. Bailey +2
  37. Cingrani -7
  38. Giles -7
  39. McGee -7
  40. Maurer -5
  41. Thornburg -5
  42. Cabrera +1
  43. Watson -8
  44. Harris -5
  45. Vizcaino -3
  46. Tolleson -5
  47. Kelley -3
  48. Kintzler -11
  49. Ziegler -16
  50. Jepsen -5

 

There is another way to analyze the 2016 save leaders performance using my 5 save type method.  Adding the number of Ultra and Power saves together gives a snapshot of how dominant a closer has been while in save situations.  Using just these two categories, the 2016 dominant save leaders were (% of saves that were dominant in parentheses):

 

Name                   Ultra & PowerSvs Combo              Percentage of overall save total that were Ultra+Power

Jansen                                 27                                         (57%)

Familia                                22                                         (43%)

Kimbrel                               19                                         (61%)

Britton                                16                                         (34%)

Chapman                            15                                         (42%)

Robertson                          14                                         (38%)

Melancon                           14                                         (30%)

Allen                                    13                                         (41%)

Ramos                                 13                                         (33%)

Colome                               12                                         (32%)

Rodriguez                           12                                         (27%)

Osuna                                  11                                         (31%)

Diaz                                     10                                         (56%)

Rondon                               10                                         (56%)

Madson                              10                                         (33%)

Dyson                                   9                                          (24%)

Gregerson                           8                                         (53%)

Oh                                         8                                          (42%)

Cishek                                 8                                          (32%)

Rodney                                8                                          (32%)

Herrera                                7                                          (58%)

Johnson                               7                                          (35%)

Casilla                                  7                                          (23%)

Iglesias                                6                                         (100%)

A Miller                               6                                          (50%)

Ottavino                              5                                          (71%)

Uehara                                 5                                          (71%)

Estevez                                5                                          (45%)

Betances                             5                                          (42%)

Thornburg                          5                                          (38%)

Rosenthal                           5                                          (36%)

Giles                                     5                                          (33%)

Papelbon                            5                                          (26%)

Davis                                    5                                          (19%)

Gomez                                 5                                          (14%)

Harris                                   4                                          (33%)

Maurer                                 4                                          (31%)

Watson                                4                                          (27%)

Cingrani                              4                                         (24%)

Jeffress                                4                                          (15%)

Street                                   3                                          (33%)

Vizcaino                              3                                         (30%)

McGee                                 3                                          (20%)

Kintzler                               3                                          (18%)

Bailey                                   2                                          (33%)

Tolleson                              2                                          (18%)

Ziegler                                 2                                          (9%)

Cabrera                                1                                          (17%)

Kelley                                   1                                          (14%)

Jepsen                                  0                                          (0%)

 

Based on these results it appears the more dominant closers fare much better than their less dominant brethren.  A quick examination of the average number of Ultra/Power saves earned by the Top 10 on the list as compared to the Bottom 10 on the list yields the following information:

Top 10 performers averaged 16.5 Ultra/Power saves in 2016.

Bottom 10 performers averaged 2 Ultra/Power saves in 2016.

 

Interestingly, if the Top 50 Saves Leaders of 2016 are examined based on total save number (i.e. the traditional list) the averages look like this:

Top 10: 14.4 Ultra/Powersave total

Bottom 10: 2.8 Ultra/Powersave total

 

Examining these two lists indicates that the ability to dominate during a save outing likely positively correlates with retaining a closer job, earning more chances at save opportunities that present themselves to a team, and ultimately compiling more saves. Conversely, a player put in a closer role with a lower level of dominance is more likely to lose a closer job, have fewer chances at save opportunities for their team, and fail to compile as many saves as the more dominant closers given the same number of save chances.

A big thanks to Baseball-reference.com and the play index.

Follow me on twitter: @doctordaver

If you haven’t already, take the Machado vs. Harper poll here.

You might also like: My two part (Part 1  / Part 2) examination of Bill James’ Top 100 First Basemen of All Time list and an evaluation of candidates since the list was last updated in 2001.

 

Advertisements

3 thoughts on “Coffee is for Closers – Evaluating the 9th Inning Guy

  1. Ian Joffe says:

    Great post, I strongly agree that the save is very outdated. However, I’m not sure if your method of solving it. Your types of saves are still very much based on oppurtunity the manager gives. Additionally, your requirements are relatively arbitrary, such as a number like “300 wins.” If you were up for the math, I would recommend a stat based on percentiles of runs, hits, K’s, BB’s or at best some mix.

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s