On Day 2 of the Society of American Baseball Reasearch (SABR) Analytics Conference, I was happy to attend a panel about Front Office Decision Making moderated by Ken Rosenthal and manned by Colorado Rockies Senior Vice President/Assistant General Manager Bill Geivett as well as Seattle Mariners Executive Vice President/General Manager Jack Zduriencik, and San Francisco Giants Vice Persident/Assistant General Manager Bobby Evans. I say happy because I was curious about how the Rockies front office worked and what kinds of data they used and happily was in a good place to get some answers.
I’ll admit that, before the panel, when I thought of the Rockies front office I thought of “Dan O’Dowd” with a capital period. Yeah, at a high level, I was aware other people are “involved”, but in my mind the bullseye stopped there. To me, there also seemed to be a difference in tone coming from the Rockies when talking about players. In contrast to a team like the A’s that might say something like “We think Jean Deux will improve because he gave up a lot of unlucky hits.”, the Rockies would say “We think he’ll get better” which is all fine but not reassuring without a “what” or “how” or “why”.
The basic idea of sabermetrics is to use evidence to back up a guess (a what) on a player’s performance. That guess is based on something happening (a how) and understanding the reasons and chances of that happening (a why). Even if the final result doesn’t match what was the guess, that doesn’t mean the sabermetric thinking was wrong. If Todd Helton has a career on-base percentage (OBP) of .400, that doesn’t mean he will always get on base four times every ten at bats. In fact, as any Rockies fan knows, he’s had quite a few slumps (and quite a few hot streaks) even though his career OBP is .414. The point is, sabermetric thinking suggests that if, to improve your team’s offense (a what) you acquire someone with a .400 OBP to replace someone with a career .300 OBP (a how), the person with a .400 OBP is more likely to get on base which should allow more runs to be scored (a why).
On the other hand, I also see the way the Rockies repeatedly value certain player abilities such as high walk rates for hitters regardless of how often they strike out and general bullpen and closer interchangeability which seem to reflect what’s popular among the sabermetric analyst crowd. So, though I saw results that looked “sabermetricish”, I wasn’t sure what the thought process was beneath those results. Thus, after breakfast, I eagerly sat down to listen to the panel with a tablet in one hand and an iced hazelnut mocha beneath my seat to get an idea from Bill Geivett of what was the proverbial bone beneath the meat.
Bill Geivett has been with the Rockies and in baseball in general for a long while, entering his 14th season with the Rockies and 26th season in professional baseball. A former All-American and minor league baseball player until a knee injury forced him into retirement, Geivett began his career as a scout and organizational instructor for the Yankees. He has also been with the Montreal Expos, Tampa Bay Devil Rays (when they were called the Devil Rays) and the Dodgers. The Rockies hired him in 2000 as their Director of Personnel and after serving in that position, the Director of Minor League Operations and the Rockies Senior VP of Scouting and Player Development, he became the Senior Vice President of Major League Operations in 2012 while also retaining the title of Assistant General Manager that he acquired in 2005.
When he became Senior Vice President, the Rockies broke tradition and gave him a field office next to ex-manager Jim Tracy, enabling the front office to be directly connected to the field staff which includes the manager and the players. One quip (of the many quips) that provided laughs during that panel was Geivett declaring (or lamenting) that he now “leads the league in offices”. Having an office in the clubhouse allows the front office to get more information about the situation while ensuring that the feelings of the field staff including the players, coaches, and managers are duly noted. Yet he is “at the same time not there” and able to keep a distance when needed. Keeping some distance can be important since, in the end, these are decisions about people who are having numbers thrown in an attempt to impartially determine their value.
Generally, the Rockies are a pretty stable organization from a player and manager standpoint, with similar faces year in and year out. The dugout, for as long as I’ve followed them, presents itself as a close-knit, comfortable community. Very rarely does discontent get to the media or cause a shouting match in the dugout. Apparently, that also has its disadvantages because one thing Geivett discovered after he created an office in the clubhouse was that players had concerns but did not want to speak in the dugout about them because they didn’t want to be seen as a distraction. Now, they are able to come to him directly if needed.
The stability of that clubhouse, while having obvious benefits especially for fans who naturally latch on to certain players, also has its disadvantages. Geivett said that “a ring is a constant reminder”. The Rockies came close in 2007 to winning that World Series ring but alas, being swept by the Red Sox meant no victory and no validation to their process. While the Rockies have tried since then with those same core players from 2007, Geivett noted that they’ve “had two last place finishes believing in that core group”. The additional challenge was that, though the core group primarily remained effective, they would also get injured. So, Geivett explained “Plan B was typically youth,” and he observed that since it’s hard to learn at the major league level, the Rockies were often competitive each year since 2007 but could not win. Thus, Geivett indicated the goal of the 2013 offseason was to acquire leadership and experience. Fun fact: Todd Helton has been to the playoffs twice, the same number of times as Brett Anderson who is 14 years younger. Boone Logan and Justin Morneau have been to the playoffs three times and LaTroy Hawkins four times.
On that note, Geivett said that the Rockies are also “pretty direct in what we like” when evaluating and acquiring players. This offseason, they wanted an impact starter who generates groundballs, throws strikes and works deep into games so that the bullpen isn’t overtaxed. Thus, they acquired Brett Anderson even though they only had two years of control and there were medical concerns which were evaluated by the Rockies doctors. He fit the bill for what they wanted and actually was available at a good cost because of those very medical concerns.
Something else to note is that Geivett uses “we” a lot. He was adamant that no general manager can do anything without approval and the days of the lone GM calling shots alone anywhere in baseball are basically over. It’s not a O’Dowd thing, but a Rockies thing.
There’s actually a simple reason for that too, as Geivett explained. There is so much information around the game these days to process and evaluate that it is virtually impossible for one person to make a decision without seeking input from analysts, scouts, coaches and ownership. The Rockies want people who fit not just their plan for run scoring and run prevention but enhance that clubhouse stability. Each potential player acquisition is evaluated with four components: contract/trade cost, medical reports, scouting (including makeup) and statistical research. Or, as Geivett called it, “There are four ways to kill a player” and the Rockies utilize members across their front office with different skill sets to shoot the holes. In order to do that, the front office has to communicate with their doctors, their analysts and with the field staff.
According to Geivett, he has been working with data and sabermetric concepts for years. Going back to 1993 as an area scout, he used to hand-calculate the Runs Created metric created by Bill James. He even stated, after the panel, that the Rockies have developed their own proprietary version of a sabermetric principle called Wins Above Replacement (WAR). WAR is generally expressed as number, positive or negative, which indicates a player’s worth to the team in terms of wins. The basic idea of WAR is to evaluate how good a player is on offense, defense and/or pitching compared to a replacement-level player like a waiver wire player or minor league veteran. For example, FanGraphs ZiPS projection system calculates, based on the chart below, that Troy Tulowitzki has a value of 5 WAR for 2014 i.e. Tulowitzki’s offense and defense will make the Rockies five wins better in 2014 than someone at AAA Colorado Springs or a generic minor league free agent.
A team with 0 WAR would probably win around 50 games, so the idea is to try to spend money (or trade talent) wisely to have as valuable a player as possible at each position. So compared to a team of replacement level players with 0 WAR, the Rockies have some definite value. As a rough estimate, if you add the WAR projected by ZiPS on the chart, the Rockies are an 80 win (50 replacement wins + 30 WAR) team. WAR also accounts for things like the offensive difference between a first baseman and a shortstop. Since it’s easier to find a good hitting minor league first baseman than a good hitting minor league shortstop, a major league first baseman has to be _that_ much better to provide value.
One of the other things that WAR also does is factor in the difference in baseball parks while valuing a pitcher. So It can say “this guy is good, he just pitches in Coors Field so he looks bad” versus “this guy is bad, he just looks good because those fly ball outs to right field would be a home run in any other park.”
There are multiple ways of calculating WAR, especially in the public domain, depending on what aspects of a player’s performance someone finds value in. I’m glad that Geivett says they their own, proprietary version of WAR. In theory, they would have studied altitude more than pretty much anybody and its possible that not just humidors, but different ways of valuing players at altitude was a result. Perhaps that might explain why there hasn’t been a free agent bust as bad as Jeff Cirillo or Mike Hampton in a long while. Unfortunately, it’s also proprietary, and it has to be so that other teams don’t take advantage of the Rockies. Other teams also have their own ways of valuing players, though they might not use batting average or ERA instead of WAR.
So I guess I’ll just be satisfied to know they thinking about it. I’ll try to be satisfied, anyway, though I do get comfort in knowing that Geivett is also a member of SABR. With the connections and research available in that organization, he would have a good idea on who to ask discreetly and/or hire to create their own version of WAR. In the meantime, not knowing the details, I’ll dub thee RWAR. Erps, wait, FanGraphs has a rWAR. Ok, I redub thee RRWAR.
So, the Rockies have methods of valuing players based on what they know they want based on those four components mentioned earlier: contract/trade cost, medical reports, scouting (including makeup) and statistical research. They also need to make sure, when evaluating a player on those four components that they are asking the right questions. As a data analyst in my “day job”, I know (and can appreciate) that it takes dialogue to properly define the question. When a season is only six months long, you don’t want to have analysts spending a month researching something that turns out to be different from what the leadership asked for just because the question was worded poorly. Thus the Rockies employ a statistical analyst team of four people, maintain discussions between the front office leadership and those analysts to define the question, then turn the analysts loose to gather the data.
A complicating factor, as Geivett talked about, is that there can be gaps in the data where even the best question returns a guess. He spent some time expressing how the Rockies targeted Jose Abreu, the Cuban slugger who signed with the White Sox, during the offseason. Though they realized the tremendous bat potential, they also were unclear about how huge that potential was because the talent in the Cuban league has been diluted with so many recent defections. They still made a competitive offer, but not one that was outside of their comfort zone based on their valuation (RRWAR?).
Even with those challenges, the Rockies attempt due diligence in valuing players and their acquisition costs through internal discussion and research. Since players are always maturing or declining, getting hurt or inventing the better curveball, evaluating current value and projecting future value never stops. That factor, Geivett adds, is also one reason why there are fewer “last minute” deals on the trading deadline then their used to be. Since more teams in baseball have better analysis departments than they used to, most teams have already done some valuation on each player and have already talked with rival teams about who they want and what the cost would be well before the deadline. Teams have also realized it is better to acquire a player sooner rather than later when possible so trades are also happening weeks or even months before the trading deadline. That also means there are fewer surprises at the trading deadline since each team has pretty much evaluated all major and the important minor league players and most likely discussed trade ideas with each other.
Ultimately, one of the final judges of a team’s success is that World Series ring. Besides the talent of the players and the maneuvers of the front office, there is still some luck involved. It is nice to know the Rockies use sabermetrics though, from the outside looking in, the only ways to judge how well they use analysis is based on their win-loss record and their ring count. The Rockies have won before, but sadly without a ring, the thing fans remember most are the last two losing seasons. The results over the last two years has not changed and from the outside looking in and without knowing the why or how, it’s hard to tell how much of the losing is because of luck versus a flaw in the logic or valuation. It is hard to say, for example, that just because a player is on a playoff team means that player contributed positively to that team or would continue to do so. The question might also be whether the Rockies are valuing certain components more or less than they should.
Nonetheless, I did walk away from the panel with a better understanding that there was a process and it involves input from multiple people analyzing and then deciding on the data, as a team. I just don’t know what RRWAR is made of except it sounds cool.
The Society of American Baseball Research has been generous enough to provide the entirety of the panel interview as well as other panel interviews. Funny at parts and insightful overall, I suggest pulling out a bag of popcorn (or for the full experience, an iced hazelnut mocha at 8 am MST) and giving this one a gander.