SSI on the Value of Spring Training Performance

Sorry. It ain't zero.

.

Q.  Can you make any sense of spring training statistics?

A.  Different stats have different levels of "noise" in them.  Ichiro's career batting average has noise in it, but it's about 2%, not counting park context.  Ichiro's career range factor has noise in it, too, but the noise is something more like (?) 30%.

Baseball sabermetricians are really cruddy at estimating the AMOUNT OF noise in a particular statistic.  A sense of proportion is almost completely lacking in the industry.  Sabermigos are great at math; they're not so great at common sense.

We mean it in a good way.

.

Q.  How much noise is in there in spring training stats?

A.  Oh ... what would it be?  60%?  Something like that.  

One of the few sabermigos who does have a sense of proportion:  Jeff Sullivan.  Read this article by him.  He points out various [huge] sources of noise in ST stats.

It is very, very challenging to make sense of a phenomenon when your metrics are diluted with 60% noise.  When you've got 60%, 70% noise, and you run a "study" (sic!) asking about the value of the stats, your study is going to suggest NO value to the metric unless you know how to design it.  

But I'll guarantee you that there is some "study" that will show the phenomenon being reflected at some level of consistency.  Give me all of the home run leaders in Arizona, past twenty years -- even first-week leaders -- and I'll guarantee that their presence is reflected in MLB regular-season slugging leaderboards, at some level of confidence.  It might be only a .20 correlation, but there will be a correlation.  Which is the same thing as saying that ST stats are not worth zero.

.

Q.  SSI thinks that spring performance is worth something?  How much?

A.  Not much, usually.  In a few cases it becomes critical.  

Kendrys Morales might be just tinkering.  Felix' results are unimportant, unless he walks a guy per inning.

Jason Bay vs Casper Wells is going to be important; both guys are playing for a job and both guys are out there competing.  Jon Garland and Jeremy Bonderman are going to be competing.

.........

The 2nd-worst thing you can do with such metrics is to assume that the noise is small - and to treat the metrics as though they are reliable with no further thought.  Many casual fans do this.  

I doubt any SSI or LL or USSM readers do this, and I hate to see the kind of condescension that assumes a web-literate fan to be unaware of the noise in ST stats.

...........

The VERY worst thing you can do is to assume that since you don't like a metric, and you don't know how to edit out the noise, and you don't know how to design a study in which correlations are found --- > then the metric must be worth ZERO.

ST stats aren't worth MUCH, in BULK.  But to imagine that every player's ST performance is irrelevant is incredibly naive.  If Jeremy Bonderman walks more guys than he strikes out, he's going home.  If he fans 20 guys and walks 2, then with the time off and re-boot, that means that he's probably going to have a good year by his own standards.

.

Q.  When are MLB teams going to wake up and ...

A.  Hold on right there, Turbo.  Just. Stop.

Consider this statement, from an author who higher on the very same web page had strictly forbidden any and all "appeals to authority":

.

Or, you know, you could use a projection based on more than just the last season’s data point. Which is what every good organization in baseball does.

.

I'll tell you something else every good organization in baseball every organization in baseball does.  Consider. Spring. Training. Performance.

We might slow down, draw a breath, and ask a simple question.

Why does every MLB organization watch ST performance carefully?  Why do all "good" MLB organizations (at least occasionally) make roster decisions based on spring results?

Org's don't just watch tools and skills.  They also watch performance and results.  All of them do. They understand about noise, understand with 100% crystal clarity.  They're out there watching, aware that ST very often misleads you, and still factoring ST performance as one variable in the equation.

Why would that be?

.

NEXT

Klat Categories: 

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <ul> <ol> <li> <i> <b> <img> <table> <tr> <td> <th> <div> <strong> <p> <br> <u>
  • Lines and paragraphs break automatically.
By submitting this form, you accept the Mollom privacy policy.

Shoutbox

Please log in or create an account to post shouts.
moethedogMuch of our OF speculation is dependent on Miller's role in '15 and beyond. It seems the M's think his footwork issues are serious enough to mean he must move off of SS, at least as a primary position. I can live with that as a transition to OF should be of no real challenge to him. So he goes to the OF. In '15 that would likely mean that he's the 4th OF and a BU SS. Lots of starts available as such, especially if he comes out of the OF blocks with some CF skills. If that is the Miller role, then Seth Smith/Rasmus talk is unneeded. But we still need that RH OF bat. Actually I would prefer we get a Smith/Rasmus guy AND Souza, still keeping Miller as an OF. It's Ackley I worry about. But whoever we get needs CF skills. Souza is going to be good. Kivlehan, too. Add miller to that group and you have a nice OF going forward.10 min 4 sec ago
SABR Mattdon't really care about defense as a value floor. If you don't hit, I don't want you.43 min 12 sec ago
okdanIf I remember right, Taro called his shot on Seth Smith several years ago, hoping the M's would target him. Looking at Souza, is he a similar kind of player? Perhaps that type of player template has a higher degree of working out. Certainly Souza's defense provides a nice floor of value in case his bat doesn't play as well.1 hour 29 min ago
TaroMontero's 2011 line in AAA for example was projected for a .261/.312/.435 line by ZIPS. He basically hasn't improved at all since then. Given his career line is .258/.302/.396 thats pretty close regressing for Safeco.8 hours 36 min ago
TaroBasically Montero was all projection, whereas Souza is good NOW with projection beyond that.8 hours 46 min ago
TaroI was huge on Montero's potential, and I was wrong. But to be fair Montero never put up the MLEs that Souza did. Souza projects as a 2-3 WAR player NOW based on what he did in AAA. Montero never did, he was incredible based on age related performance at his level and had massive power upside. But he never performed as well as Souza. His MLEs were in the 0-1 WAR range and he just never got better from there. Souza RIGHT NOW is above-average player. Montero never was.8 hours 48 min ago
moethedogI'm with Matt: Rios and Rasmus would be delightful! I'm with Taro: Souza will indeed be a player. But you could probably get both Rios and Rasmus for what Melky will cost per/year. 2X$8-$10M (with an option year) for Rasmus and something similar for Rios. Or just do Rasmus AND Souza, making all or us happy. You can get Souza fairly easily and sign Rasmus and have money to spend. Souza, btw, plays 1B, too. LoMo will need some breathers.9 hours 16 min ago
SABR MattHow much did you love Jesus Montero before we traded for him?9 hours 29 min ago
TaroJust think Souza is a player. I like his combination of athleticism, swing, and performance.9 hours 50 min ago
SABR MattRomero, Montero, Smoak, Peguero, Balentien, Clement...so...so many more. So many you won't possibly remember them all. And all but a scant tiny fraction of them have panned out. And that is not unique to the Mariners. There is a massive zero gravity moon step difference between an awesome AAA hitter and a proven major leaguer. All players have risks...but some are less risky than others. And for the record, Cruz's home run power transcends Safeco rather easily...and his hitter's park in Baltimore...has a mediocre at best HR factor to left. So...yeah...not worried about Cruz.10 hours 33 min ago
SABR MattTaro...love your insights and all...but how many guys have we promoted in the last 12 years who had great minor league numbers?10 hours 36 min ago
TaroEvery player you roster has some risk involved, included proven MLBers. Cruz will be a RH backspin 35 year old slugger with a career sub .330 OBP in hitters parks, and we've signed him for 4 years. I'd be more nervous about him than Souza in 2015, and especially beyond that. Souza has the performance in the high minors combined with insane athleticism. I think hes underrated due to being a football convert and a late bloomer.10 hours 46 min ago
SABR Mattyou don't take that sort of risk when this is the best chance you have of winning the WS. Souza can be a fourth outfielder option...not a starter.11 hours 9 min ago
GLSSouza is a risk, but a pretty good one.11 hours 11 min ago
SABR MattSouza is not reliable. That's not sufficient to this moment.11 hours 26 min ago
TaroI'm thinking Melky would want 'at least' 4 years gauranteed if hes looking for 5. If Ms stay hard at 3 they won't land him, vesting option for 4th or not. Just give him 5 with a lower AAV. 5/$55 or something in that range. Or deal for Souza (my favorite option).11 hours 57 min ago
SABR MattRasmus and Rios would be a fine platoon with CF helper potential and would be cheaper than Cabrera. I like Cabrera better even with more money involved, but Rios/Rasmus would also be fine with me.12 hours 4 min ago
Gordon GrossRios and Rasmus would be... extensive. By himself I'm not huge on Rios. A Rios/Rasmus platoon would be damaging. Playing with brass knuckles. I hope the Ms take the velvet gloves fully off this offseason. Zduriencik has waited half a decade to wade into the fight with fists flying - let's make it worth the wait. I'm fine with making Beane cry into his collapsing-budget cup o' coffee.12 hours 7 min ago
SABR MattRasmus. If you get Rios and Rasmus, the outfield aligns OK12 hours 44 min ago
moethedogThe Rios card might not bluff the Melky team guys: In '12 Rios was a heck of a player. In '11 he was a dud. In '13 and '14 he was basically a split heavy .280-.320-.420 guy with a pretty scraggly glove. There doesn't seem to be much Rios market out there right now, which might be a neat quality. You could probably get him for 2 years and not break the bank. Over the last two years Viciedo has hit RHP at a .738 and .689 clip. Rios has hit at .714 and .646 rate. Just saying. But he does continue to beat lefties up, so as a 4th OF-paired with Ackley, as Matt suggests, Rios would be great. He's been at .889 and .898 vL the last two years. That's rip-roaring. But it means that you need a RF who can play CF, as Rios no longer does.13 hours 41 min ago