SSI on the Value of Spring Training Performance

Sorry. It ain't zero.

.

Q.  Can you make any sense of spring training statistics?

A.  Different stats have different levels of "noise" in them.  Ichiro's career batting average has noise in it, but it's about 2%, not counting park context.  Ichiro's career range factor has noise in it, too, but the noise is something more like (?) 30%.

Baseball sabermetricians are really cruddy at estimating the AMOUNT OF noise in a particular statistic.  A sense of proportion is almost completely lacking in the industry.  Sabermigos are great at math; they're not so great at common sense.

We mean it in a good way.

.

Q.  How much noise is in there in spring training stats?

A.  Oh ... what would it be?  60%?  Something like that.  

One of the few sabermigos who does have a sense of proportion:  Jeff Sullivan.  Read this article by him.  He points out various [huge] sources of noise in ST stats.

It is very, very challenging to make sense of a phenomenon when your metrics are diluted with 60% noise.  When you've got 60%, 70% noise, and you run a "study" (sic!) asking about the value of the stats, your study is going to suggest NO value to the metric unless you know how to design it.  

But I'll guarantee you that there is some "study" that will show the phenomenon being reflected at some level of consistency.  Give me all of the home run leaders in Arizona, past twenty years -- even first-week leaders -- and I'll guarantee that their presence is reflected in MLB regular-season slugging leaderboards, at some level of confidence.  It might be only a .20 correlation, but there will be a correlation.  Which is the same thing as saying that ST stats are not worth zero.

.

Q.  SSI thinks that spring performance is worth something?  How much?

A.  Not much, usually.  In a few cases it becomes critical.  

Kendrys Morales might be just tinkering.  Felix' results are unimportant, unless he walks a guy per inning.

Jason Bay vs Casper Wells is going to be important; both guys are playing for a job and both guys are out there competing.  Jon Garland and Jeremy Bonderman are going to be competing.

.........

The 2nd-worst thing you can do with such metrics is to assume that the noise is small - and to treat the metrics as though they are reliable with no further thought.  Many casual fans do this.  

I doubt any SSI or LL or USSM readers do this, and I hate to see the kind of condescension that assumes a web-literate fan to be unaware of the noise in ST stats.

...........

The VERY worst thing you can do is to assume that since you don't like a metric, and you don't know how to edit out the noise, and you don't know how to design a study in which correlations are found --- > then the metric must be worth ZERO.

ST stats aren't worth MUCH, in BULK.  But to imagine that every player's ST performance is irrelevant is incredibly naive.  If Jeremy Bonderman walks more guys than he strikes out, he's going home.  If he fans 20 guys and walks 2, then with the time off and re-boot, that means that he's probably going to have a good year by his own standards.

.

Q.  When are MLB teams going to wake up and ...

A.  Hold on right there, Turbo.  Just. Stop.

Consider this statement, from an author who higher on the very same web page had strictly forbidden any and all "appeals to authority":

.

Or, you know, you could use a projection based on more than just the last season’s data point. Which is what every good organization in baseball does.

.

I'll tell you something else every good organization in baseball every organization in baseball does.  Consider. Spring. Training. Performance.

We might slow down, draw a breath, and ask a simple question.

Why does every MLB organization watch ST performance carefully?  Why do all "good" MLB organizations (at least occasionally) make roster decisions based on spring results?

Org's don't just watch tools and skills.  They also watch performance and results.  All of them do. They understand about noise, understand with 100% crystal clarity.  They're out there watching, aware that ST very often misleads you, and still factoring ST performance as one variable in the equation.

Why would that be?

.

NEXT

Klat Categories: 

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <ul> <ol> <li> <i> <b> <img> <table> <tr> <td> <th> <div> <strong> <p> <br> <u>
  • Lines and paragraphs break automatically.
By submitting this form, you accept the Mollom privacy policy.

Shoutbox

Please log in or create an account to post shouts.
IcebreakerXI can't wait until the Astros sweep us.1 hour 2 min ago
rick82There's a reason Farquhar passed all these guys up last year and became the closer. He asserted himself. Why he takes a back seat to Tommy, Charlie and Yoenis is a mystery to me.2 hours 2 min ago
Gordon GrossOur pitching has actually been pretty decent in the early going, Rick. It's our offense that is - once again - betraying us after that opening series. Gotta get the bats on track. Losing a couple close heartbreakers has changed the complexion of the early season for us. The Rangers have walked off at least 4 times in victory, and we've suffered 3 walkoff losses and a couple other heartbreaking late inning defeats. We pull those out and it's all roses while we wait for our pitching reinforcements. Now, though, we need hitters to hit - a lot. Gotta take this Astros series.3 hours 7 min ago
GLSI don't mind Furbush so much, but he's never been that guy, that when he comes in the game, that you really feel good about it. I think he has a role, but to me he's more of a 6th/7th inning guy. I really don't Furbush, Wilhelmsen, or Medina showing their faces in the 8th inning of a close game.3 hours 8 min ago
Gordon GrossI like Charlie, but he's been a slow starter in the past and is continuing that this year. It's okay that it takes him til May to get his sea-legs, but that doesn't mean I want him pitching a bunch of important April innings then. He's also never been great against righties, though he was decent last year. I don't pivot my pen around Furbush in the early going, that's for sure.3 hours 11 min ago
rick82But yeah, if you'd have told me that Elias would by default be our #2 starter on April 20, I would have expected a 7-11 record.4 hours 5 min ago
rick82Thirteen: I don't see it. It's been a while since he's been "one of the best". He's been pretty middling for about half a season at least. And I don't see any evidence of him returning to 2012 form. He has earned his way to long relief.4 hours 8 min ago
ThirteenGuys, calm down! Furbush hasn't even thrown six innings yet, and over the last two years he's been one of the best LH setup men in the sport. Don't bail on a LH setup man after six bad innings!4 hours 23 min ago
rick82I don't even want a LOOGY, if that LOOGY's name is Furbush.5 hours 4 min ago
rick82It's Farquhar, and Leone. Demote those other losers and make them earn their way back.5 hours 5 min ago
GLSWe really need that guy that can build a bridge to the closer late in the game when the score is close. Maybe that's Farquhar, I don't know really. But it sure seems like it isn't Wilhelmsen.5 hours 8 min ago
rick82These are the times that try my patience.5 hours 11 min ago
Gordon GrossLloyd for some reason thinks Wilhelmsen is better than Farquhar.5 hours 18 min ago
Gordon GrossYeah, we don't have to worry about late-season heartbreak with the M's of the last half-decade. They take care of that nonsense early.5 hours 35 min ago
GLSWhy did Lloyd put TW in for the 8th?5 hours 40 min ago
SilentpadnaYour 2011...er 2012...er 2013...er 2014 M's. In a game of inches, we give them all away in April and May. Song remains the same doesn't it? We will squash your hope as early as possible. NFL draft in three weeks.....5 hours 42 min ago
Gordon GrossYour 2014 Ms. In a game of inches we keep coming up a foot short.5 hours 48 min ago
Gordon GrossAnd now, not a strike to be found in all of Marinerland.6 hours 5 min ago
phxterryWhat happened to TW's hot fastball? Now he's throwing 90-94 instead of 94-97. And he still can't throw it for a strike.6 hours 8 min ago
Gordon GrossTom throw 4 strikes and a ball and somehow a runner walks to first. Then he gets a groundball for a double-play or force but we screw that up. If it weren't for bad luck we'd have no luck at all these days.6 hours 9 min ago