SSI on the Value of Spring Training Performance

Sorry. It ain't zero.

.

Q.  Can you make any sense of spring training statistics?

A.  Different stats have different levels of "noise" in them.  Ichiro's career batting average has noise in it, but it's about 2%, not counting park context.  Ichiro's career range factor has noise in it, too, but the noise is something more like (?) 30%.

Baseball sabermetricians are really cruddy at estimating the AMOUNT OF noise in a particular statistic.  A sense of proportion is almost completely lacking in the industry.  Sabermigos are great at math; they're not so great at common sense.

We mean it in a good way.

.

Q.  How much noise is in there in spring training stats?

A.  Oh ... what would it be?  60%?  Something like that.  

One of the few sabermigos who does have a sense of proportion:  Jeff Sullivan.  Read this article by him.  He points out various [huge] sources of noise in ST stats.

It is very, very challenging to make sense of a phenomenon when your metrics are diluted with 60% noise.  When you've got 60%, 70% noise, and you run a "study" (sic!) asking about the value of the stats, your study is going to suggest NO value to the metric unless you know how to design it.  

But I'll guarantee you that there is some "study" that will show the phenomenon being reflected at some level of consistency.  Give me all of the home run leaders in Arizona, past twenty years -- even first-week leaders -- and I'll guarantee that their presence is reflected in MLB regular-season slugging leaderboards, at some level of confidence.  It might be only a .20 correlation, but there will be a correlation.  Which is the same thing as saying that ST stats are not worth zero.

.

Q.  SSI thinks that spring performance is worth something?  How much?

A.  Not much, usually.  In a few cases it becomes critical.  

Kendrys Morales might be just tinkering.  Felix' results are unimportant, unless he walks a guy per inning.

Jason Bay vs Casper Wells is going to be important; both guys are playing for a job and both guys are out there competing.  Jon Garland and Jeremy Bonderman are going to be competing.

.........

The 2nd-worst thing you can do with such metrics is to assume that the noise is small - and to treat the metrics as though they are reliable with no further thought.  Many casual fans do this.  

I doubt any SSI or LL or USSM readers do this, and I hate to see the kind of condescension that assumes a web-literate fan to be unaware of the noise in ST stats.

...........

The VERY worst thing you can do is to assume that since you don't like a metric, and you don't know how to edit out the noise, and you don't know how to design a study in which correlations are found --- > then the metric must be worth ZERO.

ST stats aren't worth MUCH, in BULK.  But to imagine that every player's ST performance is irrelevant is incredibly naive.  If Jeremy Bonderman walks more guys than he strikes out, he's going home.  If he fans 20 guys and walks 2, then with the time off and re-boot, that means that he's probably going to have a good year by his own standards.

.

Q.  When are MLB teams going to wake up and ...

A.  Hold on right there, Turbo.  Just. Stop.

Consider this statement, from an author who higher on the very same web page had strictly forbidden any and all "appeals to authority":

.

Or, you know, you could use a projection based on more than just the last season’s data point. Which is what every good organization in baseball does.

.

I'll tell you something else every good organization in baseball every organization in baseball does.  Consider. Spring. Training. Performance.

We might slow down, draw a breath, and ask a simple question.

Why does every MLB organization watch ST performance carefully?  Why do all "good" MLB organizations (at least occasionally) make roster decisions based on spring results?

Org's don't just watch tools and skills.  They also watch performance and results.  All of them do. They understand about noise, understand with 100% crystal clarity.  They're out there watching, aware that ST very often misleads you, and still factoring ST performance as one variable in the equation.

Why would that be?

.

NEXT

Klat Categories: 

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <ul> <ol> <li> <i> <b> <img> <table> <tr> <td> <th> <div> <strong> <p> <br> <u>
  • Lines and paragraphs break automatically.
By submitting this form, you accept the Mollom privacy policy.

Shoutbox

Please log in or create an account to post shouts.
mojicianActually, scrap all that. Peacock has a reverse platoon split, and what about "Deep Count" Austin Jackson? That's why I don't hold the pencil.42 min 13 sec ago
mojicianThe Mariners should get back to their lefty roots today, my lineup: 1. Chavez CF, 2. Ackley LF, 3. Cano, 2B, 4. Seager 3b. 5. LoMo 1B, 6. Morales DH, 7. Condor, RF, 8. Sucre (the token righty) 9. Miller.45 min 8 sec ago
mojicianSome stuff about Brad Peacock: His season average is 4.74 IP per outing (This is skewed by four relief appearances) but the point is that he rarely pitches 6 innings. He has 8 six inning starts on the year, and most of these were in May. His last six inning game was on September 2, against the Angels. Peacock is a certified fivendiver. So the game plan is, we chase out Peacock with OBP in the fourth, and crack into that sweet Houston bullpen. It looks good on paper.59 min 42 sec ago
SABR MattThe only reason we're in a position to win at all right now...is the total collapse of the As.2 hours 27 min ago
merksSodo Lomo5 hours 18 min ago
IcebreakerXDaddyO: I'm scared of Young in the Juice Box. That's the matchup to fear. I'm okay going in with Walker though.5 hours 33 min ago
DaddyOrick82, "I don't want to face the Astros" -- ditto. If the M's are to sink into a hopeless position, it would be more fitting that it be at the hands of the Astros than those of the Angels. I can well imagine this offense scoring just 3 to 5 runs in the 3-game series. Let's hope not. "One game back...Could we actually do this?" -- Despite my predilection towards pessimism WHEN IT COMES TO THE M'S, a reasonable predilection based on a LONG history of failure, I guess the answer is, "Yes." I mean, all year long they have slid to the edge of the abyss, and every time just when it looks like they are doomed they hold on instead.5 hours 39 min ago
DaddyOThis team has more lives than all the cats of a cat hoarder put together. LoMo's shot was a bolt of lightning in a doldrum sea. (Well that metaphor is really not apt, because James Jones took a Kendrys Morales (!) base hit and turned it into an opportunity with his baserunning.) Anyway, we were all giving up on this team AGAIN until Jones was inserted as a pinch runner at first with two out in the 9th down 1-0. Morrison somehow magically transformed Jones' spark into the bolt that revivified the Seattle Mariners. Of course, Rodney AGAIN had to make a 3-run lead into a white knuckle affair before inducing the final out.5 hours 48 min ago
rick82One game back! Wow! Could we actually do this? Dang, I don't want to face the Astros. We get their good pitchers, too.14 hours 29 min ago
rick82Just glad we found a way to beat the Bees.14 hours 31 min ago
rick82That ump was terrible, and forced the M's to swing at junk.14 hours 31 min ago
mojicianLoMo!!!!14 hours 45 min ago
SABR MattPHHHHEEW...score some farking RUNS NOW.15 hours 1 min ago
SABR MattRetarded to walk Boesch to get to their only good hitter. Absolutely retarded.15 hours 6 min ago
mojicianThis is unbelievable.15 hours 38 min ago
SABR MattThis ump is terrible.16 hours 12 min ago
SABR Mattwell I said that before the game, Grizz...that if they couldn't win this, they shouldn't do anything worthwhile.16 hours 20 min ago
SABR MattI can't believe how Felix looks precisely identical when he's facing this cast of clowns as when he faces 1H Oakland when they were scary.16 hours 20 min ago