SSI on the Value of Spring Training Performance

Sorry. It ain't zero.

.

Q.  Can you make any sense of spring training statistics?

A.  Different stats have different levels of "noise" in them.  Ichiro's career batting average has noise in it, but it's about 2%, not counting park context.  Ichiro's career range factor has noise in it, too, but the noise is something more like (?) 30%.

Baseball sabermetricians are really cruddy at estimating the AMOUNT OF noise in a particular statistic.  A sense of proportion is almost completely lacking in the industry.  Sabermigos are great at math; they're not so great at common sense.

We mean it in a good way.

.

Q.  How much noise is in there in spring training stats?

A.  Oh ... what would it be?  60%?  Something like that.  

One of the few sabermigos who does have a sense of proportion:  Jeff Sullivan.  Read this article by him.  He points out various [huge] sources of noise in ST stats.

It is very, very challenging to make sense of a phenomenon when your metrics are diluted with 60% noise.  When you've got 60%, 70% noise, and you run a "study" (sic!) asking about the value of the stats, your study is going to suggest NO value to the metric unless you know how to design it.  

But I'll guarantee you that there is some "study" that will show the phenomenon being reflected at some level of consistency.  Give me all of the home run leaders in Arizona, past twenty years -- even first-week leaders -- and I'll guarantee that their presence is reflected in MLB regular-season slugging leaderboards, at some level of confidence.  It might be only a .20 correlation, but there will be a correlation.  Which is the same thing as saying that ST stats are not worth zero.

.

Q.  SSI thinks that spring performance is worth something?  How much?

A.  Not much, usually.  In a few cases it becomes critical.  

Kendrys Morales might be just tinkering.  Felix' results are unimportant, unless he walks a guy per inning.

Jason Bay vs Casper Wells is going to be important; both guys are playing for a job and both guys are out there competing.  Jon Garland and Jeremy Bonderman are going to be competing.

.........

The 2nd-worst thing you can do with such metrics is to assume that the noise is small - and to treat the metrics as though they are reliable with no further thought.  Many casual fans do this.  

I doubt any SSI or LL or USSM readers do this, and I hate to see the kind of condescension that assumes a web-literate fan to be unaware of the noise in ST stats.

...........

The VERY worst thing you can do is to assume that since you don't like a metric, and you don't know how to edit out the noise, and you don't know how to design a study in which correlations are found --- > then the metric must be worth ZERO.

ST stats aren't worth MUCH, in BULK.  But to imagine that every player's ST performance is irrelevant is incredibly naive.  If Jeremy Bonderman walks more guys than he strikes out, he's going home.  If he fans 20 guys and walks 2, then with the time off and re-boot, that means that he's probably going to have a good year by his own standards.

.

Q.  When are MLB teams going to wake up and ...

A.  Hold on right there, Turbo.  Just. Stop.

Consider this statement, from an author who higher on the very same web page had strictly forbidden any and all "appeals to authority":

.

Or, you know, you could use a projection based on more than just the last season’s data point. Which is what every good organization in baseball does.

.

I'll tell you something else every good organization in baseball every organization in baseball does.  Consider. Spring. Training. Performance.

We might slow down, draw a breath, and ask a simple question.

Why does every MLB organization watch ST performance carefully?  Why do all "good" MLB organizations (at least occasionally) make roster decisions based on spring results?

Org's don't just watch tools and skills.  They also watch performance and results.  All of them do. They understand about noise, understand with 100% crystal clarity.  They're out there watching, aware that ST very often misleads you, and still factoring ST performance as one variable in the equation.

Why would that be?

.

NEXT

Klat Categories: 

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <ul> <ol> <li> <i> <b> <img> <table> <tr> <td> <th> <div> <strong> <p> <br> <u>
  • Lines and paragraphs break automatically.
By submitting this form, you accept the Mollom privacy policy.

Shoutbox

Please log in or create an account to post shouts.
mojicianGoodness. The thing that is worrisome about an unknown player like Castillo, is what if there is a fatal flaw in his swing, as in, he can't hit sliders away, or a jam pitch or what have you? Jeff Clement looked good to the scouts too. $72 million is a little bit steep to take a flier on some one. Of course, if he was playing against guys like Roenis Elias, then the Red Sox have nothing to worry about.51 min 3 sec ago
SABR MattRusney Castillo signs with the Red Sox for what is essentially a 6 year contract starting next year (it starts this year but for virtually no money up front) at 12 million per year. Ouchies. At that price, I don't think I match it. Let's gets Tomas though...we need bats.3 hours 6 min ago
Bat571And Erasmo goes 8 and gives up 3 R (2 ER) on 5 hits for the Win. 6 Ks on 100 pitches - sounds like a nice job! Apparently, the first run scored on a bad relay throw from Marte on a triple, so Marte has an up-arrow and a down-arrow for his first AAA error. But then again he's hitting .371 and OPSing near .900 ! Not bad for a scrawny MIF'er !13 hours 54 min ago
Bat571Tacoma gets 2 HRs tonight - from Noriega and Marte !!14 hours 2 min ago
SABR MattThis is why the slotting system is evil.22 hours 9 sec ago
mojicianThis new scheme might explain how the agent Close talked Aiken into turning down 5 million bucks when he had a questionable elbow ligament and was unlikely to make that much in the next draft. If Nix's case is strong, the Astros have no choice but to sign both Aiken and Nix at any legal price or face total draft destruction. MLB will approve the late signing because it will be unwilling to hammer the Astros on one hand, or deny the union its legitimate case on the other.1 day 2 hours ago
mojicianThe Astros draftgate is coming into clear focus. For those not following, the Astros didn't sign the first round pick, Aiken, which puts them over slot for their fifth round pick , Nix, which causes 100 percent fines and forfeiture of two future first round picks. They didn't pay Nix so this wouldn't happen. The union is going to arbitration against the Astros. The new rumor is, It is probable that both players could still sign with the Stros, which seems cheap compared to losing two first round picks when you are trying to sell the fans on rebuilding.1 day 2 hours ago
Bat571If the Angels can't keep up their winning with the remaining pitching, they're vulnerable. And the A's keep losing (2-8 in their last 10). But Paxton and Elias have got to keep the M's in games, too. But Felix-Young-'Kuma in Boston is as good as it gets, especially with the Sawx pitching in some disarray also. Time to pick up some ground.1 day 18 hours ago
SABR Mattwow...saw the highlight of that...he's done for the year, no question. brutal. Both Oakland and Anaheim showing us their keister...but if we play like we played today, we don't deserve to win.1 day 18 hours ago
MtGrizzlyReally ugly looking knee injury for Richards.1 day 19 hours ago
MtGrizzlyGarrett Richards got taken off the field on a stretcher. Angels pitching is a wreck.1 day 19 hours ago
SABR Mattwell this is a dog of a game. Mariners playing like the bad news bears again.2 days 26 min ago
mojicianFollowing Pax on Gameday: He can't control his curve ball, his first hit in the third was from a changeup that was both hanged and tubed, the Phillies aren't swinging at any of his off speed, and Pax's pitch count is at 59 at the end of the third. The fast ball is located at both sides of the plate and velocity is topping out at 96. This start has one run five n' dive written all over it. What's wrong with that? Puzzling Pax is still better than most teams' number two starter. We haven't seen enough of the Bartender in a while anyway.2 days 1 hour ago
SABR MattPhilly is considered by many in the game to be the dumbest organization, from top to bottom, in all of baseball. Lots of folks heavily criticize their manager for not using his brain, for picking favorites, for ruining quality starts, etc. And of course, we all know what people think of Ruben Amaro.2 days 4 hours ago
moethedogWas checking out the Southern League batting leaders to get an idea of how Kivlehan rates in that league (very well, thank you) and the year that Cubbies farm hand Kris Bryant is having jumps out at you. He's the #2 pick in the draft from '13. Between AA and AAA this year (40% of his PA's at AAA) he's .333-.441-.678 with 34 2B's and 40 HR's. He does K in 27% of his PA's but he walks in 14% of them, too. One scouting report gives his extrodinary power with the most explosive rotation sine Stanton. Plays 3B with a plus arm and quick reactions, so you can add him to Addison Russel + Javier Baez and the Cubs may have a heck of an IF pretty soon.2 days 5 hours ago
moethedogMinor league report: Romero is pretty amazingly hot! In his current 12-game hitting streak he is 26-53 with 4 2B's/2 3B's/4 HR's. In his last 21 games he has had 8 3-hit games and 5 more 2 hit games. His K rate is dropping. AAA #'s for the year: .371-.405-.716.... Nice! And with Blash in AA, Kivlehan played CF last night. Don't you get the idea that the M's are really high on him? Oh, Deej whacks two more homers as Bat pointed out.2 days 5 hours ago
OBFHaving watched Chris Taylor for a while... Brad Miller is not a very good defensive SS... :( Hopefully he refinds his bat...2 days 17 hours ago
OBFHe had looked... uncomfortable... that whole inning, doing more stretches than even usual... :/ Including some weird box squaty things... hopefully just a cramp or something...2 days 17 hours ago
MtGrizzlyI hope Iwakuma didn't hurt anything on that last play2 days 17 hours ago
OBFMean while Detroit storms back from down four to tie it up... :( :( :( come on Rays!2 days 18 hours ago