I thought you amigos would find this exchange pretty cotton-pickin' interesting. Any holdouts on that $3 a month over there? ;- )
.
That's not a comment on the veracity of his claims...but he's making a REALLY important and sweeping claim not providing a scientifically verifyable experiment to demonstrate his point, and accusing people of being willfully ignorant, intentionally misleading, or just plain stubborn for not immediately seeing his genius. Doc...you've barbequed me for less.
Check...not spending my money on his site.
Actually Bill would reply that he has been providing evidence for the reliability of MLE's for 30+ years... also today at BJOL:
In my view, I have been providing evidence for my position constantly for 30 years, and the world and the sabermetric community have been explaining it away and refusing to learn for 30 years, because it requires that people re-think their established assumptions. When Juan Gonzalez came to the majors in 1991 or 1992, we published projections for him that proved to be absolutely accurate. When Jason Heyward came to the majors in 2010, we presented projections for him that proved to be extremely accurate. . . .Jason Heyard, and Reid Brignac, and Trevor Crowe, and Ian Desmond. We publish very accurate projections for a dozen or more rookies every year in the Handbook. Why is this not evidence that it is possible to do this?
:daps: Matty
Calling people dishonest fools with silly positions seems a little bit out of line. I'm not sure that James would warrant full posting priveleges at SSI.
Jeff Clements minors and college stats show complete destruction of the minors at all levels. except for a third of a season of AAA that was only so-so.
Dude never had a bad day until he got to MLB.
Here Doc. Here's a Perry Mason moment to get your $3.00 worth out of your next encounter with James.
According to Page 443 of the "Bill James Handbook 2008", which I understand, use all of James' advanced methods to forecast baseball seasons, the projection for Clement's rookie year of 2008 was as follows: .254/.325/.430
Clement's actual 2008 year .227/.295/.360.
Further, after Clement has continued to flop in the majors, he has put up some excellent numbers in the minors.
Clements AAA slashline from 2012 was .276/.340/.486. That included 35 doubles 2 triples and 16 home runs in 459 at bats.
I'd love to hear from James why Clement's big hits never came in the bigs, when they have almost always been there for him at every other level.
Was about to highlight this quote as well, but for a different reason--it is an argument from credential and anecdote rather than from logic. Those are the same types of arguments that James teaches to reject.
James is saying the MiLB production is a VERY RELIABLE method of projecting MLB production. He says it is just as reliable as MLB production is.
Few players demonstrate a multi-year ability to rake in the minors and then don't hit in the majors. Few players just fall off the cliff (other than for age or injury) once they've hit in the majors.
I don't see that as an earth-shattering assumption.
Clearly there are examples like Jeff Clement.....but there are examples like Joe Charboneau or Jose Lopez, too.
He's right about Lopez. He was a decent hitter for a while. He demonstrated that ability at AAA as a precocious 20-yr old and then followed it up for several seasons in the majors. His collapse came after that. For 5 years, Lopez was generally a productive MLB hitter. For two years he was very productive. If Jason Bay produces like Lopez did in '08 and '09 we will all be giddy. Where did the projecgtions miss? It is, of course, an inexact science so there is some margin of error. Clements exemplifies this. But you don't completly write off the model because you have marginal errors.
Hamilton's past numbers are no guarantee he does it in California. they simply say, "barring injury or rapid aging...some level of + production is a very safe bet." He may still produce, like Pujols did...but it may be the worst MLB season of his career...like Pujols.
James didn't call Doc a "dishonest fool," either. The follow-up line was, "Well, of course, you are not; You are merely doing what all of us do all of the time."
So Myers will hit in the majors, but hitters the Royals have and Shields they didn't.
Franklin and Romero are pretty fair bets to be pretty fair hitters in the majors. Absolute locks? Nope, but two more years in the minors don't make that any more secure, either.
I'm struck by how many people here were aghast at the Myers trade, because they think his potential was wasted in a swap for a TOR near-ace. Yet there seems to be a driving energy around here for us to swap our Myers-types (Franklin, Romero, Paxton, Hultzen, Walker, Montero) for just the same type of proven MLB player.
But I digress......
I don't see much controversy in James' statements. Good whacking MiLB'ers (age arc guys) almost always are decent MLB bats. Great MiLB hitters almost always become, for some period, good MLB hitters.
There is some serendipity (or Dark Matter skill set) at play, of course. But the above general rule holds true.
If it doesn't, then we're dealing with a game that is a huge Heisenberg Uncertainty Principle.
If so, then we may as well swap Romero for Jose Lopez and bring him back.
moe
Supposing that I were to ask the evidence for macroevolution across species, the reply I'd get back would be very similar to James': Check the entire body of work we've done. When I asked, "Like what," no single exhibit, taken in isolation, would be at all convincing.
..........
AS A GENERAL RULE you don't want to accept argument on authority ALONE, and you're absolutely right that James himself preaches (too humbly) that nobody should give him any extra credit for his erudition.
AS IT APPLIES TO A SENSE OF HISTORICAL PROPORTION it would be pusillanimous to deny James' feel for what and what is not typical in baseball history. It's not the end of the discussion, but it's worthy of consideration.
I think you missed the follow-on paragraph. ;- )
............
I'm very receptive to your line of argument, my friend. Just musing here, but as you know, James would respond by pointing to players who failed other transitions, such as Crawford's transition from Tampa to Boston.
Of course when Dr. D says that he takes a chess Grandmaster's intuition seriously, he is not saying that he overrules a 25-ply Rybka calculation with this intuition. In fact, this is one of the most delicious situations in chess study, when a GREAT chessplayer believes a certain thing, and then "Truth" (the Rybka chess engine) disagrees.
And Dr. D takes Bill James' sense of proportion seriously, but it goes without saying that James' intution does not override proof to the contrary.
Does Tango provide such proof here?
I did a very long post in comparing forecasts (2007-2010) by several prominent forecasters (PECOTA, ZiPS, etc), and I had broken it down in several ways. One of the breakdowns was based on "past MLB experience". And the average error in the forecasts for veterans was lower than that of part-time players which was lower than "pure rookies" (no MLB experience). In fact, the amount of error for the pure-rookies forecast was HIGHER than simply giving every pure-rookie an identical league-average forecast. It's extremely long, but this is about as detailed a test of forecasting systems that I've ever done. http://www.insidethebook.com/ee/index.php/site/article/testing_the_2007_...
Asked by: tangotiger
Answered: 12/14/2012
Thanks. I'll go look.
Most every person sleeps better in their own bed. This has been established scientifically and makes a good bit of sense. People also enjoy stories more when they are experiencing it for at least the second time. Human beings seem to thrive when they are confident their concept of the near future will closely resemble reality.
Perhaps this is one of the reasons why the contract extensions (check me on this) generally work out better than big, splashy FA adds as Sandy has argued.
Interesting as well, is the notion that a player's ability to cope with new surroundings and circumstances, as newly added FA must, carries such a great weight with James.
If this is a lightbulb, I kinda get keeping your draft picks and extending those who flourish. Also, if this is a thing, why haven't the M's benefitted from having a very localized AAA club?!
James cannot go out there and call to his own authority of having been clamoring for some greater truth for 30 years and pointing to his best examples and expect me to buy it as more than arrogant, self-serving bluster. What is needed is a study. A real study using a lot of data. What is needed is a comparison of the standard performance change magnitude (pick your favorite win stat or FIP/EqA type number and see how much it changes for guys in transition in the big leagues...and then compare that to the deltas between MLE-based projections and actual performance at the big league level for transitioning prospects). Until he does that, he just sounds like a jerk puffing his chest and proclaiming that he is right and everyone else is just not willing to see his glory.
...and gets a result that is contrary to James' wild-eyed, uber-arrogant bluster. That...is QED.
My take on this is one I tend to agree with James' position.
The trend I see most often is a tendency to OVERSTATE the reliability of MLB production over time/context changes.
I've been specifically arguing that age related decline is substantially under-appreciated by the masses for a number of years, (but it seems like every year there some new 30-year-old slugger who nearly everyone is convinced will continue raking for another 6 years).
I think James' foundation position is built on a strong foundation of understanding not just the reliability of MLEs ... but as much (or more), on the understanding that MLB production "givens" aren't nearly as "given" as is generally accepted.
I think his point on Lopez is extremely telling. Jose puts up 104 and 103 OPS+ seasons at age 24/25, yet his subsequent crash and burn is pointed squarely at his minor league performance, with little pause to ask - "how off were the projections of what was to come based on his '08/'09 MLB production?
You'd think that the fans who had watched the following players (among others) would have a better appreciation for how "weak" MAJOR league projections really are:
Lopez - crash
Sexson - burn
Beltre - sputter and re-ignite later
Ibanez - FLAME-ON!
Guti - random disease
I'm reading James' point not as a claim that MLEs are unerringly accurate ... but that they are every bit as accurate as MLB projections are.
Looking no further than the Ms 2012 roster, Olivo, Ryan, and Ichiro were all substantially below their MAJOR LEAGUE production levels (and projections).
IMO, the over-selling of the reliability of MLB Veterans (TM) is really at the core of the James' gripe. After all the angst and ire and grousing about the Ms not adding a "reliable veteran" or two in 2012 ... it was, in fact, the "reliable veterans" that were the root cause of the hitting woes for the 2012 team.
In the final analysis, I tend to side with James. MLEs are more reliable than most appreciate *AND* MLB production is far less reliable than is generally appreciated. That combination makes the skew between "prospects" and "veterans" seem larger than it really is.
I went and read the text...it's lengthy because there's a complex experimental design to explain, but the idea is to capture how reliable veteran MLB performance is and then capture how well the projection schemes do with various types of players. And the conclusion is that minor leaguers are so incredibly unpredictable that you do a better job predicting them if you assuming all of them will be AVERAGE players. That is the equivalent of having a weather prediction model and saying it would be more accurate if it always forecast precisely climatological averages for all grid points. That says that minor leaguers are are not projectable AT ALL whereas big leaguers are and veteran big leaguers are VERY projectable.
Ghost,
What I see is that wOBA for pure rookies is predicted (by marcel) to be .335 and comes in at .319. This is indeed a bit greater than the error in estimated production of other classes of players (all MLB'ers), but I would suggest that it is still pretty reliable. As well, the hiccup may be in the model and the way MILB numbers are crunched, not in the reliability of the crunching.
But I may have missed something here.
moe
First, the positive thing. I agree that it is possible we are not doing a good job understanding how minor leaguers are likely to translate to the majors. There are new discoveries every year regarding things like K rate boundaries (which types of .500 SLG minor leaguers are likely to hit in the big leagues?), contact proficiency (BABIP) and correlation to big league success, which type of power is better (lots of doubles or few doubles and lots of HRs)? And for pitchers, the same story is unfolding, but more slowly due to complications with injury. Many of the current projection schemes are too simplistic and do not account for the real reasons that production changes when you change leagues.
Second thing:
But that still does not mean that major league projections and statistics are no more reliable than minor league. While there remains uncertainty as to how best to project minor leaguers, those projections should still be considered less reliable...and Tom Tango finds that, in fact, they are less reliable. Statistically significantly less reliable to a p-value of basically 0. It may not SEEM like a lot to go from .335 average projection to .319, but that is actually a *MONUMENTAL!!* shift when you consider how many player seasons were in that sample. The weather analogy would be if the US generated global forecast had a global mean sea-level pressure bias of +0.5 millibars. Atmospheric pressure is about 1013 mb, so 0.5 seems tiny, but if you run our model for a year and it's always high by half a millibar by day five, there is something really REEEALLY wrong with the model. Something so wrong that if you just gave the climo projection for pressure, you'd have less bias and thus a better forecast. What Tango is saying is that the 16 point wOBA error is a gigantic bias given the sample size and that you actually do better if you simply assume that all minor leaguers will hit the same in their first big league season as the average minor leaguer does in reality.
I like the point that players are who they are and don't change from one level to another. Will Myers in AAA on June 10 is the same player as Will Myers in MLB on June 11.
Some players are more able to adapt to new situations, which a player will encounter in any transition. But like Dr. D has argued before, Michael Pineda was one of the best starters in MLB before he even got there. Mike Trout had the ability to be an MVP-caliber major leaguer when he was in AAA. And this is what Saberdogs lose sight of.
It seems James thinks that Myers is an upper-echelon talent regardless of where he plays.
Couldn't help but wonder from his tone, though, if his history as a Royals fan had any bearing on his opinion. ;-)
A) Players that succeed at high levels at especially young ages are solid projections compared to B) MLB mainstays that have undergone a transition between seasons (ie; traded or changed teams as free agents). Therefore, Tango's study doesn't apply since it takes all minor leaguers and compares their projections to all major leaguers. The study that would be useful would be to compare the projections of traded and free agents major leaguers to the top 100 prospects lists projections.
James was not actually just talking about great minor leaguers. He was making a MUCH broader claim about all prospects. If he was talking about a specific group of uber-prospects, then he failed to convey it and it made him sound more confident than he should have sounded, but I got this message from it "I do projections for minor leaguers, They are awesome. That is all the proof I need that my way of thinking - that minor league stats are every bit as predictive as major league stats - is right...and the fact that anyone would dare question me on that suggests they are either stupid, willfully ignorant, or just darned stubborn."
If he'd said what you actually said, there would be no controversy for me...I would still disagree pending the results of the survey I proposed earlier in this thread (which is essentially the same as the one you propose). But he didn't say what you say he said...he said something much more broad and far less humble.
James was not actually just talking about great minor leaguers. He was making a MUCH broader claim about all prospects. If he was talking about a specific group of uber-prospects, then he failed to convey it and it made him sound more confident than he should have sounded, but I got this message from it "I do projections for minor leaguers, They are awesome. That is all the proof I need that my way of thinking - that minor league stats are every bit as predictive as major league stats - is right...and the fact that anyone would dare question me on that suggests they are either stupid, willfully ignorant, or just darned stubborn."
If he'd said what you actually said, there would be no controversy for me...I would still disagree pending the results of the survey I proposed earlier in this thread (which is essentially the same as the one you propose). But he didn't say what you say he said...he said something much more broad and far less humble.
I absolutely do not see Tango testing James' biggest claim -- that the high rate of collapse for prospects is because of environmental transition (things like new park, coaching staff, etc.). From what I see, Tango's variables take absolutely no account for changing team or organization.
In fact, I dare say that his study is absolutely nothing new and that numerous others have done the same (or at least it seems so to me). I would infer that it is because the concept James has brought up is still completely over everyone else's head that they can't even hear what he's saying and can only guess at it.
In my interpretation, James' concept here seems to revolve around this anchoring question: "Is there a correlation between established MLB player collapses and established MiLB collapses? If so, what is it?" In Mariner-land terms, "What if the reason(s) Rich Aurilia, Scott Spiezio, and Chone Figgins collapsed is the same reason(s) for Jeff Clement?"
Anecdotally -- and I'm certain James has extensively tested and studied this hypothesis -- it seems that established MLB players collapse most often when they sign large FA contracts with new teams after having significant success with only one organization and prospects fail the most often when they have little-to-no adversity in the Minors.