Nate Silver, Mistaken on Overfitting
Here:
"In layman’s terms, an overfit statistical model is one that is engineered to match idiosyncratic circumstances in past data, but which is not an accurate picture and makes poor predictions as a result."
No, the problem is that it is too accurate a picture (of the past)! Instead, it is a more abstract but less accurate picture of the past which is more likely to look like the future, since it is only in certain abstract aspects that the past and future are likely to resemble each other. (and the art here is to find just which abstractions to use!)
"In layman’s terms, an overfit statistical model is one that is engineered to match idiosyncratic circumstances in past data, but which is not an accurate picture and makes poor predictions as a result."
No, the problem is that it is too accurate a picture (of the past)! Instead, it is a more abstract but less accurate picture of the past which is more likely to look like the future, since it is only in certain abstract aspects that the past and future are likely to resemble each other. (and the art here is to find just which abstractions to use!)
Looks like you and Daniel Kuehn need to email Nate Silver and set him straight: He too has been brainwashed by Karl Popper, and thinks predictive success is the criterion of a good model in the empirical sciences.
ReplyDeleteWell, predictive success *could* be the criterion of a good model, and this case it *is* Silver's criterion. (It was when I worked for the trading company as well.)
DeleteMy problem here is not that; it is that he has confused predictive success with accurate portrayal. A "rough sketch" of you at 15 may give people a much better idea of what you will look like at 45 then would a masterly portrait.
Right, I have no problem with your post here. I'm just complaining about you and Daniel from the Free Advice discussion of economic method.
Delete