Who Were the Sea Peoples?

The α = .05 cutoff for "significant" results is a case of spurious "objectivity" trumping scientific judgment. The fact that scientists have an "objective" standard to adhere to gives the appearance of being more rigorous. But consider another objective way of deciding between the "null hypothesis" and the hypothesis being tested: flip a coin. Heads, we reject the null hypothesis, tails we don't. Completely objective! We could videotape the coin flip, and all sane observers could agree as to whether we got heads or tails. Next, think about the following two cases: We do a study and find that reckless driving correlates with early death with p = .08 (greater than α). We are told to accept the null hypothesis: there is no significant correlation. We do a study and find that sunspot activity correlates with American League victories in the World Series with p = .04 (less than α). We are told to reject the null hypothesis: there is a sig...

The findings of statistical studies are usually considered "significant" when there is smaller than 5% probability that their findings were the result of mere chance in the selection of a sample to study. Keep that in mind, and let's first just consider sociologists: the American Sociological Association claims 21,000 members in its various sub-groups. Let us guess (the exact numbers don't matter for my point) that each member undertakes two statistical studies per year, and half of those show a significant correlation. That means that by chance alone, this group will produce over a thousand studies per year which appear to show a significant correlation between different phenomena, but in which the significance was really only the result of the luck of the draw in picking a sample to examine. Next let us turn our attention to the bias that exists in academic journals towards results that are  positive (no one cares much about studies that show no connectio...

We can always data mine for correlations until we get one we like. So when a socialist does a "study" proving that capitalists are psychopathic, or a libertarian "demonstrates" that lax environmental regulation makes us healthier... just ignore them. (These things might be true, but you can figure the studies themselves are worthless.) The only studies that one should pay any attention are one's where the study's author expected the opposite result, but reluctantly reports that the study contradicted his expectations. So when Roland Fryer set out to show bias in police shootings, but found none, that is worth paying attention to. Similarly, when Gregory Clark attempted to show that social mobility had increased in modern times, but found it hadn't, again, that is worth noting.