The Delicacy of Scientific Testing

This article is worthy of careful consideration. Consider these quotes:

Scientists have substantial evidence that dark matter exists and is at least five times as abundant as ordinary matter. But its nature remains a mystery.

“If it’s really dark matter, many other experiments should have seen it already,” says Thomas Schwetz-Mangold, a theoretical physicist at the Karlsruhe Institute of Technology in Germany—and none has.

“The modulation signal is there,” says Kaixuan Ni at the University of California, San Diego, who works on a dark-matter experiment called XENON1T. “But how to interpret that signal—whether it’s from dark matter or something else—-is not clear.”

The fact that many have tried and failed to repeat DAMA’s experiment shows that it is not easy, says Elisabetta Barberio at the University of Melbourne, who leads the Australian arm of SABRE. “The more one looks into their experiment, the more one realizes that it is very well done.”

So scientists have evidence of a lot of something... they know not what... kicking around the universe. Not knowing what the thing is they are looking for is, it is hard to know how to find it. They have some experiments that indicate the experimenters have found something unknown... but that unknown thing might or might not be the other unknown thing they are looking for. And other experimenters have had a very hard time duplicating those results... which leads them not to believe the original experiments were flawed, but that they were very well done!

I am not making fun of this enterprise! I am offering it as an example illustrating how inadequate any naive theory of falsification must be.