Posts tagged replication

Science takes place in a Madisonian system, where competing forces push us from all directions. The good guys don’t always win. The powerful guys push hard. The direction science moves depends on the sum of the forces on it. A well meaning push in the right direction doesn’t always end up moving science that way. The reliability project and the open science foundation have recently come into a bit of power and public influence. So far, they’ve pushed for greater reliability and more careful methods. But they’ve yet to ask “Why are scientists using poor methods?”, “What are the forces that drive reliability down?”, and “Why were past methods more reliable?” The surge in retraction is relatively new. There wasn’t an OSF in 1957, and yet, somehow my grandfather was able to publish a reliable effect.

via https://medium.com/@patrickdkwatson/better-incentives-better-science-bb0f13c2c3cb

Those who reject the 97% expert consensus on human-caused global warming often invoke Galileo as an example of when the scientific minority overturned the majority view. In reality, climate contrarians have almost nothing in common with Galileo, whose conclusions were based on empirical scientific evidence, supported by many scientific contemporaries, and persecuted by the religious-political establishment. Nevertheless, there’s a slim chance that the 2–3% minority is correct and the 97% climate consensus is wrong. To evaluate that possibility, a new paper published in the journal of Theoretical and Applied Climatology examines a selection of contrarian climate science research and attempts to replicate their results. The idea is that accurate scientific research should be replicable, and through replication we can also identify any methodological flaws in that research. The study also seeks to answer the question, why do these contrarian papers come to a different conclusion than 97% of the climate science literature?

http://www.theguardian.com/environment/climate-consensus–97-per-cent/2015/aug/25/heres-what-happens-when-you-try-to-replicate-climate-contrarian-papers

Relaxing the null hypothesis makes for great storytelling, but it’s an unsettling way to live. Which information and which perspectives we take into account, when we try to decide whether a pattern we’ve matched is real or an apophenic false alarm, affects our ability to determine whether something has gone away or whether we’ve just stopped believing in it. In the praxis of science we try to keep the false alarm rate down with things like study size and statistical inference and meta-analyses and replication, and we still get it wrong a lot.

https://medium.com/@maradydd/the-null-hypothesis-loves-you-and-wants-you-to-be-happy–3189413d8cd0