by Matthew Richter
Lately, there have been many articles, posts, podcasts, and social media videos highlighting the failure of research and science. Many are joyously pointing out the potential fraud stemming from work in behavioral economics and social psychology. And let’s not even get into the whole vaccine issues. Many are lambasting the crisis in peer-reviewed journals. Others are lamenting that concepts they have embraced, used, and sold in their consulting practices are now labeled dubious. And many are using these examples as proof that science, that research itself, isn’t reliable. That research is and has never been practical in contrast to the real work done in the field. Heck, more and more I hear, “If it works, why shouldn’t I use it, despite what research may say?”
My response: Science, and specifically the scientific method, are indeed working! Maybe not perfectly, but perfect is an impossible bar to set given how long and how many need be involved for good research. But, without replication, without good statistical modeling, we don’t catch the issues that were recently raised. In fact, without science learning more and more over time, we don’t ever grow and learn. Check out the linked (Planet Money podcast) (https://www.npr.org) for a wonderful summary of some of the current research crises.
It is the very methods, when done well that enable us to continue to challenge and question research. To not accept a single paper as gospel. Science caught these issues, as exactly it should. And independent researchers implementing good scientific methodology ensured it happened. To catch poor methodology which does exist, to identify bad analysis, and to mediate politics that get in the way of good science.
To be sure, we also have an interpretation issue, as well. My friend, Clark Quinn, called these misconceptions, where practitioners and journalists mischaracterize, short shrift, and misapply research. Or, that they ignore stated limitations and overgeneralize what may have been learned in a study.
Good scientific methodology relies on empiricism. Empiricism is the usage of processes and methods that control for the potential incorrectness, rationalism, and particularly, intuition, can lead to when observing our world. Replication, for example, is one way in which we catch errors. A single study or two does not make solid science. It’s merely a start. Good scientific methodology requires us to posit an idea called a hypothesis. Contrary to popular views of science, we don’t try to prove the hypothesis. We try to disprove it. We try to falsify it. And the more times we fail to fail, the more likely our hypothesis could be right. Our goal in science is to be less wrong. But we will never be absolutely right. The more ways we try to break the original idea, the better. But we can never prove a hypothesis. We only fail to disprove it. Hence sampling size, population demographics, factoring, various forms of validation, and other research-based tools help mitigate the “But, it works…” mindset highlighted above when it really does not.
What this means is that over time, scientific principles and ideas that look good initially fall by the wayside. But new ones build on what we learn. We get better. We get better through the controlled process of science— the intentional failure of an idea--or the inability to disprove an idea. Both are good! So, when we do disprove something g (like Learning Styles affecting learning outcomes), or other models or tools we embrace in our work, this isn’t a bad day. Rather, it is a day to celebrate. We celebrate because we learned something new, and we got closer to ultimate truth. But truth in science is like cutting a line in half repeatedly. There will always be another half to cut again. And another half still again. In other words, we never stop cutting in half. But when that line is so small you cannot see it anymore, we are probably good enough to accept the hypothesis as a scientific theory. Scientific theories, for all practical intents and purposes are day-to-day truths.
The bottom line is when we learn something is not supported by science after thinking maybe it was, we should celebrate. Throw a party! When we find out something we have long embraced and feels vital to our current work is nonetheless not supported by research, we should explore why, ask questions, and validate. Then celebrate! It means science is working exactly as it should be.
So, the next time someone knocks the good old scientific method, or claims research processes are stupid, or that we can no longer rely on science, ask them to show how science has actually been disproven.
What do you think?