Inspired by some LinkedIn posts from my mentor Matthew Richter, I dug up this previously-publsihed article.
I am skeptical of books that claim to be evidence-based. A careful review of these books reveals that the empirical evidence involves talking to three employees about their subjective feelings of improved efficacy.
We need a more precise definition of what authors mean by evidence-based.
During my graduate-school days, while trudging through statistics textbooks, I came across Robert Abelson’s provocative book, Statistics as Principled Argument. This book contains a memorable acronym MAGIC that specifies the criteria for compelling empirical evidence.
The M in the acronym stands for Magnitude. Since I am not constrained by a cute acronym, I prefer to call this criterion size. This basically suggests that bigger effects are better. If I have a technique that enables me to memorize 52 playing cards, it has a bigger and better effect than a technique for memorizing five cards.
The A in the acronym stands for Articulateness. If you prefer, we can label this criterion precision. Recalling the sequence of a shuffled deck of 52 playing cards by naming cards at specific positions and pinpointing the sequential numbers of different cards is a precise definition that meets the articulateness criterion.
The G in the acronym stands for Generality. It identifies for how broadly we can generalize the empirical conclusion. If our intervention works effectively only with middle-aged Asian men, it is not as impressive as an intervention that works with all employees.
The I in the acronym stands for Interestingness. This identifies the potential of the empirical finding to change people’s beliefs. If my intervention changes people’s beliefs about the validity of smile sheets, it is an interesting finding.
The C in the acronym stands for Credibility. This suggests my research method should be sound and disciplined and my conclusions should be logically aligned with a coherent conceptual framework.
Empirical evidence suggests that repetition is an effective technique for remembering a list of items. So here we go. Repeat after me: Magnitude, articulation, generality, interestingness, credibility; magnitude, articulation, generality, interestingness, cred….