The ongoing crisis of confidence in NHST came to a head in 2016, leading to the publication of a statement about this framework by the American Statistical Association (ASA) in their journal The American Statistician. The statement was prepared by a large group of statisticians from many different schools of statistical thought, not all of whom use significance levels in their own work or teach it to their students. This statement was intended to serve as an authoritative review of what NHST is, what it can do for scientific research, and what its limitations are. This statement is more nuanced than the other two readings above, and it was prepared by statisticians to provide guidance for researchers to avoid the misuse of NHST, without entirely abandoning the framework. The first two pages in the PDF copy of this statement review the history of this crisis, while the remaining two pages provide qualified commentary on significance, p-values, NHST, and other related matters. Read this paper here (attached), and answer the following questions:
Rather than recommending the abandonment of NHST and its ingredients (p-values, levels of significance, etc.), what does this statement recommend that researchers do?
All three of the readings discussed here mention a topic called “effect size,” and they contrast statistical significance with “scientific, human, or economic significance.” (Also review your textbook, page 199, on “practical versus statistical significance.”) Paradoxically, while effect size seems important to the authors of all of these readings, they don’t provide any really great definition of this concept. (In fact, Trafimow and Marks are wrong to discuss this concept under the heading of descriptive statistics.) The ASA’s statement comes closest to defining this concept (page 132). To the best of your ability based on these three readings, what do you think the meaning of “effect size” is?
