tag:blogger.com,1999:blog-6451967208270832502.post3662232974042937028..comments2024-03-27T03:22:41.073-07:00Comments on Psych Your Mind: (Sample) Size MattersAnonymoushttp://www.blogger.com/profile/08931064542755278772noreply@blogger.comBlogger1125tag:blogger.com,1999:blog-6451967208270832502.post-52108526219440562062014-07-09T18:32:39.896-07:002014-07-09T18:32:39.896-07:00It is good to see that you are thinking about stat...It is good to see that you are thinking about statistical power and making<br />(successful) efforts to increase sample sizes in response. (However, it is<br />disappointing that you haven’t gotten responses to this post saying things<br />like, “I’m doing the same thing,” or “Give me some advice on how I can do<br />this, too”.)<br /><br />As you comment, “In thinking about how large an N is enough, there isn't a<br />one-size-fits-all answer unfortunately.” But I hope the following<br />suggestions might help you get “more bang for your buck,” at least on<br />average, in deciding on sample size. In particular, I recommend reading<br />Muller, K. E., and Benignus, V. A. (1992), “Increasing Scientific Power<br />with Statistical Power,” Neurotoxicology and Teratology, 14, 211–219.<br /><br />One thing they point out (p. 217, “How much power is enough?) is that<br />considering the “power curve” (power plotted versus raw effect size) can<br />help in making wise decisions. For example, for a two sample t-test,<br />choosing sample size to give power .84 gives a kind of “sweet spot” for<br />the tradeoff between power and sample size. In other words, relying on 80%<br />power is an example of one-size-does-not-fit-all.<br /><br />Other things to take into account (which possibly you are aware of, but<br />possibly aren’t):<br />• When you plan more than one hypothesis test, it’s important to choose<br />sample size taking the “family-wise type I error rate” into account. This<br />means that if you wish to have an overall type I error rate of .05, you<br />will need to calculate power based on lower significance levels for<br />individual hypothesis test.<br />• Using Cohen’s standardized methods (standardized effect sizes, and<br />small/medium/large effects) is crude. Most statistical software now has<br />better methods available.<br />• These better methods require you to think about what “raw” effect size<br />you wish to be able to detect. This has the advantage of making you think<br />about practical significance as well as statistical significance.<br />• The better methods also require an estimate of standard deviation, which<br />has the advantage of prompting the researcher to consider previous studies<br />or perform a pilot study.<br /><br />I’ve got some discussion of some of the above, in the context of a couple<br />of the replications in the special issue of Social Psychology, in the<br />July 1, 3, and 6 posts at http://www.ma.utexas.edu/blogs/mks/<br /><br />-Martha SmithAnonymousnoreply@blogger.com