Via Niche Modeling
( Posting the content “as is”, for it serves as a good reminder to me whenever I go over old posts )

  1. **Get as large a sample as you can.
    **A. Large N provides for more stable measurement of variables, they are less likely to be affected by outliers.
    B. Large N also provides for distributions that are more normal, or better reflect the full range of scores in the population.
  2. **Run as few statistical tests as you can.
    **A. running several tests increase the risk of a Type 1 error
    B. focus your results as much as possible
  3. **Never report the same data twice.
    **A. all of the statistics you have learned are part of the same model, thus if one test is significant (e.g., correlation) then a different statistic will also be (e.g., regression).
    B. when doing tests of means following ANOVA, especially for analysis of interactions, include each mean in only 1 test (if possible).
  4. **When using multivariate tests, always get the most for the least.
    **A. in factor analysis, account for high percentage of variance with as few factors as possible.
    B. in multiple regression, get the highest R2 with the fewest predictors.
    C. in path analysis, specify as few paths as possible that account for most of the correlations
  5. **Use the most reliable measures possible.
    **A. always test for the reliability of scales or multi-item tests before computing a total score for the test.
    B. if a variable is unreliable, its correlations with other variables are almost always lower than they should be. Thus, you underestimate the true degree of correlation but you don’t know by how much.
  6. **Plan your analysis before collecting the data.
    **A. there are some studies whose data cannot be analyzed because the analyses were not planned in advance.
    B. control for potential problems when designing the study, not when analyzing the data.
  7. **Use statistics to support the written (verbal) argument, not to substitute for it.
    **A. also, do not write statistics that you don’t understand, it shows.
  8. **Never do multiple tests without controlling for Type 1 error.
    **A. never do several t-tests when ANOVA is appropriate
    B. never do several ANOVAs when MANOVA is appropriate
    C. never do post hoc t-tests or many correlations without adjusting alpha (or at least admitting to the risk of Type 1 error when writing them up)
  9. **Never try to prove the null hypothesis.
    **A. do not design a study to show “no difference” between means or “no correlation” between variables.
  10. **Others
    **A. Never trust a factor with less than three substantial loadings.
    B. Never interpret a correlation without looking at the scatterplot
    C. Look for outliers but never toss them out unless you know that the data are inaccurate