Moving Knowledge Forward by Strengthening the Evidence in Empirical Research
Each of the studies identified through systematic searches of the literature was reviewed using a set of standards of evidence for empirical research developed by the KMD project, drawing from numerous writings about research rigor, quality, and reporting, including efforts to address quantitative, qualitative, and mixed methodologies. A panel of mathematics and science education researchers, research methodologists, and mathematics and science education leaders assisted the KMD staff in the development and review of the standards of evidence to help ensure not only their quality, but also their broader utility. The standards of evidence examine the contribution of claims in research studies in terms of:
Adequate documentation of activities
Internal validity
Analytic precision
Generalizability/external validity
Overall fit
Warrants for claims
After applying the standards of evidence to individual studies to assess the contribution of each claim, the relationships examined in each claim were identified so that all claims about a particular relationship could be compiled. The empirical evidence across studies making similar claims was then summarized.
