Creating an Excel Tool to Help Researchers Pick and Rationalize Single-Case Experimental Design Metric

• In Event: Analysis and Meta-Analysis of Single-Case Experimental Data: Methodological Innovations and Complexities

Abstract

Throughout the past two decades, there has been increased pressure to quantify single-case experimental design (SCED) outcomes for several reasons, including laws calling for the use of evidence-based practices (e.g., Individuals with Disabilities Improvement Act of 2004), and evidence-based communities (e.g., What Works Clearinghouse, 2020) calling for the use of statistics that can be understood by group design researchers (Shadish, 2014). As a result, many different quantification metrics for SCEDs have been developed [e.g., IRD (Parker et al., 2009); Tau-U with trend adjustment (Tarlow, 2017); SMD (Hedges et al., 2012); regression-based metrics (Center et al., 1985-1986); log response ratio (Pustejovsky, 2018)]. However, there are no clear guidelines for how to pick which metric to use, as certain metrics are more appropriate for use depending on the research question(s) or data characteristics (e.g., trend in the baseline phase). Thus, researchers and practitioners may benefit from a user-friendly tool that helps them pick an appropriate SCED metric depending on their research question(s) and data characteristics.
Research Aims. The research aim is to develop a practitioner-friendly tool in Excel that can assist the user in a.) Picking an appropriate SCED metric and b.) Rationalizing the use of the SCED metric.
Method. An input/output function was created in Excel. A total of 19 different metrics and 14 different research questions and data characteristics (e.g., number of measurement occasions, outliers, etc) that may affect the appropriateness of the SCED metric were identified. Using previous SCED research, each metric was coded as being able to address or not address each research question/data characteristic. This coding was then converted into a logic code so that the user would be able to identify the research questions and data characteristics that were relevant depending on their data set, and the output would show a ranking of metrics from most appropriate to least appropriate.
Results. The Excel Tool document that was created contained three sheets; (1) instructions, (2) input/output, and (3) metric details. Using the instruction sheet, users can read about how to use the Excel Tool. Using the input/output sheet, the users can select “yes” or “no” for each of the 14 research questions/data characteristics and whether the user wants them to be accounted for when picking a metric, according to their own data set characteristics and research question(s). The output lists the recommended metrics in order of appropriateness, depending on the input. Users can click on each metric at the top, which will then take them to sheet 3, with the metric details. The metric details sheet includes information about the metric, including the founder(s) of the metric, online calculators, and examples of the metric used in the literature.

Authors

• Joelle Fingerhut, Rutgers University

• Mariola Moeyaert, University at Albany