Phase 1 Cross-Sectional Study Design and Sampling

• In Event: Symposium | Testing a Dynamic and Iterative Process of Leadership Identity and Competency Development in Youth

Short Description

The purpose of this symposium presentation is to outline the design and sampling from the initial cross-sectional analytic phase (Phase 1) within a comprehensive longitudinal examination of leadership identity and leadership competency development in youth.

Detailed Abstract

Utilizing positive youth leadership identity (PYLI; [Author] & [Author], 2018, 2019) and student leadership competencies (SLC; [Author], 2018) as variables of interest in a comprehensive longitudinal examination of leadership identity and leadership competency development in youth, an initial cross-sectional correlational study (Phase 1) evaluated the relationship between PYLI and SLCs prior to latent change score modeling, addressing Hypothesis 1: Due to the hypothesized and tested relationships between leader identity and leadership skills (e.g., Day & Sin, 2011; Lord & Hall, 2005; Miscenko et al., 2017), it is expected that holding a stronger positive youth leadership identity will be associated with higher self-ratings of leadership competency.

The competencies that make up the Youth Leadership Competency Model were selected as a semantically cohesive subset of items from SLCs ([Author], 2018). Considering the nature of (a) the sample demographics with respect to age, (b) the total number of items for which data collection was required, and (c) previous literature on inattentive/insufficient effort responding, the proposed study adopted a planned missing data design (Graham et al., 2006). For example, Baer et al. (1997) suggest that inattentive responding can increase dramatically after 100 items, while Woods (2006) notes that as few as 10 – 20% of a sample displaying careless responding behavior can impact overall model fit.

Planned missing designs directly address this issue by restricting data collection to different subsets of the variables for each participant. These subsets are pre-specified into blocks and then randomly assigned to participants. This data collection plan was proportionally balanced between scales of interest to optimize efficiency in data collection. With random assignment of item blocks and a large enough pairwise sample size between the scales, it is safe to assume the missing data are missing completely at random (MCAR), and can be analyzed using many modern and flexible missing data analysis procedures (e.g., Allison, 2003; Enders & Bandalos, 2001).

In the present study, data collection per participant ranged from 44 to 87 items with the aim of preventing inattentive responding from scales that were excessive in length. Additionally, data quality was evaluated using a self-report measure and response time (Desimone et al., 2015; Keith at al., 2017). The self-report measure asked respondents “do you commit to providing thoughtful and honest answers to the questions in this survey?” Responses were only collected and retained from participants who chose response option “I will provide my best answer.” Respondents who chose either “I will not provide my best answers” or “I can’t promise either way” (N = 100) were directed to the end of the survey. Appropriate response time was determined by collecting 150 initial responses. The average completion time was 492 seconds or approximately eight minutes. To prevent collecting responses from participants speeding through the survey and not providing quality data, a cutoff of approximately 40 percent of the completion time, three minutes, and 15 seconds, was used, resulting in 1,300 participant responses not included in the analysis. An adjusted average response time, after removing five percent of extreme values ranging from nearly 32 minutes to 333 minutes, was 536.24 seconds (approximately nine minutes).

To provide confidence in generalizability of potential findings, the data analysis goal was to collect responses from 4,000 participants, with the sample meeting two criteria. The sample needed to represent the demographic diversity of a U.S. sample of youth aged 10 – 19, while also providing subsamples large enough for data analysis testing for measurement invariance (or multi-group confirmatory factor analysis). The representative sample criteria were established based on American Community Survey estimates from 2017 (D. Drozd, personal communication, July 2, 2019). The demographic characteristics included ethnicity (Hispanic/Latino), race, gender, age, and place of residence (metro area or nonmetro area). The minimum sample size for any specific demographic category was 176 (American Indian or Alaskan Native), which is slightly below the recommendation of 200 participants for any group when conducting measurement invariance testing (Pendergast et al., 2017). All other demographic groups are above the 200-participant threshold.

A Qualtrics panel was used where adult participants with children were invited to participate in the research. The survey was accessed a total of 19,839 times. Parental consent was solicited, followed by a demographic screening question to ensure demographic targets were being met. The quota was met by 14,054 respondents, directing them to the end of the survey, and not collecting any additional data. If the quota had not been met, youth assent was also solicited. Additionally, adults fitting the target ages of 18 or 19 were invited to participate directly. A total of 385 participants did not consent to the study, and they did not provide responses. Complete data was collected from 4,000 participants.

Participants

• L.J. McElravy, University of Nebraska-Lincoln
  Presenter

• Lindsay Hastings, University of Nebraska-Lincoln
  Presenter

• Corey Seemiller, Wright State University
  Presenter

• Alex Brodersen, University of Notre Dame
  Co-Author