Paper Summary
Share...

Direct link:

Using Focus Groups to Inform Survey Development: The Dilemma of Formative Assessment

Mon, April 16, 12:25 to 1:55pm, Pan Pacific, Floor: Lobby Level, Oceanview 1&2

Abstract

This presentation focuses on the value and use of focus group interviews to inform the process of survey development for a statewide survey of 9,000 school principals across California. The objective of the survey was to elicit information about the current status of formative assessment practices in P-12 classrooms.

While the use of literature is an appropriate method of informing survey development and refinement, a growing body of literature suggests that focus groups can be utilized as a supplementary technique, particularly useful for understanding the language that populations of interest use to describe their experience and the thinking involved (O’Brien, 1993).

Therefore, in this study, both literature review and focus groups were used to inform survey development. First, a review of current (1990 to present) research literature on formative assessment practices informed survey topics. Next, focus group questions were developed, aligned with key literature findings, to gather further information from school and district practitioners as to current formative assessment practices in schools. Focus group findings were used to further refine the focus of survey items and to elucidate operational definitions of terms used in the survey.

Specifically, five focus group interviews were held with forty teachers, principals, and district administrators from schools and districts identified as demonstrating high commitment to formative assessment practices. Focus group questions targeted the identification of formative assessment instruments and practices in mathematics. Topics included: definition of formative assessment; use of formative assessment in collegial team meetings; access to formative assessment data; leadership support of formative assessment practices; impact on curriculum, instruction, professional learning, and student learning; student involvement in formative assessment, and measuring the effectiveness of formative assessment. Transcripts of the focus group interviews were analyzed thematically and quantitatively for frequency of theme repetition.

Key focus group findings included a fundamental inconsistency in the definition and understanding of formative assessment in schools. While most practitioners were clear about the difference between summative and formative use of assessments, they were not consistent in their understanding of the definition and elements of formative assessment. Practices ranging from minute-by-minute assessments of student understanding to quarterly interim assessment were included among categories of formative assessment. Additionally, there was confusion as to whether formative assessment refers to an instrument, a practice, a process, or a combination thereof. Further, the language used to refer to various types and elements of formative assessment varied across professional roles and across districts and schools. These and other critical focus group findings were utilized in developing a number of draft survey items which were presented to a nationwide panel of experts for review and feedback.

The poster will expand understanding of pivotal and controversial topics related to formative assessment and suggest clarification of definitions and language related to formative assessment that impact both researchers and field practitioners. Additionally, it will set forth considerations for the use of focus group interviews in survey design, including identification of topics, development of questions, refinement of definitions, identification of respondent groups, and tailoring language to respondent groups.

Authors