Surveys of a sensitive nature require a sensitive method of data collection: using ACASI with P3 pupils in Uganda

• In Event: Practical and Useful Tools: Collecting sensitive data, measuring resilience, and visualizing your data

Proposal

School-Related Gender-Based Violence (SRGBV) is a global phenomenon impacting more than 246 million children worldwide[1]. SRGBV’s toll is physical, emotional, and academic - impacting not only the long-term mental health of children, but their ability to learn. Yet, SRGBV remains difficult to measure due to high sensitivity and response bias. Furthermore, most measurement of SRGBV relies on Face-to-Face (FTF) survey administration, which is susceptible to increased social desirability bias. Audio Computer-Assisted Self-Interview (ACASI) allows subjects to respond to pre-recorded questions on a computerized device, providing respondents with the privacy and anonymity needed to reduce social desirability bias in sensitive surveys. While many studies have tested the feasibility and value of using ACASI in LMICs [2][3], no research has been conducted on the use of ACASI to measure school-related dimensions with a primary school-aged population. This study provides evidence that ACASI administration allows for equally, and in certain cases stronger, valid and reliable data as the FTF administration when collecting data on violence in school.

The proposed paper contains the findings from a large-scale study conducted in Uganda in 2019 where responses from surveys administered with the FTF method were compared with responses from the same surveys administered with the ACASI method. Under this study, 876 P3 students from 40 schools completed four surveys under either the FTF or ACASI administration:
-Perceptions of School Climate
-Gender Norms and Attitudes
-Social-Emotional Learning
-Experiences of Violence (bullying, corporal punishment, and sexual violence)

The purpose of this study was to explore the possibility of using ACASI with P3 students in a school environment and test ACASI against the FTF method to check for a difference in reporting in the more sensitive surveys and items.

First, we needed to ensure that using a different administration type did not lead to a change in the number and types of constructs being measured. In other words, are respondents understanding (and therefore responding to) the questions differently if they are listening to a recording of the question compared to if an assessor is reading the question out loud in front of them? Factor analysis was conducted to answer this question.

Next, we looked at the convergent validity of the surveys to assess how well the identified constructs are measured by their associated group of items. In other words, how valid are our groupings and does this change with a different administration type?

Lastly, we compared responses across administration types to see if using ACASI leads to different levels of reporting of negative perceptions or experiences. Does social desirability bias seem to play a part in student responses to sensitive questions about their school environment, their beliefs on gender, or their experiences of physical violence?

Our findings illustrate that factor structures and convergent validity do not change drastically when administering these surveys using ACASI (as compared to FTF). When comparing responses to the surveys across the two administration types, the ACASI and FTF groups look mostly similar.

While most survey responses were comparable between ACASI and FTF groups, the reporting of experiences of sexual violence differed drastically: 43% of students in the FTF group reported experiencing sexual violence in the past school term, while 77% of students in the ACASI group reported this. An item-level analysis, by gender, illustrates the severe difference in reporting between the FTF and ACASI groups.

We conclude that ACASI is therefore a better tool in measuring sensitive sub-topics of SRGBV and should be utilized over the current FTF administration. The increased variability in and increased affirmative responses of the sexual violence survey indicate that using ACASI to administer surveys that contain sensitive or culturally taboo questions can mitigate social-desirability bias. This study addresses two gaps in existing research: (1) It reveals that students as young as 10 years old are able to use the ACASI platform to complete sensitive surveys; and (2) It finds that reporting of experiences of sexual violence is drastically different when using ACASI, while other less-sensitive surveys within the school climate and SRGBV space do not show increased variability or reporting. ACASI provides a safer space in which children can feel more comfortable disclosing their experiences, knowing their privacy is safeguarded. Obtaining data through proper privacy measures, that are prone to the least amount of bias, is the first step toward determining the full breadth and depth of SRGBV. The data can be used to better inform programming that addresses the experiences and drivers of SRGBV to eradicate violence against children in schools and contribute to positive and safe learning environments.

References
[1] Plan International, https://plan-international.org/because-i-am-a-girl/violence-at-school
[2] Langhaug LF, Sherr L, Cowan FM, How to improve the validity of sexual behavior reporting: systematic review of questionnaire delivery modes in developing countries. Trop Med Int Health. 2010 Mar;15(3):362-81. doi: 10.1111/j.1365-3156.2009.02464.x.
[3] Kane, et al. Validation of a substance and alcohol use assessment instrument among orphans and vulnerable children in Zambia using Audio Computer Assisted Self-Interviewing (ACASI). Drug and Alcohol Dependence 2016. dx.doi.org/10.1016/j.drugalcdep.2016.06.026

Authors

• Maitri Punjabi, RTI International

• Julianne Norman, RTI International

• Lauren Edwards, RTI International