DIA: A Human AI Hybrid Conversational Assistant for Developing Contexts

• In Event: Innovations in Digital Support for Education in Low-Resource Contexts: Case Studies from West Africa.

Proposal

Social media messaging applications (e.g. WhatsApp, and Facebook Messenger) had already reached more than 2 billion users by 2019. The high usage among Majority World users opens the possibility of designing text-based interventions for social change. However, such interventions typically rely on experts’ knowledge (such as doctors, educators, and moderators) which is often widely distributed in low infrastructure contexts. Prior work has used a conversational agent to address the challenges of limited expert knowledge and providing personalized interactions. Expert knowledge could be scaled up using such chatbots, but more research is needed to support users who need context-specific support such as local-language interventions, or may not have regular internet connectivity.

Other recent work has used virtual communities on social media for teachers to support each other through online interactions. Still, it is unclear how these two promising research areas can translate to rural African contexts with low technology infrastructure. Additionally, teachers in these contexts are newly adopting technology and thus may require additional support to accommodate technology adoption. Therefore, there is a need to discover appropriate conversational agent designs that can support teacher communities in low-infrastructure settings.

We used an iterative design-based research (DBR) approach by working closely with teachers in rural Côte d’Ivoire to develop a technology that helps them to implement a new pedagogical program instituted by the government. From this process, we built DIA, a chatbot architecture for low-resource contexts to scale expert knowledge and support localization. DIA is a human-chatbot (humbot) hybrid system that organically learns topic-specific knowledge and local language from user interactions. Key findings in our early studies were that teachers supported each other in the community and valued community-based features in a conversational agent. This work led to the question: How does a conversational agent that supports a virtual community of practice (vCOP) impact teachers in low infrastructure settings?

To answer this question, we conducted a large-scale, yearlong study to understand the impact of community-based features at scale. This study involved a longitudinal quasi-experiment with 400 teachers in two different regions of rural Côte d’Ivoire to investigate the impact of two variations of the DIA conversational agent. In one region, a conversational agent with individual support was deployed, and in another region, a conversational agent with community support was deployed. The objective was to assess motivation, knowledge, and technology adoption changes over the school year. The findings indicated that community support positively affected motivation, enhancing agency within the community. Teacher knowledge showed some improvement, with a slight but statistically significant overall increase. Although the community condition increased technology usage, the results did not reach statistical significance. The results favor utilizing community support in conversational agents for teachers in low-infrastructure settings.

Author

• Amy Ogan, Carnegie Mellon University