From Text to Evidence: AI-Assisted Analysis and Measurement

Session Submission Type: AI Peer Learning Lab

Abstract

From Reddit to Policy Insights: Using AI to Summarize Public Opinion on Robotics Courses
Facilitator: Jialing Wu - The Ohio State University, PhD Student
Training Category: Quantitative Data Analysis
AI Tool: Claude
Participant Type: Researchers
Skill level recommended: Beginner
Recommended preparation or pre-work before session: None required. Participants who wish to follow along may bring a laptop with a Google account to access a shared Google Colab notebook. All code, prompts, and a sample dataset will be provided; no local installation is needed.

As schools expand robotics and engineering courses, teachers, parents, and students are discussing them candidly in online communities such as Reddit, often more honestly than in surveys or public hearings. This tutorial demonstrates a practical, low-cost workflow for collecting these discussions and using AI to answer two questions policy researchers often ask: What are people talking about, and how do they feel about it? The 20-minute session walks through a real project pipeline using K-12 robotics and engineering courses as the running example, in three stages: (1) collecting posts and comments from education-related subreddits (e.g., r/Teachers, r/AskEngineers) using Reddit's API with a short Python script in Google Colab; (2) using large language models and topic modeling to cluster discussions and label the main themes, such as student engagement, teacher preparedness, curriculum resources, and funding constraints; and (3) applying AI-assisted sentiment analysis to classify opinions as positive, negative, or mixed. For example, this step can distinguish enthusiasm about student engagement from frustration over being assigned to teach robotics without training or equipment. Results are then aggregated into a readable summary of public sentiment. Interactive elements: participants will suggest a policy-relevant subreddit at the start for a live demonstration, and a midway discussion invites participants to share how they gauge public opinion in their own work. Learning outcomes: participants will leave able to (a) set up Reddit data collection with minimal coding, (b) apply an AI-assisted workflow to summarize public opinions and sentiment on any policy topic, and (c) recognize key pitfalls, including sampling bias in online communities, terms-of-service and ethics considerations, and the need to validate AI-generated labels against raw data. A shared Google Colab notebook with annotated code, prompts, and a sample dataset will be provided.


******

Prototyping an AI Governance Capacity Index App Using Claude Code
Facilitator: Kun Yoon - Pennsylvania State University
Training Category: Policy Analysis & Policy Research
AI Tool : Claude
Participant Type: Researchers
Skill level recommended: Intermediate
Recommended preparation or pre-work before session: No advanced technical setup required. A Claude.ai account (free or paid) is sufficient to follow along; no coding experience is necessary, though familiarity with basic research methods (e.g., content analysis, indicator development) will help participants engage more deeply with the discussion portion.

Governments increasingly need to assess their own capacity to govern AI responsibly, yet the tools available fall short. Global indices such as Oxford Insights' Government AI Readiness Index compare countries, not the states and localities where most AI governance actually happens. Existing maturity models also rarely treat AI governance and data governance as related but distinct capacities. This session's index addresses both gaps, building on the presenter's prior comparative research on Korean local governments. This session demonstrates how Claude Code can turn a policy-theory-grounded framework into a working prototype app. Grounded in public policy capacity theory and organized as a logic model of inputs, processes, outputs, and outcomes, the index covers nine dimensions: strategy, institutional design, decision-making structure, collaborative governance, data management, AI adoption, performance management, and ethics and trust. Claude functions here not as a writing aid but as a measurement instrument, applying a pre-defined rubric to score qualitative text such as strategic plans and organizational charts. The prototype pairs this qualitative scoring with quantitative indicators pulled from open government APIs — Korea's Public Data Portal and Local Finance 365, and a U.S. state's Socrata-based open data portal. Because AI-based coding is still an emerging methodology, the session also addresses how to hold it to standards long applied to human coding: checking consistency across repeated scoring runs, and identifying what further validation, such as inter-rater agreement with human coders, would be needed before treating the index as a genuine research instrument rather than a one-off demonstration. The session runs in five parts: a brief framework overview; a video walkthrough of Claude Code scoring sample text against the rubric; a video walkthrough of the resulting dashboard comparing a U.S. state and a Korean local government; a facilitated discussion applying the workflow to participants' own cases; and a wrap-up with resources and Q&A. By the end, participants will be able to translate an academic framework into a scorable AI-assisted index, combine open government data with rubric-based qualitative coding, and identify what data access and validation this approach requires. This lab is for researchers and practitioners who want practical, replicable ways to use AI not just to write, but to build and validate research instruments.

******
From Three States to One Model: AI-Assisted Harmonization of Multi-State Medicaid Promoting Interoperability Data
Facilitator: Aaron Rapp - Winston-Salem State University, Assistant Professor of Mathematics
Training Category: Research Productivity & Workflow Automation
AI Tool : Claude
Participant Type: Researchers
Skill level recommended: Beginner
Recommended preparation or pre-work before session: Have your favorite AI assistant available to use through a web-browser or installed application. Claude or ChatGPT are preferred, but other AI models are welcome.

Comparative policy analysis across states is only as good as the comparability of the underlying data, and Medicaid's Promoting Interoperability program is a case study in just how uncomparable "public" data can be. States were never required to report meaningful use in a consistent format, so a dataset that looks straightforward for one state can be structurally incompatible with another, and can even change shape partway through its own history. This tutorial shares a practical workflow for using an AI assistant to enrich and harmonize Medicaid Promoting Interoperability data from Florida, Georgia, and North Carolina, drawn directly from the presenter's own project. The core example: Florida and Georgia reported a relatively complete row per provider; NPI, year, program year, payment amount, and EHR certification number. North Carolina split its own history in two, recording meaningful use status directly through 2014 and only a payment amount afterward, with no status field at all. Getting an AI assistant to help infer status from payment amount and map program year to meaningful use stage is a concrete, adaptable pattern for anyone whose multi-source data doesn't actually agree with itself. The session also touches on two AI-assisted API scripts used to enrich the data (vendor lookup via the ONC Certified Health IT Product List, provider demographics via NPPES) and one real limitation those scripts couldn't get around, deactivated NPIs that simply aren't retrievable through the API, as an honest example of where AI-assisted enrichment hits a wall. Designed as a fast-moving, example-first Exhibit Hall tutorial rather than a lecture, the session prioritizes showing real prompts and real harmonization logic over discussing AI in the abstract, and leaves real time for attendees to compare notes on their own multi-source data headaches. The goal is not technical mastery but a transferable habit: treating an AI assistant as a collaborator for reconciling inconsistent public data, while staying clear-eyed about what it can and can't fix. This session is aimed at researchers and analysts with some existing AI experience who want practical, adaptable examples from a real project rather than a tools overview and who have their own version of "the data that doesn't match" to bring to the discussion.

Policy Area

• AI Peer Learning Lab

Facilitators

• Jialing Wu, The Ohio State University

• Kun Yoon, Pennsylvania State University

• Aaron Rapp, Winston-Salem State University

Individual Presentations

• Presentation for Jialing Wu - Facilitator: Jialing Wu, The Ohio State University

• Kun Yoon, Presentation for - Facilitator: Kun Yoon, Pennsylvania State University

• Aaron Rapp, Presentation for - Facilitator: Aaron Rapp, Winston-Salem State University