When the Answer Is a Question: AI Socratic Questioning against Online Misinformation

Joint work with Junyu Zhang, a PhD student and the first author of this paper.

Overview

This project examines whether AI can help people evaluate online misinformation by asking a targeted Socratic question about a claim's weakest component. A preregistered, incentive-compatible randomized experiment with 606 participants compares this approach with AI warning badges, frontier-LLM verdicts, crowd-sourced fact checking modeled on Community Notes, and a no-support control.

Socratic questioning improved truth discernment, outperforming frontier-LLM verdicts and matching Community Notes in the experiment. Participants rejected more false claims without becoming more skeptical of true ones. A targeted benefit persisted after the questions were withdrawn, suggesting that question-based assistance can support users' own reasoning beyond the immediate intervention.