Study examines rationale effectiveness in AI question answering

VERA Brief

AI-generated. Grounded in the article and its cited sources.

Research on AI question answering systems found that while faithful rationales had little effect on answer accuracy, corrupted rationales significantly influenced how the AI judged support and how verifiers assessed information. This suggests that the effectiveness of sharing rationales as a verification mechanism needs to be re-evaluated.

Key facts

• Faithful rationales in AI question answering systems provided minimal gains in answer accuracy.
• Corrupted rationales substantially impacted support judgments in AI systems.
• Under specific prompts, corrupted rationales caused significant shifts in support judgments.
• Human audits revealed instances of 'corruption-overtrust' where humans rejected corrupted rationales that AI models accepted.
• The research suggests rationale sharing should be assessed for verification messaging rather than just answer accuracy enhancement.