You are here

How does AAR/AI Support Problem Solvers with Diverse Behaviors and Cognitive Styles?

Description: 
"What’s wrong with this AI?" Explainable AI (XAI) researchers are moving beyond explaining an AI’s actions, to helping users detect an AI’s failures. However this detection may not be enough—for actionability, we often need to pinpoint which part failed. We investigate how AAR/AI, a structured assessment process, supports users with diverse behaviors and cognitive styles in the context of a fault localization task in a reinforcement learning (RL) agent. In Study 1’s qualitative investigation, 17 participants engaged in diverse behaviors at all stages of sensemaking. They identified faults using behaviors ranging from ad hoc searching to consistent behavior akin to professional searchers'. Then, they confirmed faults using behaviors ranging from narrow pattern-matching approaches to specification-checking. Last, they reported faults using behaviors from "shrugging" to probing the space of actions the AI considered. We also performed a secondary analysis of 65 participants on a follow-up controlled experiment (Study 2) and disaggregated their data by their five GenderMag cognitive problem-solving styles. At each endpoint of four of the five cognitive style spectra, participants who used AAR/AI located significantly more faults than those who did not use AAR/AI. These end-points include participants with low- self-efficacy and those with high- self-efficacy; those with task-oriented motivations and those with technology-oriented motivations; those who learn by process and those who learn by tinkering; and, comprehensive information processors and selective information processors. AAR/AI also closed an inclusivity gap between risk-averse and risk-tolerant participants, further demonstrating that AAR/AI supports problem solvers with a wide diversity of behaviors and cognitive styles.
Type: 
Masters Thesis
Raw Url: 
http://ir.library.oregonstate.edu?metadataPrefix=&verb=GetRecord&identifier=ir.library.oregonstate.edu:9c67wv964
Repository Record Id: 
ir.library.oregonstate.edu:9c67wv964
Record Title: 
How does AAR/AI Support Problem Solvers with Diverse Behaviors and Cognitive Styles?
http://ir.library.oregonstate.edu/concern/graduate_thesis_or_dissertations/9c67wv964
Database: 
Resource OE Format: 
randomness