Designing Explainable AI for Healthcare Reviews: Guidance on Adoption and Trust

A new mixed-methods study highlights the critical role of **Explainable AI (XAI)** in empowering patients to navigate the vast landscape of online healthcare provider reviews. The research, published on **arXiv (2603.00072v1)**, reveals strong user demand for transparent AI systems that not only summarize patient feedback but also clearly explain their analytical outputs, suggesting that such explainability is crucial for building trust and promoting the adoption of AI tools in healthcare decision-making.

The Promise of Explainable AI in Healthcare Decisions

Addressing Information Overload for Patients

Patients increasingly turn to online reviews as a primary resource when selecting healthcare providers. However, the sheer volume of these reviews often creates an information overload, making it challenging for individuals to extract relevant insights and make informed decisions. This complexity can lead to frustration and potentially suboptimal healthcare choices. To address this, researchers evaluated a proposed **explainable AI system** designed to analyze patient reviews and provide transparent justifications for its classifications. The system aims to distill vast amounts of qualitative data into actionable insights, making the decision-making process more efficient and reliable for patients.

Unpacking User Expectations and Trust

A survey conducted as part of the study (N=60) revealed significant optimism regarding the usefulness of such an AI system. A remarkable **82% of respondents agreed** that the system would save them time, while **78% believed** it would effectively highlight essential information from reviews. This indicates a clear perceived utility for AI-driven review analysis. Crucially, the study also underscored a strong demand for **AI explainability**. An overwhelming **84% of participants considered it important** to understand *why* a review was classified in a particular way. Furthermore, **82% stated that clear explanations would significantly increase their trust** in the system's outputs. The research also found that approximately **45% of users preferred a combined text-and-visual explanation format**, suggesting a need for diverse presentation methods to enhance comprehension.

Designing for Clarity and Credibility

Core User Requirements for XAI Systems

The thematic analysis of open-ended survey responses provided deeper qualitative insights into user expectations. Key requirements identified included **accuracy, clarity, and simplicity** in the AI's explanations. Users also emphasized the need for **responsiveness**, **data credibility**, and **unbiased processing** from the AI system. These findings underscore that beyond mere functionality, the *how* and *why* of AI's operations are paramount for user acceptance in sensitive domains like healthcare.

Expert Perspectives and Technical Nuances

Complementing the user survey, interviews with AI experts offered valuable qualitative insights into the technical considerations and potential challenges associated with different explanation methods. These expert perspectives highlighted the complexities involved in designing AI systems that are not only accurate but also capable of generating explanations that are both technically sound and easily understandable by a lay audience. The insights contribute to a more holistic understanding of the technical feasibility and ethical implications of deploying XAI in real-world healthcare applications.

Why This Matters: Driving AI Adoption in Digital Health

Drawing on established frameworks like the **Technology Acceptance Model (TAM)** and theories of **trust in automation**, the study concludes that high perceived usefulness combined with transparent explanations are key drivers for the adoption of AI systems in healthcare. Conversely, complexity and inaccuracy in AI outputs and explanations are significant barriers to user acceptance. The findings provide actionable design guidance for developing **layered, audience-aware explanations** within healthcare review systems, paving the way for more trustworthy and effective digital health tools.

Key Takeaways

• Patient Demand for XAI: Patients are optimistic about AI's ability to streamline healthcare provider selection but demand transparency in its operations.
• Trust Through Transparency: Clear, understandable explanations from AI systems are crucial for building user trust, with 82% of surveyed users indicating increased trust with explanations.
• Design Principles: Core user requirements for XAI in healthcare include accuracy, clarity, simplicity, responsiveness, data credibility, and unbiased processing.
• Promoting Adoption: Perceived usefulness and transparent explanations are identified as primary drivers for the adoption of AI in healthcare, aligning with the Technology Acceptance Model.
• Actionable Guidance: The study offers practical design recommendations for creating layered and audience-aware explanations in healthcare review platforms, enhancing user experience and system credibility.