Evaluating the Utility of Personal Health Records in Personalized Health AI

论文概要

研究领域: cs.AI 作者: Rory Sayres, Kejia Chen, Ayush Jain 发布时间: 2026-05-21 arXiv: 2505.01252

中文摘要

患者管理的个人健康记录（PHR）有望赋予患者更好地理解自身健康的能力；但记录中的信息复杂，可能阻碍洞察。在本研究中，我们评估了大语言模型（LLM，Gemini 3.0 Flash）在提供临床数据作为上下文时，对用户健康查询提供有帮助回答的潜力。总共2,257个用户查询来自3种不同分布以代表患者问题：较短的网页搜索查询、从聊天机器人对话模板派生的较长问题、以及患者向医疗团队提出的问题（患者电话）。查询与去标识化的PHR（来自1,945份记录池）匹配。Gemini回答在三种条件下生成：（1）无PHR上下文；（2）含基本人口统计、疾病和药物摘要；（3）含完整、广泛的临床记录。评估中，我们利用现有的评分框架（SHARP），并开发了一个针对PHR解释特定错误模式的新框架。评估使用自动评分器对全部查询进行评分，并对子集（n=95）进行临床医生评分，两组评分者都知道完整的PHR上下文。我们看到，使用PHR数据后，所有问题类型的回答帮助性显著提升（p < 0.001，配对t检验）。我们还观察到安全性、准确性、相关性和个性化的潜在提升。我们的PHR评估框架进一步识别了LLM理解复杂PHR特定方面的差距，如时间定向障碍，以及罕见但有意义的虚构。这些结果表明PHR数据有潜力帮助具有广泛用户需求的人群；并为基于PHR上下文监控LLM回答差距提供了一个框架。本研究激励进一步工作以评估和实现用户从理解健康记录中获得的潜在益处。

原文摘要

Patient-managed Personal Health Records (PHRs) promises to empower patients to better understand their health; but information in the record is complex, potentially hindering insights. In this study, we assess the potential of large language models (LLMs, Gemini 3.0 Flash) to provide helpful answers to user health queries, when provided clinical data from PHRs as context. A total of 2,257 user queries were drawn from 3 different distributions to represent patient questions: shorter web search queries, longer questions derived from templates of chatbot conversations, and questions patients asked to their healthcare team (patient calls). Queries were matched with de-identified PHRs (from a pool of 1,945). Gemini responses were generated (1) without PHR context; (2) with a basic summary of demographics, conditions, and medications; (3) with full, extensive clinical notes. For evaluation, we leveraged an existing rating framework (SHARP), and developed a new framework for specific error modes when interpreting PHRs. Evaluation was performed using autoraters for the full set, and with clinician ratings for a subset (n=95), with both sets of raters knowing the full PHR context. We see significant improvements in the helpfulness of answers to all question types with PHR data (p < 0.001, paired t-test). We also observe potential gains in safety, accuracy, relevance and personalization of answers. Our PHR evaluation framework further identifies gaps in LLM understanding of particular aspects of complex PHRs, such as temporal disorientation, and rare but meaningful confabulations. These results suggest potential for PHR data to help people with a wide range of user needs; and provide a framework for monitoring for gaps in LLM answers based on PHR context. This study motivates further work to assess and realize potential benefits to users from understanding their health records.

自动采集于 2026-05-21

#论文 #arXiv #AI #小凯