Evaluating Conversational AI in Urgent Primary Care: A Prospective Feasibility Study of the Articulate Medical Intelligence Explorer

Highlight

• First real-world prospective evaluation of a patient-facing conversational AI system in urgent primary care.

• Zero instances of conversation safety stops and high clinical and patient-rated conversation quality.

• Positive shifts in patient attitudes towards AI post interaction, sustained after physician visits.

• Primary care providers found AI summaries helpful for visit preparation and reported potential behavior modification.

Study Background

Primary care serves as the first point of contact for many patients presenting with acute complaints, playing a critical role in timely diagnosis and management. However, rising patient volumes and complexity contribute to clinician burnout and diagnostic delays. Artificial intelligence (AI) technologies, particularly conversational AI systems, have the potential to augment clinical workflows by facilitating patient data collection and early differential diagnosis formation. Yet, implementation in real-world primary care settings remains limited, and concerns persist about safety, data quality, acceptability, and clinical utility. This study aimed to evaluate these parameters by deploying a conversational AI system, the Articulate Medical Intelligence Explorer (AMIE), in an ambulatory urgent primary care context to test its feasibility and safety before clinical encounters.

Study Design

This was a prospective, single-center, single-arm feasibility study conducted from April to November 2025. The study enrolled English-speaking adult patients (≥18 years) scheduled for a single-complaint urgent care visit. Patients interacted with AMIE up to five days before their in-person appointment. AMIE, a patient-facing conversational AI tool, engaged patients in structured clinical interviews to elicit diagnostic information. All conversations were monitored live by physician safety supervisors trained to intervene if predefined safety thresholds—such as erroneous or harmful AI outputs—were met. Transcripts and AI-generated encounter summaries were provided to primary care providers (PCPs) ahead of the visit. Outcomes included the number of supervisor-initiated conversation safety stops, conversation quality as rated independently by clinical evaluators and patients via multi-criteria scales, and attitudes toward AI assessed by surveys administered to patients and PCPs post interaction. The study was registered on ClinicalTrials.gov (NCT06911398) and funded by Alphabet.

Key Findings

A total of 114 patients consented, with 98 completing both AMIE interaction and subsequent PCP appointment. Significantly, there were zero conversation safety stops triggered by supervisors based on the preset criteria, indicating a favorable safety profile of the AI system. Nonetheless, supervisors documented one episode of hallucination by the AI—defined as generation of clinically unwarranted information—and supplemented clinical input in five cases to maintain data integrity.

Clinical evaluators rated AMIE conversations favorably in 87–100% of cases across 17 quality criteria, reflecting robust clinical content and empathetic interaction quality. Patients rated AMIE positively in 48–96% of cases along 16 evaluative dimensions, highlighting strong user engagement. Moreover, patients’ attitudes toward AI improved significantly after interaction with the system, a positive effect maintained following their physician appointment, suggesting that AI involvement enhanced overall patient openness and acceptance.

From the PCP perspective, post-visit surveys were completed for 60 encounters. Among 44 visits where PCPs reviewed the AMIE transcript before the consultation, 33 reported that the AI summary was helpful for visit preparation. Additionally, in 25 of these cases, PCPs indicated the AI input possibly influenced clinical behavior, suggesting integration of AI data into diagnostic and management reasoning. These findings underscore the potential for conversational AI tools to complement clinician workflows without replacing critical human judgment.

Expert Commentary

This pioneering prospective evaluation of a diagnostic conversational AI in urgent primary care provides a compelling proof of concept for safe deployment in ambulatory settings. The absence of safety stops aligns with rigorous oversight and predefined safety metrics, though the detection of a hallucination episode reinforces the necessity of continued human supervision during early implementation phases.

The high quality ratings by clinicians and patients suggest that well-designed AI systems can achieve both clinical relevance and patient-centered communication standards. Enhancements in patient attitudes post AI interaction highlight a key benefit in patient engagement, which could positively influence adherence and satisfaction.

However, limitations include the single-center design, English-speaking adult population, and single-complaint focus, which may limit generalizability. Longer-term studies should assess impact on diagnostic accuracy, clinical outcomes, workflow efficiency, and cost-effectiveness. Further, integration of AI outputs into electronic health records and clinician training are critical next steps.

Conclusion

This study establishes the initial feasibility, safety, and acceptability of a conversational diagnostic AI system in urgent ambulatory primary care. It demonstrates that patient-facing AI can safely collect diagnostic data, improve clinical preparation, and positively influence attitudes toward AI, thereby representing a meaningful advancement toward clinical translation. With further validation and scaling, conversational AI has the potential to augment primary care delivery and address contemporary challenges in diagnostic quality and clinician workload.

Funding and Trial Registration

This study was supported by Alphabet. The trial was registered with ClinicalTrials.gov under the identifier NCT06911398.

References

Brodeur PG et al. Conversational diagnostic artificial intelligence in ambulatory primary care: a prospective feasibility study. Lancet. 2026 Oct 8; PMID: 42849491.

Topol EJ. Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again. Basic Books; 2019.

Chen JH, Asch SM. Machine Learning and Prediction in Medicine — Beyond the Peak of Inflated Expectations. N Engl J Med. 2017 Jun 29;376(26):2507-2509.