Virtual Patient

An AI-powered virtual patient system designed to support medical education by simulating realistic clinical conversations and assessing medical students’ history-taking skills through natural spoken interaction.

A research project at Universiti Sains Malaysia (USM) focused on advancing AI-powered virtual patients for medical education. The project investigates how LLM and speech technologies can support realistic clinical dialogue, improve patient simulation, and help medical students develop effective, systematic, and patient-centred history-taking skills. Through a series of interconnected research subprojects, the project progresses from reviewing the state of the art in LLM-based virtual patients to improving text generation, developing end-to-end spoken virtual patients, and introducing trajectory-based supervision for more clinically consistent, low-latency spoken interactions.

--------------------------Team---------------------------

Subproject 01

LLM-Based Virtual Patients: A Systematic Review

Systematically review the development of LLM-based virtual patient systems for medical education, with a focus on clinical historytaking, to identify research gaps and future directions.

Subproject 02

Improving Text-Based Virtual Patients Through Beam Search

Investigate beam search and related decoding strategies to improve the quality and reliability of text-based virtual patients in clinical history-taking scenarios.

Subproject 03

SpeakPatient: End-to-End Spoken Virtual Patients

Develop end-to-end spoken virtual patients with patientlevel conditioning to maintain patient identity and case specific clinical context during natural, low-latency conversations.

Subproject 04