Authors (including presenting author) :
KM Cheung (1) BHK Yip (2) HY Yiu (1)
Affiliation :
(1) Department of Clinical Oncology, Queen Elizabeth Hospital
(2) The Jockey Club School of Public Health and Primary Care, The Chinese University of Hong Kong
Introduction :
Conducting clinical research and audit has never been more important due to the mandate by the latest Hong Kong government policies to develop Hong Kong into a global medical research and innovation hub. Our workforce has undoubtedly the brightest minds to identify meaningful gaps for clinical research, but a global shortage of experts in medical statistics has hindered the conversion of great research ideas into robust research proposals.
The growing complexity of modern healthcare research would mean the need to equip investigators with robust statistical knowledge. However, healthcare workers often face challenges in obtaining professional biostatistical advice, which is expensive and requires communication at dedicated time and venue. Young investigators often hesitate to ask questions or share preliminary, unformed ideas, which hinders the development and ideation of new studies.
Biostatistical guidance during the conceptualization of a study can enhance decisions on study design, variable selection and handling, sample size determination, and the choice of appropriate statistical tests. It can save investigators time by preventing design flaws and accelerating the progress of clinical studies.
With the advent of artificial intelligence, natural language processing, large language models and chatbots, we harnessed the new technologies and developed an AI-powered biostatistics chatbot. We postulate that AI biostatistics chatbot could improve the user's research skills and empower healthcare worker to conduct clinical studies.
Objectives :
The project "AI biostatistician" has two aims: First, to engineer an AI-powered biostatistics chatbot, providing 24x7 guidance regarding study design, variables handling and statistical test selection to researchers. Second, to assess whether the introduction of the chatbot will improve the user's clinical research skills and confidence.
Methodology :
The prototype of an AI biostatistician chatbot was developed using prompt engineering by a medical researcher-statistician, based on the base model of ChatGPT 4o. The framework of formulating a clinical study and the considerations behind was imparted in the AI model. The textual response of the AI model was tuned to show an encouraging gesture to engage and empower the user. The courtesy and the safety of the output was guarded by restricting the role and the content of the output.
The usefulness of the output was validated by professional statistician from School of Public Health and Primary Care, The Chinese University of Hong Kong.
To make this chatbot accessible to HA colleagues, KCC IT and KCC innovation committee has kindly assisted to provide an internal AI platform for the potential deployment of this chatbot.
Users were invited to Pre- and post- questionnaire to assess their interest, competency and motivation towards conducting clinical studies before and after using the AI biostatistician chatbot.
Result & Outcome :
The chatbot was successfully developed and functions correctly. Upon providing the clinical question of interest, it will guide the user step-by-step to consider about determining the outcome of interest, the covariates to collect (including suggesting what and in what form to collect), the choices of study design, statistical analyses involved and the assumptions behind, as well as sample size considerations.
The chatbot was tested by 21 trainees in the Department of Clinical Oncology, Queen Elizabeth Hospital.
Pre-questionnaire revealed that most of the trainees are interested in conducting their own clinical research and is able to identify knowledge gaps in daily clinical practice. However, they reported only moderate confidence in turning their idea into an actual proposal and awareness of the aspects involved in study formulation. While they report moderate confidence in handling outcome measurement, their confidence in choosing and handling covariates and choosing statistical tests were weak. They reported the median need of 3 years to start their own clinical study.
After using AI biostatistician, the user reported improved confidence to turn research gap into study proposal. They reported increased confidence in choosing appropriate study design, choosing and handling covariates and choosing statistical tests. They reported a shorter median time needed to start their study of 2 years.
AI biostatistician holds the promise in democratizing the access to statistical advice. Healthcare workers of any working experience and specialty could brainstorm their research ideas with this professionally curated AI chatbot, so that they can feel safe to grow their valuable yet unformed ideas into a well-considered study plan in private, without worries and undue delay.
