Amith, Muhammad

Abstract Objective Human Papillomavirus (HPV) vaccine is an effective measure to prevent and control the diseases caused by HPV. However, widespread misinformation and vaccine hesitancy remain significant barriers to its uptake. This study focuses on the development of VaxBot-HPV, a chatbot aimed at improving health literacy and promoting vaccination uptake by providing information and answering questions about the HPV vaccine. Methods We constructed the knowledge base (KB) for VaxBot-HPV, which consists of 451 documents from biomedical literature and web sources on the HPV vaccine. We extracted 202 question-answer pairs from the KB and 39 questions generated by GPT-4 for training and testing purposes. To comprehensively understand the capabilities and potential of GPT-based chatbots, 3 models were involved in this study: GPT-3.5, VaxBot-HPV, and GPT-4. The evaluation criteria included answer relevancy and faithfulness. Results VaxBot-HPV demonstrated superior performance in answer relevancy and faithfulness compared to baselines. For test questions in KB, it achieved an answer relevancy score of 0.85 and a faithfulness score of 0.97. Similarly, it attained scores of 0.85 for answer relevancy and 0.96 for faithfulness on GPT-generated questions. Discussion VaxBot-HPV demonstrates the effectiveness of fine-tuned large language models in healthcare, outperforming generic GPT models in accuracy and relevance. Fine-tuning mitigates hallucinations and misinformation, ensuring reliable information on HPV vaccination while allowing dynamic and tailored responses. The specific fine-tuning, which includes context in addition to question-answer pairs, enables VaxBot-HPV to provide explanations and reasoning behind its answers, enhancing transparency and user trust. Conclusions This study underscores the importance of leveraging large language models and fine-tuning techniques in the development of chatbots for healthcare applications, with implications for improving medical education and public health communication.

Abstract Background: HPV vaccine is an effective measure to prevent and control the diseases caused by Human Papillomavirus (HPV). This study addresses the development of VaxBot-HPV, a chatbot aimed at improving health literacy and promoting vaccination uptake by providing information and answering questions about the HPV vaccine; Methods: We constructed the knowledge base (KB) for VaxBot-HPV, which consists of 451 documents from biomedical literature and web sources on the HPV vaccine. We extracted 202 question-answer pairs from the KB and 39 questions generated by GPT-4 for training and testing purposes. To comprehensively understand the capabilities and potential of GPT-based chatbots, three models were involved in this study : GPT-3.5, VaxBot-HPV, and GPT-4. The evaluation criteria included answer relevancy and faithfulness; Results: VaxBot-HPV demonstrated superior performance in answer relevancy and faithfulness compared to baselines (Answer relevancy: 0.85; Faithfulness: 0.97) for the test questions in KB, (Answer relevancy: 0.85; Faithfulness: 0.96) for GPT generated questions; Conclusions: This study underscores the importance of leveraging advanced language models and fine-tuning techniques in the development of chatbots for healthcare applications, with implications for improving medical education and public health communication.

Abstract Objective Social determinants of health (SDoH) play critical roles in health outcomes and well-being. Understanding the interplay of SDoH and health outcomes is critical to reducing healthcare inequalities and transforming a “sick care” system into a “health-promoting” system. To address the SDOH terminology gap and better embed relevant elements in advanced biomedical informatics, we propose an SDoH ontology (SDoHO), which represents fundamental SDoH factors and their relationships in a standardized and measurable way. Material and Methods Drawing on the content of existing ontologies relevant to certain aspects of SDoH, we used a top-down approach to formally model classes, relationships, and constraints based on multiple SDoH-related resources. Expert review and coverage evaluation, using a bottom-up approach employing clinical notes data and a national survey, were performed. Results We constructed the SDoHO with 708 classes, 106 object properties, and 20 data properties, with 1,561 logical axioms and 976 declaration axioms in the current version. Three experts achieved 0.967 agreement in the semantic evaluation of the ontology. A comparison between the coverage of the ontology and SDOH concepts in 2 sets of clinical notes and a national survey instrument also showed satisfactory results. Discussion SDoHO could potentially play an essential role in providing a foundation for a comprehensive understanding of the associations between SDoH and health outcomes and paving the way for health equity across populations. Conclusion SDoHO has well-designed hierarchies, practical objective properties, and versatile functionalities, and the comprehensive semantic and coverage evaluation achieved promising performance compared to the existing ontologies relevant to SDoH.