Pilot Study of Retrieval-Augmented Generation Model in Recommending Traditional Chinese Medicine Formulations

Yan-Wo Chan, Po‐Yu Huang, Zhi-Liang Chen, C. Wang, Wen‐Chen Lin, Jung-Peng Chiu, Yi-Chun Chiu, Yun‐Hsuan Lin, E X Huang, Simon See, Kang‐Ping LinWorld 2025 · DOI · OpenAlex

Abstract

Retrieval-Augmented Generation (RAG) is a method used to optimize the output of large language models (LLMs). This study investigates the feasibility of using an LLM within a RAG framework to generate recommendations for Traditional Chinese Medicine (TCM) formulations. The study employs the mixtral-8x7b model as the LLM within the RAG architecture, utilizing clinical records from outpatient TCM visits as external data sources for generating TCM formulation recommendations. The recommendations from the RAG-based LLM are compared with those generated by the ChatGPT 3.5 model, evaluating their consistency with actual clinical prescriptions. Results indicate that the RAG-based LLM achieved an average score of 74, demonstrating a high level of alignment with clinical prescriptions across the cases studied. In contrast, the ChatGPT 3.5 model only achieved an average score of 25, primarily due to inconsistencies in the generated recommendations, which rendered them clinically unusable. The study concludes that while the RAG-based LLM shows potential in generating TCM formulation recommendations, there remains a need for improvement in the model’s accuracy.
BibTeX
@inproceedings{chan2025pilot,
  author = {Yan-Wo Chan and Po‐Yu Huang and Zhi-Liang Chen and C. Wang and Wen‐Chen Lin and Jung-Peng Chiu and Yi-Chun Chiu and Yun‐Hsuan Lin and E X Huang and Simon See and Kang‐Ping Lin},
  title = {{Pilot Study of Retrieval-Augmented Generation Model in Recommending Traditional Chinese Medicine Formulations}},
  booktitle = {World},
  year = {2025},
  doi = {10.1007/978-3-031-86323-3\_38},
}

Loading related content...

Figure