Title: Enhancing Kidney Tumor Diagnosis by Fine-Tuning the Multi-Modal Large Language Model Llama2-7B-DoctorGPT-based MiniGPT-4 with the MI210 Accelerator
AbstractAccurately interpreting medical images to distinguish between benign and malignant kidney tumors is crucial for early detection, effective treatment, and ongoing patient monitoring. In Taiwan, the ownership of medical images by hospitals and patients significantly hinders their dissemination. However, the use of offline large language models, such as Llama2-7B-DoctorGPT, which is capable of passing the US Medical Licensing Exam, can improve patient outcomes in tumor detection while safeguarding privacy. This enhancement can be achieved by fine-tuning a local multi-modal large language model (LM-LLM), Llama2-7B-DoctorGPT-based MiniGPT-4, to improve medical diagnostics for radiologists and oncologists. The computing power of the AMD Instinct MI210 Accelerator is well-suited for this task. The experimental design of this study is based on 810 high-quality annotated ultrasound images of kidney tumors from Asia University Hospital (AUH) in Taiwan and 636 ultrasound images of kidney tumors provided by the publicly available dataset on ultrasoundcases.info to fine-tune the Llama2-7B-DoctorGPT-based MiniGPT-4. Then, an additional 36 benign and 42 malignant ultrasound images of kidney tumors from ultrasoundcases.info were used for testing. The fine-tuned Llama2-7B-DoctorGPT-based MiniGPT-4 achieved test results with a sensitivity of 0.93 and a specificity of 0.86, outperforming 10 types of CNN models.
Comments