ROCmedic

0. Introduction

In recent years, advancements in medical imaging technology and artificial intelligence have significantly enhanced the capabilities of diagnostic tools used in healthcare. The integration of deep learning and GPU acceleration has revolutionized the accuracy and efficiency of medical image analysis, offering new possibilities for early detection and treatment of various diseases. ROCmedic is an innovative medical imaging analysis software developed on the AMD ROCm platform, specifically designed to meet the needs of healthcare institutions and professionals. By leveraging state-of-the-art AI techniques, ROCmedic provides comprehensive diagnostic support for electrocardiogram (ECG) detection, breast cancer analysis, and X-ray diagnosis. This technical report details the development environment, architecture, data processing workflows, and replication instructions for ROCmedic, highlighting its potential to transform clinical diagnostics through high-performance computing and advanced machine learning models.

1. ROCmedic

ROCmedic is a medical image analysis software based on the AMD ROCm platform, designed for healthcare organizations and professionals, aiming to provide efficient and accurate diagnostic tools. The software integrates the latest artificial intelligence technology and supports a wide range of functions such as electrocardiogram (ECG) detection, breast cancer detection and X-ray detection, providing comprehensive support for clinical diagnosis.

The user interface of ROCmedic is designed with clarity and functionality in mind. The main page features a clean layout where the central area, typically displaying a dynamic background when no image is uploaded, transitions to show the uploaded medical image for analysis. The left-hand navigation bar provides quick access to three primary functionalities: X-ray Diagnosis, Breast Tumor Analysis, and ECG Analysis. Users can easily select the desired function from the navigation bar and proceed to upload medical images by clicking the "Upload Images" button in the central area. This intuitive interface is built using the Vue.js framework, ensuring a responsive and seamless user experience. The image analysis workflow is powered by a Flask API backend, which integrates with AMD ROCm and PyTorch to deliver high-performance deep learning inference. By streamlining the upload and analysis process, ROCmedic enhances diagnostic efficiency and accuracy, providing valuable support for clinical decision-making.

2. Development Environments

ROCmedic leverages the computational power of the AMD Radeon W7900 for both training and inference tasks, utilizing the AMD ROCm platform for GPU acceleration. The development and deployment environment are configured with the following specifications:

• Operating System: Ubuntu 22.04 LTS, chosen for its stability and support for advanced GPU functionalities.
• Central Processing Unit (CPU): Intel Core i5-10500, providing reliable performance for managing data preprocessing and other CPU-bound tasks.
• Deep Learning Framework: PyTorch 2.3.1 with ROCm 6.0 support, enabling seamless integration with AMD's GPU acceleration capabilities and ensuring optimal performance for deep learning models.
• GPU Acceleration Platform: ROCm 6.0, which offers a robust and efficient environment for developing and deploying high-performance GPU-accelerated applications.

This configuration ensures that ROCmedic operates with maximum efficiency, leveraging state-of-the-art hardware and software to deliver precise and rapid medical image analysis.

3. Product Design Architecture

The architecture diagram of ROCmedic delineates the core functionalities, technology stack, and data processing workflows integral to the software.

In order to achieve the three functions it contains, ROCmedic's backend reasoning is a pipeline consisting of three independent models: a model for diagnosing X-ray CT images, a model for analyzing ECG images, and an analysis model for analyzing the benign and malignant nature of breast cancer trained based on medical image DICOM data.

3.1 Core Features

ROCmedic provides three primary diagnostic capabilities:

• ECG Analysis: Leveraging advanced algorithms to interpret electrocardiogram data, enabling the detection of heart conditions.
• X-ray Diagnosis: Utilizing deep learning models to analyze X-ray images, facilitating the identification of various medical conditions.
• Breast Tumor Analysis: Specializing in the detection and analysis of breast tumors, providing critical support in cancer diagnosis.

3.2 Framework and Platform Dependencies

The development of ROCmedic relies on a robust combination of hardware and software frameworks:

Deep-Learning Platform:

• PyTorch: Version 2.3.1+rocm6.0, a powerful deep learning framework that supports dynamic computational graphs and GPU acceleration.
• AMD ROCm: Version 6.0, a comprehensive platform for GPU-accelerated computing, optimizing performance for deep learning applications.
• AMD Radeon: Utilizes the AMD Radeon W7900 GPU, offering high-performance computing power necessary for training and inference tasks.

LLM Solution:

• VisualGLM-6B Multimodality Model: This model integrates multiple modalities of data, providing a robust solution for large language model applications in medical diagnostics.

3.3 Technology Stack

(1) Front-End Technology Stack

• Page Design: Ensures a user-friendly interface, allowing seamless navigation and interaction.
• Flask API Access: Establishes communication between the front-end and back-end, facilitating data exchange and processing.
• Framework (Vue): The front-end is built using the Vue.js framework, known for its simplicity and flexibility in creating interactive user interfaces.

(2) Back-End Technology Stack

• Multimodal Functional API Packaging: Encapsulates various functionalities to support multimodal data processing.
• ROCm Inference Backend Deployment: Deploys inference models on the ROCm platform, utilizing GPU acceleration for efficient processing.
• Python Flask API: Manages back-end operations and integrates with the GPU acceleration platform.

3.4 Data Types and Processing

(1) Data Types

ROCmedic processes a variety of medical imaging data:

• Medical Imaging DICOM Files: Standard format for storing and transmitting medical images.
• X-ray Structure Image Dataset: A curated dataset used for training and validating X-ray diagnostic models.
• Diagnostic Data: Includes annotations and diagnostic results essential for model training and evaluation.

(2) Data Processing

The data processing pipeline involves several critical steps to ensure data quality and model performance:

• Data Normalization Operation: Standardizes data to ensure consistency and reliability across different datasets.
• DICOM Metadata Extraction: Extracts relevant metadata from DICOM files, facilitating accurate analysis and interpretation.
• Finetuning Output Data: Prepares data for fine-tuning the large language model, optimizing it for specific diagnostic tasks.

This structured approach ensures that ROCmedic harnesses advanced GPU-accelerated deep learning techniques to deliver precise and efficient diagnostic support for medical professionals.

4. Our Work

Our work focuses on leveraging advanced data processing techniques to enhance the diagnostic capabilities of ROCmedic. Below are the key steps and methodologies we implemented:

(1) Breast Cancer Imaging Data

• We collected DICOM data from the National Biomedical Imaging Archive (NBIA) for breast cancer.
• Developed scripts to convert DICOM files into trainable image formats.
• Extracted corresponding diagnostic results and suggestions, processed into JSON files for fine-tuning models.

(2) X-ray Imaging Data

• Acquired datasets from the OpenI database, incorporating physician diagnostic results.
• Processed these into JSON files suitable for model fine-tuning.

(3) ECG Data

• Gathered ECG datasets with pre-classified diagnostic types.
• Integrated category labels with prompts, exported as JSON files for training purposes.

(4) Indexed Database (IndexedDB)

• Utilized browser-native IndexedDB to enable efficient querying of diagnostic histories, ensuring rapid access and retrieval of past diagnoses.

5. Reproduction Instructions

The following section provides detailed instructions for reproducing the data processing workflows used in ROCmedic. This includes steps for handling breast cancer, ECG, and X-ray imaging data, ensuring the data is in a suitable format for model training and fine-tuning.

5.1 Data Processing

5.1.1 Breast Tumor Data

Data Acquisition

• Download the relevant data from the Cancer Imaging Archive website, which offers data downloaders for both Linux and Windows platforms.
• Save the downloaded data onto your server or local machine. The data should appear as shown in the example below.

Data Processing

Navigate to the specified directory using the command:

cd Biochemical

Execute the following commands sequentially to process the data:

python dataprocessing.py   # This step may take a considerable amount of time.
python find_emptyimages.py
python convert_json.py
python json_normalization.py
python add_suggestion.py

These scripts will convert DICOM files into trainable image formats, add labels and prompts, and include diagnostic suggestions. The final output will be JSON files ready for model fine-tuning.

[Note]: Due to the large volume of data (646GB), full-scale data training was not feasible on our server. Users with sufficient hardware resources are recommended to perform full data fine-tuning to achieve optimal model performance.

5.1.2 ECG Data

Data Preparation

Navigate to the ECG data directory:

cd ECG

Execute the following commands to process the ECG data:

python dataprocessing.py
python add_suggestions_to_json.py

These scripts will handle the data processing, transforming it into a format suitable for model training.

5.1.3 X-ray Data

Data Handling

• The X-ray data has been pre-processed and does not require additional steps.
• However, for those interested in the processing logic, reference the code within the Xray/data directory, which contains comprehensive data handling procedures.

5.2 Reproduction Process

This section outlines the steps required to reproduce and test the individual components of ROCmedic. It includes detailed instructions for debugging and running the application, ensuring that users can effectively replicate the processes for ECG, X-ray, and Breast Tumor analysis.

5.2.1 Individual Debugging

For debugging individual components, follow the steps below. Here, ECG analysis is used as an example.

(1) Navigate to the ECG Directory:

cd ECG/VisualGLM-6B

(2) Start the Backend Service: Run the following command to initiate the backend service:

python api_ecg.py

Upon successful initiation, the backend service should display output indicating it is running on various addresses.

(3) Start the Frontend Service: In the frontend directory, start the service using:

serve -p 8080

If serve is not installed, run npm install serve first. Upon successful start, the terminal should display the service running at local and network addresses.

OR, you can simply cd in the Vite-project folder, and run the following command to start the front-end service:

npm install --legacy-peer-deps
npm run dev

With the command above, the front-end service will also get online, you can choose the way you want! 😉

(4) Access the Service in a Browser:Open the specified local address (e.g., http://localhost:8080) in a web browser. Navigate to the ECG Analysis tab, upload an image, and view the returned results.

The steps for debugging X-ray Diagnosis and Breast Tumor Analysis are similar. Ensure you select the appropriate project in the frontend during testing.

5.2.2 Full System Testing

(1) Start All APIs: To initiate all services simultaneously, execute the following script:

bash start_apis.sh

Upon successful initiation, you should see a confirmation message indicating that all services are running.

(2) Access and Use the Services: Open the web interface and interact with the three diagnostic services (ECG, X-ray, and Breast Tumor Analysis). Each service should be accessible and functional, allowing for simultaneous usage.

(3) Stop All APIs: Once testing is complete, stop all running services using:

bash stop_apis.sh

This command will terminate all active services, ensuring that resources are freed.

By following these steps, users can effectively debug and test the individual components as well as the full system of ROCmedic, ensuring that all functionalities are operating as intended.

PS: All the checkpoints files(.pt) can be found in Google Drive Link: https://drive.google.com/drive/folders/1FzqcNIGAykXaL-yM5MF45zPhWt5y9Yf2?usp=drive_link

Dataset needs to be downloaded by yourself.