Einstein Research Agent using AMD Ryzen AI PC

The process of scientific discovery is, in effect, a continual flight from wonder. - Albert Einstein

1. Abstract

Embarking on a research journey often requires detailed methods, extensive data analysis, and rigorous experimentation. Researchers face challenges like designing valid experiments within strict regulatory guidelines and addressing privacy concerns. Additionally, they spend a lot of time gathering information from various sources and sifting through long academic texts to find specific details related to their topic of interest. Generative AI, or GenAI, offers exciting opportunities to improve research practices, especially for new researchers. This project aims to tackle the complexities of research by developing a comprehensive Generative AI platform called the "Digital Research Assistant (DRA), " powered by AMD Ryzen AI powered PC. The DRA will streamline the research process, making it easier to collect and analyze data, and quickly find relevant information in academic texts, thus enhancing the overall efficiency and effectiveness of scholarly work.

2. Introduction

To gain a deeper understanding of the challenges encountered by researchers, we conducted a survey among the university research scholars at our institute. Our goal was to identify the specific problems they face during their research journey. Link to the survey conducted: https://docs.google.com/forms/d/e/1FAIpQLSetPjpWOHjqaLYXC_1u43eSM_lEPNlt_LbxPFLXPgkI0KomnA/viewform?usp=sf_link

Issues Faced by Researchers

The findings from the above survey suggest that the researchers endure several hurdles throughout their research careers. Ideation and discovering gaps in the literature are frequently difficult, worsened by the time-consuming processes of locating adequate data sources, pertinent theories, suitable methodology, and appropriate journals for publication.

Participants stated that Literature review part is very time consuming and has numerous challenges, including identifying the correct keywords, databases, and applying right search strategy to find relevant papers. Other most challenging part was found to be writing as they struggle with compiling, paraphrasing, and maintaining the originality conforming to publication rules. Also, writing it properly and synthesising the large amount of literature into clear arguments is a difficult part.

Finally, data collection and analysis are often extremely time-consuming for researchers. They struggle to find data due to concerns about personal data exposure from breaches, and they must address privacy issues when training models on large datasets containing identifiable information. Additionally, much of core research is limited to text data, a challenge that also needs to be addressed.

We asked participants how they currently manage these challenges. As shown in figure below most of the participants do not use much technology to address these issues.

As a result, there is an obvious need for a digital research agent to assist in overcoming these challenges and streamlining the research workflow.

Features of our Solution

To address the challenges posed by the complexity of research endeavors and the evolving landscape of scholarly inquiry, we propose the development of a comprehensive Generative AI platform tailored specifically for researchers, referred to as “Digital research assistant(DRA)”.

DRA will set itself apart from other solutions by providing a range of cutting-edge capabilities intended to optimize different parts of the traditional research process particularly how ideation, literature review section and data collection and experimentation sections are performed in research.

• Realistic experiments using GenAI:

Creating realistic, immersive environments for experiments will be much faster with the help of foundation models, allowing the researcher to spend more time designing, conducting, and analyzing the experiment results rather than building an ecologically valid experiment itself. For example: Generate realistic-looking text for an experiment, like fake tweets. Generate variations of an image, such as fake profile pictures. These can then be used for a specific research problem.

• Using GenAI for synthetic data generation:

Utilising synthetic data generated by generative AI offers a compelling solution to mitigate the need for collecting and storing sensitive data. This approach alleviates the burden of compliance with increasingly stringent regulatory requirements while facilitating easier data sharing among researchers and organizations. Crucially, this approach diminishes the likelihood of personal data exposure resulting from data breaches and mitigates privacy concerns associated with training models on extensive datasets containing identifiable information. For example:Replace genuine text with generated text to mask the original data, such as visual storytelling, table-to-text generation, knowledge bases to-text generation, and so on.Other example include, using generated images instead of real pictures for domain specific images for training models.

• Generating multimodal datasets:

Harnessing foundational models offers an exciting opportunity to synthesize data from different modalities, thereby enriching research capabilities. Looking forward, these models show potential for seamlessly integrating and synthesizing a wide range of multimodal datasets, paving the way for novel research avenues and fostering innovation.

Introducing Digital Research Agent(DRA)

The Digital Research Assistant (DRA) enhances research efficiency by automating tasks like experiment design and data generation, saving time and resources. It aids compliance with regulations and privacy protection by generating synthetic data, and reducing reliance on sensitive information. Additionally, DRA's ability to generate multimodal data fosters versatility and innovation, empowering researchers to explore diverse datasets and innovate within their fields.

The solution works by employing various GenAI models, mainly text generation and image generation models in the backend, and providing users an Interface to interact with the DRA, either by text or voice. The models will be fine-tuned for research use cases, which are data generation, finding research papers, analyzing research papers, & assisting in collating research from various sources.

Some of the main features of DRA include :

• Mobile and web interfaces
• Voice & text inputs Generation of text for research use cases [automated flow, with control parameters]
• Generation of images for research use cases [ automated flow, with control parameters]
• Finding information about a topic from various sources
• Summarizing information from various sources
• Creating comprehensive documentation
• Assist in the implementation of codes from research papers [error & dependencies resolution]

By using the AI engine in Ryzen AI-enabled CPUs, this assistant can run without limiting the CPU and GPU, and therefore those resources can be used for main purposes like running a project, executing research projects, simulation, etc. This will help researchers as they won’t be sharing critical ideas and information with an external service, also addressing privacy concerns, which is an important criterion for researchers.

3. Instructions/steps

We received a Ryzen AI PC from AMD to complete our project. AMD Ryzen AI is the world’s first built-in AI engine on select x86 Windows laptops. This dedicated engine is built on the AMD XDNA™ architecture and is designed to offer lower latency and better energy efficiency. This integration optimizes efficiency by offloading specific AI processing tasks such as background blur, facial detection, and eye gaze correction, freeing up CPU and GPU cycles and enhancing system efficiency.We followed these steps to set up our PC and implement our project.

Enabling IPU

Before installing the tools for the AMD AI demo we need to enable the IPU ( IPU is the old name of NPU) as shown in the figure below.For more details you can follow:https://www.hackster.io/512342/amd-pervasive-ai-developer-contest-pc-ai-study-guide-6b49d8

Source: Screenshot IPU enabling taken by the Authors

Installing Ryzen-AI software

Before installing the software we need to install the prerequisites as given in the instructions:https://ryzenai.docs.amd.com/en/latest/inst.htmlInitially we faced an issue when it came to installing Visual Studio 2019, which is not readily accessible to individuals without a corporate Microsoft Developer account. For this install.bat can be changed to work with Visual Studio 2022.

Just edit the install.bat

and change line

reg query "HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\VisualStudio\16.0" >nul 2>&1

to

reg query "HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\VisualStudio\17.0" >nul 2>&1

I left the settings at the installation default and did not add the PATH. However, there were some failures during the setup afterwards, so I ended up adding the PATH manually as shown in the figure below.

After that Download the ryzen-ai-sw-1.1.zip, Ryzen AI Software installation package(ryzen-ai-sw-1.1.zip) and extract it. The Ryzen AI Software directory will be referenced many times later during inference, so make sure to store it in a fixed location such as under the C drive, not in a temporary location.

Start CMD as administrator as before.

Running The Demo

Run the following command.

C:\Users\user\Downloads\ryzen-ai-sw-1.1\ryzen-ai-sw-1.1>conda activate ryzenai-1.1-20240730-025720.

Remember, in my case it was ryzenai-1.1-20240730-025720. replace it with your virtual environment name.

Then Let's test it.

We further proceeded with our project the complete code and instructions are also provided in the GitHub repository. Sharing the steps in brief below :

1. Download Pre-Trained Model from Hugging Face

In this step, we will download a pre-trained OPT-1.3B model from Hugging Face. You can modify the run.py script to download a pre-trained model from your or your company's repository.

cd Chatbot-with-RyzenAI-1.0\
python run.py --model_name opt-1.3b --download

The downloaded model is saved in the subdirectory \opt-1.3b_pretrained_fp32\.

2. Quantize the downloaded model from FP32 to INT8

Once the download is complete we quantize the model using the following command:

python run.py --model_name opt-1.3b --quantize

3. Evaluate the model and Deploy it with the Chatbot App

Next, we evaluate the quantized model and run it targeting AIE (IPU in Ryzen 7940HS) with the following command. Notice the model path is set to the location where the INT8 quantized model was saved during the previous step.

python run.py --model_name opt-1.3b --target aie --local_path .\opt-1.3b_smoothquant\model_onnx_int8\

For the first run, the model is automatically compiled by an inline compiler. Compilation is also a two-step process:

• First, the compiler identifies the layers that can be executed in the AIE. Then, it creates sets of subgraphs, one set for AIE and the other set for CPU with the remaining layers.
• Secondly, it creates instruction sets for each of the subgraphs targeting their respective execution unit. These instructions are executed by two ONNX execution providers (EP), one for the CPU and one for the AIE.

After that, we launched the Chatbot with the INT8 quantized model saved at the following location \opt-1.3b-smooothquant\model_onnx_int8\

python gradio_app\opt_demo_gui.py --model_file .\opt-1.3b_smoothquant\model_onnx_int8\

The Chatbot application runs on the localhost on port 1234. To open the application, we used our browser to navigate to http://localhost:1234

4. Results

Finally, we got a GUI, in which we can interact with our assistant. We can ask research-related questions, and by using some extensions/cloud GUIs, we can input some research papers and can ask the assistant questions, based on the input paper.

5. Future Work

In the future, we aim to enhance this project with features like synthetic data generation and critical thinking capabilities by fine-tuning the model. We also plan to add advanced research tools, making this system a versatile, one-stop solution for early researchers' endeavors. Stay tuned for exciting updates as we continue to push the boundaries of what's possible!

Future Updates