GIS Agent

Introduction

For university students majoring in GIS and professionals working in the GIS or surveying industry, the first step in conducting academic research or carrying out work is often to acquire various types of geographic data and use specific models and tools to process it.

However, commonly used software like ArcGIS and QGIS have certain drawbacks. For instance, their interfaces are not user-friendly, they lack explanations for specific domain terminology, and some documentation is incomplete. This results in many users spending a lot of time and effort searching for and trying different solutions on the internet when handling certain tasks with GIS software. Moreover, as a commercial software, ArcGIS requires a high licensing fee to access its full range of features, which is unaffordable for many students.

Therefore, we have fully utilized the powerful computing capabilities and extensive storage capacity of the AMD Radeon™ PRO W7900 to design an AI assistant based on LLM (Large Language Models) for geographic knowledge discovery and geospatial processing. It accepts user input in natural language, converts it into corresponding task planning, and selects appropriate tools, models, and data from predefined options to execute the tasks. The LLM then organizes and returns the execution results to the user. This project reduces reliance on complex GIS terminology and interfaces through a natural language interface, allowing users to interact with the system more intuitively and efficiently. It also automates task planning and execution, significantly reducing the time and effort required for users to process geographic data and perform spatial analysis tasks.

Flow Chart

The overall process can be roughly diveded into four parts.

First, the user can simply input the instruction. Optionally, the user can also provide some relevant data, if not, the system will decide whether to download data from the internet in the third phase.

Second, in the understanding phase, the user first interacts with the system using natural language. The system initially calls a general large language model such as gpt4 to understand and plan the user's instructions, breaking them down into multiple subtasks. It then calls a function-calling LLM such as nexusraven to retrieve the appropriate tools from a predefined tool library and constructs suitable parameters and calling sequences.

Third, in the execution phase, the system first decides whether the task need to download external data. If needed, it will use some predefined tools to download data, uses it as the input and execute the corresponding tasks with specified tools.

At last, the system will output results in multiple format. Including regular text results that discribe if the process worked; the graph of the task execution chain that is generated using networkx, which is a powerful tool to generate and render graph data; and the results directly displayed on map using leaflet, which is a leading open-source JavaScript library to create interactive maps.

Large Language Models

The used large language models is listed below. First we use GPT-4 to model and decompose tasks based on user instructions, and then we use NexusRaven to call the tools according to the task decomposition. GPT-4 can also be replaced with LLaMA 3, which may result in a slight decrease in performance.

GPT-4 is a cutting-edge large language model developed by OpenAI. It excels in understanding and generating human-like text, capable of performing a wide range of tasks such as answering questions, writing essays, summarizing content, and engaging in conversations. Website: https://openai.com/index/gpt-4/

NexusRaven is an open-source and commercially viable function calling LLM that excels in function calling capabilities. Website: https://github.com/nexusflowai/NexusRaven-V2/tree/master

Llama3 is introduced by Meta in April this year. It, demonstrates excellent performance on a wide range of industry benchmarks and offers new capabilities, including improved reasoning. Website: https://github.com/meta-llama/llama3

Predifined Tools

We have implemented over 20 commonly used algorithms in geographic information systems. By freely combining these algorithms, various complex geographic problems can be quickly and accurately solved, providing personalized and efficient geographic information analysis solutions.

• Download Data Tools use APIs provided by Amap (Gaode) and OpenStreetMap to download various types of geographic data from the internet, such as urban road network data, POI data, route planning data, and more. This ensures that users can quickly and accurately obtain the necessary geographic data, providing a solid foundation for subsequent geometric analysis, geospatial analysis, and geospatial processing.
• Geoprocessing Tools include functions such as buffer, clip, convex hull, difference, dissolve, and intersection. These tools are used to perform spatial operations and manipulations on geographic data. They help in creating buffer zones around features, clipping areas to specified boundaries, generating the smallest convex polygon that encloses a set of points, calculating the difference between geometries, merging adjacent polygons, and finding common areas between overlapping features. Together, these tools enable comprehensive spatial data processing and analysis.
• Geometry Tools include functions such as centroids, Voronoi (Thiessen) polygons, and Delaunay triangulation. These tools are used to perform geometric calculations and constructions on spatial data. Centroids determine the central point of a polygon, Voronoi polygons divide space into regions based on proximity to a set of points, and Delaunay triangulation creates a network of non-overlapping triangles connecting a set of points. Together, these tools facilitate detailed geometric analysis and spatial data structuring.
• Analysis Tools include functions such as line intersection, mean coordinates, and nearest neighbor analysis. These tools are used to perform precise geometric calculations and spatial analyses on geographic data. Line intersection identifies where lines cross, mean coordinates calculate the average location of a set of points, and nearest neighbor analysis determines the proximity relationships between spatial features. Together, these tools enable detailed examination and interpretation of geometric properties and spatial relationships within the data.

Interface description

The image below shows the main input section of our system. In the files section, users can input geographic data saved in the SHP file format. In the input text section, users can enter instructions of their desired tasks. In the Examples section, users can select one of the four provided examples, and the system will automatically fill the corresponding files and input text sections. This helps users understand how to use the system. Users can also freely modify the content of the examples according to their needs. More detailed information about the examples is provided in the next section.

The image below shows the main output section of our system. In the upper part, we directly display the system's output on the corresponding location on the map. This helps users obtain more intuitive and interactive results. The lower part contains textual output, which describes some details of the system's execution. When the system executes successfully, it will notify the user of the success and show where the results are saved. When the system execution fails, it will inform the user of the reason for the failure.

The image below shows the system's additional output section. This section displays the tools that the system ultimately executed and explains why these tools were called. Additionally, we use networkx to visualize the tool invocation chain as an image. This helps users better understand the system's decision-making process.

Examples

1. Search places that is in the path from A to B.

This example illustrates how the system can automatically search for specific places (such as subway stations) near a particular route.

By further analyzing the distribution of subway stations near the some main road, we can identify which stations in certain areas can effectively alleviate traffic pressure on the main road. Additionally, by optimizing the interchange facilities at these subway stations, more citizens can be encouraged to use public transportation, thereby reducing traffic flow on the main road.

2. Compute the convex hull of points within a certain range.

This example shows how the system computes the convex hull of some points. A convex hull is the smallest convex polygon that can enclose these points which is useful in many situations.

By calculating the convex hull of these points, we can define the management area for these points. For example, calculating the convex hull of rare plant distribution points can help us delineate the boundaries of a protected area.

3. Compute two regions' intersection.

This example demonstrates how the system calculates the overlapping area of two different regions (such as commercial and residential zones).

With this result, we can optimize public transportation facilities based on traffic flow data within the overlapping area or analyze the social service needs of the overlapping area to improve the layout of public service facilities.

4. Search locations in a specific area.

This example shows how the system identifies all specific locations within a particular area.

By finding all the subway stations in the area, we can analyze the coverage area of the subway stations and the connecting transportation options (such as buses and bike lanes), optimize the project's transportation planning, and improve transportation convenience.

Environment

GPU：AMD Radeon™ PRO W7900

System Environment：Ubuntu 20.04.5 LTS

ROCm version：6.1.6

python Environment：3.11

Dependencies Configuration

Install amdgpu

1. Download amdgpu-install

From https://repo.radeon.com/amdgpu-install/latest download the corresponding version driver installation program.

For ubuntu20.04, we need to download the program in dictionary"/ubuntu/focal".

2. Upgrade operation system

In the terminal, run commands:

sudo apt-get update
sudo apt-get dist-upgrade

3. Install amdgpu-install

go to the /Download dictionary

cd ~/Downloads

Run the install command:

sudo apt-get install ./amdgpu-install_6.1.60102-1_all.deb
sudo apt-get update

4. Install driver

Run command:

amdgpu-install

Install ROCM

Run command:

# update system
sudo apt update
# installation command of rocm
sudo apt install amdgpu-dkms rocm

Install pytorch

Create a new conda environment, specify the environment name and python version:

#  specify the environment name "rocm6.1" and python version 3.11
conda create -n rocm6.1 python=3.11
# activate the new conda environment
conda activate rocm6.1

From https://pytorch.org/ get the installation commands of the corresponding system and drive version.

For rocm6.1, run:

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.0

Verify installation, use the following commands in the command line:

python
import torch
torch.cuda.is_available()

If the terminal prints "True", then installation is successful.

Ollama Configuration

Visit the Ollama official website https://ollama.com/ and copy the installation command for the Linux system.

Enter the terminal, paste the copied command to install Ollama, and follow the prompts to enter the password as required.

After the installation is complete, enter the command to check if Ollama is installed correctly

ollama serve

Copy the port provided by the terminal and open it in the browser. If it appears as shown in the picture below, the installation is successful.

Open a new terminal and enter the following command to download the large language models.

ollama pull nexusraven:13b
ollama create nexusraven:gis-agent -f nexusraven.Modelfile

python Dependency Configuration

First, clone this gitee repository to your local directory using this command and go to this directory.

git clone https://github.com/yhycsu/gis-agent.git
cd git-agent

Then, enter the following command in the terminal to complete the Python environment setup.

pip install requirements.txt

Run Example

After completing the above configurations, navigate to the project directory and run backend.

cd git-agent
python backend.py --model gpt4 --open_api_key <your_api_key>

If you do not have openai api key, you can use alternative models from ollama. We recommend using llama3:70b, which can be downloaded via ollama, then the above command can be replaced with the following:

ollama pull llama3:70b
python backend.py --model llama3:70b

Then open another terminal and run the demo program.

python demo.py

Then open the provided link to access the demo program page. Then, you can click on the dropdown menu in the Examples section and select one of the examples to get started as shown in the videos in the above section.

Future work

This project is just a start for utilizing large language models to analyze and process geospatial tasks. The following topics can be addressed as a follow-up to improve the processing pipeline:

Broaderfile format support

At the moment the project can only receive and process data stored in shapefile format. More file format like images, tables, voices needs to be supported.

Multilingual support

Support for multiple languages, allowing users from different linguistic backgrounds to use the system.

Integration of analysis report generation:

Add functionality to produce detailed analysis reports, including charts and statistics, to help users understand the results and perform subsequent analysis.

Acknowlegements:

• Much thanks to Professor Li from Central South University for providing guidance.
• Much thanks to Hunan Yayan Shushi Company for their support.
• Much thanks to AMD for organizing the competition and providing hardware support.