Nature AI Avatars

"Nature Avatars" is a research project at the intersection of technology, art, and the different Ways of Being that exist in our world. It attempts to develop hybrid manifestations that can serve as expression medium for beyond-human entities (from individual trees to entire watersheds). Such manifestations may play the roles of ambassadors, advocates and educators while bridging the worlds of their source entities and human communities and systems.

The avatars are manifest via digital tools while maintaining a direct connection to their source entities via real time remote sensing devices. Being powered by AI, they have the ability to interpret, impersonate and articulate the state/activity of their counterpart while interacting with a user.

One of the goals of such avatars is turning technical data and literature into non-technical insights to the general public. And this is where Generative AI (more specifically Large Language Models), play a big role. By programming these models to receive, analyze, interpret and impersonate data from the natural, we create a more natural interface for humans to engage with both the data and its underlying insights.

Another aspect of their hybrid nature is that they possess dual embodiment for sure interactions: in the digital sphere, they may exist as agents embedded in varied applications and even 3D modeled characters. At the same time, they are brought to life in the physical world through artifacts and sculptures, often made from natural materials intertwined with embedded devices running their operating code.

Rio Sagrado Avatar - Prime Example

The first Nature Avatar instance, born in August 2023, is a digital/physical representation for the Rio Sagrado (Morretes-PR, Brazil), designed for educational and political purposes.

The Rio Sagrado ("Sacred River"), runs through the largest well preserved continuum of Atlantic Rainforest in Brazil, and this whole region is currently under the threat of an international railroad project that seeks to cut through many of the water sources of this river and watershed.

The local community has been mobilizing in face of such madness, with one of the strategies being the understanding of the Rio Sagrado as a living entity. In this sense, my contribution to this process lies in the creation of multimedia mediums for the expression of the River, like a short film, and the avatar itself. The latter has been developed as a manifestation of the River's history and culture via participatory processes with the local community, leading up to the design of a sculpture by a local artisan that sought to highlight the local fauna and flora in his craft. The sculpture permanently sits at the visiting center of the "Salto do Sagrado" waterfall, serving as an attraction to locals and tourists.

Embedded in the sculpture is a Raspberry Pi (model 5) that locally runs Speech-to-Text and Text-to-Speech models, controls LED effects, stream audio from and to user, and interact with an online LLM (GTP-4 from OpenAI, via Retune API calls, sponsored by GiveETH). This LLM is fine-tuned with over 30 scientific papers on the river's watershed, as well as community accounts on the local culture. And one can interact with that via the sculpture itself, or a dedicated website, or even Whatsapp. We have also created a metaverse experience for digital immersion in the Rio Sagrado alongside its avatar.

The Rio Sagrado Avatar also features prototype-level sensors that collect data from a water spring behind the sculpture, sending the data to an online database and to a small display in the sculpture.

The sensing stack is powered by an Adafruit Feather ESP32s3 with Li-Po battery and composed of: DF Robot's DS18B20 Water Temperature sensor; Gravity Analog TDS Meter; Gravity Analog Capacitive Soil Moisture Sensor; and Adafruit BME680 (Temp, Pressure, Humidity and Gas). The data is sent to a dedicated MySQL database with last 50 results displayed online at a monitoring page. Unfortunately, the Rio Sagrado Avatar itself is still not accessing the logged data.

Nonetheless, the most important thing about this whole process is that it has actually catalyzed local conversations around the recognition of the rights of the Rio Sagrado in the municipality, as showcased below during a community meeting that we had with actual lawyers who wrote draft bills for the recognition of the River's rights.

Lahn River Avatar

The second Nature Avatar, born in April 2025, is a hybrid representation for the Lahn River, developed in the context of the Planetary Agency Fellowship and Residency- alongside artist Ingvild Syntropia - as part of the Panel the Planetary Thinking at JLU in Giesen, Germany.

With a sculpture co-created by Ingvild and I with foundational support from the Magie Makerspace, the Lahn Avatar also makes use of Large Language Models as expression mediums for the cultural and scientific knowledge around the River. However, in this case, multiple open source models were combined to give life to the avatar (Mistral, Gemma 3 families), as per provision by the residency's hosting institution (JLU, via GWDG). This required the development of a unique architecture for their optimized functioning. Furthermore, the Lahn River avatar expands the first concept implementation by to include:

• Greater user interactivity via a live camera feed embedded into the avatar sculpture:

A Raspberry Pi (model 5) was also used to run Speech-to-Text and Text-to-Speech models, control LED effects, audio and video streaming and for interaction with the custom LLM framework, hosted at a virtual server at JLU's servers.

• Integrating Real-time River data monitoring with the Avatar through research-grade hydrology sensor module:

For this, we have chosen to use Atlas Scientific's Wi-Fi Aquaponics Kit, featuring research grade probes for water pH, Dissolved Oxygen, Temperature, Conductivity, as well as air CO2 concentration and Humidity. Moreover, the kit comes conveniently integrated with ThingSpeak for IoT Projects, enabling easy data storage and access via API calls. By utilizing LLM Toll Calling we are able to provide the sensor data to the Lahn Avatar upon request, in real time.

With sensors installed at the Lahnfenster river observatory in Giessen, the sculpture stands open to the public as both a poetic and political gesture as shown by Giessen's mayor, Alexander Wright, asking the first question to the river avatar during its opening at Lahnfenster.

The Lanh avatar can also be reached via a dedicated website. However, it was also made present in a first-of-its-kind digital forum, designed in partnership with Consider.it, towards structured collective deliberation between people and the River (via is avatar), around the question of What are the biggest opportunities for the Lanh's future?

Ongoing

We are actively developing this project, by both providing maintenance/upgrades to the avatars that have been developed and considering future applications. For instance, we are currently working on integrating latest speech-to-speech models to the Lahn Avatar, so as to improve German proficiency and reduce lag response time. Furthermore, we’re currently at the design phase of a new Nature Avatar. The Marumbi River Avatar.

This is a very special one because it is growing in partnership with “Terra Escola”, a nascent nature-and-community based School in the rural area of the Marumbi watershed - a school that my family and I are co-founding with other families. And we’re currently seeking help to make this vision come true. The main technical goals of this particular Avatar are:

• Monitor the physical properties and contaminants in the Marumbi river, as well as the local fauna and flora surrounding it - in the largest well-preserved continuum of one of the most biodiverse biomes in the world.
• Monitor and prevent harm from potential natural disasters in the region, which has a history of floods and landslides

As such, our vision is that this Avatar will grow with the school, co-created by the students and community as part of the school’s curriculum and programs. This way, we hope that it can function as an innovative learning platform, allowing kids and adults to learn about their bioregion and how to engage with it more consciously and regeneratively.

Finally

With all of this work, it's our hope that the Nature Avatars concept, inspired by the underlying philosophical perspective of Symbiocracy, will help us imagine and prototype emerging futures that more deeply integrate the rights, sentience and will of our more-than-human planet. Especially as we continue to apply remote sensing and AI technologies towards the understanding of beyond-human "language", and eventually towards interspecies communication. As advances in these fields continue to pile up, we imagine that the Avatars could serve as a connection and connection point between ecosystem or individual species and human communities.