Seeed Studio's SenseCAP Watcher Combines Local TinyML with Cloud LLMs to Deliver Smart Monitoring

UPDATE (9/13/2024): Seeed Studio has opened a crowdfunding campaign for the SenseCAP Watcher, which it describes as the "world's first physical AI agent for smarter spaces" — and has already surpassed its funding goal.

Reward pricing for the campaign starts at $79 for "super early bird" backers, representing a claimed 20 per cent discount over the planned retail price, with regular crowdfunding pricing set at $89. The devices, available in white and clear cases, come with a rotating stand and a "premium plan" subscription to the company's SenseCraft software suite, worth $6.90. This unlocks a usage-based payment tier, removing the limits of the software's free tier: a maximum of 200 LLM prompts per month, 15-minute gaps between image analysis requests, and a maximum of 3,000 image analysis requests per month.

The SenseCraft suite can also be deployed locally on suitable devices, Seeed has confirmed, running models on-device and keeping all data on the local network; a bundle dubbed the "Private LLM Pack" comes with a single SenseCAP Watcher and an NVIDIA Jetson AGX-based edge device on which SenseCraft can be installed.

All rewards are expected to ship in November this year, Seeed has confirmed, with more information available on the Kickstarter campaign page.

Original article continues below.

Seeed Studio has announced the impending launch of what it describes as a "physical AI [artificial intelligence] agent for smarter space," combining AI technology with the Internet of Things to build a smart gadget combining locally-running tinyML models with cloud-powered large language models (LLMs): the SenseCAP Watcher.

"We are incredibly excited to launch the SenseCAP Watcher at Embedded World 2024," says Seeed Studio Eric Pan, of the company's latest hardware release. "With the SenseCAP Watcher, we envision a more effortless way of interacting with and managing physical world. It pushes the boundaries of what a combination of tinyML at the edge and LLMs can enable, while retaining an open source and extensible design."

Designed for wall mount or desktop operation, the SenseCAP watcher includes a 1.46" circular color touchscreen display, front-facing camera and microphone, speaker, programmable RGB light, wheel interface, two USB Type-C connectors for power and charging its internal battery, and a Grove expansion slot for external add-on hardware.

The gadget is designed as a first-class device for Seeed's SenseCraft, a toolkit that combines on-device tinyML models with remote large language models — either running on remote cloud servers or on local edge devices, including Seeed's own Edge AI Box reComputer devices based on the NVIDIA Jetson system-on-module family.

The on-board models, Seeed explains, are used to monitor conditions around the device, including room occupancy and activities. For interaction, either using a companion mobile app or via voice control, the device can send queries to an LLM for natural language command operation. The company also claims the device is "self-learning and evolving," being able to train on locally-captured data, and can also be integrated into existing smart home systems.

While Seeed is showing off the device at Embedded World 2024 in Nuremberg this week, it has yet to announce pricing and availability — but is taking applications for alpha testers on its website, where interested parties can also interact with a simulated version of the SenseCAP Watcher dubbed "Nobody."