Signapse Aims to Turn a Raspberry Pi Into an Artificially Intelligent Tutor for Sign Language
Designed to automate ASL education, Signapse runs on a Raspberry Pi and monitors your progress with a CNN via the Camera Module.
A team of students from the University of Glasgow are looking to automate the process of learning sign language — by turning a Raspberry Pi into an artificially intelligent tutor using Signapse.
"I'm a student at the University of Glasgow building a Linux app for Raspberry Pi that is trying to use CNNs [Convolutional Neural Networks] to teach people the ASL (American Sign Language) alphabet," says project co-creator Ross Gardiner. "We just released the first version of our software which (although admittedly buggy) is worth sharing with interested communities."
The heart of the Signapse system is a MobileNetv2 model, trained on data from the Kaggle platform, which uses a Raspberry Pi Camera Module via OpenCV to track the user's hands. A particular letter in the ASL alphabet is flashed on-screen, and the user given the challenge to replicate it — with their effort tracked by the neural network and scored for accuracy.
"We hope," Gardiner writes, "that an eventual version of Signapse running on the Raspberry Pi can serve as an easily accessible and effective way for anyone to learn sign language and benefit those who make use of ASL everyday."
An initial release of the software, which was created by Gardiner with fellow students Adam Frew, Alban Joseph, and Lewis Russell, has been published to GitHub under the reciprocal GNU General Public License 3.
"We would love for interested people to try out our software and let us know about enhancement ideas or any bugs they may find," says Gardiner. "Please try it out!"