Talking Chalks

General Idea

TalkingChalks is an IoT system that can be used in a museum. It can help visitors during the tour providing information about the museum and the artworks. A smartband with the most suitable premade profile will be provided to each visitor, so that everyone can have a customized experience: after passing the bracelet near a statue, it will begin to tell its story according to the user's profile.

Users part of our project: when museum visitors activate the sensors with the smartband they will contribute to create an abstract painting based on their profile and appreciation of the exhibits.

Why Talking Chalks

TalkingChalks was born for the course of IoT @ Sapienza University of Rome.

During a conference with the curators of the Sapienza Museum of Classical Art we found out that they needed some systems to give some freshness to the Museum. Among the requestes and the ideas it came out that they do not have audio guides, and so we invented TalkingChalks to help the Museum giving some attractiveness for users by letting the statues - literally - talk.

TalkingChalks is the effective rensponse to what they need, and we made it real by investigating among the users on what would be the best way to make TalkingChalks real: we made surveys, tests, we added personalized profiles and appreciation recognition. Then we merged our knowledge to develop a simple and solid system to give statues a voice.

Here is TalkingChalks.

ATTENTION! About performance evaluation, the project has been only simulated cause of the global pandemic that forced us to stay in quarantine and to limit our movements through the regions. For these reasons we used real life testbed environments such as IOT-LAB and TheThingsNetwork to test the product.

Components

• STM nucleoboardwith LoRa technology

The B-L072Z-LRWAN1 LoRa®/Sigfox™ Discovery kit is a development tool develop solutions based on LoRa®, Sigfox™, and FSK/OOK technologies. This Discovery kit features the all-in-one CMWX1ZZABZ-091 open module by Murata. The module is powered by an STM32L072CZ microcontroller and SX1276 transceiver. The transceiver features the LoRa® long-range modem, providing ultra-long-range spread-spectrumcommunication and high interference immunity, minimizing current consumption. Since CMWX1ZZABZ-091 is anopen module, the user has access to all STM32L072CZ peripherals such as ADC, 16-bit timer, LP-UART, I2C, SPI, and USB 2.0 FS (supporting BCD and LPM).The B-L072Z-LRWAN1 Discovery kit includes an ST-LINK/V2-1 embedded debug tool interface, LEDs, push-buttons, antenna, Arduino™ Uno V3 connectors and USB OTG connector in Micro-B format.The LoRaWAN™ stack supports Class A, Class B, and Class C.

Here is the full Data Sheet of the kit.

• Smartband

The smartband is the key point of the entire system because it stores the premade user profiles. It also contains a NFC sensor to be identified by the board. The profile ID is sent to the LoRaWAN board and an appropirate audio guide track will be reproduced. Smartband has been chosen to activate TalkingChalks because it is easy-to-use, ready-to-use, doesn't cause isolation and it doesn't access to sensitive data of the users.

• TTN/MQTT Gateway

The Gateway is responsible of the forwarding of the incoming data from The Things Network to the Cloud. Once running, the Gateway launches a client that subscribes to the TTN's broker at the topic +/devices/+/up using as username the name of the TTN Application and as password the Applocation Access Key. Data are received from the boards as a json string, then the emitting board is identified by its ID and the payloads are so forwarded using MQTT protocol.

• Hi Fi system

Simple speakers connected to the LoRaWAN board, the main role of these instruments is to reproduce the requested audio track that explains the statue. They can be connected to the electric system or powered by batteries. We don't need as many speakers as artworks in the museum and volume must be not too loud in order to not interpose with the audio description of other statues.

• Azure IoT Hub

Azure IoT Hub by Microsoft is a managed service hosted in the cloud that acts as a central message hub for bidirectional communication between the IoT application and the device it manages. It has a lot of functionalities for all the types of applications, in our project we will use it as MQTT broker to receive messages sent by the devices through the Gateway and store the data in CosmoDB, which is an integrated service. With a free subscription and with a standard plan, we can handle a maximum of 8000 messages per day.

• CosmoDB

Azure CosmoDB is a multi-model database service for any scale. It guarantees a global distribution, high-availability and low latency. No sensitive data are stored in the database and museum curators can use these data to find out which are the most visited artworks,

How it works

We have built a demo using IoT-Lab, TheThingsNetwork, Azure IoT Hub and CosmoDB.

The Gateway starts and loads all the registered devices attached to statues. When Lora nodes send a JSON string like:

{'dev_id' : 'dev_00', 'profile_id' : 'Hugo', 'temp' : '36', 'hrate' : '72', 'timestamp' : '123'}

the Gateway receives it and looks for the device. If the device is found, then the Gateway assigns the requested profile to it and data are forwarded to the hub and to the database. Each message that the gateway receives contributes to create the abstract painting, generating a circle with the color of the profile and the dimension of the heart rate.

On the Curator's side

The curators of the Museum can access an user friendly interface to have the informations about statues and profiles displayed in a single look: by selecting the device, they will know the statue it is equipped on an the profiles that visited the statue.

End user product

We have also built a end user product to practically see how TalkingChalks works. Here the LoRa Board is graphically simulated by a smartphone and two smartbands are simulated by two distinct NFC tags. The statue is simulated by an action figure of the famous pokèmon Squirtle. In the video It is possible to see that, when the user passes his smartband [NFC tag] next to the board [Smartphone], the corresponding audio guide is activated. Audio guides are different because the two simulated users have choosen different profiles.

From these two different simulations we can see that as soon as the NFC tag is passed next to the board, the system is activated without delay and all the data are immediately stored into the database.

Full requirements and tutorial are available on the dedicated GitHub page, where in the /Demo folder is possible to find the complete code and step by step instructions on how to run both the Gateway and the LoRa board.

Evaluation

TalkingChalks objectives are:

• Provide customized information on the tour for every typology of user
• Full immersion on the tour
• Simplicity: easy-to-use and ready-to-use
• Everyone can use it, even elder people, because no smartphone and no internet connectiont are required to the final user
• Source reliability: info are provided by the museum
• Scalability: the whole application has been designed to be scalable, both for the developers and for the users.

We have carried out two surveys to analyse the habits of the museum visitors and also their relationship with our competitors (audio guides, guided tours, searching info on internet). We also asked people about their relationship with the smartphone during museum tours and which device they would prefer to use to activate Talking Chalks. We found out that the most used competitors are the audio guides, but they have many drawbacks: isolation from family and friends because you can listen to it only one at time; cost is too high; they are not so easy to use. Another competitor is the guided tour, but people cannot hear if it's crowded, the cost is too high and people feel constrained by the needs of the whole group.

In the last years internet has become another competitor. Beside source reliability and having to do a lot of different researches on different sites, what people don't really like is repeatedly checking on their phone during the tour because they feel isolated.

Survey results show that people would prefer to use the smartband to activate TalkingChalks because they wouldn't be distracted during the museum tour (smartphone is not required), it doesn't cause isolation because small groups of people can use it at the same time and it's really easy to use.

What we can deduce from the survey is the fact that TalkingChalks can offer a much more complete and satisfying experience for the user in comparison to other TC competitors.

Please check the full Evaluation on the dedicated documentation on GitHub.