Hardware components | ||||||
 |
| × | 1 | |||
| × | 1 | ||||
 |
| × | 1 | |||
 |
| × | 1 | |||
| × | 1 | ||||
Software apps and online services | ||||||
| ||||||
| ||||||
| ||||||
| ||||||
| ||||||
| ||||||
Airbnb and Uber are two new disruptive services for today's "Shareconomy" business model where the customers have shared access to a private property.
Airbnb guests, as well as regular visitors to your house (e.g. caregivers, maid services, dog walkers, FedEx/Amazon delivery personnel, etc.), would benefit from time-saving, convenient services if your house has keyless smart door lock installed (e.g. august.com and nouslogic.com smart door locks). These smart locks would send notifications to the homeowner's phone when any of the visitors' temporary access code was pressed on the keypad (or on the visitor's phone screen) to unlock the smart door lock. This project based on SensorTile microphone/accelerometer and also ST NFC (near-field communication) tag would add three additional useful features to existing smart door locks to form an integrated Home Access Platform as follows:
1. Low-cost voice messaging to enable the visitors to leave a voice mail or, with a future system upgrade, interactively communicate with the host via two-way half-duplex ST Bluevoice-link supported in SensorTile. This enables the homeowner to identify who is at his door before issuing access code to his home.
2. Motion-based notification to the homeowner's smartphone using SensorTile's accelerometer integrated on door lock PCB (when there is motion detected on the door.) In addition to lock/unlock push-notification (which is triggered when a correct shared access code is used to unlock/lock the door lock), this motion-based notification helps to detect whether there's forced entry or whether the door is actually swung open or closed. These two independent events/notifications (open/close vs. lock/unlock) could give the remote homeowner much more knowledge on what's going on at his house front door.
3. ST NFC-based smart doorbell, which, via the visitor's NFC reader-equipped smartphone (such as iPhone 7, Samsung Galaxy S5 and above) can send notifications to homeowner's phone when the visitor taps his/her phone on the NFC tagged doorbell. Such services are currently supported in NousLogic reachme.me Doorbell notification and chat service
For demo purpose, this project simulates these 3 additional features for an existing door lock system (such as NousLogic.com or August.com doorlock) by using ST SensorTile kit (with built-in microphone and accelerometer) and ST NFC tag, and iOS app running on an iPad (more on this Apple HomeKit iPad app later)
DETAILED DESCRIPTION:Let's have a walk through the system diagram below to understand how these 3 features work in this Smart Home Access System:
1. Low-Cost Voice Messaging:Refer to block diagram (in schematic section) which consists of:
** Door lock, (in interior side of house's front door),
** Keypad (on exterior side: this keypad transfers visitor's entered access code to the DoorLock over BLE Bluetooth low energy)
** iPad which serves as an Apple HomeKit Gateway bridging the local BLE-based doorlock with a remote homeowner's iPhone via Apple iCloud server, called "Home Hub". (NOTE: this is Apple's HomeKit architecture which requires an iPad to run Apple "Home app" to enable remote homeowner to monitor/control his local BLE-based doorlock)
These are the basic hardware components in an Apple HomeKit-enabled door lock. In the next paragraphs, we will describe what it takes to add the three new features.
To add voice-messaging in a very cost-effective way, we include a microphone (and an optional speaker for future 2-way audio expansion; ST accelerometer LSM303AGR is also added to support #2 feature) and reuse the existing blueNRG-MSCSP Bluetooth module in the keypad to run bv-link voice firmware (supported in STMicroelectronics FP-SNS-ALLMEMS1 3.2.0). By reusing ST BlueNRG instead of adding another WiFi module (which is typically used to support voice streaming), the system cost is highly optimized.
Similar to ST BlueMS iOS app which streams audio from SensorTile's microphone to the iPhone, our system uses a similar app to run on iPad (which also runs HomeKit Home app) to receive the visitor's audio streamed from SensorTIle' microphone/blueNRG module on Keypad at the front door. This audio buffer can be streamed in real time to homeowner's phone via NousLogic cloud server or sent as a voicemail file.
In a future upgrade, this platform can also support audio streaming in the reverse path (from remote homeowner's phone to the keypad's speaker) to form a 2-way half duplex audio system
VIDEO DEMO FOR FEATURE #1: this video shows the doorlock' Keypad (which uses SensorTile's accelerometer, microphone and bluevoice Ble module; and ST NFC tag) together with an iPAD running Apple HomeKit to support remote control of the doorlock . BLE voice streaming from Keypad/SensorTile to iPAD BlueMS app is shown
2. Motion-Based NotificationST accelerometer LSM303AGR + motion signal analysis firmware on either Keypad or Door Lock itself can detect when the door is moved (opened or closed) and send door open/closed push-notification ( and/or door locked/unlocked notification) to remote homeowner's phone via the HomeKit protocol-enabled iPad and Apple iCloud server. These two independent events/notification (open/close vs. lock/unlock) could give the remote homeowner much more knowledge on what's going on at his house front door.
PHOTO DEMO FOR FEATURE #2 which shows two push notification: for door moved event and door unlocked event
A Smart DoorLock system should be bundled with a Smart doorbell service to "ring" the homeowner when somebody is at the door pressing on the doorbell. We propose a low-cost software-based doorbell service using ST NFC tag (near field communication) which, when tapped on using the visitor's NFC-enabled phone, will use the visitor's LTE service to send push notification to homeowner's phone to start a chat service between the 2 parties. Such new NFC capabilities are added to NousLogic ReachMe Smart Doorbell app which originally uses QR code (scanned via the phone's camera). You can download and install this app from Apple Appstore and Google Playstore. the app name is Nouslogic ReachMe QR code
VIDEO DEMO for feature #3:
This video shows a software-based Smart Doorbell service that is initiated when the visitor's NFC-enabled smartphone is placed near an ST NFC tag on the door lock's Keypad .
This ReachMe Doorbell service is courtesy of NousLogic company
CODE DEMONSTRATIONHere are some codes to demonstrate the main functions of features #1 and #2:
Pushing notification for feature #2:
import UIKit
import UserNotifications
@UIApplicationMain
class AppDelegate: UIResponder, UIApplicationDelegate, UNUserNotificationCenterDelegate {
var window: UIWindow?
func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplicationLaunchOptionsKey: Any]?) -> Bool {
// Override point for customization after application launch.
if #available(iOS 10.0, *) {
let center = UNUserNotificationCenter.current()
center.delegate = self
center.requestAuthorization(options: [.sound, .alert, .badge]) { (granted, error) in
if error == nil{
DispatchQueue.main.sync {
UIApplication.shared.registerForRemoteNotifications()
}
}
}
}
else {
DispatchQueue.main.sync {
UIApplication.shared.registerUserNotificationSettings(UIUserNotificationSettings(types: [.sound, .alert, .badge], categories: nil))
UIApplication.shared.registerForRemoteNotifications()
}
}
return true
}
func application(_ application: UIApplication, didRegisterForRemoteNotificationsWithDeviceToken deviceToken: Data) {
print("token: \(deviceToken.hexEncoded())")
}
func application(_ application: UIApplication, didFailToRegisterForRemoteNotificationsWithError error: Error) {
print("didFail: \(error)")
}
func application(_ application: UIApplication, didReceiveRemoteNotification userInfo: [AnyHashable : Any]) {
print("did receive notification: \(userInfo)")
}
func applicationWillResignActive(_ application: UIApplication) {
// Sent when the application is about to move from active to inactive state. This can occur for certain types of temporary interruptions (such as an incoming phone call or SMS message) or when the user quits the application and it begins the transition to the background state.
// Use this method to pause ongoing tasks, disable timers, and invalidate graphics rendering callbacks. Games should use this method to pause the game.
}
func applicationDidEnterBackground(_ application: UIApplication) {
// Use this method to release shared resources, save user data, invalidate timers, and store enough application state information to restore your application to its current state in case it is terminated later.
// If your application supports background execution, this method is called instead of applicationWillTerminate: when the user quits.
}
func applicationWillEnterForeground(_ application: UIApplication) {
// Called as part of the transition from the background to the active state; here you can undo many of the changes made on entering the background.
}
func applicationDidBecomeActive(_ application: UIApplication) {
// Restart any tasks that were paused (or not yet started) while the application was inactive. If the application was previously in the background, optionally refresh the user interface.
}
func applicationWillTerminate(_ application: UIApplication) {
// Called when the application is about to terminate. Save data if appropriate. See also applicationDidEnterBackground:.
}
}
extension Data {
func hexEncoded() -> String {
return map { String(format: "%02.2hhx", $0) }.joined()
}
}
****************************************
For illustration purpose for this feature #2 (door motion detection), we listed the simple "Single Tap" function in ST ALLMEMS firmware . In reality we are developing more complex motion analysis algorithm for door open/close motion detection:
/**
* @brief This function enables the HW's Single Tap Detection
* @param None
* @retval None
*/
void EnableHWSingleTap(void)
{
if(TargetBoardFeatures.HWAdvanceFeatures) {
/* Disable all the HW features before */
DisableHWFeatures();
/* Enable Single Tap detection */
if(BSP_ACCELERO_Enable_Single_Tap_Detection_Ext(TargetBoardFeatures.HandleAccSensor,INT_PIN)==COMPONENT_ERROR) {
ALLMEMS1_PRINTF("Error Enabling Single Tap Detection\r\n");
} else {
ALLMEMS1_PRINTF("Enabled Sigle Tap\r\n");
W2ST_ON_HW_FEATURE(W2ST_HWF_SINGLE_TAP);
}
}
}
****************************************
Double tap on SensorTile firmware to start capturing visitor's voice as in videos (C):
/**
* @brief This function enables the HW's Double Tap Detection
* @param None
* @retval None
*/
void EnableHWDoubleTap(void)
{
if(TargetBoardFeatures.HWAdvanceFeatures) {
/* Disable all the HW features before */
DisableHWFeatures();
/* Enable Double Tap detection */
if(BSP_ACCELERO_Enable_Double_Tap_Detection_Ext(TargetBoardFeatures.HandleAccSensor,INT_PIN)==COMPONENT_ERROR) {
ALLMEMS1_PRINTF("Error Enabling Double Tap Detection\r\n");
} else {
ALLMEMS1_PRINTF("Enabled Double Tap\r\n");
W2ST_ON_HW_FEATURE(W2ST_HWF_DOUBLE_TAP);
}
#ifdef STM32_NUCLEO
#ifdef IKS01A1
if(BSP_ACCELERO_Set_Tap_Threshold_Ext(TargetBoardFeatures.HandleAccSensor,LSM6DS3_TAP_THRESHOLD_MID)==COMPONENT_ERROR) {
#elif IKS01A2
if(BSP_ACCELERO_Set_Tap_Threshold_Ext(TargetBoardFeatures.HandleAccSensor,LSM6DSL_TAP_THRESHOLD_MID)==COMPONENT_ERROR) {
#endif /* IKS01A1 */
#elif STM32_SENSORTILE
if(BSP_ACCELERO_Set_Tap_Threshold_Ext(TargetBoardFeatures.HandleAccSensor,LSM6DSM_TAP_THRESHOLD_MID)==COMPONENT_ERROR) {
#elif STM32_BLUECOIN
if(BSP_ACCELERO_Set_Tap_Threshold_Ext(TargetBoardFeatures.HandleAccSensor,LSM6DSM_TAP_THRESHOLD_MID)==COMPONENT_ERROR) {
#endif /* STM32_NUCLEO */
ALLMEMS1_PRINTF("Error setting Double Tap Treshold\r\n");
}
}
}
****************************************
Visitors' voice streaming from SensorTile (integrated on door lock's keypad) to iPad (which acts as door lock HomeKit's gateway):
/**
* @brief Process user input.
* @param None.
* @retval None.
*/
void BVL_APP_PER_Process(void)
{
BV_ADPCM_Status status;
switch (BV_APP_PER_state)
{
//...
case BV_ADPCM_STATUS_STREAMING:
{
//...
if(ready)
{
status = BluevoiceADPCM_SendData(&num_byte_sent);
if(status != BV_ADPCM_SUCCESS)
{
BV_APP_PER_Error_Handler();
}
ready = 0;
}
}
break;
//...
}
}
For the Sensortile firmware: Unless you made changes or flashed a new firmware onto the Sensortile, it should be usable as is. Here are the (optional) instructions to flash the correct FP-SNS-ALLMEMS1 firmware onto the Sensortile:
1. Download the firmware from ST website.
2. Unzip the contents of file into a folder. From now on, we will call this folder "D:\ALLMEMS1".
3. Locate the file "ALLMEMS1_ST_BL.bin" in folder "D:\ALLMEMS1\Projects\Multi\Applications\ALLMEMS1\Binary\STM32L476RG-SensorTile". Then, flash the firmware onto the SensorTile using the NUCLEO-F401RE and the software STSW-LINK004 with the following set up with the CN2 jumpers removed:
A SensorTile with a correctly flashed firmware would resemble this:
More details about this process can be found here.
For the app BlueMS, either on Android of iOS (whichever one is your favorite, your can download Android Studio here and it doesn't require a macbook), in the "AudioRecorder.java" or "W2STAudioDumpController.swift" respectively, do as you please with the audio recorded. We just save it as an audio message and notify the user.
Feature 2 (Motion-Based Notification):For illustration purposes, here, we are using single tap in FP-SNS-ALLMEMS1 and a modified version of Blue-MS app (code shown above that support single tap detection and the resulting notification message on the owner (remote) phone). As this is still work-in-progress, in a future code upgrade, we will replace single tap with a more complex algorithm to detect door being swung open or close. The setup for this feature consists of SensorTile (for processing single tap motion), HomeKit enabled iPad (acting as a gateway), Apple iCloud server (to send the notification), remote owner iPhone running Apple Home app (to receive the notification).
Step 1: Run the modified app on iPad and pair the iPad with the SensorTile.
Step 2: Shake the SensorTile to simulate door open or close.
Step 3: Via the iPad and iCloud, remote home owner's iPhone will receive this open/close motion notification (see images in previous section). Note that the other lock/unlock notification is also sent via this mechanism and data path.
Feature 3 (Instructions for Configuring ReachMe Doorbell Service via QR Code First and Then ST NFC Tag):Step 1: Familiarize yourself with how NousLogic ReachMe.Me smart door bell service works by watching this video. Here, they use QR code. Below, we'll show you how to run the same smart door bell service by using ST NFC tag.
Step 2: If you already owned a ReachMe QR code tag, read and follow the four steps in the following quick-start guide. Otherwise, you can create and print your own QR tag by downloading the "NousLogic ReachMe-QR" app for iOS or Android and follow the instructions in section 2 of this link.
Step 3: Now, you have created and configured your own ReachMe QR code tag. You can play with it by running the ReachMe QR app and scanning it using your phone camera. If you correctly configured your tag then upon scanning it, you will get the following webpage screenshot (with the door bell) on your phone. Right-click on the URL field of the webpage to copy it. You should see a URL similar to the following: https://reachme.me/r/t/t6Q2f956A3c4T904/MLkwqFrYTuCv7NW9cUtm1506067414.
See step 4 below to write this webpage's URL to your ST-NFC tag.
Step 4: Download and install the app NFC Tools onto your Android phone that is equipped with an NFC reader such as a Samsung Galaxy S5 and above. Using an ST NFC tag similar to the one shown below (ST25TA), launch the NFC Tools app and tap your phone onto the NFC tag, you should see the NFC Tools screenshot below, make sure that in the field "Technology available", "Ndef" format is shown. If so, now you are ready to write the above URL onto the NFC tag by selecting the "WRITE" tab on the app and pasting your URL into the field.
Step 5: Once the write step is completed, now you are ready to use your ST NFC tag for the same smart door bell service that you did earlier with QR code.
Comments