During this pandemic, we moved almost all of our activities in front of computers. Remote work, video conference, on-line classes etc. More and more people spend all day in front of computers. Plus, we can't live without our smartphone. We all enjoy the convenience of this digital lifestyles. At the same time, more and more people are suffering from "text neck", which can cause muscle tension, back pain and other health problems. Slouching while using computers and smartphones are the main causes for this kind of health problem.
"as the neck bends forward and down, the weight on the cervical spine begins to increase. At a 15-degree angle, this weight is about 27 pounds, at 30 degrees it’s 40 pounds, at 45 degrees it’s 49 pounds, and at 60 degrees it’s 60 pounds." (source: The Washington Post, ‘Text neck’ is becoming an ‘epidemic’ and could wreck your spine)
Unlike other bulky slouching detector devices. We wanted to build a lightweight wearable that people will use everyday. People already wear blue light filter glasses or prescription glasses for computer screen, so we decided to use the Nordic BLE(Bluetooth Low Energy) module and motion sensors to build a slouching detector glasses. It detects the forward angle of person's head and notifies them to fix their posture. It will work with smart watches, smart phones, as well as tablets and PCs. We want to help people to improve their poor posture and live a healthier lifestyle. Below is the prototype we built and a demo video.
Bluetooth Low Energy (BLE) is an efficient wireless technology that has gained immense popularity in consumer electronics. It's small size, low cost, and very low power consumption. It can operate for months or years on a coin battery. Bluetooth Low Energy is now standardized in by all modern smart phones, smart watches, tablets, PCs. This makes it perfect for our project. You can find more details about BLE on Wikipedia.
HardwareThanks to Nordic Semiconductor. This idea was chosen to receive free hardware Bundle A(nRF5340 DK + Adafruit Display + nRF52840 Dongle)
nRF5340 DK is an excellent Bluetooth Low Energy development kit. It comes with nRF5340 SOC, on-board SEGGER J-Link debugger/programmer, 4x Two-Wire Interfaces (TWI) which are compatible with I²C standard. And all interfaces, and GPIOs are available via headers and edge connectors. It's easy to debug software and connect sensors to nRF5340 DK. While nRF52840 Dongle doesn't have debugger support, but it's small and it has similar features as nRF5340. We decide to use nRF5340 DK as our main software development platform. And we use nRF52840 Dongle to build our prototype.
MPU-6050 is a low power, low cost and high-performance motion sensor. It combines a MEMS 3-axis gyroscope and a 3-axis accelerometer. This compact sensor uses I²C to communicate, it can be easily connect to nRF5340/nRF52840 with two wires, plus power and ground. To make life easier and focus on our main idea, we use a MPU-6050 Breakout Board which support both 3.3V and 5V power supply. We use this sensor to measure the gravity direction and calculate neck bends forward angle.
Wiring/SolderingAs mentioned above MPU-6050 speaks I²C, it only need 4 wires(SCL, SDA, VCC, GND). nRF5340 DK comes with female pin headers. We used 4 male-to-female jumper wires to connect them without doing any soldering.
Technically, It's possible to map SCL, SDA pins to any available GPIO pins on nRF5340. We used the I²C (i2c1) default pins on nRF5340 DK board. Any VCC and GND pins on board can be used to power MPU-6050.
SCL <-------> P1.03
SDA <-------> P1.02
nRF52840 Dongle has 15 GPIO accessible from castellated solder points along the edge. It requires a little bit soldering work. It's similar to nRF5340 that SCL, SDA pins can be mapped to any available GPIO pins. We decided use P0.17 for SCL and P0.15 for SDA.
SCL <-------> P0.17
SDA <-------> P0.15
nRF Connect for Desktop is Nordic's free tool. It helps install and set up software development environment quickly. When starting nRF Connect for Desktop a central launcher is shown. There'are several Apps available there. Toolchain Manager and Programmer are required for this project. Toolchain Manager is used to install, update and launch IDE and SDK, while the Programmer is used to program nRF52840 Dongle(it doesn't have on-board SEGGER J-Link debugger/programmer).
nRF Connect SDK includes all the tools for developing, building and debugging firmware for most Nordic's wireless devices. Once the SDK is installed, the development environment is ready.
click Open IDE, the SEGGER Embedded Studio (SES) will be opened. The SES is a Powerful IDE recommended by Nordic Semiconductor, and it offers a Free Commercial license for Nordic SDK users.
Open SDK directory, we found Zephyr, a small real-time operating system supported by Linux Foundation. It inherited some features from the Linux kernel. For instance, devicetree is used by Zephyr to describe hardware resources. We are able to use the same C code for both nRF5340 DK and nRF52840 dongle by using different devicetree overlays.
Zephyr has drivers for most common bus/devices. We expected it had the I²C driver, but did not expect the MPU-6050 driver. To our surprise, it has MPU-6050 driver, plus it has a mpu6050 sample project in zephyr/samples/sensor/mpu6050. We will use it later in this project.
Our hardware and development environment are ready. It's time to build a simple firmware to test BLE. We first start with BLE peripheral sample project.
Back to SEGGER Embedded Studio, click File menu, then Open nRF Connect SDK Project...
Change Projects to "peripheral" and Board Names to "nrf5430dk_nrf5430_cpuapp", click OK, it will open that sample solution.
click Build menu, select Build Solution to build the firmware.
Wait couple seconds to finish building the firmware. Once it's done. It will show "Build complete" in the Output window. If there's any error, read the error message, fix it and try to build again.
Once the firmware is build, connect nRF5340 DK to PC with a USB cable. In "Debug" menu, click "Go" to download and run the firmware.
We can setup a breakpoint and Step in/out a function using SEGGER Embedded Studio. in Project Explorer, find and open main.c file. click the left side of editor window to set breakpoint. It just like other using other IDE.
Once the firmware is downloaded to the board and running, we use nRF Connect on a smartphone to test the BLE.
As we mentioned above, Zephyr already has a sample project called mpu6050. It's located in zephyr/samples/sensor/mpu6050. The project is not shown in the nRF Connect Options. And It was built for another Nordic board, but it can be modified to work with nRF5340 DK or nRF52840 dongle.
Go to zephyr/samples/sensor/mpu6050/boards, create devicetree overlay files for nRF5340 DK or nRF52840 dongle using the content below.
nrf5340dk_nrf5340_cpuapp.overlay
&i2c1 {
mpu6050@68 {
compatible = "invensense,mpu6050";
reg = <0x68>;
status = "okay";
label = "MPU6050";
int-gpios = <&gpio0 11 GPIO_ACTIVE_HIGH>;
};
};
nrf52840dongle_nrf52840.overlay
&i2c0 {
sda-pin = <15>;
scl-pin = <17>;
mpu6050@68 {
compatible = "invensense,mpu6050";
reg = <0x68>;
status = "okay";
label = "MPU6050";
int-gpios = <&gpio0 11 GPIO_ACTIVE_HIGH>;
};
};
In Open nRF Connect SDK Project... menu, click "..." on Projects, select the project location zephyr/samples/sensor/mpu6050/ . Change the Board Name and click OK to open the project.
Repeat the Build and Run process as the above project. For this project. The sensor data is output through printf() function. We can see the data through COM port using Putty. Below is the Putty setting and output.
MPU-6050 sensor data
We have built and tested both BLE and MPU-6050 sensor projects. Now we just need to combine both projects together and modify the BLE so we can read the sensor data via BLE.
Our final code is attached at the end of this project. We use BLE notify to send MPU-6050 sensor data out.
bt_gatt_notify(NULL, &stsensor_svc.attrs[3], data, sizeof(data));
The same characteristic also support READ/WRITE. It's used to change the notify frequency.
static ssize_t write_ct(struct bt_conn *conn, const struct bt_gatt_attr *attr,
const void *buf, uint16_t len, uint16_t offset,
uint8_t flags)
{
uint8_t *value = attr->user_data;
if (offset + len > sizeof(period_ms))
{
return BT_GATT_ERR(BT_ATT_ERR_INVALID_OFFSET);
}
memset(value, 0, sizeof(period_ms));
memcpy(value + offset, buf, len);
LOG_INF("change notify freq: %d ms", period_ms);
if (notify_enable && k_timer_remaining_get(&my_timer) > 0) {
k_timer_stop(&my_timer);
k_timer_start(&my_timer, K_MSEC(period_ms), K_MSEC(period_ms));
}
return len;
}
After combining BLE and MPU6050 code into one project, we built and tested the code on nRF5340 DK. Then we move it to nRF52840 dongle, which just used Open nRF Connect SDK Project... to open the same project and change the Board Name to nRF52840 dongle(Make sure the nrf52840dongle_nrf52840.overlay file is exist).) After we finished building the firmware, we plug the nRF52840 dongle on to computer and try to download the firmware to the dongle. But the SEGGER Embedded Studio can't download it to nRF52840 dongle. As we mentioned above, the dongle doesn't have on-board SEGGER J-Link debugger/programmer. But we can use the Programer to program the dongle.
Open the Programmer, Drag and drop the zephyr.hex file from build_nrf52840dongle_nrf52840/zephyr/ folder to the File memory layout area. chose the right device, then click "Write" to program the dongle.
Once it's programmed, we can use the nRF Connect app to verify BLE and the sensor worked well.
We use Reactjs to build web app and smart phone app. And threejs is used to visualize the data. It can be another long blog for the app development. Here we only show how to run the Web app.
Use git to clone the repository
git clone --depth=1 https://github.com/xuminready/web_ble_motion_sensor.git
change to the web_ble_motion_sensor project folder
cd web_ble_motion_sensor
install software dependency
npm install
start the app
npm start
Web BLE is only supportted by Chrome and Edge. Once the app is opened, click the button to find and connect to the our nRF5340 or nRF52840 device.
React Native is used to port this app to smart phone. If anyone is interested in port ing this web app to smart phone app , please go to React Native Docs.
Here's our final prototype demo:
Next step- Make is smaller: Because of the original contest schedule. We decided to use the nRF52840 dongle instead of building our PCB board. We plan to design our PCB in next step, so it can be very small and lightweight.
- Improve slouch detector accuracy: The way we detect slouch is very primitive algorithm. We plan to use neural network to improve the slouch detection.
- Battery: Both nRF8340 and nRF52840 can be powered by a small button battery. And the nRF52840 dongle can be connect to External regulated source. But we decided to use the battery when we design our PCB.
Min, Kate, Victor, and Li Lingneng
Comments