Big Red Button

The inspiration to this project was my grandmother, who due to her deteriorating vision is finding it progressively harder to navigate the ever-changing menu screens and small fonts on her mobile phone to stay in touch with us. These days it's easy to forget that a lot of the technology that we use intuitively is very hard to use for the older generations who grew up with cord phones and black and white TVs.

Social isolation is one of the biggest issues facing the aging community in England, and with the current COVID-19 lockdown this problem is only worsening. Not being able to visit your family who live across the country/globe is sad, but not being able to contact them first because you cannot remember or figure out how to call them is even sadder. According to the NHS:

more than 2 million people in England over the age of 75 live alone, and more than a million older people say they go for over a month without speaking to a friend, neighbour or family member.

Unfortunately, the statistic above will only have worsened in the context of this pandemic. Most phone companies try their best to make their tech accessible to the elderly with large text and text to speech software, but this does not take away from the base level complexity of the phone itself. Often, simple is better.

The Project

Video chat is nothing short of magic, and this project aims to remove all the hassle in setting up a video call so that it is accessible at literally the push of a button. Essentially, it is 'the big red button' which lets senior citizens stay in touch with their family easily - just hearing their family's voice or seeing their face can brighten up their day.

The project uses a Raspberry Pi and Google's duo software to establish a video call between the Pi and another device. Pressing the button triggers the Pi to set up the video chat to a hardcoded contact - it's that simple! No single finger typing of numbers and contact names, or navigating through new apps. Just push the button and the video chat will be established. For larger families, more than one button can be interfaced! The added benefit is that as the Pi is HDMI compatible, it can be hooked up to a large television or a dedicated screen/monitor in the users' house - completely relieving the video call experience from mobile phone hardware! The main idea is to have a screen that can be idle, display a digital painting, or be of some other use (hosted on the Pi) until the button is pressed to trigger a video call.

This prototype shows the basics of how such a system would work, demonstrating only the automation of the core functionality. The following is a 30-second demo

The Hardware

The project has 4 main hardware components. The Pi which is the computing unit, a push button which acts as input and triggers the process, speaker/mic to register sound and webcam/camera to capture video. The speaker/mic is connected through the AUX port, the camera through the DSI, and the button via the GPIO. Certain cameras and microphones can also be interfaced through the USB. Similarly, certain displays have built-in speakers and microphones - so additional ones would not be needed. See the schematic below:

The hardware setup is simple - the thing that took me the longest was making the button. I already had a 5mm push button but I really wanted to drive home the concept of a large button. Turns out that larger buttons are relatively expensive to buy (about £10.) I aimed for this project to be as lightweight and cheap as possible so I decided to make a simple one myself using cardboard from an old shoebox and half of a Pokeball that was lying around. The hope is that once I have access to a 3D printer I can polish this. All the push button needs to do is bridge the gap between a 5V supply to the GPIO pin, and I used the metal on the crocodile clips to do this. You can see the steps briefly below:

As you can see, the metal on the crocodile clips completes the circuit - allowing the GPIO to register a high signal and trigger the rest of the process. Note that the metal is only exposed for the sake of the video - in reality only a very small amount of metal should be exposed, preferably in an enclosed space.

Software

I used Google Duo over Skype or Zoom due to the latter two being a pain to set up on the Pi. This project aims to be as simple as possible. The video call can be initiated from the web interface but Duo can also be downloaded to the Pi using Chromium. This adds a Duo icon to the Pi's desktop and in my opinion, makes it easier to access.

(Note: you have to be signed into Duo's web interface to be able to download the app, strange I know.)

I already had my number associated with my Duo account so I did not need to set anything up, and I used my Dad's number (which was already registered on Duo too) to test with. You may have to set this up if using it for the first time, though Duo apparently allows calls to phones without the app installed too.

Once Duo was installed, I needed a way to automatically open it and make a call. Unfortunately there was no API for it, but I had heard of PyUserInput which is a module for automating keystrokes and mouse clicks. I downloaded this using the command:

sudo pip install PyUserInput

The package is fairly intuitive and simple to use. Mouse and keyboard objects can be initialized as follows

from pymouse import PyMouse
from pykeyboard import PyKeyboard

m = PyMouse()
k = PyKeyboard()

You can then specify some coordinates for the mouse to click on - I intend to leave the Duo app running to remove the hassle and time of opening the app every time.

I could not get the example mouse tracking script working so I wrote a quick one myself to find the coordinates of the search bar.

from pymouse import PyMouse
from pykeyboard import PyKeyboard

m = PyMouse()
k = PyKeyboard()

if __name__ == '__main__':
    for i in range(5000):
        print(m.position())

Once I found the coordinates I was able to automate the click on the Duo search bar

from pymouse import PyMouse
from pykeyboard import PyKeyboard

m = PyMouse()
k = PyKeyboard()

if __name__ == '__main__':
    
    m.click(960, 100, 1)

After the click, PyUserInput needs to type the name or number of the contact (my dad in this case) and press enter which you can do as follows

from pymouse import PyMouse
from pykeyboard import PyKeyboard
from time import sleep

m = PyMouse()
k = PyKeyboard()

if __name__ == '__main__':
    
    m.click(960, 100, 1)
    sleep(0.5)
    k.type_string('dad') # change this to the contact you want 
    sleep(0.5)
    k.tap_key('Return')

Note that it is essential to include the sleep commands to give the program time to type and navigate as expected - the last thing you want is your gran calling the wrong contact!

Finally, I used the mouse tracking function to find the position of the video call button and then programmed the mouse to click it.

from pymouse import PyMouse
from pykeyboard import PyKeyboard
from time import sleep

m = PyMouse()
k = PyKeyboard()

if __name__ == '__main__':
    
    m.click(960, 100, 1)
    sleep(0.5)
    k.type_string('dad') # change this to the contact you want 
    sleep(0.5)
    k.tap_key('Return')
    sleep(1.0) #reduce this to make system faster 
    m.click(1053,826,1)

And that was it for the automation side. Now that the video call was being made automatically from a script- I needed to interface the giant button I had made. This was simple enough by utilizing the GPIO module built into the Pi.

Finally, to put it all together I stuck the whole thing inside a while loop and made the video call function triggered within an if statement. The final script looks like this:

from pymouse import PyMouse
from pykeyboard import PyKeyboard
from time import sleep
import RPi.GPIO as GPIO

#mouse and keyboard objects
m = PyMouse()
k = PyKeyboard()

#GPIO setup
GPIO.setmode(GPIO.BOARD)
GPIO.setup(11,GPIO.IN,pull_up_down=GPIO.PUD_DOWN)

if __name__ == '__main__':
    while True:
        #print(GPIO.input(11))
        if GPIO.input(11) == 1: #If button is pressed 
            m.click(960, 100, 1)
            sleep(0.5)
            k.type_string('dad') # change this to the contact you want 
            sleep(0.5)
            k.tap_key('Return')
            sleep(1.0) #reduce this to make system faster
            m.click(1053,826,1)

Next Steps

The project greatly simplifies the hassle of video calling, and can also be performed with the phone being elsewhere in the house. More buttons can be interfaced to hardcode more than one contact on the same system - in my case there will be two, one for my grandmother to contact my dad and one to contact my aunt.

Moreover, as the system is now running on the Pi, the Cron scheduler can be used to schedule calls, for example at 12 pm every Sunday. This approach provides routine, and something for the user to look forward to every week.

A project like this also has the potential to be packaged into a tablet-sized device at an overall cost of about £50, severely undercutting the market as similar devices are more than ten times this price.

The system still needs a way to accept calls; this can be done by interfacing a button of a different color with a different label.

There are still edge cases that need to be tackled when taking this project further from a prototype:

• What happens if the button is pressed twice?
• What if there is no internet connection?
• What if either of the contact's number change?

If wifi cuts out, this will be flagged by Duo on the screen that the system is running on. If contact numbers change - they can be changed on Duo's web portal away from the Pi by anyone who has access to the account.

The issue of the button being pressed twice can be easily mitigated by making the script block any GPIO reads while the video call is active. The script already has an essence of this. When the first button press is recorded the video call initiation process is blocking so no other presses will be registered until the call is active. This is especially good at eliminating any duplicate button readings due to shaky hands, which becomes increasingly common with age.

Final Notes

Using this system myself, I have noticed that the audio quality via Duo is excellent. The video also streams pretty well (network dependant of course) but there is still room for improvement. Perhaps a better camera or a more powerful Pi will ease the strain on the processor. All in all the project achieves what I had intended; it allows people to make video calls easily using a lightweight and cheap solution.