Oculus: The Smart Camera

Updated for 2019 China-US Young Maker Competition

Introduction

Recently my friend presented me a couple of smart cameras from some well known manufacture, and I got an idea. How is it complex to build yours? I had Raspberry Pi Camera and decided to investigate what software and components do I need to build my own smart camera. I checked several projects and decided to enhance my project with touchscreen to control camera, simple motion detection and cloud storage support. And an optional bonus if you need it: OpenCV face recognition.

Required Hardware

So to start we assembling we need:

• 3.5" touchscreen (optional)
• Raspberry Pi Case to handle toucscreen (or simple case if no screen)
• Camera module (I used NoIR version but you can use any RPi camera)
• Flex cable
• Camera mounting case

Assembly is pretty simple: install camera module, place it inside camera case (for protection and mounting), install toucscreen and put it together inside the case and tighten the screws.

Pre-installation

Before we download and install all other components we need to install Raspbian OS. There are couple options:

• Install Raspbian with Pixel desktop (you have GUI and can easily setup, WiFi, SSH etc.)
• Install Raspbian Light and setup through terminal commands or modifying micro SD image

IMPORTANT:

If you are going to use touchscreen and Chromium (further in article) you need to install Raspbian with Pixel desktop to avoid issues with installation of missing libraries to support Chromium GUI with GPIO touchscreen.

If you prefer to use Raspbian Light as it saves disk space and you actually do not need GUI for this project below is example how to setup it to connect to WiFi and be accessible through SSH. If you have Linux you can do this directly with SD card otherwise you need to connect your Raspberry Pi to display and setup it through terminal commands. If you use Windows you can try Ext2Fsd tool to modify Raspbian partition on SD card, did not test this solution well.

• After you burn the image on the SD card you need to make few more changes. We want to enable ssh and provide the wireless connection details before we insert the media in the Raspberry Pi.
• Enable SSH by creating a blank file named ssh in the boot partition. When the sd card image is created, it creates 2 partition on the media. One partition is labelled as boot and the second partition is the root Raspbian partition
• Setup the WiFi. Go to the root partition of the media and navigate to etc/wpa_supplicant. Add the WiFi configuration to the wpa_supplicant.conf file as per the following format. Append the following text in the configuration file by replacing the appropriate values of ssid, password and key_mgmt.

network={
    ssid="SSID"
    psk="password"
    key_mgmt=WPA-PSK
}

Now let's create oculus directory, copy required archive from GitHub and unzip it:

mkdir ~/oculus
cd ~/oculus
wget https://github.com/UBayouski/RaspberryPiSmartCamera/archive/master.zip
unzip -j master.zip

Make *.sh files executable:

chmod +x end_box.sh
chmod +x run_kiosk.sh

Enabling Camera

We are going to use camera and we need to double check if it is enabled, and if it is not we need to complete a simple step-by-step setup:

sudo raspi-config

And enable camera under "Interfacing Options".

IMPORTANT:

You need to reboot your RPi after enabling camera to finish the setup:

sudo reboot

or alternatively:

sudo shutdown -h now

Installing RPi-Cam-Web-Interface

I spent many days playing with different solutions: Motion, MotioneEye, MotionEyeOS (which was a pretty good and simple camera "firmware" however I needed more flexibility in customization of the OS), streaming through VLC. Unfortunately I experienced different issues with all of them (I do not blame the software probably I used it in a wrong way).

And... after different setups and tries I found Pi-Cam-Web-Interface which did the job!

So first of all I would highly recommend to call below commands to be sure we have all recent updates and restart your RPi:

sudo apt-get update
sudo apt-get upgrade

Now depending on your web server preference (apache, nginx, lighttpd) you are recommended to install it (you can do his during set up of your camera web interface, however it is recommended to do it in advance to avoid conflicts between different web servers). I would recommend Apache as it did not cause any troubles to set up RPi-Cam-Web-Interface.

sudo apt-get install apache2

Now we need to install PHP package as RPi-Cam-Web-Interface uses it.

IMPORTANT:

Depending on Raspbian version (before Stretch or after) you may need to install PHP 5 if it is before Raspbian Stretch (mostly Jessie):

sudo apt-get install php5

or PHP 7 if it is Raspbian Stretch:

sudo apt-get install php

If you do not have git run:

sudo apt-get install git

Now let's install RPi-Cam-Web-Interface and run it:

git clone https://github.com/silvanmelchior/RPi_Cam_Web_Interface.git
cd RPi_Cam_Web_Interface 
./install.sh

You will see something like this:

Some explanations:

• Cam subfolder: it is a folder (/var/www/camera) to store media and URL path (http://your_pi_camera_ip:8080/camera) to access your camera by web (default is html)
• Autostart: start HTTP server on Raspberry Pi boot
• Server: HTTP server to be used (default is apache)
• Webport: HTTP port to access your camera (default is 80)
• User: authorized user to access your camera (if you would like to extend camera to internet access)
• Password: password for authorized user
• jpglink: yes if you need external software to access images
• phpverison: you should select depending on version you installed previously, however I recommend to use Raspbian Stretch and PHP 7 as it is the latest one for today and I didn't face any issues with it

Now click "Yes" and Start it:

IMPORTANT:

If after clicking "Yes" you see errors like:

mmal: mmal_vc_component_create: failed to create component 'vc.ril.camera' (1:ENOMEM)
mmal: mmal_component_create_core: could not create component 'vc.ril.camera' (1)
mmal: Failed to create camera component

Please double check you enabled camera and reboot

Now open up any browser on any computer in your network and enter the URL to access the camera web site. This will be http://ipAddress:port/subfolder. If the port had been left at default 80 during install then this may be left out. Similarly the subfolder (default html) can be left out if that was cleared during the install.

So for a port 80, no subfolder install the URL becomes http://ipAddress

All additional information and details can be found here.

IMPORTANT:

To start motion detection you need to set Motion detect mode to Internal in Camera Settings.

Installing 3.5" Touchscreen

If you do not have this screen (remember it is HDMI "free" screen for Raspberry Pi based only on GPIO and drivers) or do not need it you can skip reading this section.

We are going to use 3.5" touchscreen built on Waveshare brand.

Surprisingly this time installation of this screen was really simple. Looks like manufacturer improved drivers for it. Let's start!

Download driver to preferable folder:

wget http://www.waveshare.com/w/upload/0/00/LCD-show-170703.tar.gz

Looks like there is new version:

wget https://www.waveshare.com/w/upload/3/34/LCD-show-180331.tar.gz

Now unarchive it and set executable mode:

tar xvf LCD-show-*.tar.gz
cd LCD-show/
chmod +x LCD35-show

Install the driver and it toggles the mode to LCD display:

./LCD35-show

IMPORTANT:

• Network connection is required while installing driver to your Pi, or else the touchscreen won't work properly, I noticed if you loose connection during installation calibration may be broken which requires a bit complex calibration later or reinstall Waveshare drivers
• This LCD may not work after apt-get upgrade, in such cases, please edit the config.txt file in the SD card and remove this statement: dtoverlay=ads7846
• Power off and on your Raspberry Pi, not software restart
• If camera or RPi-Cam-Web-Interface stopped working after installation, please re-enable camera and reboot
• If it does not work re-run RPi-Cam-Web-Interface

Additional information about Waveshare touchscreen and its driver can be found here.

Use Chromium to Control Camera

In my solution I decided to use Chromium browser in kiosk mode (aka full screen) because it is simple and lightweight and HTML page with some controls and it effectively fits small 3.5" screen. It uses desktop shortcut to automate Chromium browser with pre-selected URL/page in kiosk mode.

If you do not have chromium browser run:

sudo apt-get install chromium-browser

Now, let's install tinny application to remove mouse cursor, which is called unclutter:

sudo apt-get install unclutter

Now copy/move desktop shortcut to run chromium with our preferable page in kiosk mode:

mkdir -p ~/.config/autostart/
cp ~/oculus/oculus.desktop ~/.config/autostart/

And copy PHP page to be used with touchscreen:

sudo cp ~/oculus/mobile.php /var/www/YOUR_FOLDER/

This page handles empty view with "Start streaming" button which leads to camera view and control buttons. This was done to reduce network bandwidth load. After you restart your Raspberry Pi you should see something like this:

You probably will see some "first run" notifications like "Set chromium as a default browser" etc. when Chromium kiosk launches for the first time.

Sync with Google Drive

I chose Google Drive as cloud storage for my smart camera videos and pictures. You can choose any other, and it is really straightforward because for syncing with cloud storage we are going to use rclone and it supports really many clouds.

The setup process and reason for using rclone I described in my other article, you can use it as manual for installation and setup of rclone.

After you install and setup rclone copy macros to handle cloud sync/copy/move and set www-data as owner and group (RPi-Cam-Web-Interface works under them):

sudo cp ~/oculus/end_box.sh /var/www/YOUR_FOLDER/macros/
sudo chown www-data:www-data /var/www/YOUR_FOLDER/macros/end_box.sh

Please double check you copied rclone.config under /var/www where RPi-Cam-Web-Interface was installed and set www-data as owner and group:

sudo mkdir -p /var/www/.config/rclone
sudo cp ~/.config/rclone/rclone.conf /var/www/.config/rclone
sudo chown -R www-data:www-data /var/www/.config

IMPORTANT:

I faced an issue my macro end_box.sh did not run, I spent plenty of time to figure out an issue. Your macro should run in background or execute all commands in background.

• That is why in RPi-Cam-Web-Interface config you can see ampersand (&) before end_box.sh (by default) if you changed it you can not set ampersand again except of restoring settings (may be only my case, or some bug), so I would advise to not change it unless strong need

• Also if you check end_box.sh file you also notice ampersand at the end of the command (just to be sure we are safe in case we lost ampersand in settings)

rclone -v move $source remote:oculus --include $file &

Extend Access to Internet

To access your smart camera from Internet you can use couple options: the most common and popular port forwarding and something more modern and secure network tunneling. There is nothing complex and new about the first option, but if you want to stick with the second option you can check my article about remot3.it I provided complete guide how to install and setup it. It is really simple and provides you with great and flexible control over your camera.

Stable Work

After more than six months usage I found entire solution can experience some issues based on many factors: Google Drive sync, Internet connection, Motion detection etc.

UPDATED: Finally I found the reason of unstable work... micro SD card :( It's really pity it took time and effort to try different approaches to fix it when the real reason was related just... micro SD card. Please do not use this brand micro SD cards. It does not work for highly intensive write sequences projects. When I replaced it with other brand card, it resolved all issues. I am going to leave my previous workarounds, may be you can find them useful for other solutions.

ARCHIVED: So, I added cron job to reboot it daily (may be it requires less frequent reboot, to be updated):

crontab -e

Append the following entry:

0 10 * * * sudo reboot

This will reboot your camera project daily at 10am.

Raspbian is a robust and stable operating system kernel, but there are instances where it can panic, be it due to bad hardware or bad software. It does not happen often, but it can happen. I configured Raspbian to automatically reboot on a kernel panic by making a small modification to a configuration file that tweaks many kernel operating parameters:

sudo nano /etc/sysctl.conf

Add the following line:

kernel.panic = 5

This tells the kernel that if it encounters a panic, it is to reboot the system after a 5 second delay. By default, the kernel will never reboot when it encounters a panic, but with the above setting you can force it to.

Face Recognition

Does this project look too simple for you? OK let's add useful and interesting challenge - add face recognition supported by OpenCV. It is pretty simple to install and use. First let's install it:

sudo apt-get install python-opencv 
sudo apt-get install python-matplotlib

Now you have two options (all files can be found in downloaded Oculus archive):

• Read an image: opencv_image.py
• Read video: opencv_video.py

To use it, simply call them with python and pass the file path:

python ~/oculus/opencv_image.py /var/www/some_image.jpg

I would recommend read image which can be taken during motion detection, it is much faster because processing motion detected video on Raspberry Pi can take appropriate time. The processed file is saved to script's folder (by default to oculus) with _output suffix.

OpenCV works pretty good and can be tuned depending on conditions and required result. So, what can you do with this? You can read image and send processed by OpenCV library result to email, or upload it to Google Drive using Rclone (mentioned above). or add other logic to end_box.sh macro (remember to put & at the end of your command to execute it in background and to avoid blocking entire execution).

python ~/oculus/opencv_image.py $1*.jpg &
rclone -v move $1* remote:oculus --delete-after --include *.mp4 &

I did not mention exact usage because it is completely up to you and in fact it is more for fun, extra knowledge and gaining expertise.

Afterwords

What is the best conclusion for this project? I believe getting notified your mobile phone when motion detections is triggered and video is uploaded to Google Drive. Good news we do not need to implement new mobile app, and there is really easy way to setup it with IFTTT platform. I uploaded an article related to this topic, so you can easily use it to setup notifications for your mobile phone. Take a look on Bonus section which is dedicated to Google Drive and Notifications on your mobile phone.

Now all set!