OpenCV Based Fire Detection System with Blues

Introduction

Fire detection systems are essential for protecting lives and properties from the devastating effects of fire. However, conventional fire detection systems that use smoke or flame sensors have some limitations, such as high false alarm rates, slow response times, and low sensitivity in outdoor environments. To address these challenges, computer vision technology can be used to detect fire more accurately and efficiently. Computer vision is the field of study that enables machines to process and understand visual information, such as images and videos. By applying computer vision techniques, such as image processing, machine learning, and deep learning, we can design fire detection systems that can:

• Recognize fire and smoke patterns in various scenarios, such as indoors, outdoors, aerial, or satellite images.
• Distinguish fire and smoke from other similar objects, such as clouds, fog, or dust.
• Estimate the location, size, and intensity of fire and smoke regions.
• Alert the authorities and the public in real time and provide relevant information for fire prevention and control.

System Overview

The proposed system leverages OpenCV (Open Source Computer Vision Library), a state-of-the-art computer vision and machine learning software library, to detect fire in various scenarios. The system can process images and videos from different sources and identify fire and smoke regions using image processing, machine learning, and deep learning techniques. The system can also estimate the location, size, and intensity of the fire and provide relevant information for fire prevention and control. Furthermore, the system can send an alert to a remote fire station in real time, enabling a faster and more effective response.

Components

The main components of this system include:

• Camera: A camera is used for continuous surveillance of the area.
• OpenCV and Python: OpenCV library along with Python is used for image processing and fire detection.
• Raspberry Pi: These are used as the main processing unit.
• Blues Raspberry Pi Notecarrier Hat and Notecard:To send data to cloud platforms.

Working Flow

The working of the system can be broken down into the following steps:

• Image Capture: The camera continuously captures video frames of the monitored area.
• Image Processing: The captured frames are processed using OpenCV. The system is trained to recognize the unique characteristics of fire, such as color, flickering, and irregular shape.
• Fire Detection: If fire-like characteristics are detected in the processed frames, the system identifies it as a fire.
• Alert Generation: Upon fire detection, the system triggers an alarm in the building. Simultaneously, it sends an alert message of the incident to a remote fire station via the Cellular+WiFi module.

Procedure:

Step 1: Install Raspberry Pi OS on your microSD card

The first step is to install the operating system for your Raspberry Pi on your microSD card. You can use any OS that supports remote access, but I recommend using Raspberry Pi OS, the official OS from the Raspberry Pi Foundation.

To install Raspberry Pi OS, you will need to use the Raspberry Pi Imager tool, which is available for Windows, Mac, and Linux. You can download it from the Raspberry Pi website.

After downloading and installing the Raspberry Pi Imager, launch it and follow these steps:

• Insert your microSD card into your computer’s SD card reader.
• Click the Choose OS button and select Raspberry Pi OS (64-bit) or Raspberry Pi OS (other) depending on your Pi model.
• Click the Choose Storage button and select your microSD card from the list.
• Click the Next button and then click the Edit Settings button from the pop-up window.
• Fill in all the fields on the General tab, such as hostname, username, password, wireless LAN (if you want to use Wi-Fi), and locale settings.
• On the Services tab, toggle enable SSH to on and select Use password authentication. This will allow you to access your Pi via SSH later.
• Click the Save button and then click the Write button to start writing the OS image to your microSD card. This may take a few minutes, depending on the speed of your card and your internet connection.

Step 2: Enable remote access on your Raspberry Pi

After the OS image is written to your microSD card, eject your microSD card from your computer and insert it into your Raspberry Pi. Then, plug in the power supply and wait for your Pi to boot up.

Step 3: Find the IP address of your Raspberry Pi

The next step is to find the IP address of your Raspberry Pi on your network. This is the address that you will use to connect to it from another computer.

There are different ways to find the IP address of your Raspberry Pi, but one of the easiest methods is to use a tool called mobaxterm, which is available for Windows, Mac, and Linux. You can download it from the mobaxterm website.

After downloading and installing mobaxterm, launch it and follow these steps:

• Click the Start button to scan your network for active devices. This may take a few seconds, depending on the size of your network.
• Look for a device that has the hostname that you set for your Raspberry Pi in step 1. For example, if you set the hostname to raspberry pi, look for a device that has raspberrypi.local as its name.
• Note down the IP address of that device. It should be something like 192.168.1.X, where X is a number between 1 and 255.

If you can’t find your Raspberry Pi using mobaxterm, you can try other methods, such as logging into your router’s web interface and looking for the list of connected devices, or using a command line tool like ping or nmap.

Step 4: Connect to your Raspberry Pi remotely

The final step is to connect to your Raspberry Pi remotely from another computer on the same network, using a VNC or SSH client.

To use an SSH client, you don’t need to enable anything on your Raspberry Pi, as SSH is already enabled by default. You can use any SSH client to connect to your Raspberry Pi. I recommend using the mobaxterm. Create a new session with the following details:

• Enter the IP address of your Raspberry Pi in the Host Name field.
• Make sure the Port is set to 22 and the Connection type is set to SSH.
• Click the Open button to start the connection.
• Enter the username and password that you set for your Raspberry Pi in step 1 when prompted.

You should see the command line interface of your Raspberry Pi on your computer screen. You can use your keyboard to type commands and run programs on your Pi.

Step 5: Install OpenCV on Raspberry Pi

To install OpenCV on your Raspberry Pi is to use the package manager (APT). This is the easiest and fastest way, but it may not give you the latest version of OpenCV. To install OpenCV via APT, follow these steps:

• Open a terminal on your Raspberry Pi or connect to it via SSH.

Update your system and upgrade your packages with the following commands:

sudo apt update, sudo apt upgrade

• Install OpenCV and its dependencies with the following command:

sudo apt install python3-opencv

• Wait for the installation to complete. It may take a few minutes, depending on your internet speed and your Pi model.
• Verify that OpenCV is installed correctly by running the following command:

apt show python3-opencv

You should see some information about the OpenCV package, such as its version, description, and dependencies.

Step 6: Fire Detection Algorithm Setup

The fire detection system uses the HSV (Hue, Saturation, Value) color algorithm to detect fires. The algorithm involves the following steps:

• RGB to HSV Conversion: The image is converted from RGB (Red, Green, Blue) color space to HSV.
• Apply a Threshold Mask: A threshold mask is applied to isolate the colors representing fire.

The HSV representation of color allows for easier isolation of colors. In HSV, hue determines the color, saturation determines the color’s intensity, and value determines the lightness of the image.

Here is the complete python script to detect the fire.

import cv2

fire_cascade = cv2.CascadeClassifier('fire_detection.xml')

cap = cv2.VideoCapture(0)

while(True):
    ret, frame = cap.read()
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    fire = fire_cascade.detectMultiScale(frame, 1.2, 5)

 for (x,y,w,h) in fire:
        cv2.rectangle(frame,(x-20,y-20),(x+w+20,y+h+20),(255,0,0),2)
        roi_gray = gray[y:y+h, x:x+w]
        roi_color = frame[y:y+h, x:x+w]
        print("Fire Detected")

    cv2.imshow('frame', frame)
 if cv2.waitKey(1) & 0xFF == ord('q'):
 break

Step 7: Cellular IoT Setup

Blues notecard is a device-to-cloud data pump that reduces the complexity of building connected solutions with a secure, reliable cellular or Wi-Fi connection. It can be easily integrated with Raspberry Pi using a development board called Notecarrier, which exposes the pins on the Notecard to the Raspberry Pi.You can use the Notecarrier Pi Hat.

You can directly connect the Notecarrier and Notecard with Raspberry Pi.

Once installed the Notecarrier and the Notecard, we need to enable the I2C on Raspberry pi. For that, open the SSH and type “sudo raspi-config” then set up the I2C.

Then enable the I2C communication.

Then reboot the raspberry pi.

Step 8: Blues integration with OpenCV

Next, we need to install a notehub library to communicate with the Notecard.

Here is the sample python sketch to communicate with Notehub via I2C.

"""note-python I2C example.

This file contains a complete working sample for using the note-python
library with an I2C Notecard connection.
"""
import json
import sys
import os
import time

sys.path.insert(0, os.path.abspath(
                os.path.join(os.path.dirname(__file__), '..')))

import notecard  # noqa: E402

productUID = "com.your-company.your-project"

if sys.implementation.name != 'cpython':
 raise Exception("Please run this example in a CPython environment.")

from periphery import I2C  # noqa: E402


def NotecardExceptionInfo(exception):
 """Construct a formatted Exception string.

    Args:
        exception (Exception): An exception object.

    Returns:
        string: a summary of the exception with line number and details.
    """
    s1 = '{}'.format(sys.exc_info()[-1].tb_lineno)
    s2 = exception.__class__.__name__
 return "line " + s1 + ": " + s2 + ": " + ' '.join(map(str, exception.args))


def configure_notecard(card):
 """Submit a simple JSON-based request to the Notecard.

    Args:
        card (object): An instance of the Notecard class

    """
    req = {"req": "hub.set"}
    req["product"] = productUID
    req["mode"] = "continuous"

 try:
        card.Transaction(req)
 except Exception as exception:
        print("Transaction error: " + NotecardExceptionInfo(exception))
        time.sleep(5)


def get_temp_and_voltage(card):
 """Submit a simple JSON-based request to the Notecard.

    Args:
        card (object): An instance of the Notecard class

    """
    temp = 0
    voltage = 0

 try:
        req = {"req": "card.temp"}
        rsp = card.Transaction(req)
        temp = rsp["value"]

        req = {"req": "card.voltage"}
        rsp = card.Transaction(req)
        voltage = rsp["value"]
 except Exception as exception:
        print("Transaction error: " + NotecardExceptionInfo(exception))
        time.sleep(5)

 return temp, voltage


def main():
 """Connect to Notcard and run a transaction test."""
    print("Opening port...")
 try:
        port = I2C("/dev/i2c-1")
 except Exception as exception:
 raise Exception("error opening port: "
                        + NotecardExceptionInfo(exception))

    print("Opening Notecard...")
 try:
        card = notecard.OpenI2C(port, 0, 0, debug=True)
 except Exception as exception:
 raise Exception("error opening notecard: "
                        + NotecardExceptionInfo(exception))

 # If success, configure the Notecard and send some data
    configure_notecard(card)
    temp, voltage = get_temp_and_voltage(card)

    req = {"req": "note.add"}
    req["sync"] = True
    req["body"] = {"temp": temp, "voltage": voltage}

 try:
        card.Transaction(req)
 except Exception as exception:
        print("Transaction error: " + NotecardExceptionInfo(exception))
        time.sleep(5)

main()

In this python script you have to add your product UID for that to navigate to Blues Notehub and create a new project.

and create a new project.

Here is the complete sketch to detect fire and initiate the data transmission.

import cv2
import json
import sys
import os
import time


fire_cascade = cv2.CascadeClassifier('fire_detection.xml')
cap = cv2.VideoCapture(0)

sys.path.insert(0, os.path.abspath(
                os.path.join(os.path.dirname(__file__), '..')))

import notecard  # noqa: E402

productUID = "xxxxxxxxxxxxxxxxx"

if sys.implementation.name != 'cpython':
 raise Exception("Please run this example in a CPython environment.")

from periphery import I2C  # noqa: E402

def NotecardExceptionInfo(exception):
 """Construct a formatted Exception string.

    Args:
        exception (Exception): An exception object.

    Returns:
        string: a summary of the exception with line number and details.
    """
    s1 = '{}'.format(sys.exc_info()[-1].tb_lineno)
    s2 = exception.__class__.__name__
 return "line " + s1 + ": " + s2 + ": " + ' '.join(map(str, exception.args))


def configure_notecard(card):
 
    req={"req":"card.wifi","ssid":"xxxxx","password":"xxxxx"}
    card.Transaction(req)
    time.sleep(5)


    req = {"req": "hub.set"}
    req["product"] = productUID
    req["mode"] = "continuous"

 try:
        card.Transaction(req)
 except Exception as exception:
        print("Transaction error: " + NotecardExceptionInfo(exception))
        time.sleep(5)


def get_temp_and_voltage(card):
    temp = 0
    voltage = 0

 try:
        req = {"req": "card.temp"}
        rsp = card.Transaction(req)
        temp = rsp["value"]

        req = {"req": "card.voltage"}
        rsp = card.Transaction(req)
        voltage = rsp["value"]
 except Exception as exception:
        print("Transaction error: " + NotecardExceptionInfo(exception))
        time.sleep(5)

 return temp, voltage


def main():
 """Connect to Notcard and run a transaction test."""
    print("Opening port...")
 try:
        port = I2C("/dev/i2c-1")
 except Exception as exception:
 raise Exception("error opening port: "
                        + NotecardExceptionInfo(exception))

    print("Opening Notecard...")
 try:
        card = notecard.OpenI2C(port, 0, 0, debug=True)
 except Exception as exception:
 raise Exception("error opening notecard: "
                        + NotecardExceptionInfo(exception))

 # If success, configure the Notecard and send some data
    configure_notecard(card)
 
 
 while(True):
      ret, frame = cap.read()
      gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
      fire = fire_cascade.detectMultiScale(frame, 1.2, 5)

 for (x,y,w,h) in fire:
        cv2.rectangle(frame,(x-20,y-20),(x+w+20,y+h+20),(255,0,0),2)
        roi_gray = gray[y:y+h, x:x+w]
        roi_color = frame[y:y+h, x:x+w]
        print("fire is detected")
 
        temp, voltage = get_temp_and_voltage(card)
 
        req = {"req": "note.add"}
        req["sync"] = True
        req["body"] = {"temp": temp, "voltage": voltage,"status":"fire_detected","Alarm":1}

 try:
          card.Transaction(req)
 except Exception as exception:
          print("Transaction error: " + NotecardExceptionInfo(exception))
          time.sleep(10)

      cv2.imshow('frame', frame)
 if cv2.waitKey(1) & 0xFF == ord('q'):
 break
main()

Now you can run the python script from the ssh.

Now you can look at your data in Blues notehub.

Step 9: Dashboard Creation

Next, we need to add the dashboards to view our data. For dashboards we are going to use Ubidots. Here is a complete step by step guide from Blues to integrate the Blues with Ubidots.

Routing Data to Cloud - Blues Developers

Here is the final device dashboard with all the device data.

Conclusion

OpenCV for fire detection and Cellular IoT for alerting are two cutting-edge technologies that work together to create a robust and reliable fire safety system. By using image processing and machine learning algorithms, this system can identify fire hazards in real time and send video alerts to the remote fire alarm control unit. This way, the system can prevent fire disasters from escalating and notify the relevant authorities and stakeholders as soon as possible.