System for collecting information about the behavior of pets

I wonder what our pets do during the day. Especially when we are not at home. I decided to address this issue.

The type of pets – cats and cats in the amount of 4-5 copies (these are those that have settled permanently, others periodically visit).

The territory of the habitat of pets is a house with a plot of 6 acres.

A task.

Register the location of animals with the help of cameras and save the frames. Next, make a microfilm from the frames and send it to telegram at the end of the day.

I tell you what was planned, what happened, what did not.

Equipment.

Considering that cats spend most of their time on the site, I decided to use the installed video surveillance system of 4 cameras with a registrar to track pets.

A very simple Falcon Eye recorder for 4 cameras. Four analog cameras. The main thing is that the registrar can send a stream to the rtsp network, which can be received on a computer and search for animals in the frames.

Capturing rtsp stream from cameras in python

The registrar sends the rtsp stream to the address

rtsp://<login>:<password>@ < registrar ip address>:554/mode=read&id= < camera n>&ids=1

Install the operating system on the Raspberry pi 4, then Open Info and NS2 according to the instructions https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_raspbian.html

Installing additional packages

pip3 install imutils
pip3 install numpy
pip3 install pytelegrambotapi

Downloading the model files

mobilenet-ssd.caffemodel and mobilenet-ssd.prototxt

We are writing a script for capturing video from 4 cameras, running the image through the detector and saving the image to the cam directory<n>/<d-n-Y> when detecting an object (cat)

Connecting libraries

from imutils.video import VideoStream
from imutils.video import FPS
import numpy as np
import sys
import argparse
import imutils
import time
import cv2
from urllib.request import urlopen
from datetime import datetime
import os

Variables

tekcamera=1
pathSaveImg="/home/petin/python3_prgs_1/OpenVino01"
cameras=[[0,0,0,0,0],
[0,0,0,0,0],
[False,True,False,True,False],
["","Camera1","Camera2","Camera3","Camera4"]]
for i in range(1,5):
cameras[1][i] = 'rtsp://admin:191066@192.168.0.109:554/mode=real&idc='+str(i)+'’&ids=1'
fps = FPS().start()

# getting command line arguments

# --prototxt path to the mobilenet-ssd.prototxt file

# --model path to the mobilenet-ssd.caffemodel model file

# --show output of images from cameras to windows

# - c minimum accuracy of object detection

ap = argparse.ArgumentParser()
ap.add_argument("--prototxt", required=True,
help="path to Caffe 'deploy' prototxt file")
ap.add_argument("--model", required=True,
help="path to Caffe pre-trained model")
ap.add_argument("--show", required=True,
help="Show cv2.imshow)")
ap.add_argument("-c", "--confidence", type=float, default=0.2,
help="minimum probability to filter weak detections")
args = vars(ap.parse_args())

# loading the model

CLASSES = ["background", "aeroplane", "bicycle", "bird", "boat",
"bottle", "bus", "car", "cat", "chair", "cow", "diningtable",
"dog", "horse", "motorbike", "person", "pottedplant", "sheep",
"sofa", "train", "tvmonitor"]
COLORS = np.random.uniform(0, 255, size=(len(CLASSES), 3))
print("[INFO] loading model...")
net = cv2.dnn.readNetFromCaffe(args["prototxt"], args["model"])
# processing Neural Compute Stick
net.setPreferableTarget(cv2.dnn.DNN_TARGET_MYRIAD)

# initialization of receiving a stream from cameras

print ("[INFO] starting video stream...")
for i in range(1,5):
cameras[0][i] = cv2.VideoCapture(cameras[1][i])
print("OK")
time.sleep(5.0)
detected_objects = []

# cycle

while(1):
  tekcamera = tekcamera+1
  if tekcamera+==5:
    tekcamera=1
    logfile=open("last.txt","w+")
    ftime=datetime.now()
    str1=.strftime("%d-%m-%Y % %H:%M:%S\n ")
    logfile.write(str1)
    logfile.close()
  if cameras[2][tekcamera] == False:
    continue
  # getting frames from a stream
  ret, frame = cameras[0][tekcamera].read()
  frame = imutils.resize(frame, width=800)
  # grab the frame dimensions and convert it to a blob
  (h, w) = frame.shape[:2]
  blob = cv2.dnn.blobFromImage(cv2.resize(frame, (300, 300)), 0.007843, (300, 300), 127.5)
  # pass the blob through the network and obtain the detections and
  # predictions
  net.setInput(blob)
  detections = net.forward()
  # processing of detection results
  print("******************")
  for i in np.arange(0, detections.shape[2]):
    confidence = detections[0, 0, i, 2]
    idx = int(detections[0, 0, i, 1])
    #if confidence > args["confidence"] and idx==15 :
    if confidence > args["confidence"] and (idx==8 or idx==12) :
      # save files
      ftime=datetime.now()
      if(os.path.exists(pathSaveImg+"/cam"+str(tekcamera)+"/"+ftime.strftime("%d-%m-%Y"))==False):
        os.mkdir(pathSaveImg+"/cam"+str(tekcamera)+"/"+ftime.strftime("%d-%m-%Y"))
      f=cv2.imwrite(pathSaveImg+"/cam"+str(tekcamera)+"/"+ftime.strftime("%d-%m-%Y")+"/_"+ftime.strftime("%H:%M:%S.%f")+".jpg", frame)
      print("write file = ",f)
      # extract the index of the class label from the
      # `detections`, then compute the (x, y)-coordinates of
      # the bounding box for the object
      #idx = int(detections[0, 0, i, 1])
      box = detections[0, 0, i, 3:7] * np.array([w, h, w, h])
      (startX, startY, endX, endY) = box.astype("int")
      # draw the prediction on the frame
      label = "{}: {:.2f}%".format(CLASSES[idx], confidence * 100)
      print(confidence," ",idx," - ",CLASSES[idx])
      detected_objects.append(label)
      cv2.rectangle(frame, (startX, startY), (endX, endY),COLORS[idx], 2)
      y = startY - 15 if startY - 15 > 15 else startY + 15
      cv2.putText(frame, label, (startX, y),cv2.FONT_HERSHEY_SIMPLEX, 0.5, COLORS[idx], 2)
      # show the output frame
  if args["show"] == "True":
    cv2.imshow(cameras[3][tekcamera], frame)
  key = cv2.waitKey(1) & 0xFF
  # exit using the ‘q ' key
  if key == ord ("q"):
    break
  fps.update()
  fps.stop()
  print("[INFO] elasped time: {:.2f}".format(fps.elapsed()))
  print("[INFO] approx. FPS: {:.2f}".format(fps.fps()))
  # do a bit of cleanup
  cv2.destroyAllWindows()

Running the script for execution

cd /home/pi/python 3_pigs_1/OpenVino01
python 3 pets04.py --proto txt mobilenet-sd.proto txt --model mobilenet-sd.safe mode --show True -c 0.3

Images are saved in the folders cam<n>/<d-m

Be sure to do the autoloading of the script

Assembling a video from images

It is necessary to collect videos from the received images.

Installing the ffmpeg package

sudo apt install ffmpeg

And running the following commands

for camera 1

ffmpeg -f image2 -pattern_type glob -i /home/pi/python3_prgs_1/OpenVino01/cam1/$(date +%d-%m-%Y)/'*.jpg' -y /home/pi/python3_prgs_1/OpenVino01/cam1-$(date +%d-%m-%Y).mp4 –r 10

for the camera 2

ffmpeg -f image2 -pattern_type glob -i /home/pi/python3_prgs_1/OpenVino01/cam2/$(date +%d-%m-%Y)/'*.jpg' -y /home/pi/python3_prgs_1/OpenVino01/cam2-$(date +%d-%m-%Y).mp4 –r 10

etc.

-r 10 is 10 frames per second (making a timelapse )) )

These commands must be placed in cron (run at the end of the day)

By the end of the day, I form 4 files-cam<n>-<d-m-Y>. mp4

Sending data to Telegram

Creating a Telegram bot. For those who do not know, we write a message to the user @BotFather and in the dialog we come up with the name of the bot - <name>bot

A message is received with a token for the http api, which must be saved

Create a group and add a bot to it

Now sending messages to the bot

I wrote the program in python

Installing the

pip3 install pytelegrambotapi library

And we are writing a program.

import telebot
from telebot import types
from datetime import datetime, timedelta
from pathlib import Path

keyboard1 = [["", "Camera 1", "Camera 2", "Camera 3", "Camera 4","Don't"], [ " ", "cam1", "cam2", "cam3", "cam4","no"]]

bot = telebot.TeleBot('your-token');

@bot.message_handler(content_types=['text', 'document', 'audio'])
def get_text_messages(message):
  chatId=message.chat.id
  print(chatId)
  key_1=[0,0,0,0,0,0]
  ftime=datetime.now()
  print(ftime.day," ",ftime.hour)
  day=ftime.strftime("%d-%m-%Y")
  if message.text == "/help":
    bot.send_message(message.from_user.id, " Hi, here you can see what my cats did during the day 9-00-18-00. Type /video")
  elif message. text = = "/video":
    msg= "here you can see what my cats did for the day 9-00-18-00" +day
  bot.send_message(message.from_user.id, msg)
  keyboard = types.InlineKeyboardMarkup(); # keyboard
  for i in range(1,6):
    #button
    key_1[i] = types.InlineKeyboardButton(text=keyboard1[0][i], callback_data=keyboard1[1][i]);
    #adding a button to the keyboard
    keyboard.add(key_1[i]);
    bot.send_message(message.from_user.id, "Choose a camera", reply_markup=keyboard)
  else:
    bot.send_message(message.from_user.id" I don't understand you. Write /help.")

@bot.callback_query_handler(func=lambda call: True)
def callback_worker(call):
  ftime=datetime.now()
  day=ftime.strftime("%d-%m-%Y")
  if call.data.find("cam") < 0:
    bot.send_message(call.message.chat.id, 'No');
  else:
    if ftime.hour >= 18:
      bot. send_message(call.message.chat.id, 'Video from the camera '+call. data.replace ("cam","")+'. Wait ....');
      videofile = Path(call.data+'-'+day+'.mp4')
      if videofile.is_file():
        video = open(call.data+'-'+day+'.mp4', 'rb')  
        bot.send_video(call.message.chat.id, video)
        bot. send_message(call.message.chat.id, "Loaded")
      else:
        bot. send_message(call.message.chat.id, 'The video is missing !!!');
    else:
      bot. send_message(call.message.chat.id, 'Viewing the video of the current day only after 18: 00, Viewing for the previous day');
      ftime=datetime.now()- timedelta(days=1)
      dayold=ftime.strftime("%d-%m-%Y")
      videofile = Path(call.data+'-'+dayold+'.mp4')
      if videofile.is_file():
        video = open(call.data+'-'+dayold+'.mp4', 'rb')
        bot.send_video(call.message.chat.id, video)
        bot. send_message(call.message.chat.id, "Loaded")
      else:
        bot. send_message(call.message.chat.id, 'The video is missing !!!');

bot.polling(none_stop=True, interval=0)

Launching the bot

cd /home/pi/python3_prgs_1/OpenVino01
python3 pets_send_telegram.py

Commands for the bot /help and / video

When receiving the video command, the menu with the camera selection

When choosing a camera, sending a video to the bot of the current day (after 18-00) or the previous one

Choosing places to install cameras

And the most unpredictable thing is the choice of places where to install cameras to detect seals. Here is only a trial and error method. Installing the cameras

We connect it to the registrar

The network has a router, a registrar, a Raspberry pi over Wi-Fi. Access to the Internet

Run the script for receiving data from cameras and detecting

cd /home/pi/python 3_pigs_1/OpenVino01
python 3 pets04.py --proto txt mobilenet-sd.proto txt --model mobilenet-sd.safe mode --show True -c 0.3

And the bot script

cd /home/bin/python3_prgs_1/OpenVino01
python 3 pets_send_telegram.py

Setting up cron

We go to Telegram, select the camera

When selecting a camera, sending a video to the bot of the current day (after 18-00) or the previous one