Fire Rescue UAV Drone

The Fire Rescue UAV Drone

By Fin Mead (Meadeor) for the HoverGames Drone competition

Summary

With the increasing threat of climate change, greater populations and consequently, more extreme environmental events - the need for better emergency rescue services is paramount. Fire deaths have tripled and response times have almost doubled in some areas, leading to the idea that new technologies such as unmanned aerial vehicle (UAV) drones could help reduce deaths, decrease response times and provide aid, in many forms, to the emergency services.

Introduction

Devices such as a drone can do many things that rescue teams cannot and produce important data to help operations. Technology has been used in the past to assist the emergency services in the UK, and in many other countries on a global scale. An example of this is how camera dogs are used to perform reconnaissance, assess a situation, search an area and find missing people. These canines are fitted with cameras and sensors. The video feed along with other data is transmitted to the dog’s handler, giving them information about the situation and environment before engaging other resources - like people. These advanced K-9 Units can be used on small scale scenarios like a house fire or property search as well as on a larger scale operation; perhaps by search and rescue teams following a natural disaster like a forest fire, tsunami or earthquake.

A better-unmanned system which can be used in these same ways (and more) without risk of animal and human lives is with UAV drones and rovers. These devices can gather masses of information and allow for a more prepared and effective mission, operation or search party. The problem is mainly due to the technology being very expensive as well as it being a very new field of technological development. However, in recent years, there have been major developments in drone and autonomous technology. Commercial drones have become so popular that the UK government have now implemented a licensing law for larger or more advanced vehicles (similar to a car license plate system). Another emerging field is autonomy and artificial intelligence. The advances made by companies like Tesla with their actively sensing cars and autonomous controls have created a valuable demand for a smarter and more connected world. Therefore, it seems natural that we should explore how these new technologies can aid police, fire and rescue services in order to save lives and the environment.

Research Review

I began my research by simply aggregating and collating any statistics and evidence concerning the deficiency of the Fire Brigade and emergency services. I also tried to find examples of this in other countries. As well as this, I looked into whether a drone project similar to mine is already in development.

The first thing I found when researching the UK’s emergency services, is that there is a lot of political controversy surrounding it. For many reasons, the emergency services have been the victim of many budget cuts and increased working hours due to job reductions. For instance, the fire service was cut 15% this last year without warning, triggering the closure of multiple stations. Similar examples were found for the ambulance service, NHS, police and other services. These funding cuts and consequential problems mean that the quality of care and ability to perform are put under significant strain and could cost lives.

I was able to find multiple examples of drones and robotics being used in emergency situations, many cases involve using drones to find missing people. A drone helped locate a missing woman in a wooded area in Vilas County, Wisconsin. A mountain biker who fell off a cliff in Sedona, Arizona was rescued by the fire department after a drone located the victim. Drones can also provide other forms of assistance, a drone transported food to hunters during a river rescue. Drones are beginning to make massive improvements to the logistics and strategy that emergency services and response teams, there are examples of the helping in search and rescue operation, fire management missions and for transporting devices, medicine and other important items. An MIT project partnered with NASA in developing autonomous drones that can help in search and rescue scenarios. The delivery company DHL are developing a parcel-copter that has successfully delivered medication to patients in a fraction of the time it would take conventional delivery systems. So it’s not surprising that the Federal Aviation Administration (FAA) aerospace report for 2016 to 2036, 7 million drones will fly the skies by 2020.

In order to begin researching the components, design and specific features of my drone, I summarised the key issues and problems which I wanted to solve as well as any additional abilities which could improve the emergency services further.

The main problems:

Increasing response time

A drone can be deployed to a location faster, and autonomously, so officers/personnel can focus on the task. A drone is not governed by road traffic, speed limits or lanes. (Airspace/traffic operates differently)

Budget cuts

I am aiming to develop this drone at a fraction of the cost of most commercial and government used drones. Most drones are £10, 000 minimum - I will be building with a sub £700 budget. This ensures that it would be affordable to any station which may require one.

Reduced workforce/personnel

Using an autonomous drone requires little to no interaction. It will be able to search large areas which would normally require many people - in far less time.

Poor communication

The drone will be equipped with deployable radio comms as well as a video and data feed, meaning firefighters can see where a fire is spreading, ambulances can see any casualties, rescue teams can calculate arrival times and analyse terrain and search teams can find a missing person. These are just a few applications. The drone enables a greater level of communication - with anyone.

Risk of death

Why send the fire brigade into an unstable building or blazing forest fire when a drone could be used. A drone can be used so that lives are only put at risk when there is an absolute necessity.

Environment

Emergency services can take hours and even days to reach a call due to mountainous terrain, rivers, and low visibility. Search parties can’t cross bodies of water and must stick to pathways or risk getting lost themselves. None of these factors adversely affect a drone’s abilities.

Natural disasters can inhibit emergency services; earthquakes shut down cities, tsunamis destroy fast coastlines and fires compromise large areas of last and emit harmful fumes. A drone can face all of this without compromise.

Specialist features:

A possible feature of the drone could be the ability to deploy devices, resources and relay information to victims and other service people.

Another feature could involve tracking a fire using cameras and then using drone telemetry to calculate how it will spread, and relay that information.

A drone can provide pin-point GPS accuracy to help emergency services find a location.

With multiple cameras, finding people becomes far easier - and after developing software, can be achieved autonomously.

Planning ahead; firefighters, police officers and EMTs can plan how to operate on-route to a scene thanks to the drones video feed and environmental data.

A drone can also provide assistance to victims and teams. Surveying the areas for obstacles, people, other threats and lead the way to emergency service stations, outposts, units and further help.

A drone also offers the ability to spread information via the internet by posting to social media sites like Twitter and Instagram, all whilst on the scene, involved in other operations.

INTERVIEW WITH A RETIRED POLICE OFFICER:

It took a while to set up but I got to talk to a retired police officer (I believe he was a sergeant) about the emergency services and he explained how the police work with the fire brigade and ambulance service during a forest fire and missing person scenario.

InterviewNotes:

"When working with the Fire Brigade to stop a forest fire we try to find out where the fire is moving and find the front. Vegetation and trees are then removed to stop it spreading further - sacrificial dead trees are used to keep the fire from spreading (blocks fire)"

A drone could help find out wind direction and motion of fire to make this process more efficient by telling fire brigade and police where to begin blocking or removing material and where to surround and begin operations.

"When trying to find a missing person, depending on the environment, location and person of interest - usually a radius is created from a point, perhaps their last seen location or where their possessions were found / clue they were there. If items of clothing or personal belongings are found within the circle, the new location becomes the epicentre of a new circle and the search spreads out again from there."

A drone could help in the search, this is because a drone can scan and search an area far quicker than a person and arguable in more detail and with more precision. A drone can also help guide a person back to help as drones can be spotted from a long way away and are loud - as well as this, a drone could be fitted with devices designed to make people on the ground aware of its location (lights/beacon/speaker).

With all of this information from my research and the interview in-mind, I created a list of features which I will attempt to implement into my project.

Proposed Features

NOTE - It's unlikely I will be able to implement all of these features or perhaps not fully due to college, the given time-frame and any issues getting components to work together

• GPS tracking
• Telemetry feed
• Automated flight
• Fire detection and tracking camera (PixyCam2,3D printed Pan and Tilt servo system, Raspberry Pi)
• Deployable oxygen unit (Adapted oxygen tank, Release mechanism)
• Deployable radio communications (Portable HAM radio, Release mechanism)
• Custom Deployable unit (Cargo storage system, Cargo release)
• Guidance system (LED visual guidance, Siren - audio guidance, RPi)
• Deployable signalling device (Flare/LED strobe, Release mechanism)
• Wind/Fire direction monitoring (Wind direction sensor, Display in QGroundControl, Post to social media)
• Social media capabilities (Fire status alerts, Image uploading)

Future features/developments

Fire detection using cameras with image recognition. The fire’s colour signature is picked up by the IR program and then it calculates the fire spread/direction and rate using onboard companion computer and telemetry data e.g. wind-speed. This information is relayed to the emergency services and operator whilst simultaneously being posted to social media. Live updates, almost instantaneously.

The ‘line of fire’ can also be tracked precisely via GPS and the PixyCam2 vector recognition. The drone flies along the path of the fire, plotting it via GPS. This keeps the Fire Brigade and Emergency Services updated on the fire's behaviour and current position.

DEVELOPMENT LOG

I began by building the HoverGames Drone Kit - I didn't document building the kit because it seemed unnecessary considering it's made by NXP and I simply followed the very detailed gitbook (Thank you!) like everyone else.

Thanks to the kit, I can already tick three features on my list.

GPS tracking✔ (Pixhawk4 GPS module)

Telemetry feed✔ (Holybro Telemetry Radio)

Automated flight✔ (PX4 Autopilot)

Using the PixyCam2 to 'see' fire.

The first thing I wanted to start developing was fire recognition/tracking. I began by playing around with the pixcam2 using Pixymon on my computer. I was able to get it working to detect colores/shapes. I then used two 9 gram servo create a pan and tilt system. I made a quick pan/tilt mechanism out of cardboard, and it worked great. The PixyCam2 would track signatures quite well, and the servos would adjust to keep the learned object in focus.

I now wanted to get it to track fire - I started by seeing if the PixyCam2 could track the overexposure of a naked flame.

With the drone successfully tracking fire, I decided to upgrade it's mount and servo mechanism. Using fusion360, I modelled and 3D printed a mounting plate with a hinge and servo slots. (See CAD files) - I mounted it to the drone with some extra nuts and bolts leftover from the kit build.

With the pixy mounted to the drone, and able to track fire. I began to look into how I could make this portable. I was still using my computer to both power and run the pixycam2. I needed to use a microcomputer to control the pixy - although it is able to track on its own - in order to do anything with that tracking data, I needed a companion computer on the drone - I chose the Raspberry Pi 3 B+.

Raspberry Pi - PixyCam2 fire detection

I ran into a number of issues getting the pixy to work on my Pi. Firstly, I had problems installing the dependencies for the pixy, and after finally getting that to work, I had more issues trying to get the python wrapper working so I could run the pixy and control it through python scripts - lots of errors - segmentation errors in particular - but eventually I got it to run.

When I got the demo python scripts to run, I began looking into codding my own script solution - for fire. But before that, I took a break to start my alert system (aka guidance system).

Guidance System

At first, I wanted my visual guidance light to be a small hockey-puck shaped object, but this would also involve soldering a dozen LEDs together, figuring out the voltage, getting the right resistor and then power source. I figured this out after I 3D printed my first light design (made with fusion360 - see CAD repository) I simply couldn't make it that small.

I then realised also, that if I wanted a light that was going to be visible through forest canopies, smoke and in the day - I would need something more powerful than regular LEDs.

I used to work at a Repairs Garage, and from a broken landrover I was able to salvage its brake light (The one at the top, in the middle). Very bright and red, yet only 12 volts! It had been sitting in a drawer for a few months but luckily it still worked.

I needed to use a relay (on hardware list) in order to power the panel sufficiently. I also added a buzzer which I would use later. The light was in this thick plastic housing shown in the Landrover picture, I removed this and measured the circuit board and 3D modelled a sleek, lightweight function backing for the panel (see CAD repository - all models made with fusion 360, now I can stop repeating myself!) and then secured the lensing from the original plat in top!

After this, I wired up my new audio-light module to the Raspberry Pi. (See schematics section) Here's what that looked like!

Now, I was ready to code!!!!

Guidance System

I started by trying to get each component working separately with the Raspberry Pi, then I could simply merge the code together.

Buzzer

The buzzer was fairly simple to program with the Pi. I already had GPIO knowledge from previous projects, and so I knew how to set up a GPIO in python. You can see my test script in the code repository called 'led_test.py'. Even though I used it for the LED panel, it uses the same concept; intervals of turning on and off GPIOs using 'time.sleep'.

LED panel with relay

For the relay, I used a similar method of setting up a GPIO (port 21) connection and then turning it on and off in intervals - also see 'led_test.py' in the code repository for this. The lights would blink red very quickly, it was very bright!

You can see them combined in the led_test.py also, there are two files.

You will see these two components combined in later code also - but first, I needed to develop another part of the project - social media communication.

Twitter API and the Raspberry Pi

Before you can even begin posting to twitter, you have to apply for your own special set of credentials (keys) to use the API. It took a while to get my developer account but when I got approved, I was given my tokens/keys, I now had access to the Twitter API!

Note - You will notice a file called auth.py in my repository, this file is used to hold your keys and tokens - the file doesn't contain the keys though because they belong to my account - you can easily paste your own keys/tokens if you have an account and it'll work just fine.

To access the API from my Pi, I installed the python library for accessing the Twitter API. It's called Tweepy. I then wrote some code to simply post a preexisting text to twitter, I did this a number of times. You can actually see every tweet via the drone on the twitter page here. In fact, every post was made using the Raspberry Pi and API requests.

I also was able to post images to Twitter, you can see this in the 'twitter.py' file in the repository. This will be important in future. I may be able to post images of the fire - live to Twitter.

Now that I had the API working on my Raspberry Pi and I was able to post tweets and photos to twitter, I now wanted to combine each component to work together.

Fire detection, audio-visual drone alert triggering and social media posting as a procedure

After developing all these features, it was time to merge them together to create create a script, which could launch alert procedures like the LED panel, buzzer and twitter posting when the PixyCam2 recognises a fire.

I was able to look at the python demo scripts for the pixy and find out how to access the block recognizing code. This allowed me to use that code and add my procedures into the return statement at the end of the script. So I made a new script with the basic signature recognition code I needed from the demo script and began inserting my code for twitter posting and the LED panel and buzzer components. You can see the resultant script in the repository. It's called 'pixyFireDetectionAndTweet.py'.

Here is a test video from later in the project which shows it working, so you may see new parts on the drone that haven't been evidenced yet.

I disabled the fire-alert twitter posting part of the code for this test, I was worried it might be taken seriously and also I believe there are rules against spreading false information on twitter. I didn't want to get my developer license taken away!

I've been keeping track of my progress with this project using a spreadsheet, and updating it as I complete features. Here's an example.

Now that we have these components all working properly, we can tick off more features from my list.

• Fire detection and tracking camera (PixyCam2,3D printed Pan and Tilt servo system, Raspberry Pi)✔
• Guidance system (LED visual guidance, Siren - audio guidance, RPi)✔
• Social media capabilities (Fire status alerts, Image uploading)✔

Adaptive Payload Transport and Deployment system

The next big thing on my feature list is the idea to be able to transport and deposit resources, devices and objects from the drone.

In a forest fire, the drone could transport packs of respirator masks to survivors in the fire, even provide an aerosol oxygen tank for temporary air supply to help people get moving in toxic smoke. The drone could also transport a radio or smartphone to people who are stranded and create a direct channel of communication between the emergency services and the victims (very important). Not only this, you could deliver medicine, first aid kits and other resources. I've been calling such items 'Deployables'.

Release mechanism - all parts are 3D printed

The release mechanism is a key part of the payload transport mechanism. It will keep the payload securely attached to the drone until the drone lands, and it is released by the servo arm or it can perform a mid-air supply drop (Do not do this with an oxygen aerosol!).

I have made release mechanisms before so I understand how they work. I jumped into fusion360 and began modelling a release mechanism for the servo. You can see all the mentioned model files in the CAD repository. It's basically an enclosed hook which is retractable by the servo. The servo is connected to one of the auxiliary ports on the drone receiver/FMU to allow me servo control from the radio transmitter.

I then create two plates which sandwiched together. One side (payload) has a square loop for the hook and four 'nipples' which sit inside impressions on the other plate. This made the connection far more secure with no wiggle room to loosen the connection.

I also had to make a small spacer to allow clearance for the battery.

Here's what the whole module looks like mounted:

I used velcro tape to create a velcro surface on the bottom of the orange payload plate - velcro is actually very strong! This will also make it easier for people to remove the payload item from the plate as there are no bolts or screws. Good luck finding a screwdriver in a forest fire! In order to attach an object to the payload plate, you just need velcro on it. This means radios, medicine tins and first aid bags etc... are now all easily deployable to survivors, injured civilians or unreachable victims.

I even made a sleeve to fit an aerosol oxygen tank in. I don't have one, but they are the normal aerosol diameter so I used a deodorant can as reference. It's a mesh design so it isn't heavy.

You can also fit the lightbar module from earlier to the payload plate as it too, has velcro on it! You can then detach the Lightbar take the RPi plugs out and the 9v battery will power it portably, this can now be used as a signalling device!

Note - I designed all these parts with strength, lightweight and functionality in mind. I wanted to preserve as much excess thrust so It can carry more!

You can see the drone finally flying in this next video! Sorry for the poor video quality but they the file was compressed very heavily when sent over the web from my mother's phone to me. You may still be able to see the red slashing led bar toward the end of the video - it was far brighter in person - and I was also only running it on a 9 volt, I normally connect it to the drone battery which actually overvolts it to 14 volts.

With the 'Adaptive Payload Transport and Deployment system' part done, I can now tick even more features of the list!

• Deployable oxygen unit (Adapted oxygen tank, Release mechanism)✔
• Deployable radio communications (Portable HAM radio, Release mechanism)✔
• Custom Deployable unit (Cargo storage system, Cargo release)✔
• Deployable signalling device (Flare/LED strobe, Release mechanism)✔

Final touches

I also modelled and 3D printed a dome to cover the FMU and other valuable components. Then I also designed and printed some additional landing gear mounts for a camera and for the Guidance system so they were all secure. (See CAD repository) - I have a dual rig on my radio transmitter so I have a small monitor I can see the camera live feed as well as my phone with QGroundControl and the holybro telemetry radio attached to my phone case with velcro!

Some things I wanted to do but ran out of time

The Raspberry Pi as a companion computer

I really wanted to get the Raspberry Pi connected to the FMU as a companion computer - this would have allowed for fully autonomous fire tracking! Now, this isn't particularly difficult, it just takes time and was tedious as I keep running into errors. However, just as a proof of concept, I will talk through how I would have developed such features which were dependant on this connection.

Autonomous Fire Tracking

I would have used Mavlink to transfer positional data and windspeed to the Raspberry Pi. Then, I would have implemented that data into the python script which manages the drone. With this, I could have the drone circle the position of the fire, follow a fast-moving fire and provide an approximate direction for which the fire will spread to via wind-speed telemetry data. Autonomous fire tracking also means that there's no need for a drone operator, so more people can be engaged in helping fight the fire and save lives.

Precise social media updates

With the Pi connected to the FMU, I would be able to access GPS and telemetry data and insert that into my twitter API posting code. This means the drone could post the fire's current location and the windspeed - which suggests the way it will travel. With the companion computer, I could also have had the drone autonomously surveying the forest, and when it spots a fire starting - alert the Fire department via twitter! This could prevent massive forest fires. You could also have the colour signature of a lost person's clothing on the PixyCam and post to twitter the drone's GPS location and a picture if it picks up anything which resembles that signature - speeding up the Search and Rescue process and improving the possibility of finding a missing person.

However, I can partially tick off fire tracking with wind direction/speed because I was able to display the current drone wind speed and direction in QGroundControl, so the drone operator could know the exact wind-speed and direction and relay that information to Fire Rescue teams.

• Wind-direction/speed monitoring✔

PixyCam2 fire-vector tracking

One way I wanted to develop the PixyCam2 was with a 'line of fire' recognition script. This would have allowed me to track the front of a spreading fire - which was mentioned in the possible features section.

Line-of-fire GPS tracking

I would have liked to use the PixyCam2's built-in line recognition intelligence to try and recognise a line of fire. This would have made it possible to have the Pi command the drone to fly along that line and even procedurally log it's location on that line, sending it back to the QGroundControl or over the web. This would create a plotted map of fire data - which could be updated live, as the fire spreads!

Raspberry Pi 4G Shield

A 4G shield for my Pi would have made twitter posting fully operational because while testing my drone I had to have it close enough to Wifi or tethered to my phone meaning that once It went approximately 50 meters from either source, the internet connection dropped. A 4G shield would have been very easy to set up but I simply couldn't get it delivered in time - the one I required only shipped from China and took up to 40 days to arrive!

A 4G shield would also have allowed for text updates. this means fire marshals and firemen could receive text messages from the drone when it spots the fire or when the drone senses a directional change in the fire. So instead of twitter, the fireman could quickly get information via SMS text messages.

Other components in the HoverGamed drone kit

I would have loved to explore the capabilities of the NXP Rapid IoT prototyping kit and thermal sensor but I simply ran out of time! I can imagine using the Rapid IoT for lots of things like transferring flight data via NFC or even line of fire map data. Perhaps use it to sense if the drone is falling, log pressure changes due to the fire's creating thermal pockets and also transmit air quality data to the emergency services to say if it is breathable etc... So many possibilities!

In summary

I believe I have accomplished a lot for a one-man team, whilst juggling my final year of college and work simultaneously!

I have a drone which can sense fire and track it! Then send out a tweet saying so whilst triggering an audio-visual alert on the drone.

I have a guidance system for people who are lost or require assistance getting from one place to another, which is very bright and easy to spot in any conditions.

I've made a portable signalling device which allows people to get the attention of a search party, aircraft or emergency personnel.

I've also built a modular payload transport system with lots of possibilities like transporting supplies, dropping off packages and delivering devices like radios and respirators.

Completed Features

• GPS tracking
• Telemetry feed
• Automated flight
• Fire detection and tracking camera (PixyCam2,3D printed Pan and Tilt servo system, Raspberry Pi)
• Deployable oxygen unit (Adapted oxygen tank, Release mechanism)
• Deployable radio communications (Portable HAM radio, Release mechanism)
• Custom Deployable unit (Cargo storage system, Cargo release)
• Guidance system (LED visual guidance, Siren - audio guidance, RPi)
• Deployable signalling device (Flare/LED strobe, Release mechanism)
• Wind/Fire direction monitoring (Wind direction sensor, Display in QGroundControl)
• Social media capabilities (Fire status alerts, Image uploading)

Thank you

I'd like to thank NXP and Hackster.io for running this competition. In the current political and environmental climate - it is important to be looking for ways to help preserve the environment and of course, save lives. With an ever-increasing population, more and more forest fire like the current events in Australia and California and the Amazon rainforest, it is necessary to be looking for a better solution. I have had the idea to use a drone to help combat environmental hazards like air pollution and deforestation but it wasn't until I found this competition that I was able to fully understand the opportunities that a small UAV aircraft can have.

I'd like to thank PixyCam, more specifically, Jesse French who I talked back and forth with over email a lot, they helped me get the PixyCam2 to work with my Raspberry Pi.

I'd also like to thankFliteTest for making me aware of the competition! I'm a long time viewer and if it wasn't for them promoting it, I'm not sure I'd have found out about it!

Finally, I'd like to thank my mum for helping me film the test videos and taking some photos, as well as my grandparents for letting me use their garden as my own personal test facility!

Here are some final images:

Thanks for reading!