Published June 16, 2023 © CC BY-NC

ReFound

A novel idea to efficiently improve the process of retaining lost items in campuses and schools

IntermediateFull instructions provided7 hours351

Safety & Security Award

Smart Campus 2023

Things used in this project

Hardware components

DFRobot Hackster EEDU Kit - Getting started with IoT

DFRobot HUSKYLENS AI Camera

Software apps and online services

Arduino Web Editor

Arduino IoT Cloud

Hand tools and fabrication machines

Soldering iron (generic)

Solder Wire, Lead Free

Soldering Iron Holder, With Sponge

Story

Imagine you are in your school or university campus, and it is last period when you realize you have lost your water bottle. In a typical scenario, you might have to check all of your eight classes to find out where you lost it, but if you have a smart campus equipped with ReFound, you won't have to go through all the hassle.

ReFound is a novel product to efficiently improve the process of retaining lost items on campuses and schools. ReFound uses Artificial Intelligence (AI) Object Detection algorithms to detect when an item is placed in front of it on the lost and found table and updates the ReFound dashboard with the item name so that the user can easily retrieve it. Additionally, it plays calming music in the background to relieve the user's stress about losing the item and gives an audio signal so the user can more easily locate the lost and found table.

ReFound combats the problem of finding lost items and improves the following factors of the campus:

Sustainability -If the item is lost and we can not find it, we have to buy a new item, depleting the Earth's resources; ReFound increases the chances of the item being found and reduces the chances of resource wastage
Safety & Security - reduces the chances of the item being stolen as all the lost items will be in a centralized location where they are monitored by the DFRobot HuskyLens camera
Transportation - ReFound reduces traffic congestion in the hallways as one does not have to travel to every classroom to search for their item; they just need to pick it up at the lost and found table when they see it in the ReFound dashboard
Accessibility - makes it easier for individuals with disabilities to retrieve their lost items as there is less travel distance
Student Engagement - the calming music emotionally supports the students while at the same time is an audio signal for students to easily locate the lost and found table

How it works

Labeled Diagram of ReFound

Above are some of the main components of ReFound.

HuskyLens🐶📷AI Module - camera takes picture of item on lost and found table and runs object classification model to determine the object type
FireBeetle🔥🪲 ESP32 - main microcontroller of ReFound - responsible for taking output from Object Classification model running on HuskyLens AI Module and sending it to Arduino IoT Cloud ReFound Dashboard.
DF Player Pro💿 & Speaker🔊 - plays calming audio in background to relieve stress of user and allow students to more easily audibly locate where the lost and found table is.

Once the FireBeetle ESP32 sends the data to Arduino Cloud, it can be viewed in the ReFound dashboard

Arduino IoT Cloud is at the core functionality of ReFound in that it allows for the user to view easily in the customized dashboard what objects are lying on the lost and found table, and creates an easy way for the DFRobot FireBeetle ESP32 to send the result of the AI inference to the user.

Demo Video

Now let's see a demo video of how ReFound works:

How to DIY

1. Solder the header pins to the DFRobot Player Pro

2. Connect the components as shown in the following circuit diagram:

ReFound Schematic

3. Open Arduino IoT Cloud and register a new FireBeetle-ESP32 device

4. Make sure to record the secret keys in your notes

5. Install the Arduino Create Agent

6. Create the `message` and `Object_ID` variables

7. Configure your WiFi Network Credentials

8. Put code in device sketch tab and upload it to device (make sure FireBeetle-ESP32 is connected to PC)

9. Create dashboard like the one below:

The project is complete! Train the machine learning model on the DFRobot HuskyLens AI in image classification mode and then you can deploy ReFound in your school or university and see it in action, making the process of finding lost items more efficient.

References

https://wiki.dfrobot.com/Hackster_EEDU_Kit_AI_IoT_Environmental_Sensing

https://wiki.dfrobot.com/HUSKYLENS_V1.0_SKU_SEN0305_SEN0336

https://wiki.dfrobot.com/DFPlayer_PRO_SKU_DFR0768

https://docs.arduino.cc/arduino-cloud/getting-started/iot-cloud-getting-started

Code

Main

#include "thingProperties.h"
#include "HUSKYLENS.h"
HUSKYLENS huskylens;
//HUSKYLENS green line >> SDA; blue line >> SCL
void printResult(HUSKYLENSResult result);

void setup() {
  // Initialize serial and wait for port to open:
  Serial.begin(9600);
  // This delay gives the chance to wait for a Serial Monitor without blocking if none is found
  delay(1500);

  // Defined in thingProperties.h
  initProperties();

  // Connect to Arduino IoT Cloud
  ArduinoCloud.begin(ArduinoIoTPreferredConnection);

  /*
     The following function allows you to obtain more information
     related to the state of network and IoT Cloud connection and errors
     the higher number the more granular information you’ll get.
     The default is 0 (only errors).
     Maximum is 4
  */
  setDebugMessageLevel(2);
  ArduinoCloud.printDebugInfo();
  Wire.begin();
  while (!huskylens.begin(Wire))
  {
    Serial.println(F("Begin failed!"));
    Serial.println(F("1.Please recheck the \"Protocol Type\" in HUSKYLENS (General Settings>>Protocol Type>>I2C)"));
    Serial.println(F("2.Please recheck the connection."));
    delay(100);
  }
  pinMode(D6, OUTPUT);
}

void loop() {
  ArduinoCloud.update();
  // Your code here
  if (!huskylens.request()) Serial.println(F("Fail to request data from HUSKYLENS, recheck the connection!"));
  else if (!huskylens.isLearned()) Serial.println(F("Nothing learned, press learn button on HUSKYLENS to learn one!"));
  else if (!huskylens.available()) Serial.println(F("No block or arrow appears on the screen!"));
  else
  {
    // Serial.println(F("###########"));
    while (huskylens.available())
    {
      HUSKYLENSResult result = huskylens.read();
      printResult(result);
    }
  }
}

void printResult(HUSKYLENSResult result) {
  if (result.command == COMMAND_RETURN_BLOCK) {
    object_ID = result.ID;
    if (object_ID == 2) {
      message = "mobile phone";
    } else if (object_ID == 1) {
      message = "water bottle";
    } else if (object_ID == 3) {
      message = "bracelet";
    } else if (object_ID == 4) {
      message = "jewel";
    } else if (object_ID == 5) {
      message = "mouse";
    } else {
      message = "pencil case";
    }
  }
}

/*
  Since ObjectID is READ_WRITE variable, onObjectIDChange() is
  executed every time a new value is received from IoT Cloud.
*/
void onObjectIDChange()  {
  // Add your code here to act upon ObjectID change
}

/*
  Since ObjectType is READ_WRITE variable, onObjectTypeChange() is
  executed every time a new value is received from IoT Cloud.
*/
void onObjectTypeChange()  {
  // Add your code here to act upon ObjectType change
}

/*
  Since Message is READ_WRITE variable, onMessageChange() is
  executed every time a new value is received from IoT Cloud.
*/
void onMessageChange()  {
  // Add your code here to act upon Message change
}