Salt Water Tracker (SWT)

Sea levels are rising. Salt seawater is penetrating severely into inland rice paddies in Vietnam Mekong delta and other regions on earth. It affects not only plants but also millions of people who live in the area. Leaves turn yellow and people get more diseases because of salt in water.

A temporary solution now is to use river water and rain water to regulate salt level and wash salt back to the ocean. In the wet seasons, farmers store rainwater and river water in reservoirs. In the dry seasons, no drain drops and less upstream water cause salt seawater to intrude into rivers and vast canal water distribution networks. During these dry seasons, farmers use reservoir water for cooking and drinking, and to pump reservoir water to paddies to regulate the salt water acceptable for plants. One of the key successes for protecting people's health and regulating water for plants is to monitor the salt levels in water. One of the key successes for protecting people health and regulating water for plants is to monitor the salt levels in water.

Currently, farmers use mouth to taste salty water or hand held devices to measure salt levels which are inaccurate and ineffective for large areas

This project integrates a salt water sensor system on a Hovergames drone, making the drone become an amazing effective saltwater tracking tool. The Hovergames drone and its associate app QGroundControl is able to map salinity of salt seawater in rivers and vast canal water distribution networks to help farmers to know if water is safe for cooking and drinking, and when and what amount of reservoir water to use for regulating water for plants.

Step One: Project Planning

The figure above is the sensor system for the saltwater tracker. The system includes a host unit (blue) and a sensor module (yellow). The retractable reel allows the sensor module to be dropped down into water while the drone hovering above the water surface. The sensor module is expected to be floated on the water surface, gathering water condition from salt and other water condition sensors and sending it wirelessly back to the host. The host can pass the data to the Flight Management Unit (FMU) to send to a remote control station on ground for real time data monitoring. The sensor data can be stored on the sensor unit for post-processing.

For easy and safe navigation of the drone from places to places, the sensor module needs to be pulled up in a safe position.

A camera can be used to take images/videos of water. Processing image data can provide useful additional water type and condition where the sensor module is deployed. The camera can also be used to monitor the drop-down and pull-up process of the sensor module.

Step Two: Part Sourcing

Below is the list of the main components to be used for the sensor system

1. Salinity sensor: two electrodes and associated circuit

2. Water color and transmissibility sensor: TCS34725

3. Water temperature sensor probe: 3950 NTC

4. Air quality sensor: BME688

5. Two unit of Microcontroller and Bluetooth: Xiao nRF52840

6. Rechargeable 3.7V li-po battery.

7. Printer circuit board and miscellaneous components.

8. Continuous servo motor: FS90R

9. Limited switch for detecting if the sensor module in safe retract location.

10. Reel and string

11. Cables, wires, connectors, nuts and bolts, and brackets.

Below is link to my video of the kit received from NXP

Step Three: Documenting the Build Hardware

1. Test the motor and propeller thrust

Each of motor can draw up to 21 A at 12 V when maximum throttle.

2. Assemble the drone and the first test

Later I found out that one of the motor drivers, Readytosky, was unintentionally programmed with different parameters. That caused the crash

3. Build and test the water sensor system

4. Back up slides

Step Four: Documenting the Build Software

The diagram above indicates there are four blocks of software need to be developed:

. Arduino for the Host Unit

. Arduino for the Sensor Module

. MavLink command for FMUK66-E

. Image and video capture and process

1. Arduino code for the host unit

. The host controller manage Bluetooth connection with the Sensor Module

. Receive sensor data from the Sensor Module and sent to the FMU telemetry2 via UART

The host generate RC PWM signal to control a continuous motor

. The host controller also control the drop-down and pull-up. Monitoring the limited switch to know when the Sensor Module is in safe position when pulling up. Monitoring the salinity sensor value to know when the Sensor module is in water.

/*********************************************************************
 This is an example for our nRF52 based Bluefruit LE modules

 Pick one up today in the adafruit shop!

 Adafruit invests time and resources providing this open source code,
 please support Adafruit and open-source hardware by purchasing
 products from Adafruit!

 MIT license, check LICENSE for more information
 All text above, and the splash screen below must be included in
 any redistribution
*********************************************************************/

/*
 * This sketch demonstrate the central API(). A additional bluefruit
 * that has bleuart as peripheral is required for the demo.
 * 
 * Modified by Nghia Tran
 * File name: HostMode_BT.ino
 * Date: 3/24/2023
 * Serial: NAVQ plus interface. USB-C port
 * Serial1: FMUK66-e interface. pin 6: UART-Tx, pin 7: UART-Rx
 * Serial motor: pin 10: PWM
 * Limited switch: pin 8: Digital input
 */
#include <bluefruit.h>
#include <Servo.h>


BLEClientBas  clientBas;  // battery client
BLEClientDis  clientDis;  // device information client
BLEClientUart clientUart; // bleuart client

// Servo motor 
Servo myservo;  // create servo object to control a servo
int val;        // Servo value
int continousMotorPin = 10;
int limitSwitchPin = 8;
unsigned long time_now = 0;
unsigned long time_now2 = 0;
int drop_period = 40000;
int pull_period = 30000;



// Serial1 used for FMUK66-e interface
const byte NumChars = 10;
char ReceivedChars[NumChars];
char TempChars[NumChars];  
 // variables to hold the parsed data
char MessageFromPC[NumChars] = {0};
int IntegerFromPC1 = 0;
int IntegerFromPC2 = 0;
boolean NewData = false;
int limitSwichState = 0;
char chCommandStart;
char chCommandValue;
char chCommandEnd;
int pullSpeed = 70;
int dropSpeed = 115;
int stopSpeed = 89;
int motorState = 0;



bool inWater = false;     // Sensor module in water?
char buf[20+1] = { 0 };   // buffer data sent form bluetooth sensor module

void setup()
{
  Serial.begin(115200);     //Interface to NAVQ plus
  Serial1.begin(921600);    //Interface to FMUK66-e

//  while ( !Serial ) delay(10);   // for nrf52840 with native usb

  Serial.println("Bluefruit52 Central BLEUART Example");
  Serial.println("-----------------------------------\n");
  
  // Initialize Bluefruit with maximum connections as Peripheral = 0, Central = 1
  // SRAM usage required by SoftDevice will increase dramatically with number of connections
  Bluefruit.begin(0, 1);
  
  Bluefruit.setName("Bluefruit52 Central");

  // Configure Battery client
  clientBas.begin();  

  // Configure DIS client
  clientDis.begin();

  // Init BLE Central Uart Serivce
  clientUart.begin();
  clientUart.setRxCallback(bleuart_rx_callback);

  // Increase Blink rate to different from PrPh advertising mode
  Bluefruit.setConnLedInterval(250);

  // Callbacks for Central
  Bluefruit.Central.setConnectCallback(connect_callback);
  Bluefruit.Central.setDisconnectCallback(disconnect_callback);

  /* Start Central Scanning
   * - Enable auto scan if disconnected
   * - Interval = 100 ms, window = 80 ms
   * - Don't use active scan
   * - Start(timeout) with timeout = 0 will scan forever (until connected)
   */
  Bluefruit.Scanner.setRxCallback(scan_callback);
  Bluefruit.Scanner.restartOnDisconnect(true);
  Bluefruit.Scanner.setInterval(160, 80); // in unit of 0.625 ms
  Bluefruit.Scanner.useActiveScan(false);
  Bluefruit.Scanner.start(0);             // 0 = Don't stop scanning after n seconds


  // Servo motor
 //  myservo.attach(continousMotorPin);  // attaches the servo on pin 9 to the servo object
   pinMode(limitSwitchPin, INPUT);
   myservo.write(stopSpeed); 
   myservo.detach(); 
}

/**
 * Callback invoked when scanner pick up an advertising data
 * @param report Structural advertising data
 */
void scan_callback(ble_gap_evt_adv_report_t* report)
{
  // Check if advertising contain BleUart service
  if ( Bluefruit.Scanner.checkReportForService(report, clientUart) )
  {
    Serial.print("BLE UART service detected. Connecting ... ");

    // Connect to device with bleuart service in advertising
    Bluefruit.Central.connect(report);
  }else
  {      
    // For Softdevice v6: after received a report, scanner will be paused
    // We need to call Scanner resume() to continue scanning
    Bluefruit.Scanner.resume();
  }
}

/**
 * Callback invoked when an connection is established
 * @param conn_handle
 */
void connect_callback(uint16_t conn_handle)
{
  Serial.println("Connected");

  Serial.print("Dicovering Device Information ... ");
  if ( clientDis.discover(conn_handle) )
  {
    Serial.println("Found it");
    char buffer[32+1];
    
    // read and print out Manufacturer
    memset(buffer, 0, sizeof(buffer));
    if ( clientDis.getManufacturer(buffer, sizeof(buffer)) )
    {
      Serial.print("Manufacturer: ");
      Serial.println(buffer);
    }

    // read and print out Model Number
    memset(buffer, 0, sizeof(buffer));
    if ( clientDis.getModel(buffer, sizeof(buffer)) )
    {
      Serial.print("Model: ");
      Serial.println(buffer);
    }

    Serial.println();
  }else
  {
    Serial.println("Found NONE");
  }

  Serial.print("Dicovering Battery ... ");
  if ( clientBas.discover(conn_handle) )
  {
    Serial.println("Found it");
    Serial.print("Battery level: ");
    Serial.print(clientBas.read());
    Serial.println("%");
  }else
  {
    Serial.println("Found NONE");
  }

  Serial.print("Discovering BLE Uart Service ... ");
  if ( clientUart.discover(conn_handle) )
  {
    Serial.println("Found it");

    Serial.println("Enable TXD's notify");
    clientUart.enableTXD();

    Serial.println("Ready to receive from peripheral");
  }else
  {
    Serial.println("Found NONE");
    
    // disconnect since we couldn't find bleuart service
    Bluefruit.disconnect(conn_handle);
  }  
}

/**
 * Callback invoked when a connection is dropped
 * @param conn_handle
 * @param reason is a BLE_HCI_STATUS_CODE which can be found in ble_hci.h
 */
void disconnect_callback(uint16_t conn_handle, uint8_t reason)
{
  (void) conn_handle;
  (void) reason;
  
  Serial.print("Disconnected, reason = 0x"); Serial.println(reason, HEX);
}

/**
 * Callback invoked when uart received data
 * @param uart_svc Reference object to the service where the data 
 * arrived. In this example it is clientUart
 */
void bleuart_rx_callback(BLEClientUart& uart_svc)
{

  Serial.print("[RX]: "); 
  while ( uart_svc.available() )
  {
    uart_svc.read(buf, sizeof(buf)-1);
    Serial.print(buf);
    
    if(buf[1] == ','){
      if(buf[0] == '0'){
        inWater = false;
        Serial.println("IN AIR");
      }
      else if(buf[0] == '1'){
        inWater = true;
        Serial.println("IN WATER");
      }
    }
  }
  
  Serial.println();
}


void RecvWithStartEndMarkers()
{
    static boolean RecvInProgress = false;
    static byte ndx = 0;      // index
    char StartMarker = '$';
    char EndMarker = '\n';
    char rc;          // received data

    while (Serial1.available() > 0 && NewData == false)
    {
        rc = Serial1.read();               // test for received data
        Serial.println(rc);
        if (RecvInProgress == true)
        {         // found some!!
            if (rc != EndMarker)          // <> end marker
            {
                ReceivedChars[ndx] = rc;  // 1st array position=data
                ndx++;                    // next index 
                if (ndx >= NumChars)      // if index>= number of chars
                { 
                    ndx = NumChars - 1;   // index -1
                }
            }
            else                          // end marker found
            {
                ReceivedChars[ndx] = '\0'; // terminate the string  
                RecvInProgress = false;
                ndx = 0;                  // reset index
                NewData = true;           // new data received flag
            }
        }

        else if (rc == StartMarker)       // signal start of new data
        {
        RecvInProgress = true;
        }
    }
}



void loop()
{

  while(1){

//    if(inWater==true) 
//      Serial.println("IN WATER");
//     else 
//      Serial.println("IN AIR");

    limitSwichState = digitalRead(limitSwitchPin);

    // Droping down the sensor module
    if(motorState == 1){
      
      //If sensor module touch water, stop motor
      if(inWater == true){
        myservo.detach();
      }
      else{ //keep motor run for some period
        if(millis() >= time_now + drop_period){
        time_now += drop_period;
        myservo.detach(); 
        motorState = 3;
        }
      }
    }

    // Pull up the sensor module
    else if(motorState == 2){
      //If sensor module hit the limited switch, stop motor
      if(limitSwichState == 0){
        myservo.detach();
      }
      else{ //keep motor run for some period
        if(millis() >= time_now + pull_period){
        time_now += pull_period;
        myservo.detach(); 
        motorState = 3;
        }
      }
    }
    
    // Turn off motor
    else
      myservo.detach(); 

    // Send sensor data back to FMU every 10ms
    if(millis() >= time_now2 + 10){
        time_now2 += 10;
        Serial1.println (buf); 
    }
    
    RecvWithStartEndMarkers();            // test for serial input
    if (NewData == true)                  // input received
    {
        Serial.println("******************");
        Serial.println(ReceivedChars);
        if(ReceivedChars[0] == 'd'){
          Serial.println("drop");
          motorState = 1;
          myservo.attach(continousMotorPin);
          myservo.write(dropSpeed);
          time_now = millis();
        }
        else if(ReceivedChars[0] == 'p'){
          Serial.println("pull");
          motorState = 2;
          myservo.attach(continousMotorPin);
          myservo.write(pullSpeed);
          time_now = millis();
          }

        else if(ReceivedChars[0] == 's'){
          Serial.println("stop");
          motorState = 3;
          myservo.detach();
        }
        else
          myservo.detach();

        NewData = false;                  // reset new data
    }


  }
  
  //Bluetooth process
  if ( Bluefruit.Central.connected() )
  {
    // Not discovered yet
    if ( clientUart.discovered() )
    {
      // Discovered means in working state
      // Get Serial input and send to Peripheral
      if ( Serial.available() )
      {
        delay(2); // delay a bit for all characters to arrive
        
        char str[20+1] = { 0 };
        Serial.readBytes(str, 20);
        
        clientUart.print( str );

 //       Serial.println("water_sensor_command");
      }
    }
  }

//    if (Serial1.available())
//    {
// 
//      String water_sensor_command = Serial1.readString();
//      chCommandStart = water_sensor_command[0];
//      chCommandValue = water_sensor_command[1];
//      chCommandEnd = water_sensor_command[2];
//
//      Serial.println(water_sensor_command);
//      Serial.println(chCommandStart);
//      Serial.println(chCommandValue);
//      Serial.println(chCommandEnd);
//      
//    }

  
}

2. Arduino code for the sensor module unit

. The controller generates 5 kHz square wave as excitation signal for salinity sensor. The signal is sent down to water on one electrode and received by another electrode. The controller receives analog value of the signal, processes it and provides value associate with salinity in water.

. The controller initializes and read all sensor data of water temperature sensor, color and transmissibility sensor, air quality sensor.

. Establish bluetooth connection with the host unit and transfers the sensor data

/*********************************************************************
 This is an example for our nRF52 based Bluefruit LE modules

 Pick one up today in the adafruit shop!

 Adafruit invests time and resources providing this open source code,
 please support Adafruit and open-source hardware by purchasing
 products from Adafruit!


Adafruit_nRF52_Arduino core
  Built by Khoi Hoang https://github.com/khoih-prog/nRF52_PWM
  
 MIT license, check LICENSE for more information
 All text above, and the splash screen below must be included in
 any redistribution

Modified: Nghia Tran
Date 3/27/2023
File Name: SensorModule_BT

Sensor Module includes:
1. Water Salinity sensor
2. Water Temperature sensor
3. Water Color sensor: R, G, G, and transmitvity
4. Air quality sensor: pressure, temperature, humidity, gas resistance

*********************************************************************/
#include <Wire.h>
#include "Adafruit_TCS34725.h"
#include "bme68xLibrary.h"
#include "nRF52_PWM.h"
#include <bluefruit.h>
#include <Adafruit_LittleFS.h>
#include <InternalFileSystem.h>


// BLE Service
BLEDfu  bledfu;  // OTA DFU service
BLEDis  bledis;  // device information
BLEUart bleuart; // uart over ble
BLEBas  blebas;  // battery

// Temperature Sensor
int sensorTempPin = A0;    // select the input pin for the temperature 3950 NTC
int sensorTempValue = 0;   // variable to store the value coming from the sensor

// Salt Sensor
int sensorSaltPin = A1;    // select the input pin for the salinity
int sensorSaltValue = 0;   // variable to store the value coming from the sensor
//int pulseSaltPin = 2;      // Pulse 490 Hz output

//#define _PWM_LOGLEVEL_       4
#define pulseSaltPin       2
#define USING_TIMER       false   //true
//creates pwm instance
nRF52_PWM* PWM_Instance;
float frequency = 1000.0f;
float dutyCycle = 50.0f;

int adcMidVal = 470;
int adcInAir = 470;
int adcInWater = 550;
int adcVal;
int saltSensorVal;
bool inWater = false;
unsigned long time_now = 0;





// Color sensor
Adafruit_TCS34725 tcs = Adafruit_TCS34725(TCS34725_INTEGRATIONTIME_50MS, TCS34725_GAIN_4X);
int whiteLEDPin = 6;

// Air quality sensor
#define NEW_GAS_MEAS (BME68X_GASM_VALID_MSK | BME68X_HEAT_STAB_MSK | BME68X_NEW_DATA_MSK)
#ifndef PIN_CS
#define PIN_CS SS
#endif

Bme68x bme;



void setup()
{
  Serial.begin(115200);

#if CFG_DEBUG
  // Blocking wait for connection when debug mode is enabled via IDE
  while ( !Serial ) yield();
#endif
  
  Serial.println("Bluefruit52 BLEUART Example");
  Serial.println("---------------------------\n");

  // Setup the BLE LED to be enabled on CONNECT
  // Note: This is actually the default behavior, but provided
  // here in case you want to control this LED manually via PIN 19
  Bluefruit.autoConnLed(true);

  // Config the peripheral connection with maximum bandwidth 
  // more SRAM required by SoftDevice
  // Note: All config***() function must be called before begin()
  Bluefruit.configPrphBandwidth(BANDWIDTH_MAX);

  Bluefruit.begin();
  Bluefruit.setTxPower(4);    // Check bluefruit.h for supported values
  //Bluefruit.setName(getMcuUniqueID()); // useful testing with multiple central connections
  Bluefruit.Periph.setConnectCallback(connect_callback);
  Bluefruit.Periph.setDisconnectCallback(disconnect_callback);

  // To be consistent OTA DFU should be added first if it exists
  bledfu.begin();

  // Configure and Start Device Information Service
  bledis.setManufacturer("Adafruit Industries");
  bledis.setModel("Bluefruit Feather52");
  bledis.begin();

  // Configure and Start BLE Uart Service
  bleuart.begin();

  // Start BLE Battery Service
  blebas.begin();
  blebas.write(100);

  // Set up and start advertising
  startAdv();

  Serial.println("Please use Adafruit's Bluefruit LE app to connect in UART mode");
  Serial.println("Once connected, enter character(s) that you wish to send");

  // Color sensor TCS34725
  if (tcs.begin()) {
  //Serial.println("Found sensor");
  } else {
    Serial.println("No TCS34725 found ... check your connections");
    while (1); // halt!
  }

  // White LED light source pin
  pinMode(whiteLEDPin, OUTPUT);
  digitalWrite(whiteLEDPin, HIGH);

  // Air quality sensor BME688
  SPI.begin();
  /* Initializes the sensor based on SPI library */
  bme.begin(PIN_CS, SPI);

  if(bme.checkStatus())
  {
    if (bme.checkStatus() == BME68X_ERROR)
    {
      Serial.println("Sensor error:" + bme.statusString());
      return;
    }
    else if (bme.checkStatus() == BME68X_WARNING)
    {
      Serial.println("Sensor Warning:" + bme.statusString());
    }
  }

    /* Set the default configuration for temperature, pressure and humidity */
  bme.setTPH();

  /* Heater temperature in degree Celsius */
  uint16_t tempProf[10] = { 100, 200, 320 };
  /* Heating duration in milliseconds */
  uint16_t durProf[10] = { 150, 150, 150 };

  bme.setSeqSleep(BME68X_ODR_250_MS);
  bme.setHeaterProf(tempProf, durProf, 3);
  bme.setOpMode(BME68X_SEQUENTIAL_MODE);

  Serial.println("TimeStamp(ms), Temperature(deg C), Pressure(Pa), Humidity(%), Gas resistance(ohm), Status, Gas index");


  //For salt sensor, assigns PWM frequency of 1.0 KHz and a duty cycle of 50%
  PWM_Instance = new nRF52_PWM(pulseSaltPin, frequency, dutyCycle);

}

void startAdv(void)
{
  // Advertising packet
  Bluefruit.Advertising.addFlags(BLE_GAP_ADV_FLAGS_LE_ONLY_GENERAL_DISC_MODE);
  Bluefruit.Advertising.addTxPower();

  // Include bleuart 128-bit uuid
  Bluefruit.Advertising.addService(bleuart);

  // Secondary Scan Response packet (optional)
  // Since there is no room for 'Name' in Advertising packet
  Bluefruit.ScanResponse.addName();
  
  /* Start Advertising
   * - Enable auto advertising if disconnected
   * - Interval:  fast mode = 20 ms, slow mode = 152.5 ms
   * - Timeout for fast mode is 30 seconds
   * - Start(timeout) with timeout = 0 will advertise forever (until connected)
   * 
   * For recommended advertising interval
   * https://developer.apple.com/library/content/qa/qa1931/_index.html   
   */
  Bluefruit.Advertising.restartOnDisconnect(true);
  Bluefruit.Advertising.setInterval(32, 244);    // in unit of 0.625 ms
  Bluefruit.Advertising.setFastTimeout(30);      // number of seconds in fast mode
  Bluefruit.Advertising.start(0);                // 0 = Don't stop advertising after n seconds  
}

int loop_cnt = 0;
//String strTemp, strSalt, strColorR, strColorG, strColorB, strAirTemp, strPressure, strHumid, strGasRes;
String stringAllData;
bme68xData data;
uint8_t nFieldsLeft = 0;


void loop()
{
 
  //Set 5 KHz pulse for salinity sensor
  PWM_Instance->setPWM(pulseSaltPin, 5000.0f, 50.0f);
  
  while(inWater == false){

    // read the value from the salt sensor in 1 second
    adcVal = analogRead(sensorSaltPin);
    
    if(adcVal >= adcMidVal)
      saltSensorVal = adcVal;
    else
      saltSensorVal = adcMidVal + (adcMidVal-adcVal);
 
    delay(10);

    // If 1 second passed
    if(millis() >= time_now + 1000){
      time_now += 1000;
      // print out to check
      Serial.println("In Air");
      //Serial.println(saltSensorVal);
      
      // Sensor in water
      if (saltSensorVal > adcInWater){
        inWater = true;
      }


      // read the value from the temperature sensor probe in water:
      sensorTempValue = analogRead(sensorTempPin);
      //Serial.println(sensorTempValue);
      
      // Color sensor TCS34725
      float red, green, blue;
      tcs.setInterrupt(false);  // turn on LED
      delay(60);                // takes 50ms to read
      tcs.getRGB(&red, &green, &blue);
      tcs.setInterrupt(true);  // turn off LED
      //Serial.print("R: "); Serial.print(int(red)); 
      //Serial.print("\tG: "); Serial.print(int(green)); 
      //Serial.println("\tB: "); Serial.print(int(blue));

      if (bme.fetchData())
      {
        bme.getData(data);
       // Serial.print(String(millis()) + ", ");
       // Serial.print(String(data.temperature) + ", ");
       // Serial.print(String(data.pressure) + ", ");
       // Serial.print(String(data.humidity) + ", ");
       // Serial.println(String(data.gas_resistance));
      }

      //stringAllData = "T:"+String(sensorTempValue) + ", S:"+String(sensorSaltValue)+\
      //", R:"+String(int(red))+", G:"+String(int(green))+", B:"+String(int(blue))+\
      //", A:"+String(data.temperature)+", P:"+String(data.pressure)+", H:"+String(data.humidity)+", G:"+String(data.gas_resistance);

      stringAllData = "0," + String(saltSensorVal)+ "," + String(sensorTempValue) + "," +\
      String(int(red)) + "," + String(int(green)) + "," + String(int(blue)) + "," + \
      String(data.temperature) + "," + String(data.pressure) + "," + String(data.humidity) + "," + String(data.gas_resistance);        
      
      Serial.println(stringAllData);
      int str_len = stringAllData.length() + 1;
      char charAllData[str_len];
      stringAllData.toCharArray(charAllData, str_len);
      bleuart.write( charAllData, str_len );
    }
   }
   
   while(inWater == true)
    {

    // read the value from the salt sensor in 1 second
    adcVal = analogRead(sensorSaltPin);
    
    if(adcVal >= adcMidVal)
      saltSensorVal = adcVal;
    else
      saltSensorVal = adcMidVal + (adcMidVal-adcVal);
 
    delay(10);

    // If 1 second passed
    if(millis() >= time_now + 1000){
      time_now += 1000;
      // print out to check
      Serial.println("In Water");
      //Serial.println(saltSensorVal);

      // Sensor in water
      if (saltSensorVal < adcInWater)
      inWater = false;

      // read the value from the temperature sensor probe in water:
      sensorTempValue = analogRead(sensorTempPin);
      //Serial.println(sensorTempValue);
      
      // Color sensor TCS34725
      float red, green, blue;
      tcs.setInterrupt(false);  // turn on LED
      delay(60);                // takes 50ms to read
      tcs.getRGB(&red, &green, &blue);
      tcs.setInterrupt(true);  // turn off LED
      //Serial.print("R: "); Serial.print(int(red)); 
      //Serial.print("\tG: "); Serial.print(int(green)); 
      //Serial.println("\tB: "); Serial.print(int(blue));

      if (bme.fetchData())
      {
        bme.getData(data);
        //Serial.print(String(millis()) + ", ");
        //Serial.print(String(data.temperature) + ", ");
        //Serial.print(String(data.pressure) + ", ");
        //Serial.print(String(data.humidity) + ", ");
        //Serial.println(String(data.gas_resistance));
      }

      //stringAllData = "T:"+String(sensorTempValue) + ", S:"+String(sensorSaltValue)+\
      //", R:"+String(int(red))+", G:"+String(int(green))+", B:"+String(int(blue))+\
      //", A:"+String(data.temperature)+", P:"+String(data.pressure)+", H:"+String(data.humidity)+", G:"+String(data.gas_resistance);

      stringAllData = "1," + String(saltSensorVal)+ "," + String(sensorTempValue) + "," +\
      String(int(red)) + "," + String(int(green)) + "," + String(int(blue)) + "," + \
      String(data.temperature) + "," + String(data.pressure) + "," + String(data.humidity) + "," + String(data.gas_resistance);        
      
      Serial.println(stringAllData);
      int str_len = stringAllData.length() + 1;
      char charAllData[str_len];
      stringAllData.toCharArray(charAllData, str_len);
      bleuart.write( charAllData, str_len );
    }
   }


  // Forward from BLEUART to HW Serial
  while ( bleuart.available() )
  {
    uint8_t ch;
    ch = (uint8_t) bleuart.read();
    Serial.write(ch);
  }
}

// callback invoked when central connects
void connect_callback(uint16_t conn_handle)
{
  // Get the reference to current connection
  BLEConnection* connection = Bluefruit.Connection(conn_handle);

  char central_name[32] = { 0 };
  connection->getPeerName(central_name, sizeof(central_name));

  Serial.print("Connected to ");
  Serial.println(central_name);
}

/**
 * Callback invoked when a connection is dropped
 * @param conn_handle connection where this event happens
 * @param reason is a BLE_HCI_STATUS_CODE which can be found in ble_hci.h
 */
void disconnect_callback(uint16_t conn_handle, uint8_t reason)
{
  (void) conn_handle;
  (void) reason;

  Serial.println();
  Serial.print("Disconnected, reason = 0x"); Serial.println(reason, HEX);
}

3. Code runs on FMUK66-E

This code was written and compile using MCUXpresso IDE. The compiler produce nxp_fmuk66-e_default.bin and nxp_fmuk66-e_default.px4.

I used QGroundControl to load the nxp_fmuk66-e_default.px4 to the FMU.

From the Analyze Tools, I can command the FMU using by typing command "water_sensor -[option]"

Below is the list of commands

-d, --command-drop command to drop down the water sensor-p, --command-pull command to pull up the water sensor-r, --command-read command to read water sensor data-i, --command-image command NAVQ plus to perform image processing-v, --command-image command NAVQ plus to perform image processing-s, --command-stop command stop all

The water_sensor.c code run Process_Option and execute command based on options set.

For example, to drop down the Sensor Module, the code open a serial port (telemetry2) and send command to the host unit. The host unit receives the drop-down command and enable the motor to turn the reel releasing the string.

To capture image or video, the "water_sensor -i" command can be used. The FMU will send message to the NAQ

/****************************************************************************
 *
 *   Copyright (c) 2019 PX4 Development Team. All rights reserved.
 *
 *   This work based on the original work of Cliff Brake
 *   cbrake@bec-systems.com released under the MIT license below.
 *   See https://github.com/cbrake/linux-serial-test
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *
 * 1. Redistributions of source code must retain the above copyright
 *    notice, this list of conditions and the following disclaimer.
 * 2. Redistributions in binary form must reproduce the above copyright
 *    notice, this list of conditions and the following disclaimer in
 *    the documentation and/or other materials provided with the
 *    distribution.
 * 3. Neither the name PX4 nor the names of its contributors may be
 *    used to endorse or promote products derived from this software
 *    without specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
 * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
 * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
 * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
 * COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
 * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
 * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
 * OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
 * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
 * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
 * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 * POSSIBILITY OF SUCH DAMAGE.
 *
 ****************************************************************************/

// SPDX-License-Identifier: MIT
/*
 * Permission is hereby granted, free of charge, to any person
 * obtaining a copy of this software and associated documentation files
 * (the "Software"), to deal in the Software without restriction,
 * including without limitation the rights to use, copy, modify, merge,
 * publish, distribute, sublicense, and/or sell copies of the Software,
 * and to permit persons to whom the Software is furnished to do so,
 * subject to the following conditions:

 * The above copyright notice and this permission notice shall be
 * included in all copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
 * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
 * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
 * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
 * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
 * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
 * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 * SOFTWARE.
 *
 */

/*
 * To use this test
 * 1) Build the https://github.com/cbrake/linux-serial-test on
 *    a linux box.
 *
 * 2) Add serial_test to the DUT board's default.cmake
 * 3) Connect an FTDI cable from the Linux box to the DUT's Telem port.
 *
 * 4) run ./linux-serial-test -e -b 921600 -p /dev/serial/by-id/usb-FTDI_... -c
 * on the linux box (Use control C to exit)
 *
 * 5) Run  serial_test -e -b 921600 -p /dev/ttyS2 -c on DUT
 *    Use ESC or control C to exit'
 *
 * Modified: Nghia Tran
 * Date: 3/25/2023
 * File Name: water_sensor.c
 *
 */

#if defined(__PX4_NUTTX)
#include <px4_platform_common/px4_config.h>
#include <px4_platform_common/getopt.h>
#endif

#include <stdio.h>
#include <termios.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/ioctl.h>
#include <fcntl.h>
#include <stdlib.h>
#include <strings.h>
#include <string.h>
#include <poll.h>
#include <getopt.h>
#include <time.h>
#if defined(__PX4_NUTTX)
#include <nuttx/serial/serial.h>
#else
#include <linux/serial.h>
#endif
#include <errno.h>

/*
 * glibc for MIPS has its own bits/termios.h which does not define
 * CMSPAR, so we vampirise the value from the generic bits/termios.h
 */
#ifndef CMSPAR
#define CMSPAR 010000000000
#endif

// command line args
struct cli_args_t {
	int          _baud;
	char        *_port;
	int          _divisor;
	int          _rx_dump;
	int          _rx_dump_ascii;
	int          _tx_detailed;
	int          _stats;
	int          _stop_on_error;
	int          _single_byte;
	int          _another_byte;
	int          _rts_cts;
	int          _2_stop_bit;
	int          _parity;
	int          _odd_parity;
	int          _stick_parity;
	int          _dump_err;
	int          _no_rx;
	int          _no_tx;
	int          _rx_delay;
	int          _tx_delay;
	int          _tx_bytes;
	int          _rs485_delay;
	unsigned int _tx_time;
	unsigned int _rx_time;
	int          _ascii_range;
	int			 _command_drop;		// command to drop down the water sensor
	int			 _command_pull;		// command to pull up the water sensor
	int			 _command_read;		// command to read water sensor data
	int			 _command_image;	// command NAVQ image processing
	int			 _command_video;	// command NAVQ video stream
	int			 _command_stop;		// commnad stop all
};

struct g_mod_t {

	unsigned char  _write_count_value;
	unsigned char  _read_count_value;
	int            _fd;
	unsigned char *_write_data;
	ssize_t        _write_size;

	// keep our own counts for cases where the driver stats don't work
	long long int _write_count;
	long long int _read_count;
	long long int _error_count;
};


static void display_help(void)
{
#if defined(__PX4_NUTTX)
	printf("Usage: water_sensor [OPTION]\n"
#else
	printf("Usage: linux-water-sensor [OPTION]\n"
#endif
	       "\n"
	       "  -h, --help\n"
	       "  -d, --command-drop    command to drop down the water sensor\n"
		   "  -p, --command-pull	command to pull up the water sensor\n"
		   "  -r, --command-read	command to read water sensor data\n"
		   "  -i, --command-image	command NAVQ plus to perform image processing\n"
		   "  -v, --command-image	command NAVQ plus to perform image processing\n"
		   "  -s, --command-stop	commnad stop all\n"
		   "example: water_sensor -d\n"
	       "\n"
	      );
}


static void process_options(int argc, char *argv[],  struct cli_args_t *g_cl)
{
#if defined(__PX4_NUTTX)
	int myoptind = 1;
	const char *myoptarg = NULL;
	int ch;

	while ((ch = px4_getopt(argc, argv, "hdpivs", &myoptind, &myoptarg)) != EOF) {
		switch (ch) {
#else
#define myoptarg optarg

	for (;;) {
		int option_index = 0;
		static const char *short_options = "hdpivs";
		static const struct option long_options[] = {
			{"help", no_argument, 0, 0},
			{"d", no_argument, 0, 'd'},
			{"p", no_argument, 0, 'p'},
			{"r", no_argument, 0, 'r'},
			{"i", no_argument, 0, 'i'},
			{"v", no_argument, 0, 'v'},
			{"s", no_argument, 0, 's'},
			{0, 0, 0, 0},
		};

		int c = getopt_long(argc, argv, short_options,
				    long_options, &option_index);

		if (c == EOF) {
			break;
		}

		switch (c) {
#endif

			case 0:
			case 'h':
				display_help();
				exit(0);
				break;

			case 'd':
				g_cl->_command_drop = 1;
				g_cl->_command_pull = 0;
				break;

			case 'p':
				g_cl->_command_drop = 0;
				g_cl->_command_pull = 1;
				break;

			case 'r':
				g_cl->_command_read = 1;
				break;

			case 'i':
				g_cl->_command_image = 1;
				break;

			case 'v':
				g_cl->_command_video = 1;
				break;

			case 's':
				g_cl->_command_stop = 1;
				break;


		}
	}

}


static void setup_serial_port(int baud, struct g_mod_t *g_mod, struct cli_args_t *g_cl)
{
	struct termios newtio;
	struct serial_rs485 rs485;

	if (g_mod->_fd == -1) {
//		g_mod->_fd = open(g_cl->_port, O_RDWR | O_NONBLOCK);
		g_mod->_fd = open("/dev/ttyS1", O_RDWR | O_NONBLOCK);

		if (g_mod->_fd < 0) {
			perror("Error opening serial port");
			free(g_cl->_port);
			exit(1);
		}
	}

	bzero(&newtio, sizeof(newtio)); /* clear struct for new port settings */

#if defined(__PX4_NUTTX)
	tcgetattr(g_mod->_fd, &newtio);
	cfsetspeed(&newtio, baud);
	newtio.c_cflag = CS8 | CLOCAL | CREAD;
#else
	/* man termios get more info on below settings */
	newtio.c_cflag = baud | CS8 | CLOCAL | CREAD;

#endif

	if (g_cl->_rts_cts) {
		newtio.c_cflag |= CRTSCTS;
	}

	if (g_cl->_2_stop_bit) {
		newtio.c_cflag |= CSTOPB;
	}

	if (g_cl->_parity) {
		newtio.c_cflag |= PARENB;

		if (g_cl->_odd_parity) {
			newtio.c_cflag |= PARODD;
		}

		if (g_cl->_stick_parity) {
			newtio.c_cflag |= CMSPAR;
		}
	}

	newtio.c_iflag = 0;
	newtio.c_oflag = 0;
	newtio.c_lflag = 0;

	// block for up till 128 characters
	newtio.c_cc[VMIN] = 128;

	// 0.5 seconds read timeout
	newtio.c_cc[VTIME] = 5;

	/* now clean the modem line and activate the settings for the port */
	tcflush(g_mod->_fd, TCIOFLUSH);
	tcsetattr(g_mod->_fd, TCSANOW, &newtio);

	/* enable/disable rs485 direction control */
	if (ioctl(g_mod->_fd, TIOCGRS485, (int) &rs485) < 0) {
		if (g_cl->_rs485_delay >= 0) {
			/* error could be because hardware is missing rs485 support so only print when actually trying to activate it */
			perror("Error getting RS-485 mode");
		}

	} else if (g_cl->_rs485_delay >= 0) {
		rs485.flags |= SER_RS485_ENABLED | SER_RS485_RTS_ON_SEND | SER_RS485_RTS_AFTER_SEND;
		rs485.delay_rts_after_send = g_cl->_rs485_delay;
		rs485.delay_rts_before_send = 0;

		if (ioctl(g_mod->_fd, TIOCSRS485, (int) &rs485) < 0) {
			perror("Error setting RS-485 mode");
		}

	} else {
		rs485.flags &= ~(SER_RS485_ENABLED | SER_RS485_RTS_ON_SEND | SER_RS485_RTS_AFTER_SEND);
		rs485.delay_rts_after_send = 0;
		rs485.delay_rts_before_send = 0;

		if (ioctl(g_mod->_fd, TIOCSRS485, (int) &rs485) < 0) {
			perror("Error setting RS-232 mode");
		}
	}
}


//****************************************************************
// Main program
//****************************************************************
#if defined(__PX4_NUTTX)
int water_sensor_main(int argc, char *argv[])
{
	printf("water sensor app\n");
#else
int main(int argc, char *argv[])
{
	printf("Linux serial test app\n");
#endif

char *cmd;
unsigned char rb[1024];

	struct cli_args_t g_cl = {
		._single_byte  = -1,
		._another_byte = -1,
		._rs485_delay  = -1,
		._command_drop = -1,
		._command_pull = -1,
		._command_read = -1,
		._command_image = -1,
		._command_video = -1,
		._command_stop = -1,
	};

	struct g_mod_t g_mod = {
		._fd           = -1,
	};

#if defined(__PX4_NUTTX)
	memset(&g_cl, 0, sizeof(g_cl));
	g_cl._single_byte = -1;
	g_cl._another_byte = -1;
	g_cl._rs485_delay = -1;
	g_cl._command_drop = -1;
	g_cl._command_pull = -1;
	g_cl._command_read = -1;
	g_cl._command_image = -1;
	g_cl._command_video = -1;
	g_cl._command_stop = -1;
	memset(&g_mod, 0, sizeof(g_mod));
	g_mod._fd = -1;
#endif

	process_options(argc, argv, &g_cl);

	if(g_cl._command_drop == 1 && g_cl._command_pull == 0)
		cmd = "$d\n";
	else if(g_cl._command_drop == 0 && g_cl._command_pull == 1)
		cmd = "$p\n";
	else if(g_cl._command_image == 1)
		cmd = "$i\n";
	else if(g_cl._command_video == 1)
		cmd = "$v\n";
	else if(g_cl._command_stop == 1)
		cmd = "$s\n";
	else
		cmd = "$h\n";

	// Set up serial port
	int baud = B921600;
	setup_serial_port(baud, &g_mod, &g_cl);

	// Write to serial port, sending command to sensor module and NAVQ+
	write(g_mod._fd, cmd, strlen(cmd));

	if(g_cl._command_read == 1){

		// Read sensor data from serial port
		int count = read(g_mod._fd, &rb, sizeof(rb));

		// Send sensor data to terminal
		int i;
		for (i = 0; i < count; i++) {
			printf("%c", rb[i]);
		}
	}
	return 0;
}

4. Build and code for Companion computer NAVQ - Plus

Objectives: to analyze and determine the water content for salinity, clarity, quality… in the canals before water can be pumped into rice paddy fields, using the drone or drones as remote device to take water samples from many locations then analyze these water samples for further processing along with environmental data such as ambient air quality, air temperature, humidity, water sample time stamp, water level relative to the canal banks and so on.

Software Image downloaded and built:

1. Images:

Here are links to images that can be downloaded and used on the NavQPlus.

https://github.com/rudislabs/navqplus-create3-images/releases/

for Image: ”navqplus-image-22.04-230127.wic.bz2” then unzip to

“navqplus-image-22.04-230127.wic” file

Utilize software: ”Win32Diskimager.exe” to write “navqplus-image-22.04-230127.wic” file to micro SD card.

Refer to “https://nxp.gitbook.io/8mpnavq/navqplus-user-guide/quickstart/flashing-with-new-firmware”

2. Plug micro-SD card above to the NAVQ+ board and energize the power On.

3. Plug in provided USB Serial/UART console cable and boot initial steps can be seen

4. Configure the USB serial/UART thru putty.exe (Windows) as:

a. Baud rate: 115200

b. COM port is provided by device manager (Win)

c. Flow control: = default XON/OFF to None

d. Refer to “https://nxp.gitbook.io/8mpnavq/navqplus-user-guide/quickstart/serial-console”

5. To login:

a. Default username: user

b. Default password: user

6. Please change this user’s password, type passwd, recommended

7. Type “echo -e "d\n2\nn\np\n2\n196608\n\n\nw" | sudo fdisk /dev/mmcblk1 && sudo resize2fs /dev/mmcblk1p2” to expand the entire micro SD card, since the image file is typical smaller than your actual card. For my case I used 128GB card for the image above therefore, this command will allow me to use 128gb storage, theoretically.

a. Then reboot,

b. Re login and type df-h to see entire micro-SD card storage

8. Set your time zone, for instant to run: “timedatectl set-timezone America/Los_Angeles” for someone like us in California. Please use correct time zone for your area

9. Install other software to run commands:

a. sudo apt update

b. cd

c. Refer to “https://nxp.gitbook.io/8mpnavq/dev-guide/software”

d. mkdir ROS2

e. cd ROS2

f. wget https://raw.githubusercontent.com/rudislabs/nxp_install/main/install.sh

g. chmod +x install.sh

h. ./install.sh

i. Setting up the MicroDDS Agent/MicroROS:

i. git clone https://github.com/eProsima/Micro-XRCE-DDS-Agent

ii. cd Micro-XRCE-DDS-Agent

iii. mkdir build && cd build

iv. cmake ..

v. make

vi. sudo make install

j. AI/ML: “https://nxp.gitbook.io/8mpnavq/dev-guide/software/ai”

i. USE_GPU_INFERENCE=0 ./label_image --external_delegate_path=/lib/libvx_delegate.so -m mobilenet_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt

k. OpenCV: “https://nxp.gitbook.io/8mpnavq/dev-guide/software/opencv”

i. Install if not existed: “sudo apt install python3-opencv”

ii. To access the Google Coral Camera(s) on NavQ+ in OpenCV, you may use the following VideoCapture instantiation:

iii. “cap = cv2.VideoCapture('v4l2src device=/dev/video3 ! video/x-raw,framerate=30/1,width=640,height=480 ! appsink', cv2.CAP_GSTREAMER)”

l. GStreammer: https://nxp.gitbook.io/8mpnavq/dev-guide/software/gstreamer

i. Take an Image: “gst-launch-1.0 -v v4l2src num-buffers=1 device=/dev/video3 ! jpegenc ! filesink location=capture.jpeg”

*) Image named “capture.jpeg”

ii. Record a video: “gst-launch-1.0 v4l2src device=/dev/video3 ! imxvideoconvert_g2d ! "video/x-raw, height=640, width=480, framerate=30/1" ! vpuenc_h264 ! avimux ! filesink location='/home/user/video.avi'”

**) capture video “video.avi”

m. Flashing the eMMC is optional, to increase the performance of the system, However, we stopped at this to use micro-SD card as main storage throughout this process.

Additional hardware setup: due to no physical ethernet of the NAVQ+, we installed USB C hub to 4x USB type A. Also, plug in USB type A dongle ethernet to allow LAN access within this network via computer laptop to ease the setup steps above. Typically, use PUTTY to SSH thru IP of the board,

On board image Analysis:

With on board of this NAVQ+ board, images were captured by on board camera and analyzed using Python codes:

For visual viewing images:

import cv2
 import numpy as np
 
 
img = cv2.imread('water2.png')
 while True:
    cv2.imshow('Water',img)
 if cv2.waitKey(1) & 0xFF == 27:
 break
 
 cv2.destroyAllWindows()

This visualization of Figure 1 image is shown above and it was determined that color of water is brown along with vegetation on the edge shore lines. This information of the above was transmitted back for further processing.

import cv2
 import numpy as np
 
 
img = cv2.imread('watercolor1.jpg')
img.shape
#output (79, 156,3)
#convert to RED GREEN BLUE
Cv2.cvtColor(img,cv2.COLOR_BGR2RGB)
while True:
    cv2.imshow('Water',img)
 if cv2.waitKey(1) & 0xFF == 27:
 break
 
 cv2.destroyAllWindows()

This is a python code running NAVQ plus. It receives message sent from Host Unit Controller. The Host unit can command the NAVQ plus to capture image or video during on the flight for post processing.

# This is a sample Python script.

# Press ⌃R to execute it or replace it with your code.
# Press Double ⇧ to search everywhere for classes, files, tool windows, actions, and settings.
from cv2 import *
import time
import numpy as np
import subprocess
from serial import *
#from itertools import izip
#import errno
#import h5py
import os




def captureImage():
    cam_port = 3
  cam = VideoCapture(cam_port)
  if not (cam.isOpened()):
  print("Could not open video device")
  # To set the resolution
    #cam.set(CAP_PROP_FRAME_WIDTH, 640)
    #cam.set(CAP_PROP_FRAME_HEIGHT, 480)

    # reading the input using the camera
  result, image = cam.read()

  # If image will detected without any error,
    # show result
  if result:
 # showing result, it take frame name and image
        # output
        #imshow("GeeksForGeeks", image)

  timestr = time.strftime("%Y%m%d-%H%M%S")

        filename = 'image' + timestr + '.png'
  # saving image in local storage
  imwrite(filename, image)

  # If keyboard interrupt occurs, destroy image
        # window
        #waitKey(0)
        #destroyWindow("GeeksForGeeks")

    # If captured image is corrupted, moving to else part


def video_capture():
  # cap = VideoCapture(3)
  cam_port = 3
  cam = VideoCapture(cam_port)
  if not (cam.isOpened()):
  print("Could not open video device")

    start_time = time.time()

  # Define the codec and create VideoWriter object
  fourcc = VideoWriter_fourcc(*'XVID')
    timestr = time.strftime("%Y%m%d-%H%M%S")
    filename = 'video' + timestr + '.avi'
  out_video = VideoWriter.open(filename, fourcc, 20.0, (640, 480))

  # loop runs if capturing has been initialized.
  flag = True
    while (flag):
  # reads frames from a camera
        # ret checks return at each frame
  ret, frame = cam.read()

  # Converts to HSV color space, OCV reads colors as BGR
        # frame is converted to hsv
  hsv = cvtColor(frame, COLOR_BGR2HSV)

  # output the frame
  out_video.write(hsv)
        stop_time = time.time()
  if (stop_time >= start_time + 10):
  print (start_time, stop_time)
            flag = False

# Close the window / Release webcam
  cam.release()
# After we release our webcam, we also release the output
  out_video.release()


def command_capture():
  #subprocess.run(['ls', '-l'], capture_output=True, text=True).stdout
    #subprocess.run(['sudo', 'gst-launch-1.0','v4l2src', 'device=/dev/video3 ! imxvideoconvert_g2d !', ["video/x-raw, height=640, width=480, framerate=30/1"], '! vpuenc_h264 ! avimux ! filesink','location =', './video.avi'], capture_output=True, text=True).stdout
    #subprocess.run(['gst-launch-1.0', 'v4l2src device=/dev/video3 ! imxvideoconvert_g2d ! "video/x-raw, height=640, width=480, framerate=30/1" ! vpuenc_h264 ! avimux ! filesink', 'location="/home/user/video.avi"'])
  subprocess.call("./capVideo.sh",shell=True)


ser = Serial(
  port='/dev/ttymxc2',
  baudrate=115200
)

ser.isOpen()
#print ('Enter your commands below.\r\nInsert "exit" to leave the application.')

input = 1
ch1=ch2=ch3=""
flag1 = True
while flag1:
  if (ser.read(1).decode('ASCII') == "$"):
        ch2 = ser.read(1).decode('ASCII')
  if(ch2 =="i"):
            ch3 = ser.read(1).decode('ASCII')
  if (ch3 == '\n'):
  print("image\n")
                captureImage()
  elif (ch2=="h"):
            ch3 = ser.read(1).decode('ASCII')
  if(ch3=="\n"):
  print("video\n")
  #video_capture()
  command_capture()
  else:
  print('do nothing')

        flag1 = False
#    if out != '':
#        print(">>" + out)

ser.close()

Note of Building the FMUK66-E

Instruction 1:

https://docs.px4.io/main/en/dev_setup/building_px4.html

Build PX4 - Autopilot
>git clone https://github.com/PX4/PX4-Autopilot.git --recursive
>bash ./PX4-Autopilot/Tools/setup/ubuntu.sh

Build PX4 firmware
>cd PX4-Autopilot
>make px4_sitl jmavsim (Build simulation FMU JMAVSIM)
>make nxp_fmuk66-e_default (Build firmware for FMUK66-E board)

Run Simulation
. Run QGroundControl
. Run JMAVLINK by typing
>cd ~/src/px4-firmware && make px4_sitl jmavsim (simulation FMU JMAVSIM)

Instruction 2:
https://nxp.gitbook.io/hovergames/developerguide/tools/toolchain-installation

Build PX4 - Firmware
> mkdir -p ~/src
>cd ~/src && git clone --recursive https://github.com/PX4/Firmware.git px4-firmware
>cd ~/src/px4-firmware && bash ./Tools/setup/ubuntu.sh
>cd ~/src/px4-firmware && make nxp_fmuk66-v3_default (previous version of FMU)
>cd ~/src/px4-firmware && make nxp_fmuk66-e_default (current version of FMU)
>cd ~/src/px4-firmware && make px4_sitl jmavsim (simulation FMU JMAVSIM)

The JMAVSIM simulation was very helpful. I could practice creating a mission in following steps:
. Take off, add way points, and return

Step Five: The Final Test - Successful

We were almost ready to perform a real test using the test-and-running sensor module, but...

And unfortunately - Crash and Burn

This drone project is totally new to Hoa Phan and I. We started it with almost zero knowledge about a flying drone. We went through up until now by late nights and weekend independent and team work. Thank you Hoa.

My other friend, Thien Phung, and relatives supported us with food and encouragement. Thank you all.

We are not sure that we can make the Hovergames drone revived but for sure we became drone lovers now. We probably will procure another Hovergames drone for each of us. My brother wishes he could buy one to support his work for large area surveillance. My son wants to see more drone flight tests in wild areas.