Dexter: The All-In-One Smart Home Bot

Dexter: The Ultimate Home Assistant Robot

Introduction

Meet Dexter, the ultimate home assistant robot designed to change how we interact with our homes. Dexter is not just a cleaning robot—it's a smart and capable helper that can perform a variety of household tasks. With its advanced features, Dexter can navigate your home, pick up and place objects, and even recognise faces and gestures.

The Inspiration Behind Dexter

The idea for Dexter came from our desire to create a home automation device that could do much more than just clean. While robotic vacuum cleaners like Roomba are great for cleaning, we wanted something that could handle a wide range of household tasks, making our lives easier and more efficient. That's why we decided to create Dexter—a versatile and intelligent home assistant.

What Dexter Can Do

Dexter is a three-wheeled robot equipped with a plethora of features designed to assist with various household tasks. Here are some of the key capabilities of Dexter:

1. Advanced Navigation with SLAM and LIDAR: Dexter uses Simultaneous Localization and Mapping (SLAM) technology in conjunction with a LIDAR system to navigate your home with precision. This allows Dexter to map out your living space, avoid obstacles, and move efficiently from room to room.

2. Robotic Arm for Versatile Tasks: Unlike standard cleaning robots, Dexter features a robotic arm that can pick up, place, and manipulate objects. Whether it's picking up toys from the floor, placing dishes in the sink, or handling other household chores, Dexter’s arm is designed for versatility and precision.

3. Smart Vision and Object Detection: Equipped with an advanced camera system, Dexter streams live video feed to your laptop. This system uses computer vision to detect faces, recognize gestures, and identify household objects. Dexter can respond to your gestures, recognize family members, and even identify when your pet needs attention.

4. Pet Detection and Interaction: Dexter isn’t just a helper for humans. With pet detection capabilities, Dexter can identify and interact with your pets, ensuring they are safe and even playing with them when needed.

5. Gesture Recognition: Using advanced gesture recognition, Dexter can understand and respond to your hand signals. You can command Dexter to perform tasks or navigate to specific locations in your home without needing to use voice commands or a remote control.

6. Comprehensive Home Integration: Dexter integrates seamlessly with other smart home devices. It can communicate with your home security system, smart thermostats, lighting systems, and more to create a cohesive and intelligent home environment.

How Dexter Work:

1. Navigation and Mapping:

Dexter employs ROS 2 Humble for advanced navigation and mapping capabilities, making it an autonomous robot. The LiDAR system, integrated using ROS 2 nodes, continuously scans the environment to generate real-time occupancy grids and 3D point clouds. Utilizing Simultaneous Localization and Mapping (SLAM) algorithms, such as Gmapping or Cartographer, Dexter constructs detailed maps of its surroundings. These maps are visualized in RViz for real-time monitoring and debugging. Precise localization is achieved through sensor fusion techniques, combining LiDAR data with wheel odometry and IMU inputs. ROS 2’s Nav2 stack facilitates path planning, dynamic obstacle avoidance, and route optimization, ensuring Dexter navigates efficiently and safely within the home environment, completely autonomously.

2. Robotic Arm Control:

Dexter’s robotic arm features multiple degrees of freedom, driven by high-torque servo motors and controlled via ROS 2 action servers. The MoveIt! framework is employed for motion planning and control, allowing for intricate manipulation tasks. Dexter’s arm can execute precise pick-and-place operations, leveraging inverse kinematics (IK) solvers to compute optimal joint trajectories. The arm’s end-effector, equipped with a gripper or specialized tool, can securely grasp and manipulate a variety of household objects. ROS 2’s real-time communication ensures synchronized and smooth arm movements, while sensor feedback from encoders and force sensors enhances dexterity and precision.

3. Computer Vision and Machine Learning:

Dexter’s camera system streams live video at 640x480 resolution, interfaced through ROS 2 nodes using image_transport for efficient data handling. The camera feed is processed on a connected laptop using OpenCV and TensorFlow, integrated with ROS 2 for seamless data flow. Computer vision algorithms detect faces, recognize gestures, and identify objects using pre-trained convolutional neural networks (CNNs). The image frames are published to ROS 2 topics, where subscriber nodes perform further analysis and trigger appropriate responses. Machine learning models are continually refined through ROS 2’s lifecycle management, enabling Dexter to adapt and improve its perception capabilities over time.

4. Pet and Gesture Recognition:

Dexter employs advanced image recognition techniques to distinguish pets from humans, utilizing YOLO (You Only Look Onc for real-time object detection. Gesture recognition is facilitated through the integration of Mediapipe, which tracks hand landmarks and interprets gestures. ROS 2 nodes handle the processing pipeline, publishing recognized gestures to relevant topics. This enables Dexter to respond to hand signals and other non-verbal commands, enhancing user interaction. The system is designed to be robust, capable of operating under varying lighting conditions and dynamic environments.

5. User Interface and Control:

Users interact with Dexter through a user-friendly interface implemented in rqt, providing real-time control and monitoring capabilities. The interface displays the live video feed from Dexter’s camera, alongside visualizations from RViz, showing the current map and Dexter’s position within it. Users can issue commands, assign tasks, and monitor Dexter’s performance through this interface. Teleoperation is facilitated via ROS 2’s teleop_twist_keyboard or joystick interfaces, allowing manual control when needed. Task automation and scheduling are managed through ROS 2’s action servers and clients, providing a seamless and intuitive user experience.

Implementation Details:

ROS 2 Humble: Dexter’s core functionality is built on ROS 2 Humble, leveraging its robust communication framework and real-time capabilities.

Publisher-Subscriber Model: Sensor data (LiDAR, camera, IMU) and control commands are managed through ROS 2’s publisher-subscriber model, ensuring efficient data exchange and system modularity.

Gazebo Simulation: Dexter’s design and functionality are validated in the Gazebo simulation environment, enabling comprehensive testing and refinement before deployment.

RViz Visualization: Real-time visualization of sensor data, maps, and Dexter’s state is provided through RViz, facilitating debugging and system monitoring.

rqt Imaging: The rqt_image_view plugin is used for displaying the live camera feed, enhancing the development and testing of computer vision algorithms.

Why We Decided to Create Dexter

Our motivation for creating Dexter stemmed from a desire to push the boundaries of what home automation can achieve. We envisioned a future where robots are not just tools but companions that enhance our daily lives. By integrating advanced navigation, robotic manipulation, and intelligent interaction capabilities, we aimed to create a robot that is truly helpful in a wide range of scenarios. Dexter is designed to be a reliable and indispensable part of your household.

Conclusion

Dexter is more than just a robot; it's could be our future. By combining the latest in robotics, computer vision, and smart home integration, Dexter can not only make daily tasks easier but also redefine the role of robots in our lives.

Folder Link with all the Pictures and Videos of Dexter

https://drive.google.com/drive/folders/1dOayg0DQPPq8AvxcvO0d2vquiOrU_lbC?usp=sharing

https://drive.google.com/file/d/1TASs6VSFgeM5Da4VK1ouYMG9RqhG8_M-/view?usp=sharing