2019-2020 Robot Capstone Project
Member : 유호연, 김민우, 배종학, 이현우, 최수진, 황지원
Directed by Prof. 한재권
BADA is a Social Robot made for Hearing-impaired people. The word 'BADA' is derived from "Buddy for Auditory Detection Assistance" and South Korean first assistance dog's name.
This project is inspired by hearing dog which can assists hearing-impaired people by acting from the situation of things and modeled on their behaviors. If BADA detects dangerous sounds such as Fire alarm, this robot firstly send sound information to BADA Web page and KAKAO Talk message for people in thier home and displays pictogram of the sounds source and find person who live in. However, to prepare in situations that people cannot interact with those interfaces, BADA will bump into him/her to delivery sound information.
BADA can hear 6 sounds source that is selected on the basis of Korean hearing-dog education guide. these are Fire Alarm, Door Knocking, Water Boiling, Water, Baby crying and Door Bell. By utilizing YAMNet which is a machine learning algorithm of sound recognition, this detection system can be constructed.
RTAB Map, one of the state of the arts 3D SLAM package, helps BADA roam around house.
Raspberry pi 3B+, NUC, ReSpeaker Microphone Array, Coral USB accelerator, Intel Realsense D435, T265, Adafruit Dot LED Matrix, Arduino mega
wget https://github.com/joan2937/pigpio/archive/master.zip
unzip master.zip
cd pigpio-master
make
sudo make install
Odometry Information of Bada. Pose and velocity are the Position and Velocity of Bada in Global reference frame.
The linear and angular velocity of Bada in Bada's Local reference fream. Twists.linear.x is the forward velocity [m/s], Twists.angular.z is the angular velocity [rad/s].
Detected sound signal. type: std_msgs/String
Realtime sound detection informations. JSON array contains detected sound data with [name, probability] format.
type: std_msgs/String
format: {'data': <JSON array>}
var detectionData = JSON.parse(msg.data);- refactoring
- Architecture Design
- Coding
- Debug
- Encoder value setting
- Publish Twist message
- PID Control
- Code Refactoring
- Dead Reckoning Information (Odometry / Pose)
- Code Refactoring
- TF message Generation
- Trajectory
- Navigation
- TF Change
- Making roslaunch
- Measure the direction of sound using TDoA Sensor
- Merge TDoA sensor and mobile robot.
- Create movement toward where sound is ocurred.
- YamNet
- Make Yamnet as ROS Program.
- Pointcloud using RealSense T265 & D435
- Making SLAM Information
- GMAPPing
- Navigation
- Interaction Design
- Connect WebAPP with ROS
- Visualizing SLAM Information
- What if the person sleeps.
- What if the person is far from their phone.
- How to transfer information about danger situation.