This project is inspired by people that are ashamed by their mental health issue and fail to get the help they need. And with today's technology where human facial features and expressions can be extracted, detected, and recognized through computer algorithms, examining a person's face can be one method of effectively discerning their genuine mood and reactions, therefore help the counselor, doctor, or teacher better assess someone's mental states using their facial expressions.
This website assists all kinds of professionals at works in the public health sector that deals with people with mental health issues like doctors, teachers, and counselors. It detects emotional expression from the person and provides information about the person like whether if the person is under stress. This website accepts videos and breaks it down by frames. It then uses google cloud vision API to detect facial expressions of the person and generates a large data set representing a person's mood over time. The data set is then analyzed and visualized in a graph to shows the person's mental states and changes in mood over time.
We first make a website using JavaScript that accepts videos in mp4 and breaks it down by frames, forming an array of images using open cv. The images are then analyzed through the Vision API from Google Cloud Platform that detects facial expressions of the person and generates a 2D list representing a person's different emotion in scale. Finally, we visualize the data using matplotlib and numpy which is push back to the website.
Using Google Cloud Platform, Python, Vision API, HTML, CSS, JavaScript, JQuery and OpenCV.
Getting a snapshot of different scenes in a video was a difficult part. Everyone in the team was new to the Google Cloud services and the platform.
We were able to implement image analysis with Google Vision API - a world-leading technology - and utilized a wide spectrum of developing tools.
Building with a team, and tackling social issues that will affects many people in the society.
We would integrate real-time analysis of a person's emotion without having to upload files.
The API used to record videos requires a secure origin: HTTPS or localhost. We used localhost. You can run python3 -m http.server to use get a local host up and running. It would run on localhost:8000 by default. Check here for more info.