Face Mask D&R

Face mask Detection & Actors Recognition

Index

1. What is FMD&R?
2. System Dependencies
3. Quick Start
4. How it Works
5. Evaluation and Results
6. Conclusions and future develop
7. Credits

What is fmdr

FMD&R is an application developed for an "Agent Systems" exam at University Aldo Moro of Bari Italy.
its goal is to detect face mask on images of faces and eventually recognize:
'Andrew Garfield', 'Angelina Jolie', 'Anthony Hopkins', 'Ben Affleck', 'Beyonce Knowles' with or without face mask protections

Extra Functionalities

• Google Scraper:
  Google scraper is a script that uses a special library to be able to download google images related to certain keywors. This library proved to be useful for obtaining further images of the reference actors for the purpose of creating a test dataset containing images different from those used in the training stage of the classifiers
• Wear Mask:
  wear mask is a script useful for creating a dataset containing images of faces to which a facial mask has been affixed. This script proved to be very useful in order to train the classifier to recognize faces with the mask. it is based on the use of Dlib to be able to identify the facial points useful for identifying the coordinates where the mask should have been (nose, mouth and chin)

• Requirements and Dependencies

  Python 3 or higher (tested on 3.7)
  opencv-python~=4.2.0.34
  dlib~=19.19.0
  numpy~=1.18.2
  Pillow~=7.1.1
  Keras~=2.3.1
  utils~=1.0.1
  matplotlib~=3.2.1
  seaborn~=0.10.1
  scikit-learn~=0.22.2.post1
  mtcnn~=0.1.0
  face_recognition
  keras-vggface
  

• Quick Start

  1 - Clone this repository
  2 - Install all dependencies with "pip3 install -r requirements.txt"
  3 - Download SysAgDatasets and SysAgModels from : https://drive.google.com/drive/folders/15BUd3s0lUbZjG_Ck8vrawa5sx94pSjAW?usp=sharing
  4 - Execute "main.py"
  5 - Enjoy with Face Mask D&R
  

• How it works

  Project Presentation inside "/doc/Presentazione FMD&R.pptx"
  

  ---

  1) Mask Detection
  

  This project trains a CNN to detect the presence of a face mask in a face image. CNN is trained on a dataset containing faces with and without a mask. The script is able to start via command line on images, videos and webcams
  
  

  

  ---

  2) Face Recognition
  

  This project based on the pre-trained classifier VGG16 is able to fine-tune the images of the dataset containing only the faces of the actors, and make a prediction on which of the 5 actors it can be.
  
  

  ---

  3) Masked Face Recognition
  

  This project based on the pre-trained classifier VGG16 is able to fine-tune the images of the dataset containing only the eye line returned by MaskCropper.py. He is therefore able to recognize the identity of the image provided between one of the 5 masked actors used.

  Andrew Garfield
  
  
  

  Angelina Jolie
  
  
  

  ---

  4) Mask Cropper
  

  this script processes the images returned by WeraMask.py, subtracts the color of the face mask and identifies the cut point that separates the image into two sections: eye line section (useful for recognizing the subject) and the "artificial" mask section
  
  Cropped EyeLine for Masked actors recognition
  
  

  Face Mask image for color range
  
  

• Evaluation and Results

  VGG16 results on Actor clean faces Recognition

  
  

  
  

  VGG16 results on Actor masked faces (EyeLinesCrop) Recognition

  
  

  
  

• Conclusions and future develop

FMD&R is able to recognize one of the five actors in the dataset starting from an image of a face without or equipped with a face mask using only the eye area.
A possible future development could involve the use of more accurate techniques for the recognition of the exposed face area such as the distance between the eyes and the nose etc.
The MaskCropper script is based on the blue color range (typical of the blue_masck.png used) which makes the crop on images with masks of different color ineffective; to solve this problem, it would be possible to identify the area of the image with greater concentration of color to more precisely identify the "cut point" and obtain the eye area of the face.

• Credits

  Developed and Designed by:

  Chrism1c, mpia3 and mircocrit