The deep neural network is trained on the fused textual and visual features to classify the memes as either hateful or non-hateful. Fine-tuning techniques are applied to further enhance the model's performance by adapting the pre-trained models to better capture the nuances of hate speech in memes. Once the model achieves satisfactory performance, it can be deployed as an application or integrated into social media platforms to automatically detect and moderate hateful memes. Regular updates to the training dataset, model retraining, and fine-tuning are conducted to ensure the system remains effective in identifying evolving forms of hate speech. Ethical considerations are taken into account throughout the project to ensure fairness, user privacy, and prevention of misuse. The ultimate goal of this system is to contribute to a safer online environment by accurately classifying and mitigating the spread of hateful memes.
Dataset used - https://hatefulmemeschallenge.com/#download
TSNE Plot for Unimodal Task of Image Classification -
TSNE PLOT for Unimodal Task of Text Classification -
Multimodal Architecture
TSNE Plot for MUltimodal Task of Text Classification-
