MemeCraft

A brief description of what the project does.

Installation

First, install the dependencies:

• git+https://github.com/huggingface/transformers
• tokenizers==0.13.3
• ipywidgets
• openai
• accelerate==0.20.3
• torch==2.0.1
• nltk
• protobuf==3.20.*
• langdetect
• requests urllib
• gensim==3.8.0
• scikit-learn==0.22
• spherecluster
• numpy==1.23.5

Features

• The dataset folder contains memes generated by MemeCraft and baseline models.
• The script folder houses Python code for extracting text descriptions, generating meme text and detect hateful memes.

Usage

1. [vlm_text_description_generation.py] - Extracts text descriptions.
2. [llm_meme_text_generation.py] and [vlm_meme_text_generation.py] - Generate contextual meme text using LLM or VLM. [prompt_demonstration.py] - Prompt demonstration examples.
3. [text_overlay.py] - Overlays text onto meme templates.
4. [hateful_memes_detection.py] - Identifes and excludes hateful memes.

Citation

@article{hanw2024memecraft, author = {Han wang, Roy Ka-Wei Lee}, title = {MemeCraft: Contextual and Stance-Driven Multimodal Meme Generation}, year = {2024} }

@misc{singh2020mmf, author = {Singh, Amanpreet and Goswami, Vedanuj and Natarajan, Vivek and Jiang, Yu and Chen, Xinlei and Shah, Meet and Rohrbach, Marcus and Batra, Dhruv and Parikh, Devi}, title = {MMF: A multimodal framework for vision and language research}, howpublished = {\url{https://github.com/facebookresearch/mmf}}, year = {2020} }

Contact Information

For questions or feedback, email [[email protected]].