Visual Question Answering (VQA)- MultiModular Architecture

Featurization of image and question, Feature fusion, Answer generation for Multimodal System

Multilingual models for seven languages, visit BhavikArdeshna Hugging Face

🤗 Models for Experimentation:

Hugging Face 🤗

Text Transformers (for encoding questions):
- BERT (Bidirectional Encoder Representations from Transformers)
- RoBERTa (Robustly Optimized BERT Pretraining Approach)
Image Transformers (for encoding images):
- ViT (Vision Transformer)
- BEiT (Bidirectional Encoder representation from Image Transformers)

Text Transformer	Image Transformer	Wu & Palmer Score	Accuracy	F1
BERT	ViT	0.263	0.23	0.010
BERT	BEiT	0.283	0.214	0.032
RoBERTa	ViT	0.273	0.239	0.019
RoBERTa	BEiT	0.299	0.259	0.03

My Research Works

We have investigated the efficacy of cascading adapters with transformer models to leverage high-resource language to improve the performance of low-resource languages on the question answering task. We trained four variants of adapter combinations for - Hindi, Arabic, German, Spanish, English, Vietnamese, and Simplified Chinese languages. We demonstrated that by using the transformer model with the multi-task adapters, the performance can be improved for the downstream task. Our results and analysis provide new insights into the generalization abilities of multilingual models for cross-lingual transfer on question answering tasks.
[1] Hariom A. Pandya, Bhavik Ardeshna, Dr. Brijesh S. Bhatt Cascading Adaptors to Leverage English Data to Improve Performance ofQuestion Answering for Low-Resource Languages
```
@misc{pandya2021cascading,
      title={Cascading Adaptors to Leverage English Data to Improve Performance of Question Answering for Low-Resource Languages}, 
      author={Hariom A. Pandya and Bhavik Ardeshna and Dr. Brijesh S. Bhatt},
      year={2021},
      eprint={2112.09866},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```
Gujarati QA Benchmark
Under Review