Webpage_Saliency_Prediction

In this project, we have done a focused study on the saliency analysis for webpages using FiWi dataset and propose a Fully Convolutional Network (FCN) based architecture to predict the saliency of webpages, following this implementation. Given an input Webpage image, the proposed model is capable of generating its saliency map.

P.S. This whole project has been performed and tested on Google Colab. Colab Access

Guidelines to use this Repository

DIRECTORY ARCHITECTURE:

Can be seen here

A. REQUIREMENTS:

To avoid version conflicts,

install tensorflow version 1.15.0 from Tensorflow

install keras version 2.2.5 from keras

B. USING PRE-TRAINED MODEL FOR WEBPAGE SALIENCY ANALYSIS:

1. Download this repository
2. Download the Pre-trained model for FCN-16s from here
3. Provide correct path for the downloaded model. Run Model_Performance_on_Test_Images.ipynb
4. For computing AUC, CC and NSS scores, codes are provided in Prediction_and_Analysis folder. From Evaluation Metric Scripts run the auc_calculation.py for computing AUC score, and cross_correlation_and_nss_calculation.py for calculating CC/NSS score.

C. FOR TRAINING MODEL WITH YOUR OWN DATA:

1. Download GDI dataset from here
2. Download FiWi dataset from here
3. From Model Training Scripts folder, start training GDI model at first by running "GDI_FCN_TRAIN.ipynb" with GDI dataset. Alternatively this pre-trained model can be downloaded from here
4. Run heatmapGeneration.py from Initial_Data_Preprocessing folder for generating Heatmaps from FiWi eye fixation data, in a sparate folder "finalHeatMap"
5. Run "TestImageGenerator.ipynb" to divide the data set into train and test by randomly separating 8 images from stilumi and generated heatMap. Move thiese folders to the Evaluation Data directory.
6. After training, start training FiWi data by running "website_saliency_prediction_final.py", from the same folder. P.S. Please check the proper paths of the FiWi dataset stimuli,generated Heatmaps and pretrained GDI model to avoid path errors.
7. Save the model (change the path for saving model) and use the saved model by followning section B.

D. MACHINE LEARNING ALGORITHMS FOR WEBSITE SALIENCY PREDICTION:

Run the scripts in folowing order to get the saliency predictions using random forest and SVM.

1. Run "TestImageGenerator.ipynb" to divide the data set into train and test by randomly separating 8 images from stilumi and generated heatMap. Create two folders for the same. These 8 images can be used for predicting model's performance for unknown data.
2. Run the saliency.py script for the training data set. This will create folders for intensity, channel orientation and subchannel conspicuity maps. This script will also give saliency predictions as per itti et al. in a seperate folder.
3. Run the datafilegeneration.py to generate a data file in csv format. This file carries the information of various conspicuity maps at each pixel of images in the training set.
4. Run the datasegregation.py file to generate one more csv file which is used by random forest regressor and SVM for generating saliency maps.
5. Run saliency.py and datafilegeneration.py for the test data.
6. Finally, run the svm.py and randomforest.py scripts to get the saliency predictions on the test dataset.