Skip to content

Using PySpark to perform the ETL process to extract the dataset, transform the data, connect to an AWS RDS instance, and load the transformed data into pgAdmin. Then using PySpark, Pandas, & SQL to determine if there is any bias toward favorable reviews from Vine members in the dataset.

Notifications You must be signed in to change notification settings

NedaAJ/Amazon_Vine_Analysis

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Amazon_Vine_Analysis

You can find the Analysis file here: Amazon_Reviews_ETL.ipynb | VineReviews_Analysis.ipynb

Analysis Overview

This project examines the Amazon Vine service to see if there is a bias favouring positive Vine member reviews. PySpark is used to extract the dataset, convert the data, connect to an AWS RDS instance, load the changed data into pgAdmin, and calculate various metrics in the analysis. We concentrated on video game reviews in the United States.

Results

  • Vine Reviews Summary:

  • Non-vine reviews Summary:

Summary

There could be biased among "Star Ratings". There were 40,471 records that did not have vine reviews, but only 94 records that did have vine reviews.The Vine programme received a 5-star rating from 51 percent of recordings. In comparison, about 38% of records not included in the Vine programme received a 5-star rating. Given the fact that there were much fewer recordings with Viners than there were records without Viners, the Vine/Star-Rating ratings might be skewed. For future studies, We might also look at the statistical distribution of the star rating for Vine and non-Vine reviews (mean, median, and mode).

Contact:

About

Using PySpark to perform the ETL process to extract the dataset, transform the data, connect to an AWS RDS instance, and load the transformed data into pgAdmin. Then using PySpark, Pandas, & SQL to determine if there is any bias toward favorable reviews from Vine members in the dataset.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published