Market Sentiment Analysis

This project analyzes market sentiment by scraping news articles from Moneycontrol, summarizing the content, and performing sentiment analysis to extract meaningful insights. The processed data includes sentiment scores and summaries, which can be saved for further analysis.

---

Features

• Web Scraping: Collects market news articles for a specified number of days.
• Summarization: Summarizes the content of news articles using an LLM.
• Sentiment Analysis: Analyzes the sentiment (positive, negative, neutral) of the articles using FinBERT.
• Data Export: Optionally saves the processed data with summaries and sentiment scores to a CSV file.

---

Table of Contents

1. Setup
2. Usage
3. Project Structure
4. Dependencies
5. Example Output
6. License

---

Setup

1. Clone the Repository

git clone https://github.com/SandyDRawat/Market_sentiment_analysis

2. Install Dependencies

Install the required Python libraries:

pip install -r requirements.txt

3. Configure API Keys

Store your API keys(Hugging Face) in .env file:

HUGGINGFACE_API_KEY=your_huggingface_api_key

---

Usage

1. Run the streamlit App

Launch the streamlit interface to analyze sentiment interactively:

streamlit run app.py

2. Use the get_data Function

To scrape, summarize, and analyze sentiment programmatically:

from your_project_module import get_data

# Fetch and process articles for the last 3 days
data = get_data(no_of_days=3, download_csv=True)

---

Project Structure

.
├── data_intake/
│   ├── get_urls.py          # Fetches article URLs
│   ├── web_crawler.py       # Scrapes article content
├── data_preprocess/
│   ├── add_summary.py       # Adds summaries to articles
│   ├── add_sentiment.py     # Adds sentiment analysis
├── app.py                   # Streamlit application
├── main.py                  # Main script for fetching and processing data
├── requirements.txt         # List of dependencies
├── README.md                # Project documentation
└── .env                     # Environment variables (not tracked in Git)

---

Dependencies

• Python 3.8 or later
• Libraries:
  • streamlit
  • pandas
  • transformers
  • python-dotenv
  • langchain
  • torch
  • scipy

Install all dependencies using:

pip install -r requirements.txt

---

Example Output

Sample DataFrame

Title	Content	Summary	Positive	Negative	Neutral	Predicted Label	Score
Market gains momentum today...	Full article content	Summarized content	0.65	0.20	0.15	Positive	0.45

Saved CSV File

Processed data is saved to: D:/projects/Market sentiment analysis/saved_data/

---