Telegram Crawler - parser for connections between Channels in Telegram. It can detect connections such as:
Forwards, Mentions and direct links to other Channels.
Telegram Crawler is the parser for connections between Channels in Telegram. It is written in Python and mostly based on telethon. This project requires separate back-end. Back-end based on PostgresQL and FastAPI, so that it allows multiple parsers running in parallel to increase scanning speed.
These instructions will get you a copy of the project up and running on your local machine.
This project written in Python, so you have to make sure you have Python installed on your machine.
Make sure, Python is operative and you can proceed to installation.
Also this project requires separate back-end to be working already before running this project. Make sure to set it up before proceeding with installation.
Clone this repo:
git clone https://github.com/Antcating/TelegramCrawler.git
Go to the project folder
cd TelegramCrawler
The appropriate thing to do is to create virtual environment for this project. Newer Linux distros are forcing users to create virtual environments and not install Python packages system-wide
Create virtual environment for this project in the working directory
python -m venv .
Activate virtual environment
source bin/activate
Install all the dependencies
pip install -r requirements.txt
For parser to work you have to provide Telegram API_HASH and API_ID. To get them visit my.telegram.org/ and create new application. After that you will get your Telegram API_ID and API_HASH on top of the application page. More info here. Paste your API_ID and API_HASH into config.ini file, located in the root of the project directory.
You are done with installation. Now we proceed to running the parser
Before running the project make sure, that backend is running on 127.0.0.1. After you confirmed, that backend is running, you can proceed to running this project.
python src/main.py
If the installation was successful, you would be asked to enter mobile number for the account, that you created API_ID and API_HASH. This is regular Telegram procedure, you are creating new session for your Telegram account.
After you completed Telegram login, would be presented with question about starting point of the parser. After that parser should be running without any problems.
You can change your starting channel prompt in the config.ini file on the row START_CHANNEL_USERNAME.
Also, if you are deploying your backend on external server and not locally - you can specify ip:port of your server on the row SERVER.
- Telethon - Telegram Backend
- @Antcating - Author