HUJI Concatenator

Table of Contents

• HUJI Concatenator
  • Table of Contents
  • Getting Started
    • Prerequisites
    • Installing
    • Configuration
      • PATHs
      • Data characteristics
      • Specific configuration
      • LOG
        • Telegram
  • Usage
    • Manual use
      • Using Python
      • Using bat
    • Scheduling
  • Linux support

Getting Started

To set up packet concatenation, you have to ensure that you have fresh Python installation (>=3.10).

Prerequisites

This project has scheduling scripts, which could help automate packet concatenation. The scheduling implemented using bat scripts so it is only compatible with Windows systems.

Installing

A step by step series of guide that tell you how to get a packet concatenation running.

Clone source code of the project

git clone https://github.com/Antcating/DAS-Prisma-Concat.git
dir DAS-Prisma-Concat

Create virtual environment for Packet Concatenation:

python -m venv .venv_concat

Activate virtual environment (bash):

.venv_concat\Scripts\activate

Install required modules

pip install -r requirements_concat.txt

Configuration

Before running the project you have to configure parameters in the config.ini file.

PATHs

LOCALPATH is absolute PATH to the directory which will contain uncompressed mseed files.

NASPATH_final is absolute PATHs to the NAS directory which will contain concatenated hdf5 files.

Data characteristics

UNIT_SIZE is length of the packet including redundancy (in second). By default 20. SPS is expected time frequency after data downsamling (in Hz). By default 100. DX is expected spatial spacing after data downsampling (in m). By default 9 m.

Specific configuration

CHUNK_SIZE length of chunks processed by Packet Concatenation after concatenation (in seconds)

Ensure that you provided properly updated config.ini and you ready to go.

LOG

Used for configuring internal logger. Logger file is located in LOCALPATH\log.

• LOG_LEVEL variable defines the log level of the file logger, and is set to INFO by default.

• CONSOLE_LOG is a boolean (True/False) that defines whether console output should be logged. It is set to True by default.

• CONSOLE_LOG_LEVEL defines the log level of the console logger. If not provided, it defaults to INFO.

Telegram

Enables exception logging to Telegram using TelegramBotAPI

• USE_TELEGRAM a boolean (True/False) that enables or disables Telegram logging.

• token a string that represents the token provided by Telegram to operate the bot.

• channel a string that represents the Telegram channel where notifications will be sent.

Usage

Script will scan LOCALPATH directory (provided in config.ini) for subdirectories. Script will not write to this directories to ensure that last modification time is not changed (crussial for sorting directories, as they are names in UUID format).

Manual use

Using Python

If you want to run project once, you can do this using Python directly:

Ensure you activated virtual environment

.venv_concat\Scripts\activate

Run Packet Concatenation

python main.py

Using bat

Project provide bat wrapper around Python project, which sets up PATH, virtual environment by itself.

To use bat script, you will have to setup it: In the concat.bat in root directory of the project change PROJECT_PATH to absolute PATH of root directory of this project.

Scheduling

As the target system is Windows, the concatenation can be schedulled using a Task Scheduler. The service is defined in the concat.bat file and started at the specified time.

Make sure to complete Using Bat. We will use concat.bat in your scheduling script.

Linux support

The project is not tested on Linux, but it should work. Project provides bash script concat.sh which is equivalent to concat.bat and systemd service and timer to schedule packet concatenation. Please refer to Febus Concatenator for more information.