IoT Workload Generator

Monitoring Backend's Performance Through Simulating Metrics of IoT Sensors

This project is an HTTP workload generator that simulates metrics of IoT sensors. Furthermore, it utilizes these metrics to monitor the performance of a web service. It consists of a client and an example backend server. A complete example can be found in the sample_setup directory.

Client

The client simulates sensors and sends their metrics as HTTP requests to the configured endpoint. Its goal is to simulate an actual workload which is usually generated by sent by users/devices. During this load testing operation, it monitors the service's performance. Multiple factors, such as response time distribution and server error rate, define the service's performance. These monitoring metrics can be aggregated, written down to a file, or served to Prometheus.

Sensor Types

Each sensor has an interval attribute which is the time-delta between two consecutive metrics generation operations.

• Temperature: generates numbers from a normal variable with random $\mu \in (-30,30)$ and $\sigma=10$.
• Device: generates numbers from a guassian distribution with sepecified $\mu$ and $\sigma$.
• GPS: returns latitude and longitude coordinates from a random path specified in assets; As if someone walks back and forth in this random path.
• ASD: generates data from a wave file specified in assets and SPS (samples per second) specified in config. The returned portion of the wave file is determined randomly.
• Camera: generates random frames from a camera with specified bitrate and fps.

Configuration

Example Configuration File

# initial and end waiting time in seconds
wait:
  start: 3
  end: 3
metrics:
  method: "file" #The method for dumping performance metrics from monitored service. values can either be "file" or "prometheus"
# running schedule
schedule:
  - sensors: 10 #number of sensors for this step
    time: 70 # the duration of this step in seconds
  - sensors: 5
    time: 70
  #This schedule means simulating ten sensors for 70 seconds and then five sensors for 70 seconds. One can also simulate 0 sensors to simulate sleeping.
# sensor configurations
sensors:
  - type: temp # temperature sensor
    interval: 1.0 # seconds
  - type: device
    mean: 5.0 #float
    sigma: 1.0 #float
  - type: gps
    interval: 1.0
  - type: asd
    sps: 24 #int
  - type: camera
    fps: 15 #int
    bitrate: 50000 #int

Deployment

This workload generator can be deployed as a docker container.

Docker Compose Example

iot:
 image: ghcr.io/nimaafshar/workload-generator-iot:latest
 pull_policy: always
 depends_on:
   - backend
 expose:
   - "8000"
 environment:
   HOST_URL: http://backend:5000/sensor/add
   TZ: "Asia/Tehran"
   IOT_LOG_LEVEL: 20 # info
 volumes:
   - ./config/iot/:/app/config/  # replace the config file
   - ./iot_results/:/app/run/    # see results

Monitored Metrics from The Service

The client can aggregate these metrics and dump them in a CSV file. This CSV file is saved in the folder /app/run and has the following columns.

time_till,average_sensors,requests,err_rate,average_response_time

The client can also serve metrics to Prometheus:

• iotwg_web_response_time (Type: Summary): The response time for each request sent.
• iotwg_web_request_length (Type: Summary): The response length in bytes for each request sent.
• iotwg_web_failed_requests (Type: Counter): The number of failed requests.
• iotwg_sensor_count (Type: Gauge): The current number of sensors.

Example Visualization Of These Metrics In Grafana

Example Backend Server

This module is an example backend server to which the workload generator can be connected. It inserts received metrics into a MongoDB database and can return stored metrics. This backend service is built using python, gunicorn, and Flask

Endpoints

• Insert Data POST /sensor/add
• Retrieve Data GET /sensor/query/<int:page>

Deployment

This backend server can be deployed as a docker container.

web:
    image: ghcr.io/nimaafshar/workload-generator-web:latest
    pull_policy: always
    depends_on:
      - db
    expose:
      - "5000"
    sysctls:
      net.core.somaxconn: 30000
    environment:
      TZ: "America/Montreal"
      WEB_WORKLOAD_CONFIG_MONGODB_USERNAME: root
      WEB_WORKLOAD_CONFIG_MONGODB_PASSWORD: password
      VIRTUAL_HOST: web.local
      VIRTUAL_PORT: 5000
    deploy:
      resources:
        limits:
          cpus: 1.0
    logging:
      driver: none

Reference

Please cite the following papers if you use this project:

1. Rasolroveicy, M., & Fokaefs, M. (2020, November). Dynamic reconfiguration of consensus protocol for IoT data registry on blockchain. In Proceedings of the 30th Annual International Conference on Computer Science and Software Engineering (pp. 227-236).

2. Ramprasad, B., Fokaefs, M., Mukherjee, J., & Litoiu, M. (2019, June). EMU-IoT-A Virtual Internet of Things Lab. In 2019 IEEE International Conference on Autonomic Computing (ICAC) (pp. 73-83). IEEE.