deploy-GAISSA

Summary

Guidelines to deploy AI models in different cloud providers aligned with green AI goals.

Repository Structure

The repository is structured as follows:

- app
  | API, schemas
- models
  | This folder contains our trained or pretrained models
- notebooks
  | This folder contains the jupyter notebooks
- reports
  | Generated PDFs, graphics and figures to be used in reporting
- utils
  | Python functions
- manuals
  | self-contained manuals
- requirements.txt: The dependencies of our implementation

Guide:

1. API creation. Guide to create an API to deploy ML models.
2. Add pretrained model. Guide to add pretrained ML models (from HuggingFace, hdf5 format, pickle format) to do inferences through an API.
3. Deploy ML models in a cloud provider (General). Guide to deploy ML models using an API in a cloud provider.
4. Deploy in Virtech. Virtech setup, Guide to deploy ML models using an API in an AWS VM.
5. AWS. AWS setup, Guide to deploy ML models using an API in an AWS VM.
6. GCP. GCP setup, Guide to deploy ML models using an API in an GCP VM.
7. Azure. Azure setup, Guide to deploy ML models using an API in an Azure VM.
8. FAQ. Documentation with problems arised during deployments.
9. Other. Other notes.

Cloud providers*

* Initial proposed cloud providers

- Amazon Elastic Compute Cloud (Amazon EC2) from Amazon Web Services (AWS)
  | URL: https://aws.amazon.com/
- Azure Virtual Machines from Microsoft Windows Azure
  | URL: https://azure.microsoft.com/
- Google Compute Engine from Google Cloud Platform (GCP)
  | URL: https://cloud.google.com/
- Virtech, UPC cloud provider (By OpenNebula)
  | URL: https://www.fib.upc.edu/es/la-fib/servicios-tic/cloud-docente-fib
  | URL: https://opennebula.io/

Amazon EC2

Google Compute Engine

Virtech, UPC cloud provider

Azure Virtual Machines

Models*

* Initial proposed models

• BERT model
• T5
• CodeGen
• Pythia-70m
• CNN model
• Codet5p-220m

Text Generation

• BERT

  • https://huggingface.co/bert-base-uncased

• T5

  • https://huggingface.co/t5-base

Computer Vision

• CNN model
  • https://github.com/fjdurlop/guided-retraining/tree/main/models

Code Generation

• CodeGen
  • https://huggingface.co/Salesforce/codegen-350M-mono
• Pythia-70m
  • https://huggingface.co/EleutherAI/pythia-70m
• Codet5p-220m
  • https://huggingface.co/Salesforce/codet5p-220m

API

See manuals/01_create_api.md to check how to create an API to deploy ML models.

FastAPI

ML frameworks

• TensorFlow
• PyTorch

ML model formats

• ONNX
• h5, complete model
• h5, weights only
• Pickle

ML task

• CV
• NLP
• ...

Roles

Role: ML Engineer

• Data engineer: Manage DBs
• Data scientist: Train ML models
• BI: Dashboards, analytics, BI
• ML Engineer: SE --deploy--> ML systems

Energy tracking metrics

• codecarbon
• ...

Future work

• Track energy efficiency.
• Trade-off between green-AI related metrics and accuracy.
• Monitor models' performance

References

See manuals/references

---

ToDo:

• Add info cloud providers
• Add FastAPI info

---