Official code repository for GATK versions 4 and up

Spark RAPIDS plugin - accelerate Apache Spark with GPUs

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

Final project for the course 'Architecture for Large Data Volumes', taught in the Bachelor's program in Data Science at ITAM

This construct builds some elements for you to quickly launch an EMR Serverless application. After submitting the Emr Serverless job, you could also launch an EMR notebook via cluster template to check the outcome from the EMR Serverless application.

Apache Spark - A unified analytics engine for large-scale data processing

Apache DataFusion Comet Spark Accelerator

New generation decentralized data lake and a streaming data pipeline

Host a Docker container for the Spark history server / Spark UI of AWS Glue jobs

In this project, we focusing on generating an orders fact table from the dataset provided by Olist in order to analyze the sales performance of the company.

YTsaurus is a scalable and fault-tolerant open-source big data platform.

Extensible Python SDK for developing Flyte tasks and workflows. Simple to get started and learn and highly extensible.

🧙 Build, run, and manage data pipelines for integrating and transforming data.

Apache Doris is an easy-to-use, high performance and unified analytics database.

Academic course

Web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more.

Customized Jupyter Spark Docker images with everything you need

DoEKS is a tool to build, deploy and scale Data & ML Platforms on Amazon EKS

Simple and Distributed Machine Learning

Practice course on Big Data