Trino CI CD

Pipeline performance

Job duration

The heaviest jobs are product tests because they:

• set up complex environments: a Trino cluster, made of a coordinator and some workers, and other dependencies, like Hive
• generate test data
• execute a lot and/or complex queries

TODO to confirm the above, profile tests to be able to get average times for different steps.

Queue

Trino's CI/CD pipelines are implemented as Github Workflows that use Github's public runners. Because there's a limit of concurrent jobs, runs are often queued for a long time before they execute.

The solution is to make the jobs shorter and (in PRs only) skip ones not relevant to the change.

• Make CI only build/test modules with any changes