AdoptOpenJDK TSC and Knowledge Base

Contribution

We welcome any contributions big or small from experienced OpenJDK folks to those who are brand new to software projects!

To start, make sure you sign up to the AdoptOpenJDK Slack, say hello and please read the Contribution Guide.

Knowledge Base

Overview

The AdoptOpenJDK Build Farm is three things:

1. A Repository of professionally built, tested and supported OpenJDK binaries - A place for end users to download professionally built and tested OpenJDK binaries, see Support for our full support details.
2. A Community of builders - A place where those who build and test OpenJDK come together to share common code and practices.
3. Infrastructure as Code - To host, build, test and deploy variants of OpenJDK (aka Java)! This infrastructure as code is designed to be usable by any person or organisation wishing to build a derivative build farm or parts of one.

See https://www.adoptopenjdk.net for in depth details of the motivation, who's involved, the sponsors and much more!

Dependent Projects

These projects are located in the following repositories in rough order of importance with regards to understanding how the build farm is put together and works. Each repository maintains its own set of issues, pull requests and documentation (edit the Image Source at Google and export to make changes.)

• Technical Steering Committee (TSC) - The Technical Steering Committee and Knowledge Base starting point
  • TSC-confidential) - To hold any confidential information for the TSC, e.g. Sponsorship Agreements
  • security - A private repo for the security team
    • moderation - A private repo for moderation requests
    • openjdk-security - A private repo for (adopt)openjdk vulnerability investigations
    • secrets - A private repo for secrets
  • buzz - Community Outreach and Sponsors
• openjdk-infrastructure - Infrastructure as Code and documentation for build farm hosts
  • email - For handling all org emails (mailgun)
  • openjdk-jenkins-helper - An auto label generator for Jenkins
• openjdk-build - Code and instructions for building Adopt OpenJDK Binaries
  • openjdk-Icedtea-web - A mirror of Iced tea web, the OSS replacement for Java Web Start and the Browser plugin.
  • openjdk-docker - Scripts for creating Docker images of OpenJDK binaries
  • openjdk-docker-build-tools - Helper scripts for openjdk-docker
  • openjdk-installer - Installer files for creating platform packaged OpenJDK binaries
  • homebrew-openjdk - HomeBrew TAP repo for all macOS OpenJDK binaries
  • jdk-api-diff - A diff tool that reports what API changes there are between versions
• openjdk-tests - Instructions for all testing at AdoptOpenJDK, code for app, performance and regression testing of AdoptOpenJDK Binaries and CI test automation scripts
  • openjdk-test-tools - Tools for openjdk testing (not to be confused with stf which is the tooling for system tests)
  • openjdk-systemtest - Code and instructions for system and load testing AdoptOpenJDK Binaries
  • stf - The System Test Framework, a harness for executing openjdk-systemtest
  • bumblebench - A microbenchmarking test framework for Adopt OpenJDK
• openjdk-website - Code and instructions for https://www.adoptopenjdk.net
  • openjdk-website-next - The Typescript and React re-write for https://www.adoptopenjdk.net
  • openjdk-api - Code and instructions for https://api.adoptopenjdk.net
    • openjdk-api-java-client - A Java client for our API
  • openjdk-website-backend - Code for pulling the GitHub releases API into the website
  • openjdk-dashboard - AdoptOpenJDK Dashboard for download stats at https://dash.adoptopenjdk.net

AdoptOpenJDK Source and Distribution Repos

The following diagram lists the source and distro repos for AdoptOpenJDK (edit the Image Source at Google and export to make changes.)

Clones of OpenJDK Forests

When an OpenJDK variant is mercurial based or AdoptOpenJDK needs to maintain its own patches then we have a clone of that source:

• openjdk-jdk8u
• openjdk-aarch64-jdk8u
• openjdk-jdk9u
• openjdk-jdk10u
• openjdk-jdk11u
• openjdk-jdk12u
• openjdk-jdk13u
• openjdk-jdk
• openjdk-jfx
• openjdk-amber

AdoptOpenJDK binary repos

These GitHub Repos are where builds that pass the pipeline are published to. The API (and subsequently) website poll these repos to make binaries available.

• openjdk8-binaries - AdoptOpenJDK nightly and release binaries for OpenJDK 8 (HotSpot VM and Eclipse OpenJ9 VM)
• openjdk9-binaries - AdoptOpenJDK nightly and release binaries for OpenJDK 9 (HotSpot VM and Eclipse OpenJ9 VM)
• openjdk10-binaries - AdoptOpenJDK nightly and release binaries for OpenJDK 10 (HotSpot VM and Eclipse OpenJ9 VM)
• openjdk11-binaries - AdoptOpenJDK nightly and release binaries for OpenJDK 11 (HotSpot VM and Eclipse OpenJ9 VM)
• openjdk12-binaries - AdoptOpenJDK nightly and release binaries for OpenJDK 12 (HotSpot VM and Eclipse OpenJ9 VM)
• openjdk13-binaries - AdoptOpenJDK nightly and release binaries for OpenJDK 13 (HotSpot VM and Eclipse OpenJ9 VM)
• openjdk-binaries - AdoptOpenJDK nightly and release binaries for OpenJDK (Latest) (HotSpot VM and Eclipse OpenJ9 VM)

OpenJDK binary repos

These GitHub Repos are where builds from the OpenjDK update projects (led by Red Hat) are published to. The API (and subsequently) website poll these repos to make binaries available.

• openjdk8-upstream-binaries - OpenJDK nightly and release binaries for OpenJDK 8 (HotSpot VM)
• openjdk11-upstream-binaries - OpenJDK nightly and release binaries for OpenJDK 11 (HotSpot VM)

Private repos

Due to security or licensing concerns the following repos are private. Please raise an issue on the Infrastructure Project if you think you need access.

• TSC-confidential) - To hold any confidential information for the TSC, e.g. Sponsorship Agreements
• moderation - For holding moderation requests
• secrets - For holding some secrets
• security - For the security team
• openjdk-security - A private repo for (adopt)openjdk vulnerability investigations
• openjdk-website-staging - for staging website PR's

Workflow

The following diagram is a very simplified view of how a build progresses through a pipeline (edit the Image Source at Google and export to make changes.)

NOTE: Please ensure you read the documentation below and in the appropriate repos to get a full understanding

The workflow to source, build, test and deploy variants of OpenJDK is as follows.

1. Get the OpenJDK variant source code

We source variants and versions of OpenJDK from a variety of source repositories. Add a new build variant describes the typical work flow of getting a new variant up and running. Please also see Platform Costs as variants with high impact on the build farm will require funding.

1. OpenJDK HotSpot Mirrors - GitHub mirrors of OpenJDK forests are created in the AdoptOpenJDK org. For example: openjdk-jdk8u, openjdk-jdk11u. Please open an openjdk-TSC issue if you'd like a new variant added.
   1. git-hg Jobs Update Mirrors - Our Jenkins CI runs git-hg jobs to regularly update those various clones of OpenJDK forests. See their job configurations in Jenkins and the openjdk-build repo for details.
2. OpenJDK OpenJ9 - Eclipse OpenJ9 source is built from IBM Runtimes, e.g. openj9-openjdk-jdk8, openj9-openjdk-jdk9
3. OpenJDK SAP - SAP source is built from e.g. openjdk-sap-jdk10
4. OpenJDK OpenJFX - GitHub mirror of OpenJDK JFX lives at AdoptOpenJDK - openjdk-jfx

2. Choose the branches to build from

Each OpenJDK variant has a canonical branch that is built:

1. OpenJDK HotSpot - The dev branch of AdoptOpenJDK contains extra patches over and above master (which is the exact clone of the OpenJDK forest). dev is used to build AdoptOpenJDK binaries
2. OpenJDK OpenJ9 - Branches such as openj9-0.8 are used to build OpenJ9 Releases for Java 8 and openj9 is used to build OpenJ9 for Java 9.
3. OpenJDK SAP - The sapmachine branch is used to build SAP for Java 10
4. OpenJDK OpenJFX - TODO

3. Build, Test and Deploy Pipeline

Our Jenkins Pipelines (source code) build binaries and then push them through a test pipeline (NOTE Test pipeline is skipped for certain unstable combinations).

NOTE: Once the Test jobs have stabilised, the pipeline described below will change so that only tested binaries are released to Nightly and Release repos for consumption by the Website and/Or API. See The Quality Bar Discussion for details.

3.1 Build

Builds are run on the Supported Platforms. The Jenkins leader sends the build jobs to the Jenkins followers based on a tagging system configured in the Jenkins jobs, e.g. centos6&&x64&&build. See openjdk-build and openjdk-infrastructure for details.

3.2 Test

Note that other OpenJDK binaries (such as the openjdk8-upstream-binaries and openjdk11-upstream-binaries) can be put through pipelines entering at this Test phase.

Tests are run on the Supported Test Platforms. The Jenkins leader sends the test jobs to the Jenkins followers based on a similar tagging system to build. See openjdk-tests and openjdk-infrastructure for details.

1. Builds are tested - The tests in openjdk-tests are executed and tests results are posted to TODO. Tests include, but are not limited to the jtreg tests that come with OpenJDK itself.
2. Builds are system tested - System Tests from openjdk-systemtests are executed and test results are posted to TODO
3. Builds are external tested - External Tests are executed and test results are posted to TODO
4. Builds are performance tested - Performance Tests from bumblebench are executed and test results are posted to TODO

3.3 Deploy Binaries

TODO: We're missing the SAP and OpenJFK deployments here.

NOTE Future versions of this workflow will show the status of testing and meta data about how the binary was built.

1. Binaries are deployed - Using the OpenJDK Release Tool (from the openjdk-webiste-backend project) in order to:

   1. deploy them to the various binary repos: (openjdk8-binaries, openjdk9-binaries, openjdk10-binaries, openjdk11-binaries, openjdk12-binaries, openjdk13-binaries, openjdk-binaries).

2. Binaries made available - Binaries are made available for download via the website and api gateway. See openjdk-website, openjdk-api and openjdk-api-java-client projects for more details.

The TSC

List of TSC Responsibilities

The TSC exercises autonomy in setting up and maintaining procedures, policies, and management and administrative structures as it deems appropriate for the maintenance and operation of these projects and resources.

Included in the responsibilities of the TSC are:

• Managing the tangible assets of the projects such as machines, code, etc, and the intangible assets such as policies, brand, etc for the listed projects and resources

• Meeting monthly to discuss progress and other TSC issues

• Setting and maintaining standards covering contributions of code, documentation and other materials including infrastructure such as build machines

• Facilitating code and binary releases: types, schedules, frequency, delivery mechanisms

• Acquiring resources and ensuring they are maintained in a functional and secure manner for the benefit of the project.

• Resolving overall technical direction for the AdoptOpenJDK organization, including high-level goals and low-level specifics regarding features and functionality

• Maintaining a relationship with the AdoptOpenJDK security group for dealing with vulnerabilities in an appropriate manner

• Setting and maintaining appropriate standards for community discourse via the various mediums under TSC control, including but not limited to the Website, Slack, GitHub comments

TSC Members

TSC members can nominate new members at any time. Candidates for membership tend to be people who have a competency for community management and high tolerance and patience for process minutiae as the TSC delegates most of its responsibilities to other teams.

Current Members

Director	Chair
karianna - Martijn Verburg (jClarity)	gdams - George Adams (jClarity)

	Members (listed alphabetically)
cmdc0de - Demetrius Comes (GoDaddy)	gdams - George Adams (jClarity)	jerboaa - Severin Gehwolf (Red Hat)
johnoliver - John Oliver (jClarity)	karianna - Martijn Verburg (jClarity)	smlambert - Shelley Lambert (IBM)
sxa555 - Stewart X Addison (IBM)	tellison - Tim Ellison (IBM)	vielmetti - Edward Vielmetti (Packet)

Rules for TSC Membership

1. No single organization can occupy more than half the seats on the TSC.

2. The appointing of TSC members occurs on an annual basis and is based on a do-ocracy.

3. Candidates with the intention of becoming a member of the TSC should briefly outline where they'd like to see the project going - all in a transparent manner that is available to the public.

Election Process

Each community member has a Single Transferable Vote (STV). There are tools to manage STV voting: https://en.wikipedia.org/wiki/Single_transferable_vote#Voting

Thanks! The AdoptOpenJDK Community.