[email protected] redownloading jdk on every run #632

Ivanuil · 2024-05-09T13:31:38Z

Description:
[email protected] redownloading jdk on every every job in a pipeline while executing on a self-hosted Linux runner (about 15min on every job).

Task version:
v4.2.0

Platform:

Ubuntu
macOS
Windows

Runner type:

Hosted
Self-hosted

Repro steps:
Workflow file:

build:
    runs-on: self-hosted
    steps:
      - uses: actions/checkout@v4
        with:
          persist-credentials: false
      - name: set up JDK 17
        uses: actions/[email protected]
        with:
          distribution: 'oracle'
          java-version: 17
          check-latest: false
          cache: 'maven'
      - name: Maven Verify
        run: mvn clean compile

  codestyle:
    needs: build
    runs-on: self-hosted
    steps:
      - uses: actions/checkout@v4
      - name: set up JDK 17
        uses: actions/[email protected]
        with:
          distribution: 'oracle'
          java-version: 17
          check-latest: false
          cache: 'maven'
      - name: Maven Verify
        run: mvn checkstyle:check

  test:
    needs: codestyle
    runs-on: self-hosted
    steps:
      - uses: actions/checkout@v4
      - name: set up JDK 17
        uses: actions/[email protected]
        with:
          distribution: 'oracle'
          java-version: 17
          check-latest: false
          cache: 'maven'
      - name: Maven Verify
        run: mvn -B clean verify
      - name: Upload coverage reports to Codecov
        uses: codecov/[email protected]
        with:
          token: ${{ secrets.CODECOV_TOKEN }}
          slug: Ivanuil/ImageEditor

Setup JDK log:

Run actions/[email protected]
  with:
    distribution: oracle
    java-version: 17
    check-latest: false
    cache: maven
    java-package: jdk
    server-id: github
    server-username: GITHUB_ACTOR
    server-password: GITHUB_TOKEN
    overwrite-settings: true
    job-status: success
    token: ***
Installed distributions
  Trying to resolve the latest version from remote
  Resolved latest version as 17
  Trying to download...
  Downloading Java 17 (Oracle) from https://download.oracle.com/java/17/latest/jdk-17_linux-x64_bin.tar.gz ...
  Extracting Java archive...
  /usr/bin/tar xz --warning=no-unknown-keyword --overwrite -C /home/ivanuil/actions-runner/_work/_temp/6ec944e[2](https://github.com/Ivanuil/ImageEditor/actions/runs/9017688035/job/24776650890#step:3:2)-cb2c-4b87-ad79-8f27ace92481 -f /home/ivanuil/actions-runner/_work/_temp/7f52[3](https://github.com/Ivanuil/ImageEditor/actions/runs/9017688035/job/24776650890#step:3:3)1a7-7ee0-4356-a299-19e1080a6[4](https://github.com/Ivanuil/ImageEditor/actions/runs/9017688035/job/24776650890#step:3:4)80
  Java 17 was downloaded
  Setting Java 17 as the default
  Creating toolchains.xml for JDK version 17 from oracle
  Overwriting existing file /home/ivanuil/.m2/toolchains.xml
  
  Java configuration:
    Distribution: oracle
    Version: 17
    Path: /home/ivanuil/actions-runner/_work/_tool/Java_Oracle_jdk/17/x[6](https://github.com/Ivanuil/ImageEditor/actions/runs/9017688035/job/24776650890#step:3:6)4
  
Creating settings.xml with server-id: github
Overwriting existing file /home/ivanuil/.m2/settings.xml
maven cache is not found

Post setup JDK log:

Post job cleanup.
/usr/bin/tar --posix -cf cache.tzst --exclude cache.tzst -P -C /home/ivanuil/actions-runner/_work/ImageEditor/ImageEditor --files-from manifest.txt --use-compress-program zstdmt
Cache Size: ~3[2](https://github.com/Ivanuil/ImageEditor/actions/runs/9017688035/job/24776650890#step:7:2)1 MB (336496180 B)
Cache saved successfully
Cache saved with the key: setup-java-Linux-maven-a1b758b65f9e0e80afb7bef1758b722af2701f1dbe542c6f8a9f6a83933ce403

The text was updated successfully, but these errors were encountered:

aparnajyothi-y · 2024-05-10T13:48:11Z

Hello @Ivanuil, Thank you for creating this issue and we will look into it :)

Sebagai · 2024-05-15T03:19:53Z

Haha

abelmatos · 2024-05-24T07:58:31Z

I'm using the default installed version java 21.
there is a 17 too
https://github.com/actions/runner-images/blob/ubuntu22/20240516.1/images/ubuntu/Ubuntu2204-Readme.md#java
e.g.

        # https://github.com/actions/runner-images/blob/ubuntu22/20240516.1/images/ubuntu/Ubuntu2204-Readme.md#java
      - name: Switch to Java 17
        run: echo "JAVA_HOME=$JAVA_HOME_17_X64" | tee -a $GITHUB_ENV ;  sudo update-java-alternatives --set $JAVA_HOME_17_X64;

Will be cool this action to take advantage of the pre-existent java versions already installed

oscar-b · 2024-05-31T09:52:03Z

We're seeing the same thing on our self hosted runners. Shouldn't it re-use /opt/actions-runner/_work/_tool/Java_Oracle_jdk/17/x64 if it's already there? Or do we need to setup a tools cache like Githubs runners? This is all quite poorly documented for us running self hosted runners. Any idea @aparnajyothi-y

oscar-b · 2024-05-31T11:10:27Z

Try specifying java-version: 17.0.11, the action doesn't use the cache if it's not an exact version.

https://github.com/actions/toolkit/blob/main/packages/tool-cache/src/tool-cache.ts#L553

Ivanuil · 2024-05-31T14:55:28Z

Try specifying java-version: 17.0.11, the action doesn't use the cache if it's not an exact version.

That did the trick. Thanks)

mahabaleshwars · 2024-06-03T06:40:59Z

Hello @Ivanuil, I've tried to reproduce the issue but didn't encounter any problems. Here's a screenshot for you to look at. Could you please outline the steps that led to this issue, so I can attempt to recreate it?

oscar-b · 2024-06-03T07:41:50Z

@mahabaleshwars
Your screenshot says: "Java 17 was downloaded"

It should say: "Resolved Java 17.0.11 from tool-cache" when it re-uses the cache.

Look at the code I pointed to above, using "17" as the version will disqualify that dir in _work/_tool/Java_Oracle_jdk. However, if you'd happen to have both 17 and 17.0.1, it will pass the check that it has any caches and 17 will find its directory anyway. The logic there is quite badly implemented.

mahabaleshwars · 2024-06-14T04:38:43Z

Hello @Ivanuil,
If you're using the setup-java action and specify a major version like java-version: 17, the action will try to resolve it to the latest minor version. If a new minor version is released (say 17.0.12), it won't match the cache key for 17.0.11 and the JDK will be downloaded again.
To ensure that the exact version is cached and used across workflows, it's recommended to specify the exact JDK version you want to cache in your workflow file, like java-version: 17.0.11. This way, the cache key will remain consistent across workflow runs.

oscar-b · 2024-06-14T10:07:03Z

Hello @Ivanuil Yeah but that's not how it works in reality, unfortunately. See my comments above.

LetapF · 2024-07-11T07:45:50Z

something is downloading every time that i build something

jobs:
  build_android:
    if: "contains(github.event.head_commit.message, 'build-android')"
    name: Build Android
    runs-on: self-hosted
    steps:
      - name: Check out repository code
        uses: actions/checkout@v4
      - name: use Node.js
        uses: actions/setup-node@v4
        with:
          node-version: "18.19.1"
      - name: Setup Java
        uses: actions/[email protected]
        with:
          java-version: 17.0.11
          distribution: "oracle"
          check-latest: false
          cache: "gradle"
      - name: Setup Android SDK
        uses: android-actions/setup-android@v3

Ivanuil · 2024-07-11T08:01:21Z

something is downloading every time that i build something

jobs:
  build_android:
    if: "contains(github.event.head_commit.message, 'build-android')"
    name: Build Android
    runs-on: self-hosted
    steps:
      - name: Check out repository code
        uses: actions/checkout@v4
      - name: use Node.js
        uses: actions/setup-node@v4
        with:
          node-version: "18.19.1"
      - name: Setup Java
        uses: actions/[email protected]
        with:
          java-version: 17.0.11
          distribution: "oracle"
          check-latest: false
          cache: "gradle"
      - name: Setup Android SDK
        uses: android-actions/setup-android@v3

Same for me :( Everything worked as it was supposed to for a month. And a few days ago the action started redownloading something on every run. It also usually fails to download (percentage get stuck at some point) and i have to cancel that run

Ivanuil · 2024-07-11T08:02:36Z

My job looks like this:

- name: set up JDK 17
        uses: actions/[email protected]
        with:
          distribution: 'oracle'
          java-version: 17.0.11
          check-latest: false
          cache: 'maven'

mahabaleshwars · 2024-07-17T11:07:31Z

Hi @Ivanuil, I've attempted to reproduce the issue but I didn't encounter the problem you described. Could you provide a public repository where the issue is reproducible?

Ivanuil · 2024-07-18T19:25:10Z

Hi @Ivanuil, I've attempted to reproduce the issue but I didn't encounter the problem you described. Could you provide a public repository where the issue is reproducible?

Sure, repo: https://github.com/Ivanuil/FriendAlertBot
action: https://github.com/Ivanuil/FriendAlertBot/actions/runs/9856712473

mahabaleshwars · 2024-07-31T09:09:20Z

Hello @Ivanuil,

I have forked the repository ""FriendAlertBot"" but am still unable to reproduce the issue.
For your reference, I am sharing my workflow: workflow link
I noticed that the cache is not visible in your repository, whereas it is being saved in my repository and setup-java is using it for the next run.

aparnajyothi-y · 2024-08-14T15:05:00Z

Hello @Ivanuil, Please let us know if you still have any concerns on the above comment.

Ivanuil · 2024-08-14T17:23:15Z

the cache is not visible in your repository

Thank you for the info. Could you give me some advice on how to set up cahche in my reposityory so that everything works? :)

mahabaleshwars · 2024-08-30T05:51:25Z

Hello @Ivanuil,

Could you enable debugging in your repository by adding the following secrets:
ACTIONS_RUNNER_DEBUG: true and ACTIONS_STEP_DEBUG: true ?

Additionally, could you share your runner configuration file and its log so we can examine if it is contributing to this issue?

mahabaleshwars · 2024-09-12T12:46:53Z

Hello @Ivanuil, were you able to debug with the commands mentioned above?
Please share the logs that will help us identify the root cause more effectively.

mahabaleshwars · 2024-09-25T04:49:11Z

Hello @Ivanuil,

Caching Mechanisms for Java in GitHub Actions

Dependency Caching
Method: Cache dependencies using the actions/cache action. When to Use: Useful for caching Maven or Gradle dependencies to speed up subsequent builds.

Maven Dependency Caching: workflow
Gradle Dependency Caching: workflow

Tool Caching
Method: Cache the Java distribution itself. When to Use: Reduces the time to download and install the Java version on each run.
Example: Cache AdoptOpenJDK

name: Cache AdoptOpenJDK
uses: actions/cache@v2
with:
path: ~/.sdkman/candidates/java
key: ${{ runner.os }}-adoptopenjdk-${{ hashFiles('**/sdkmanrc') }}
restore-keys: |
${{ runner.os }}-adoptopenjdk-

Build Caching
Method: Cache the actual build outputs. When to Use: Speeds up the build process by reusing previous build outputs.
Example: Cache Build Output

name: Cache Build Output
uses: actions/cache@v2
with:
path: build
key: ${{ runner.os }}-build-${{ hashFiles('**/*') }}
restore-keys: |
${{ runner.os }}-build-

Each method has its own use cases and benefits, such as reducing build times and improving efficiency. Choose the appropriate method based on which part of the build process you want to cache.

Maven Caching
With Maven, you can cache the local Maven repository (~/.m2/repository). This saves the downloaded dependencies across different workflow runs.
Example: workflow
Gradle Caching
With Gradle, you can cache the Gradle user home directory (~/.gradle/caches). This saves the downloaded dependencies and other build artifacts across different workflow runs.
Example: workflow
SBT Caching with setup-java
Sample Workflow for SBT Caching:
Example: workflow

Comparison

actions/cache: More flexible, allowing for custom paths and keys, which can be optimized for specific project structures.
cache (maven/gradle/sbt): Automatically handles standard paths, potentially resulting in quicker cache restores.

Choose the appropriate caching mechanism based on your project's needs and the level of customization required.

mahabaleshwars · 2024-10-09T06:39:14Z

Hello @Ivanuil, We are closing this issue now. Please feel free to reach out to us for any other concerns.

Ivanuil added bug Something isn't working needs triage labels May 9, 2024

aparnajyothi-y removed the needs triage label May 10, 2024

mahabaleshwars self-assigned this May 15, 2024

Ivanuil closed this as completed Jun 14, 2024

Ivanuil reopened this Jul 11, 2024

mahabaleshwars closed this as completed Oct 9, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[email protected] redownloading jdk on every run #632

[email protected] redownloading jdk on every run #632

Ivanuil commented May 9, 2024

aparnajyothi-y commented May 10, 2024

Sebagai commented May 15, 2024

abelmatos commented May 24, 2024 •

edited

Loading

oscar-b commented May 31, 2024

oscar-b commented May 31, 2024

Ivanuil commented May 31, 2024

mahabaleshwars commented Jun 3, 2024

oscar-b commented Jun 3, 2024

mahabaleshwars commented Jun 14, 2024

oscar-b commented Jun 14, 2024

LetapF commented Jul 11, 2024

Ivanuil commented Jul 11, 2024

Ivanuil commented Jul 11, 2024

mahabaleshwars commented Jul 17, 2024

Ivanuil commented Jul 18, 2024

mahabaleshwars commented Jul 31, 2024

aparnajyothi-y commented Aug 14, 2024

Ivanuil commented Aug 14, 2024

mahabaleshwars commented Aug 30, 2024

mahabaleshwars commented Sep 12, 2024

mahabaleshwars commented Sep 25, 2024

mahabaleshwars commented Oct 9, 2024

[email protected] redownloading jdk on every run #632

[email protected] redownloading jdk on every run #632

Comments

Ivanuil commented May 9, 2024

aparnajyothi-y commented May 10, 2024

Sebagai commented May 15, 2024

abelmatos commented May 24, 2024 • edited Loading

oscar-b commented May 31, 2024

oscar-b commented May 31, 2024

Ivanuil commented May 31, 2024

mahabaleshwars commented Jun 3, 2024

oscar-b commented Jun 3, 2024

mahabaleshwars commented Jun 14, 2024

oscar-b commented Jun 14, 2024

LetapF commented Jul 11, 2024

Ivanuil commented Jul 11, 2024

Ivanuil commented Jul 11, 2024

mahabaleshwars commented Jul 17, 2024

Ivanuil commented Jul 18, 2024

mahabaleshwars commented Jul 31, 2024

aparnajyothi-y commented Aug 14, 2024

Ivanuil commented Aug 14, 2024

mahabaleshwars commented Aug 30, 2024

mahabaleshwars commented Sep 12, 2024

mahabaleshwars commented Sep 25, 2024

mahabaleshwars commented Oct 9, 2024

abelmatos commented May 24, 2024 •

edited

Loading