Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gradle parallel build support #1001

Open
czp3009 opened this issue Aug 8, 2022 · 1 comment
Open

Gradle parallel build support #1001

czp3009 opened this issue Aug 8, 2022 · 1 comment

Comments

@czp3009
Copy link

czp3009 commented Aug 8, 2022

When building project with multiple sub-projects, gradle parallel builds are key to improving CI speed.

In this case, gradle will try to build all sub-projects at the same time.

If there are multiple sub-projects to deploy (multiple services), these sub-projects will all have the task downloadCloudSdk.

This is a file download task, if gradle start downloading at the same time, the task will fail.

fail message:

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':default-service:downloadCloudSdk'.
> java.nio.file.FileAlreadyExistsException: /root/.cache/google-cloud-tools-java/managed-cloud-sdk/downloads/google-cloud-sdk.tar.gz

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':example-service:downloadCloudSdk'.
> java.nio.file.FileAlreadyExistsException: /root/.cache/google-cloud-tools-java/managed-cloud-sdk/downloads/google-cloud-sdk.tar.gz

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

There are several workaround, but none of them are perfect:

  1. Not using parallel builds. This results in very slow builds, especially in large projects.
  2. Use an image with gcloud already installed as a CI build image. This may result in the gcloud not being up to date, and the build image containing the compiler needed for the build, which may not been created by the CI user.

For tasks that require requests to external resources, such as the file system, gradle provides an API to avoid execute multi time: https://docs.gradle.org/current/userguide/build_services.html

I don't know much about gradle plugins, can you please see if you can use build service to solve this problem.

@elefeint
Copy link
Contributor

elefeint commented Aug 9, 2022

Thanks! We'll keep it open as an enhancement request.

@JoeWang1127 JoeWang1127 transferred this issue from GoogleCloudPlatform/app-gradle-plugin Jan 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants