Skip to content

Gradle parallel build support #1001

@czp3009

Description

@czp3009

When building project with multiple sub-projects, gradle parallel builds are key to improving CI speed.

In this case, gradle will try to build all sub-projects at the same time.

If there are multiple sub-projects to deploy (multiple services), these sub-projects will all have the task downloadCloudSdk.

This is a file download task, if gradle start downloading at the same time, the task will fail.

fail message:

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':default-service:downloadCloudSdk'.
> java.nio.file.FileAlreadyExistsException: /root/.cache/google-cloud-tools-java/managed-cloud-sdk/downloads/google-cloud-sdk.tar.gz

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':example-service:downloadCloudSdk'.
> java.nio.file.FileAlreadyExistsException: /root/.cache/google-cloud-tools-java/managed-cloud-sdk/downloads/google-cloud-sdk.tar.gz

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

There are several workaround, but none of them are perfect:

  1. Not using parallel builds. This results in very slow builds, especially in large projects.
  2. Use an image with gcloud already installed as a CI build image. This may result in the gcloud not being up to date, and the build image containing the compiler needed for the build, which may not been created by the CI user.

For tasks that require requests to external resources, such as the file system, gradle provides an API to avoid execute multi time: https://docs.gradle.org/current/userguide/build_services.html

I don't know much about gradle plugins, can you please see if you can use build service to solve this problem.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions