Share Kotlin DSL build results using remote build cache

[matejdro]

Expected Behavior

Store Kotlin DSL compilation results as a gradle build cache entry that can then be uploaded to remote build cache and shared between machines.

Current Behavior (optional)

Currently Kotlin DSL compilation results only get cached locally and cannot be easily transferred into new machine.

Context

On CI, every build starts with clean container/virtual machine (without existing .gradle folder). That means that even with 100% build cache coverage, first build on CI will always take a while, since it still needs to compile and apply all kts build files. It would be nice if, instead, those could be downloaded from the build cache like all other compiled files.

Obvious issue here is that settings.gradle.kts usually contains the configuration for the build cache, so that would still need to be compiled locally. But all other build files should be able to be downloaded from the build cache.

Related to #13510, gradle/kotlin-dsl-samples#902 and #9225

[cobexer]

Thank you for your interest in Gradle!

This is already the behavior. This issue will be closed.

---

If you see different behavior please add a reproducer to this issue.

[matejdro]

@cobexer Here is how to reproduce this:

1. Download https://docs.gradle.org/current/samples/sample_sharing_convention_plugins_with_build_logic.html Kotlin DSL sample
2. Run ./gradlew tasks --build-cache and wait for everything to compile and store into build cache.
3. Run ./gradlew tasks --scan and look at build scan under Performance -> Configuration. Compilation should be at 0.000s, as expected
4. Delete ~/.gradle/caches/8.5/kotlin-dsl folder and stop the daemon (to simulate running this on another machine where there is no kotlin dsl cache yet)
5. Run ./gradlew tasks --build-cache --scan and look at build scan under Performance -> Configuration. There should be zero script compilation, since scripts should be loaded from build cache:
   

in addition, build cache tab will not show any cache hits.

6. Try clearing kotlin-dsl again and run ./gradlew tasks --scan (without build cache this time) - compilation step will take similar amount of time than before, so cache was not used.

Now, of course this is a trivial example and that tiny compilation time does not affect total build time here (we have to use build scans to just notice the difference), but with large projects with lots of modules, this can make a sizeable difference.

[ov7a]

Did you configure the remote build cache? We don't see that in your reproducer steps.

[matejdro]

Above steps are for local build cache (it appears it behaves the same) to make it easier to reproduce (you can do it on every local machine). But you can just as easily use remote build cache instead by setting it up in the project's settings.gradle.kts after step 1.

[ov7a]

@matejdro Unfortunately, there is still not enough information to reproduce it.

---

As stated in our issue template, a minimal reproducible example is a must for us to be able to track down and fix your problem efficiently. Our available resources are severely limited, and we must be sure we are looking at the exact problem you are facing.

If we have a reproducer, we may be able also to suggest workarounds or ways to avoid the problem.

The ideal way to provide a reproducer is to leverage our reproducer template. You can also use Gradle Project Replicator to reproduce the structure of your project.

[matejdro]

I've outlined complete steps. What part did you get stuck on?

[ov7a]

As it was mentioned before, this scenario should work. It is covered by tests. So, please, provide a proper self-contained reproducer for the issue.

[matejdro]

Have you tried it?

I'm not sure how I could make it more self contained. Do you want me to make a shell script that would to all the steps automatically?

[ov7a]

@matejdro Please, use our reproducer template. It has a GitHub action to run. You can edit it so your shell script demonstrating the issue is executed

[matejdro]

Thanks, will try.

Although, above steps require user to manually check times on build scans. How can I automate build scan checks?

[ov7a]

@matejdro you can check Gradle's output (probably with more verbose logs level) instead of publishing a Scan

[matejdro]

@ov7a Here you go https://github.com/matejdro/kotlin-compilation-build-cache

[ov7a]

We investigated the reproducer you provided.

What happens here is expected behavior.

There is no local build cache for script compilation. There is only a local script compilation cache, which is shared for all projects on your machine. The location of this cache is $GRADLE_HOME$/caches/8.5/kotlin-dsl.

When you clear that folder, you effectively clear the whole local cache of your KTS compilation results. So it is expected that it will take longer to complete the Gradle build after you remove the cache.

With the remote cache enabled, the build will request the remote cache by a file's hash and unpack it if there is a hit. This of course excludes settings.gradle.kts where you define the remote cache.

[matejdro]

How can I make the reproducer for the remote build cache? I can't just add a random build cache server.

[ov7a]

You can use our docker image for that.

[matejdro]

Thanks for your help. @ov7a I have update the reproducer to use remote build cache and the discrepancy is still there.

[ov7a]

Thanks @matejdro!

---

This issue needs a decision from the team responsible for that area. They have been informed. Response time may vary.

[asodja]

Hey @matejdro, I took a look at the reproducer and it seems it's missing a flag to enable build cache. Build cache is by default disabled, so you have to enable it either by adding --build-cache to a command line or by adding org.gradle.caching=true to gradle.properties file even if you have settings for build cache in settings.gradle.kts file.

After that it looks like entries are loaded from cache on a second run when running:

rm -rf ~/.gradle/caches/8.5/kotlin-dsl .gradle/8.5  && ./gradlew help --no-daemon --build-cache --info --scan 

Also build scan reports:

Does that fix your issue or I missed something?

[matejdro]

My reproducer contains --build-cache on all gradle commands:

https://github.com/matejdro/kotlin-compilation-build-cache/actions/runs/7208272256/workflow

[asodja]

Oh yeah, didn't see it is used in the workflow, thanks. But I see that build cache is used when compiled kts files are removed.

So I think the problem is not that build cache not being used, but it might be build cache overhead, e.g. we have to unpack the result. We also do some extra work like instrumentation that is not stored in the build cache (should be fixed with #27721), although I tested it and it seems there is still difference.

Is this a problem just for your CI setup or also locally? I would expect that on CI downloading dependencies with clean container is the slowest operation. Or do you cache dependencies somehow?

[matejdro]

We:

• Copy over ~/.gradle/wrapper folder
• Copy over ~/.gradle/caches/modules-2 folder (which contains all dependencies)
• Cache build steps using remote build cache that is located on the premises (so very low network latency)

That means that when there is a very minor change (so not much needs to build), configuration step is pretty big percentage of a build time.

Our ultimate goal is also somehow sharing configuration cache (#13510), but until then we want to make configuration as fast as possible and as far as I can see, building kotlin files seems to consume big chunk of that.

[asodja]

I see.

We miss some documentation what is safe to cache on ephemeral CI. If it helps you in any way this is what gradle/gradle-build-action is caching: cache-extract-entries.ts#L354-L361. This build action can also store cc cache few lines below, and I think this is the closest to the cc sharing atm (but I am not familiar with that part of code).

[matejdro]

thanks for the link. I've tried caching kotlin-dsl folder, but it did not seem to make any meaningful difference (which might point to the fact that the issue is not actually kotlin DSL and slowdown is caused by something else as you've mentioned above).

Configuration cache caching is further complicated by the fact that it is now encrypted. #27055 will seemingly help, but as far as I can see, it has not been included in a recent release yet.

[asodja]

If you want fast startups you probably want to also copy cache for:

• caches/*/generated-gradle-jars/*.jar -> we generate some API jars on the first run for Kotlin DSL and this is the location where we cache them
• before 8.6: caches/jars-*/* and after 8.6 also caches/transforms-*/* -> we do some bytecode instrumentation on first run for plugin jars so we can detect configuration cache violations and this is the location where we cache them

[asodja]

I will close this issue now since it seems build cache works for Kotlin DSL.

If there is any other issue with Gradle feel free to open a new issue, or if you have any other question you can also ask on Slack or forum, see https://gradle.org/help.