cache size does not match keepBytes in buildkit.toml

[dyume99]

Hello, I found that the cache usage of buildkit was too high. When I used buildctl du to check, the size that could be reclaimed was 0, which does not match with the keeyBytes in my buildkit.toml.

buildctl du -v
-----
Reclaimable:	0B
Total:		539.53GB

my buildkit.toml is

[log]
  format = "text"

[grpc]
 address = [ "tcp://0.0.0.0:1234" ]
 debugAddress = "0.0.0.0:6066"

[history]
  maxAge = "48h"
  maxEntries = 10000

[worker.oci]
  enabled = true
  snapshotter = "auto"
  rootless = false
  noProcessSandbox = false
  gc = true
  cniPoolSize = 64

  # keep build cache in 12 hours
  [[worker.oci.gcpolicy]]
    filters = [ "type==source.local", "type==exec.cachemount", "type==source.git.checkout"]
    keepBytes = "200GB"
    keepDuration = "12h"

  # if previous policies were insufficient start deleting internal data to keep build cache under cap
  [[worker.oci.gcpolicy]]
    all = true
    keepBytes = "20%"

When I drilled down to see the exact cache usage, I found that all entries reclaimable was set to false:

ID:		ervg9n06wm02tjvzc9xsm2t4g
Parents:	6cl3mau09xcljkmtieaq83nmg;rnumhttyzaidr34enviasu400
Created at:	2024-12-03 09:27:07.444138637 +0000 UTC
Mutable:	false
Reclaimable:	false
Shared:		false
Size:		0B
Description:	[stage-2 3/6] LINK COPY --from=xxxx --chown=1000:1000 --link /a /b
Usage count:	1
Last used:	2024-12-03 09:29:13.815128022 +0000 UTC
Type:		regular

After I restarted the buildkit service, the reclaimable value changed to total and the total cache size gradually decreased

Reclaimable:	149.15GB
Total:		149.15GB

I'm wondering if this is something I have set something wrong. If I have set it wrong, how can I correct it? Thank you very much

[jedevc]

Hm, if reclaimable is set to false, it's an issue with the cache record still being marked as in-use I suspect. Nothing wrong with the cache policy.

Is the cache record in use before you restart the server? If the buildkit instance is processing a build that requires those records, then those records will not be marked as reclaimable.

[dyume99]

Hm, if reclaimable is set to false, it's an issue with the cache record still being marked as in-use I suspect. Nothing wrong with the cache policy.

Is the cache record in use before you restart the server? If the buildkit instance is processing a build that requires those records, then those records will not be marked as reclaimable.

I looked at som cache entries and found that they were created a few days ago, and their Last used was also a few days ago:

ID:		zgpsmf2icbiidnkews49v3nob
Created at:	2024-12-02 11:31:24.437966614 +0000 UTC
Mutable:	false
Reclaimable:	false
Shared:		false
Size:		10.18GB
Description:	[stage-3 3/7] COPY --from=stage-0 --chown=1000:1000 --link /opt/xxx /opt/xxx/
Usage count:	1
Last used:	2024-12-02 11:43:19.747273024 +0000 UTC
Type:		regular

Are some caches incorrectly marked as in use? And i want to know if there is a way to force buildkit to delete caches that exceed the threshold.
Thanks for your reply.

[tonistiigi]

Records should be reclaimable: false only if a build is running. If you see otherwise then please post reproduction steps.

[dyume99]

Records should be reclaimable: false only if a build is running. If you see otherwise then please post reproduction steps.

I didn't see any builds in progress, how can I confirm that the entries with reclaimable set to false are being used?