support building private repos

[avsm]

Is it possible to plumb through private GitHub repo building using ocluster at the moment?

I tried modifying (a local) ocurrent-deployer with a GitHub app that has access to a private repository. That lets the deployer query the repository, but then the actual build when submitted to a ocluster pool fails to clone the repository.

As far as I can tell, the best way to do that is to pass the GH access token through to the git clone in the clone URL. This probably needs support in ocluster to hide the token from the logs...

[dra27]

@tmcgilchrist - had you experimented with this a while back? The major concern - at least for the “main” cluster, is that the trust model for the workers changes as soon as we have them receiving secrets.

[tmcgilchrist]

It’s possible to get it working I had a prototype working last year. There’s a feature request on ocaml-ci for it as well ocurrent/ocaml-ci#323 The two concerns were the security of private repos (as David mentioned) we cache everything in a shared directory and don’t really care about sharing git repos, and no one was strongly asking for it at the time. What is your use case for it?

…

On Sat, 14 May 2022 at 8:52 pm, David Allsopp ***@***.***> wrote: @tmcgilchrist <https://github.com/tmcgilchrist> - had you experimented with this a while back? The major concern - at least for the “main” cluster, is that the trust model for the workers changes as soon as we have them receiving secrets. — Reply to this email directly, view it on GitHub <#179 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABJXOPAFFFLQ2SDKMDJBT3VJ7Y6FANCNFSM5V55D6IA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[avsm]

My usecase is pretty straightforward: got a bunch of private repos that need building and deploying. The builds are pushed to the github container registry (which has the advantage that the access token used can both clone the repository and push to the ghcr.io container registry with the same access privs as the source repository).

The deployer works great; the only hitch is that I can't farm out the builds to an ocluster instance.

[tmcgilchrist]

If you trust the ocluster instance then we could just plumb through the right information and get it working, however that is somewhat of a hack but perhaps is enough in this case. For my original use case, I just wanted to be able to build private GitHub repos and didn't care too much about whether the source was available on the cluster. The only security was not having the repo as a public repo on GitHub.

A better solution would be making the private git repos truely private, maybe by having a private git cache store per GH user or some other segregation of the repositories. We haven't really investigated this area of the design space.

[avsm]

Agreed that plumbing through the auth info would be enough in the first instance. Can do more sophisticated ocluster reservation later.