[Feature Request]: Authentication of gemini pro models using service account file on google cloud

[ghost]

Is your feature request related to a problem? Please describe.

All enterprise cloud projects use service account file keys to authenticate. Autogen llm configuration does not support the service account file keys at the moment, rather authenticates with api keys. Pleaase make this feature available.

Describe the solution you'd like

Autogen assistant and Userproxy agents to support LLM configurations using service account files.

Additional context

No response

[ekzhu]

Thanks! If you know how to do this, could you submit a PR?

[ghost]

Hello team, is there any update on this feature?

[ekzhu]

cc @BeibinLi

[BeibinLi]

@sadiqgpasha89 Thanks for this feature request. Since I don't have access to any enterprise account, do you have any pointers (links, documentations, etc.) on how to use the service account file keys from enterprise? Thanks ahead~

[ghost]

Hello All,Kindly find the attached doc link for service account keys: https://cloud.google.com/docs/authentication#service-accountsAnd https://cloud.google.com/docs/authentication/provide-credentials-adc#local-keyRegards,Sadiq PashaSent from my iPhoneOn 6 May 2024, at 22:28, Beibin Li ***@***.***> wrote: @sadiqgpasha89 Thanks for this feature request. Since I don't have access to any enterprise account, do you have any pointers (links, documentations, etc.) on how to use the service account file keys from enterprise? Thanks ahead~ —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[ekzhu]

@sadiqgpasha89 this is a difficult one for us to do as we don't have access to the account. We are replying on the open-source community for this one.

[arjun-g]

Gemini is available as part of 2 separate offerings of Google - Google AI and Vertex AI. Vertex AI is available for enterprise as part of the Google Cloud with some additional capabilities. (https://medium.com/@martareyessuarez25/what-are-vertex-ai-and-google-ai-studio-the-artificial-intelligence-development-tools-that-already-666bdfe85dbd#:~:text=Difference%20between%20Vertex%20AI%20and%20Google%20AI%20Studio)

The GeminiClient available in AutoGen can only support Google AI's Gemini as Vertex AI use different python library "vertexai". I have built a seperate VertexClient for my use. If I want to give PR should I build it as a seperate client or merge the functionalities with the existing GeminiClient ?

[sonichi]

Gemini is available as part of 2 separate offerings of Google - Google AI and Vertex AI. Vertex AI is available for enterprise as part of the Google Cloud with some additional capabilities. (https://medium.com/@martareyessuarez25/what-are-vertex-ai-and-google-ai-studio-the-artificial-intelligence-development-tools-that-already-666bdfe85dbd#:~:text=Difference%20between%20Vertex%20AI%20and%20Google%20AI%20Studio)

The GeminiClient available in AutoGen can only support Google AI's Gemini as Vertex AI use different python library "vertexai". I have built a seperate VertexClient for my use. If I want to give PR should I build it as a seperate client or merge the functionalities with the existing GeminiClient ?

@BeibinLi @marklysze do you have a suggestion?

[marklysze]

Looking at the migration steps from Gemini AI to Vertex AI, they do seem to be fairly similar, main difference is authentication.

However, if @arjun-g has already built a library, it may be more straightforward to have two separate clients. For a developer to change their code base from GeminiClient to VertexClient it should be simple.

[arjun-g]

I have a pending PR #2793. I will raise a PR for VertexClient once this is approved.

[luxzoli]

I have created a PR which solves this issue as well: 2805

[BeibinLi]

@luxzoli Thanks for your great PR! I will take a look at it.
@arjun-g Please also take a look at luxzoli's PR. For your function calling PR, can you add a few test cases, similar to luxzoli's?

[luxzoli]

@BeibinLi thanks a lot! Meanwhile I have added an additional configuration parameters for the compute regions in my PR 2805, when using Gemini via VertexAI and a service account key file (or simply google default auth even with personal Google Cloud account), which can also be an important enterprise requirements.