Skip to content

Latest commit

 

History

History
151 lines (100 loc) · 6.23 KB

CONNECTING_TO_AICORE.md

File metadata and controls

151 lines (100 loc) · 6.23 KB
Raw

Connecting to AI Core

Table of Contents

The AI SDK uses the Destination concept of the SAP Cloud SDK to connect to AI Core. This allows for a seamless integration in BTP and offers a wide variety of managing the credentials for the connection.

Using a Service Binding

By default, the AI SDK will look for an aicore service binding in the environment of the application.

When running on SAP BTP, create an instance of the AI Core service and bind it to your application. The AI SDK will automatically detect the service binding and use it to connect to AI Core.

If no service binding could be found (and no other destination has been given), the AI SDK will throw an exception:

Could not find any matching service bindings for service identifier 'aicore'

To run your code outside SAP BTP (e.g. locally), you will have to provide the credentials manually.

Providing a Service Binding Locally

For local development, you can provide the service credentials locally in various ways.

Using CAP Hybrid Testing

When developing a CAP application, you can use the hybrid testing approach. Assuming your AI Core service instance is called aicore, the following command runs a Maven command with the credentials of the service instance:

cds bind --to aicore --exec mvn spring-boot:run

Using the AICORE_SERVICE_KEY Environment Variable

You can also provide the service credentials in the AICORE_SERVICE_KEY environment variable. First, create a service key for your AI Core service instance. Copy the resulting JSON object.

Then set the environment variable AICORE_SERVICE_KEY to be defined for however you want to run your code. How exactly to set this depends on how you run your code (e.g. via IDE, command line etc.) and potentially on your operating system.

Here are three examples of how this can be done:

  1. Using a `.env` file

    Create a .env file in the working directory from which you run your code. Add the following line:

    AICORE_SERVICE_KEY={ "clientid": "...", "clientsecret": "...", "url": "...", "serviceurls": { "AI_API_URL": "..." } }

    [!IMPORTANT] The value of AICORE_SERVICE_KEY must be a single line, so remove any line breaks from the service key JSON.

  2. Using the command line on Mac OS

    Run your code with the following command:

    export AICORE_SERVICE_KEY='{ "clientid": "...", "clientsecret": "...", "url": "...", "serviceurls": { "AI_API_URL": "..." } }'
mvn ...
  3. Using PowerShell on Windows

    Run your code with the following command:

    $env:AICORE_SERVICE_KEY='{ "clientid": "...", "clientsecret": "...", "url": "...", "serviceurls": { "AI_API_URL": "..." } }'
mvn ...
  4. Using an IntelliJ run configuration

    In IntelliJ, go to Run > Edit Configurations... > Environment variables. Add a new environment variable with the name AICORE_SERVICE_KEY and paste the service key as value.

    For more information, see the official IntelliJ documentation.

Using a Destination from the BTP Destination Service

You can define a destination in the BTP Destination Service and use that to connect to AI Core.

How to create a Destination in the SAP BTP Cockpit

  1. Create a service key for your AI Core service instance.

  2. Create a new Destination in the SAP BTP Cockpit with the following properties:

  • Name: my-aicore

  • Type: HTTP

  • URL: [serviceurls.AI_API_URL]

  • Proxy Type: Internet

  • Authentication: OAuth2ClientCredentials

  • Client ID: [clientid]

  • Client Secret: [clientsecret]

  • Token Service URL Type: Dedicated

  • Token Service URL: [url]

    Fill in the values for URL, client ID, client secret, and token service URL from the service key JSON.

To use the destination, ensure you have created an instance of the BTP Destination Service and bound it to your application.

Destination destination = DestinationAccessor.getDestination("my-aicore").asHttp();
AiCoreService aiCoreService = new AiCoreService().withBaseDestination(destination);

Now you can use the AiCoreService as follows:

// To create an OpenAiClient:
var openAiDestination = aiCoreService.getInferenceDestination().forModel(OpenAiModel.TEXT_EMBEDDING_3_LARGE);
var client = OpenAiClient.withCustomDestination(openAiDestination);

// To create an OrchestrationClient:
var orchestrationDestination = aiCoreService.getInferenceDestination().forScenario("orchestration");
var client = new OrchestrationClient(orchestrationDestination);

// To create any of the com.sap.ai.sdk.core.client.* classes (e.g. DeploymentApi):
new DeploymentApi(aiCoreService);

Important

The destination obtained from BTP destination service will expire once the contained OAuth2 token expires. Please run the above code for each request to ensure the destination is up-to-date. Destinations are cached by default, so this does not come with a performance penalty. To learn more, see the SAP Cloud SDK documentation.