DefaultAzureCredential in azidentity v1.8.0 rejects Azure Container Instance's IMDS

[bianyifan]

Bug Report

• What happened:
  When using DefaultAzureCredential in github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.8.0 on Azure Container Instance, it throws error ManagedIdentityCredential: unexpected response to IMDS probe.
  Azure Container Instance implements same IMDS token request protocol, but its response to SDK's probe request is different with the authentic IMDS. Authentic IMDS returns {"error":"invalid_request","error_description":"Required metadata header not specified"}, but ACI's mock IMDS only returns plain string Required metadata header not specified or not correct.
  azidentity v1.7.0 works. Checking whether the response to probe request is a valid JSON is added in DefaultAzureCredential continues after an unexpected IMDS response #23273.
• What did you expect or want to happen?
  Azure SDK should work on Azure Container Instance, by accepting the probe response to be a string, or... coordinate with ACI team so ACI also returns a JSON for probe request.
• How can we reproduce it?
  Deploy this bicep template to get an ACI instance which simulates the probe request sent by azidentity SDK:

resource aci 'Microsoft.ContainerInstance/containerGroups@2024-05-01-preview' = {
  name: 'test'
  location: resourceGroup().location
  identity: { type: 'SystemAssigned' }
  properties: {
    sku: 'Standard'
    containers: [
      {
        name: 'test'
        properties: {
          image: 'mcr.microsoft.com/azurelinux/base/core:3.0'
          command: ['curl', '-v', 'http://169.254.169.254/metadata/identity/oauth2/token']
          resources: { requests: { cpu: 1, memoryInGB: 1 } }
        }
      }
    ]
    osType: 'Linux'
    restartPolicy: 'Never'
  }
}

Output:
In comparison, this is the response of an authentic IMDS server, by running the same curl command on a VM:

root [ / ]# curl -v http://169.254.169.254/metadata/identity/oauth2/token
*   Trying 169.254.169.254:80...
* Connected to 169.254.169.254 (169.254.169.254) port 80
> GET /metadata/identity/oauth2/token HTTP/1.1
> Host: 169.254.169.254
> User-Agent: curl/8.8.0
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 400 Bad Request
< Content-Type: application/json; charset=utf-8
< Server: IMDS/150.870.65.1544
< x-ms-request-id: 3bca7686-9e4e-4911-8092-3571a9ce7ebb
< Date: Wed, 25 Dec 2024 06:21:59 GMT
< Content-Length: 88
<
* Connection #0 to host 169.254.169.254 left intact
{"error":"invalid_request","error_description":"Required metadata header not specified"}

[chlowell]

Thanks for reporting this! I don't see a good way to make DefaultAzureCredential do the right thing in this scenario. Reverting #23273 is easy but would regress other applications. Alternatively, we could add a special case for ACI, but that would use a fragile mechanism like matching the error string because we can't identify ACI at runtime. I'll give this some more thought and discuss it with the rest of the team after the holidays. In the meantime, you can avoid this by using ManagedIdentityCredential instead.