[Issue] [aspire] azd infra synth ignored by azd deploy in azdo ci/cd pipeline

[Bpflugrad]

• Make sure you've installed the latest version using instructions in the wiki

Output from azd version
Run azd version and copy and paste the output here:
azd version 1.9.0 (commit 651394c3ddcfadff194d177f8b0ddf06fe3752bf)

Describe the bug
azd infra synth has been executed, and the generated templates in the AppHost project are committed and pushed to the build branch, but azd deploy generates the infra itself, ignoring any manual changes.

To Reproduce

• Create a new Aspire Starter App.
• Run azd init
• Add state: remote: backend for remote configuration.
• Add .azdo/pipelines/azure-dev.yaml
• Run azd pipeline config
• Permission created azconnection service principal with Storage Blob Contributor on your remote configuration blob storage.
• Update project to use AddConnectionString in AppHost.
• Use AddReference on the ApiService resource.
• Add Aspire.Azure.Storage.Blobs to ApiService.
• Run azd env set AZURE_BLOB_STORAGE_CONNECTION "" to add string to env.
• Run azd infra synth
• Edit apiservice.tmpl.yaml to replace '{{ securedParameter "BlobStorageConnection" }}' with {{ .Env.AZURE_BLOB_STORAGE_CONNECTION }}
• Edit main.bicep to add output AZURE_BLOB_STORAGE_CONNECTION string = BlobStorageConnection
• Push to trigger build.

Build will fail at azd deploy with message:
ERROR: failed deploying service 'apiservice': failing invoking action 'deploy', failed executing template file: template: containerApp.tmpl.yaml:21:19: executing "containerApp.tmpl.yaml" at <securedParameter "BlobStorageConnection">: error calling securedParameter: parameter BlobStorageConnection not found

This indicates that securedParameter "BlobStorageConnection" is still in the template. Note that containerApp.tmpl.yaml is the file name, which was the file name from azd 1.8.2.

Log message indicates that new template was generated, not the one created by azd infra synth:
2024/05/09 23:45:47 service_target_dotnet_containerapp.go:195: generating container app manifest from /home/vsts/work/1/s/.\AspireInfraExample.AppHost\AspireInfraExample.AppHost.csproj for project apiservice

Expected behavior
Infra created by azd infra synth should be used.

Environment
Information on your environment:
- dotnet 8.0.300-preview.24203.14
- Version 17.10.0 Preview 7.0
- azd version 1.9.0 (commit 651394c)

Additional context
Complete azure.yaml:

# yaml-language-server: $schema=https://raw.githubusercontent.com/Azure/azure-dev/main/schemas/v1.0/azure.yaml.json

name: AspireInfraExample
services:  
  app:
    language: dotnet
    project: .\AspireInfraExample.AppHost\AspireInfraExample.AppHost.csproj
    host: containerapp
state:
  remote:
    backend: AzureBlobStorage
    config:
      accountName: azdconfiguration
      containerName: aspireinfraexample

Complete azure-dev.yaml:

trigger:
- master
- staging

pool:
  vmImage: ubuntu-latest

steps:
  - task: DotNetCoreCLI@2
    displayName: Install Aspire Workload with Preview
    inputs:
      command: custom
      custom: 'workload'
      arguments: 'install aspire --include-previews --source https://api.nuget.org/v3/index.json'

  - task: setup-azd@0 
    displayName: Install azd

  - pwsh: |
      azd config set auth.useAzCliAuth "true"
    displayName: Configure AZD to Use AZ CLI Authentication.

  - task: AzureCLI@2
    displayName: Provision Infrastructure
    inputs:
      azureSubscription: azconnection
      scriptType: bash
      scriptLocation: inlineScript
      inlineScript: |
        azd provision --no-prompt
    env:
      AZURE_SUBSCRIPTION_ID: $(AZURE_SUBSCRIPTION_ID)
      AZURE_ENV_NAME: $(AZURE_ENV_NAME)
      AZURE_LOCATION: $(AZURE_LOCATION)

  - task: AzureCLI@2
    displayName: Deploy Application
    inputs:
      azureSubscription: azconnection
      scriptType: bash
      scriptLocation: inlineScript
      inlineScript: |
        azd deploy --no-prompt --debug
    env:
      AZURE_SUBSCRIPTION_ID: $(AZURE_SUBSCRIPTION_ID)
      AZURE_ENV_NAME: $(AZURE_ENV_NAME)
      AZURE_LOCATION: $(AZURE_LOCATION)

[Bpflugrad]

I was trying to follow the workaround for azd deploy not accepting secured parameters in *.tmpl.yaml the same way bicep does described in #3597.

[weikanglim]

@Bpflugrad To confirm your findings:

1. Does local azd deploy work? From the issue, it seems like this is only happening inside the CI pipeline?
2. If 1 is true, I wonder if the old version if azd is still being installed. The debug logs should have in the first line of the log, it looks something like
   2024/05/09 18:43:55 main.go:59: azd version: 1.8.0 (commit 8246323c2472148288be4b3cbc3c424bd046b985)

[vhvb1989]

same as #3850

please see: #3850 (comment) .

[microsoft-github-policy-service]

Hi @Bpflugrad. Thank you for opening this issue and giving us the opportunity to assist. We believe that this has been addressed. If you feel that further discussion is needed, please add a comment with the text “/unresolve” to remove the “issue-addressed” label and continue the conversation.

[Bpflugrad]

Hi @vhvb1989,

While my goal to get the secured value to work in deployment pipelines is the same, this seems like a distinct issue where the azd infra synth output is not honored.
I would have expected output like:
2024/05/09 23:45:47 service_target_dotnet_containerapp.go:186: using container app manifest from /home/vsts/work/1/s/.\AspireInfraExample.AppHost\infra\apiservice.tmpl.yaml
but get:
2024/05/09 23:45:47 service_target_dotnet_containerapp.go:195: generating container app manifest from /home/vsts/work/1/s/.\AspireInfraExample.AppHost\AspireInfraExample.AppHost.csproj for project apiservice

Doesn't seem like a duplicate request.

Big thanks for your info on #3850 though! Didn't mean for this to be terse.

[vhvb1989]

@Bpflugrad , thank you for your follow up.

There is indeed, an issue.
When you run azd init on Windows, the path for the AppHost is persisted in azure.yaml with \ , like project: .\sp.AppHost\sp.AppHost.csproj

Then, when azd runs on CI/CD, if the pipelines uses linux, azd is not transforming the path to a linux path, so it is not finding the /infra folder in the appHost and the synthetized yaml files; so it is generating them from the project.

Seems like a regression after moving the yaml files from each individual service path, to an infra folder in the AppHost. Since you were manually updating the yaml files before and that was working for you, I think this is a regression on azd. @ellismg

[ellismg]

There is indeed, an issue.
When you run azd init on Windows, the path for the AppHost is persisted in azure.yaml with \ , like project: .\sp.AppHost\sp.AppHost.csproj

azd goes out of its way to do this on Windows, on the assumption that using native path separators would feel more natural, instead of us just always generating partial paths like ./sp.AppHost/sp.AppHost.csproj even on Windows.

But it does feel like this decision doesn't play well with the fact that even if you are on Windows, CI is likely Linux.

I think we should just ensure we allow using / as the path separator for partial paths, even on Windows, and then have our generated azure.yaml use / here.

This feels in line with the aspire manifest, which uses / as the path separator, even when generated on Windows.

filepath.ToSlash will likely be useful for us here.

[rajeshkamal5050]

cc @Petermarcu @coolcsh