Variables created in preprovision hook with azd env set are not available in the same script

[dayland]

Output from azd version
azd version 1.11.0 (commit 5b92e06)

Describe the bug
I have set up a shell script to run in preprovision hook.. when running this script I am unable to use the variables I have set using azd env set for conditional checks in the same script without manually loading the .azure/<environment>/.env in.

To Reproduce
Steps to reproduce the behavior...
Assume you have the following azure.yaml:

name: mytestapp
metadata:
    template: [email protected]
infra:
  provider: terraform
hooks:
  preprovision:
    posix:
      shell: sh
      continueOnError: false
      interactive: true
      run: ./scripts/prepare-tf-variables.sh
services:
    web:
        project: ./app/backend
        host: appservice
        language: py

my prepare-tf-variables.sh script is:

#!/bin/sh

# Set up all the required environment variables for the deployment
# Default values - you can override these using "azd env set"
# -------------------------------------------------------------------------------------------------------
if [ -z $AZURE_ENVIRONMENT ]; then
    echo "Azure Environment not set, defaulting to AzureCloud"
    azd env set AZURE_ENVIRONMENT AzureCloud
else
    echo "Azure Environment: $AZURE_ENVIRONMENT"
fi

# Azure Cloud specific variables. These are the default values for AzureCloud and AzureUSGovernment
# -------------------------------------------------------------------------------------------------------
if [ $AZURE_ENVIRONMENT = "AzureCloud" ]; then
    azd env set arm_template_schema_mgmt_api "https://schema.management.azure.com"
    azd env set azure_portal_domain "https://portal.azure.com"
elif [ $AZURE_ENVIRONMENT = "AzureUSGovernment" ]; then
    azd env set arm_template_schema_mgmt_api "https://schema.management.usgovcloudapi.net"
    azd env set azure_portal_domain "https://portal.azure.us"
fi

I want the user to only have to specify the AZURE_ENVIRONMENT variable and then conditionally set additional variables based on it. However, this does not work because AZURE_ENVIRONMENT is not recognized in the if condition. I have had to modify the script to do the following for the conditional parts to work...

#!/bin/sh

# Set up all the required environment variables for the deployment
# Default values - you can override these using "azd env set"
# -------------------------------------------------------------------------------------------------------
if [ -z $AZURE_ENVIRONMENT ]; then
    echo "Azure Environment not set, defaulting to AzureCloud"
    azd env set AZURE_ENVIRONMENT AzureCloud
else
    echo "Azure Environment: $AZURE_ENVIRONMENT"
fi

# Get the defaults set above, as azd env set doesn't actively load the variable into memory
eval $(azd env get-values | sed 's/^/export /')

# Azure Cloud specific variables. These are the default values for AzureCloud and AzureUSGovernment
# -------------------------------------------------------------------------------------------------------
if [ $AZURE_ENVIRONMENT = "AzureCloud" ]; then
    azd env set arm_template_schema_mgmt_api "https://schema.management.azure.com"
    azd env set azure_portal_domain "https://portal.azure.com"
elif [ $AZURE_ENVIRONMENT = "AzureUSGovernment" ]; then
    azd env set arm_template_schema_mgmt_api "https://schema.management.usgovcloudapi.net"
    azd env set azure_portal_domain "https://portal.azure.us"
fi

Expected behavior
I would expect that as I add values to the AZD environment using azd env set they would be available for immediate use.

Environment
Information on your environment:
* Linux Debian devcontainer [python-3.12-bookworm]

Additional context
Add any other context about the problem here.

[v-xuto]

We are following up on this issue. cc @rajeshkamal5050

[Menghua1]

@dayland Azd's underlying layer loads environment variables from .env file before executing hook script and then passes it as environment of shell script. Therefore, any changes made to environment variables during script execution will not be reflected in the running environment of script.
Currently, we have not found a better way to modify the azd code to solve this issue.

In addition, for the script file, you can also change it to the following:

#!/bin/sh

# Set up all the required environment variables for the deployment
# Default values - you can override these using "azd env set"
# -------------------------------------------------------------------------------------------------------
if [ -z "$AZURE_ENVIRONMENT" ]; then
    echo "Azure Environment not set, defaulting to AzureCloud"
    azd env set AZURE_ENVIRONMENT "AzureCloud"
    azd env set arm_template_schema_mgmt_api "https://schema.management.azure.com"
    azd env set azure_portal_domain "https://portal.azure.com"
elif [ "$AZURE_ENVIRONMENT" == "AzureUSGovernment" ]; then
    azd env set arm_template_schema_mgmt_api "https://schema.management.usgovcloudapi.net"
    azd env set azure_portal_domain "https://portal.azure.us"
elif [ "$AZURE_ENVIRONMENT" == "AzureCloud" ]; then
    azd env set arm_template_schema_mgmt_api "https://schema.management.azure.com"
    azd env set azure_portal_domain "https://portal.azure.com"
else
    echo "Azure Environment: $AZURE_ENVIRONMENT"
fi

[dayland]

Thanks for the follow up. I did find in the documentation where it shows an example of loading the ENV file into a script as needed, although not called out in the documentation itself. I guess this is more of a future consideration for enhancement that would align with assumptions that azd env set VAR is similar or equivalent to export VAR, even though they are not.

[Menghua1]

@dayland To clarify, the azd env set command only modifies the environment variables in the local .env file, and does not export the environment variables in the .env file to the hook's script environment.

When executing hooks, azd will first load environment variables from the .env file into the hooks environment, and then execute the hooks script content. After executing the hook script, the environment variables in the local .env file will be reloaded into the hooks again. During the script execution, the environment variables in the script will not change unless the current script ends.

We think this design is reasonable at present. If you have more ideas, we can also discuss.

[Menghua1]

@rajeshkamal5050 When setting environment variables using azd env set in hooks scripts, users expect these variables to take effect immediately in the same script, similar to how export VAR works.

We think the behavior is reasonable design. Do you think this is an enhancement issue?