External Prompting (#3364)

This change enables `azd` to delegate prompting
behavior to an external host, modeled the same way it delegates
authorization behavior.

VS is the current consumer of external prompting, with VSCode 
soon to follow.

This change also introduces a new concept of a `PromptDialog`,
in which prompts are batched into a dialog. This is currently in-use
by VS integration.

Fixes #3591

cli/azd/cmd/container.go

@@ -122,7 +122,7 @@ func registerCommonDependencies(container *ioc.NestedContainer) {
 			Stdin:  cmd.InOrStdin(),
 			Stdout: cmd.OutOrStdout(),
 			Stderr: cmd.ErrOrStderr(),
-		}, formatter)
+		}, formatter, nil)
 	})
 
 	container.MustRegisterSingleton(

cli/azd/docs/external-prompting.md

@@ -0,0 +1,95 @@
+# External Prompting
+
+## Problem
+
+During operations, `azd` may need to prompt the user for information. For example, during `init` for an Aspire application, we prompt the user to select which services should be exposed to the Internet. The first time `azd provision` is run for an environment, we ask the user to select an subscription and location. In addition, the IaC we provision for an application may have parameters which `azd` needs to prompt the user for.
+
+Today, this prompting happens via the terminal - we have a set of methods that can be called on `input.Console` that allow different types of prompts:
+
+- `Prompt`: Asks the user for a single (freeform) text value.
+- `Select`: Asks the user to pick a single value from a list of options.
+- `MultiSelect`: Asks the user to pick zero or more values from a list of options.
+- `Confirm`: Asks the user to confirm and operation with a given message.
+
+The implementation of this interface uses a go library to provide a terminal experience (using ANSI escape sequences to provide a nice terminal interaction model) with a fallback to raw text input when the user is not connected to a proper terminal.
+
+This is a reasonable experience for users interacting with `azd` via their terminal.  However, `azd` also supports being used within IDEs (today Visual Studio Code, tomorrow Visual Studio as well) and there our terminal based prompting strategy is not ideal. VS Code is forced to run us in an interactive terminal and the user has to interact with `azd` via its terminal interface or specifically craft their calls of `azd` to not prompt the user.  In Visual Studio, AZD is run in a background process, so no terminal interaction is possible.
+
+In both cases it would be ideal if `azd` could delegate the prompting behavior back to the caller.  This document outlines a solution, which provides a way for an external tool to provide a remote service that `azd` interacts with when it needs to prompt the user for information.
+
+## Solution
+
+Similar to our strategy for delegating authentication to an external host, we support delegating prompting to an external host via a special JSON based REST API, hosted over a local HTTP server.  When run, `azd` looks for two special environment variables:
+
+- `AZD_UI_PROMPT_ENDPOINT`
+- `AZD_UI_PROMPT_KEY`
+
+When both are set, instead of prompting using the command line - the implementation of our prompting methods now make a POST call to a special endpoint:
+
+`${AZD_UI_PROMPT_ENDPOINT}/prompt?api-version=2024-02-14-preview`
+
+Setting the following headers:
+
+- `Content-Type: application/json`
+- `Authorization: Bearer ${AZD_UI_PROMPT_KEY}`
+
+The use of `AZD_UI_PROMPT_KEY` allows the host to block requests coming from other clients on the same machine (since the it is expected the host runs a ephemeral HTTP server listing on `127.0.0.1` on a random port). It is expected that the host will generate a random string and use this as a shared key for the lifetime of an `azd` invocation.
+
+The body of the request contains a JSON object with all the information about the prompt that `azd` needs a response for:
+
+```typescript
+interface PromptRequest {
+    type: "string" | "password" | "directory" | "select" | "multiSelect" | "confirm"
+    options: {
+        message: string // the message to be displayed as part of the prompt
+        help?: string // optional help text that can be displayed upon request
+        choices?: PromptChoice[]
+        defaultValue?: string | string[] | boolean
+    }
+}
+
+interface PromptChoice {
+    value: string
+    detail?: string
+}
+```
+
+The `password` type represents a string value which represents a password. The host may want to use a different UI element (perhaps one that uses `***` instead of characters) when prompting.
+
+The server should respond with 200 OK and the body that represents the result:
+
+```typescript
+interface PromptResponse {
+    status: "success" | "cancelled" | "error"
+
+    // present when status is "success"
+    value?: string | string[]
+
+    // present when status is "error"
+    message?: string 
+}
+```
+
+### Success 
+
+When the host is able to prompt for the value a response with the `status` of `success` is sent.
+
+When the type is `confirm` the value should be either `"true"` or `"false"` (a string, not a JSON boolean) indicating if the user confirmed the operation (`"true"`) or rejected it (`"false"`). Note that a user rejecting a confirm prompt still results in a `"success"` status (the value is simply `"false"`). In the case of `multiSelect` an array of string values is returned, each individual value is a value from the `choices` array that was selected by the user.
+
+### Cancelled
+
+The user may decline to provide a response (imagine hitting a cancel button on the dialog that is being used to present the question, or a user hitting something like CTRL+C in a terminal interaction to abort the question asking but not the entire application).
+
+In this case the `status` is `cancelled`.
+
+`azd` returns a special `Error` type internally in this case, which up-stack code can use.
+
+### Error
+
+Some error happened during prompting - the status is `error` and the `message` property is a human readable error message that `azd` returns as a go `error`.
+
+Note that an error prompting leads to a successful result at the HTTP layer (200 OK) but with a special error object. `azd` treats other responses as if the server has an internal bug.
+
+## Open Issues
+
+- [ ] Some hosts, such as VS, may want to collect a set of prompts up front and present them all on a single page as part of an end to end - how would we support this? It may be that the answer is "that's a separate API" and this solution is simply focused on "when `azd` it self is driving and end to end workflow".

cli/azd/internal/repository/detect_confirm_test.go

@@ -200,6 +200,7 @@ func Test_detectConfirm_confirm(t *testing.T) {
 						Stdin:  strings.NewReader(strings.Join(tt.interactions, "\n") + "\n"),
 						Stdout: os.Stdout,
 					},
+					nil,
 					nil),
 			}
 			d.Init(tt.detection, dir)

cli/azd/internal/repository/infra_confirm_test.go

@@ -210,6 +210,7 @@ func TestInitializer_infraSpecFromDetect(t *testing.T) {
 						Stdin:  strings.NewReader(strings.Join(tt.interactions, "\n") + "\n"),
 						Stdout: os.Stdout,
 					},
+					nil,
 					nil),
 			}
 

cli/azd/internal/repository/prompt_util.go

@@ -12,18 +12,6 @@ import (
 	"github.com/azure/azure-dev/cli/azd/pkg/input"
 )
 
-// dirSuggestions provides suggestion completions for directories given the current input directory.
-func dirSuggestions(input string) []string {
-	completions := []string{}
-	matches, _ := filepath.Glob(input + "*")
-	for _, match := range matches {
-		if fs, err := os.Stat(match); err == nil && fs.IsDir() {
-			completions = append(completions, match)
-		}
-	}
-	return completions
-}
-
 // tabWrite transforms tabbed output into formatted strings with a given minimal padding.
 // For more information, refer to the tabwriter package.
 func tabWrite(selections []string, padding int) ([]string, error) {
@@ -47,9 +35,8 @@ func promptDir(
 	console input.Console,
 	message string) (string, error) {
 	for {
-		path, err := console.Prompt(ctx, input.ConsoleOptions{
+		path, err := console.PromptDir(ctx, input.ConsoleOptions{
 			Message: message,
-			Suggest: dirSuggestions,
 		})
 		if err != nil {
 			return "", err

cli/azd/internal/vsrpc/server_service.go

@@ -47,11 +47,11 @@ func (s *serverService) InitializeAsync(
 	session.rootContainer = s.server.rootContainer
 
 	if options.AuthenticationEndpoint != nil {
-		session.authEndpoint = *options.AuthenticationEndpoint
+		session.externalServicesEndpoint = *options.AuthenticationEndpoint
 	}
 
 	if options.AuthenticationKey != nil {
-		session.authKey = *options.AuthenticationKey
+		session.externalServicesKey = *options.AuthenticationKey
 	}
 
 	return &Session{

cli/azd/internal/vsrpc/server_session.go

@@ -12,6 +12,7 @@ import (
 	"github.com/azure/azure-dev/cli/azd/internal"
 	"github.com/azure/azure-dev/cli/azd/pkg/auth"
 	"github.com/azure/azure-dev/cli/azd/pkg/environment/azdcontext"
+	"github.com/azure/azure-dev/cli/azd/pkg/httputil"
 	"github.com/azure/azure-dev/cli/azd/pkg/input"
 	"github.com/azure/azure-dev/cli/azd/pkg/ioc"
 	"github.com/azure/azure-dev/cli/azd/pkg/lazy"
@@ -29,9 +30,9 @@ type serverSession struct {
 	// rootPath is the path to the root of the solution.
 	rootPath string
 	// root container points to server.rootContainer
-	rootContainer *ioc.NestedContainer
-	authEndpoint  string
-	authKey       string
+	rootContainer            *ioc.NestedContainer
+	externalServicesEndpoint string
+	externalServicesKey      string
 }
 
 // newSession creates a new session and returns the session ID and session. newSession is safe to call by multiple
@@ -120,7 +121,7 @@ func (s *serverSession) newContainer() (*container, error) {
 		}),
 	})
 
-	c.MustRegisterScoped(func() input.Console {
+	c.MustRegisterScoped(func(client httputil.HttpClient) input.Console {
 		stdout := outWriter
 		stderr := errWriter
 		stdin := strings.NewReader("")
@@ -133,7 +134,13 @@ func (s *serverSession) newContainer() (*container, error) {
 				Stdin:  stdin,
 				Stdout: stdout,
 				Stderr: stderr,
-			}, &output.NoneFormatter{})
+			},
+			&output.NoneFormatter{},
+			&input.ExternalPromptConfiguration{
+				Endpoint: s.externalServicesEndpoint,
+				Key:      s.externalServicesKey,
+				Client:   client,
+			})
 	})
 
 	c.MustRegisterScoped(func(console input.Console) io.Writer {
@@ -156,8 +163,8 @@ func (s *serverSession) newContainer() (*container, error) {
 
 	c.MustRegisterScoped(func() auth.ExternalAuthConfiguration {
 		return auth.ExternalAuthConfiguration{
-			Endpoint: s.authEndpoint,
-			Key:      s.authKey,
+			Endpoint: s.externalServicesEndpoint,
+			Key:      s.externalServicesKey,
 		}
 	})
 

cli/azd/pkg/infra/provisioning/bicep/bicep_provider.go

@@ -1962,6 +1962,11 @@ func (p *BicepProvider) ensureParameters(
 
 	configModified := false
 
+	var parameterPrompts []struct {
+		key   string
+		param azure.ArmTemplateParameterDefinition
+	}
+
 	for _, key := range sortedKeys {
 		param := template.Parameters[key]
 
@@ -1987,35 +1992,81 @@ func (p *BicepProvider) ensureParameters(
 		configKey := fmt.Sprintf("infra.parameters.%s", key)
 
 		if v, has := p.env.Config.Get(configKey); has {
-
-			if !isValueAssignableToParameterType(p.mapBicepTypeToInterfaceType(param.Type), v) {
+			if isValueAssignableToParameterType(p.mapBicepTypeToInterfaceType(param.Type), v) {
+				configuredParameters[key] = azure.ArmParameterValue{
+					Value: v,
+				}
+				continue
+			} else {
 				// The saved value is no longer valid (perhaps the user edited their template to change the type of a)
 				// parameter and then re-ran `azd provision`. Forget the saved value (if we can) and prompt for a new one.
 				_ = p.env.Config.Unset("infra.parameters.%s")
 			}
+		}
 
-			configuredParameters[key] = azure.ArmParameterValue{
-				Value: v,
+		// No saved value for this required parameter, we'll need to prompt for it.
+		parameterPrompts = append(parameterPrompts, struct {
+			key   string
+			param azure.ArmTemplateParameterDefinition
+		}{key: key, param: param})
+	}
+
+	if len(parameterPrompts) > 0 {
+		if p.console.SupportsPromptDialog() {
+
+			dialog := input.PromptDialog{
+				Title: "Configure required deployment parameters",
+				Description: "The following parameters are required for deployment. " +
+					"Provide values for each parameter. They will be saved for future deployments.",
 			}
-			continue
-		}
 
-		// Otherwise, prompt for the value.
-		value, err := p.promptForParameter(ctx, key, param)
-		if err != nil {
-			return nil, fmt.Errorf("prompting for value: %w", err)
-		}
+			for _, prompt := range parameterPrompts {
+				dialog.Prompts = append(dialog.Prompts, p.promptDialogItemForParameter(prompt.key, prompt.param))
+			}
 
-		if err := p.env.Config.Set(configKey, value); err == nil {
-			configModified = true
+			values, err := p.console.PromptDialog(ctx, dialog)
+			if err != nil {
+				return nil, fmt.Errorf("prompting for values: %w", err)
+			}
+
+			for _, prompt := range parameterPrompts {
+				configKey := fmt.Sprintf("infra.parameters.%s", prompt.key)
+				value := values[prompt.key]
+
+				if err := p.env.Config.Set(configKey, value); err == nil {
+					configModified = true
+				} else {
+					// errors from config.Set are panics, so we can't recover from them
+					// For example, the value is not serializable to JSON
+					log.Panicf(fmt.Sprintf("warning: failed to set value: %v", err))
+				}
+
+				configuredParameters[prompt.key] = azure.ArmParameterValue{
+					Value: value,
+				}
+			}
 		} else {
-			// errors from config.Set are panics, so we can't recover from them
-			// For example, the value is not serializable to JSON
-			log.Panicf(fmt.Sprintf("warning: failed to set value: %v", err))
-		}
+			for _, prompt := range parameterPrompts {
+				configKey := fmt.Sprintf("infra.parameters.%s", prompt.key)
+
+				// Otherwise, prompt for the value.
+				value, err := p.promptForParameter(ctx, prompt.key, prompt.param)
+				if err != nil {
+					return nil, fmt.Errorf("prompting for value: %w", err)
+				}
 
-		configuredParameters[key] = azure.ArmParameterValue{
-			Value: value,
+				if err := p.env.Config.Set(configKey, value); err == nil {
+					configModified = true
+				} else {
+					// errors from config.Set are panics, so we can't recover from them
+					// For example, the value is not serializable to JSON
+					log.Panicf(fmt.Sprintf("warning: failed to set value: %v", err))
+				}
+
+				configuredParameters[prompt.key] = azure.ArmParameterValue{
+					Value: value,
+				}
+			}
 		}
 	}
 

cli/azd/pkg/infra/provisioning/bicep/prompt.go

@@ -6,6 +6,7 @@ import (
 	"fmt"
 	"strconv"
 
+	"github.com/Azure/azure-sdk-for-go/sdk/azcore/to"
 	"github.com/azure/azure-dev/cli/azd/pkg/account"
 	"github.com/azure/azure-dev/cli/azd/pkg/azure"
 	"github.com/azure/azure-dev/cli/azd/pkg/input"
@@ -16,6 +17,40 @@ import (
 	. "github.com/azure/azure-dev/cli/azd/pkg/infra/provisioning"
 )
 
+// promptDialogItemForParameter builds the input.PromptDialogItem for the given required parameter.
+func (p *BicepProvider) promptDialogItemForParameter(
+	key string,
+	param azure.ArmTemplateParameterDefinition,
+) input.PromptDialogItem {
+	help, _ := param.Description()
+	paramType := p.mapBicepTypeToInterfaceType(param.Type)
+
+	var dialogItem input.PromptDialogItem
+	dialogItem.ID = key
+	dialogItem.DisplayName = key
+	dialogItem.Required = true
+
+	if help != "" {
+		dialogItem.Description = to.Ptr(help)
+	}
+
+	if paramType == ParameterTypeBoolean {
+		dialogItem.Kind = "select"
+		dialogItem.Choices = []input.PromptDialogChoice{{Value: "true"}, {Value: "false"}}
+	} else if param.AllowedValues != nil {
+		dialogItem.Kind = "select"
+		for _, v := range *param.AllowedValues {
+			dialogItem.Choices = append(dialogItem.Choices, input.PromptDialogChoice{Value: fmt.Sprintf("%v", v)})
+		}
+	} else if param.Secure() {
+		dialogItem.Kind = "password"
+	} else {
+		dialogItem.Kind = "string"
+	}
+
+	return dialogItem
+}
+
 func autoGenerate(parameter string, azdMetadata azure.AzdMetadata) (string, error) {
 	if azdMetadata.AutoGenerateConfig == nil {
 		return "", fmt.Errorf("auto generation metadata config is missing for parameter '%s'", parameter)