feat: add templating and custom config file to complete (#89)

<!-- Thank you for submitting a Pull Request. Please fill out the
template below.-->
## Overview/Summary

This PR introduces templating for the config file of the complete
module. It also introduces the specific config file parameter.

This PR has a dependency on:
Azure/ALZ-PowerShell-Module#100

## This PR fixes/adds/changes/removes

1. #85 

### Breaking Changes

`additional_files` has been changed to a hidden input, so users will no
longer be prompted fore this.

## Testing Evidence

Will trigger e2e tests.

Ran an end to end test locally and here is the successful plan run with
a custom config file, also tested with an empyy value for the config
file and confirms it defaults to the `config.yaml` file as expected:

![image](https://github.com/Azure/alz-terraform-accelerator/assets/1612200/2aaa0261-a6ec-4e72-8653-b28bfcd0f33e)


![image](https://github.com/Azure/alz-terraform-accelerator/assets/1612200/109b00ec-fdf2-45c0-95d7-c33f0c9ff2a6)


![image](https://github.com/Azure/alz-terraform-accelerator/assets/1612200/61c561e3-f8b7-4440-aff9-72ca7fecc0cc)


## As part of this Pull Request I have

- [x] Checked for duplicate [Pull
Requests](https://github.com/Azure/alz-terraform-accelerator/pulls)
- [x] Associated it with relevant
[issues](https://github.com/Azure/alz-terraform-accelerator/issues), for
tracking and closure.
- [x] Ensured my code/branch is up-to-date with the latest changes in
the `main`
[branch](https://github.com/Azure/alz-terraform-accelerator/tree/main)
- [x] Performed testing and provided evidence.
- [x] Updated relevant and associated documentation.

---------

Co-authored-by: Jared Holgate <[email protected]>

.github/workflows/end-to-end-test.yml

@@ -150,7 +150,6 @@ jobs:
 
           $Inputs["apply_approvers"] = "" # if ($versionControlSystem -eq "github") { "${{ vars.VCS_APPROVER_GITHUB }}" } else { "${{ vars.VCS_APPROVER_AZUREDEVOPS }}" }
           $Inputs["root_parent_management_group_display_name"] = "Tenant Root Group"
-          $Inputs["additional_files"] = ""
 
           $Inputs["subscription_id_connectivity"] = "${{ vars.ARM_SUBSCRIPTION_ID }}"
           $Inputs["subscription_id_identity"] = "${{ vars.ARM_SUBSCRIPTION_ID }}"

bootstrap/.config/ALZ-Powershell.config.json

@@ -121,6 +121,11 @@
             "Type": "Valid",
             "Description": "A valid CIDR range e.g '10.0.0.0/16'",
             "Valid": "^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(/(3[0-2]|[1-2][0-9]|[0-9]))$"
+        },
+        "configuration_file_path": {
+            "Type": "Valid",
+            "Description": "A valid yaml or json configuration file path e.g. './my-folder/my-config-file.yaml' or `c:\\my-folder\\my-config-file.yaml`",
+            "Valid": "^.+\\.(yaml|yml|json)$"
         }
     }
 }

bootstrap/azuredevops/main.tf

@@ -8,12 +8,14 @@ module "resource_names" {
 }
 
 module "files" {
-  source                     = "./../modules/files"
-  starter_module_folder_path = local.starter_module_folder_path
-  pipeline_folder_path       = local.pipeline_folder_path
-  pipeline_files             = var.pipeline_files
-  pipeline_template_files    = var.pipeline_template_files
-  additional_files           = var.additional_files
+  source                            = "./../modules/files"
+  starter_module_folder_path        = local.starter_module_folder_path
+  pipeline_folder_path              = local.pipeline_folder_path
+  pipeline_files                    = var.pipeline_files
+  pipeline_template_files           = var.pipeline_template_files
+  additional_files                  = concat(var.additional_files)
+  configuration_file_path           = var.configuration_file_path
+  built_in_configurartion_file_name = var.built_in_configurartion_file_name
 }
 
 module "azure" {

bootstrap/azuredevops/variables.tf

@@ -108,7 +108,7 @@ variable "root_parent_management_group_display_name" {
 }
 
 variable "additional_files" {
-  description = "Additional files to upload to the repository. This must be specified as a comma-separated list of absolute file paths (e.g. c:\\config\\config.yaml or /home/user/config/config.yaml)|19"
+  description = "Additional files to upload to the repository. This must be specified as a comma-separated list of absolute file paths (e.g. c:\\config\\config.yaml or /home/user/config/config.yaml)|hidden"
   type        = list(string)
   default     = []
 }
@@ -147,6 +147,18 @@ variable "target_subscriptions" {
   type        = list(string)
 }
 
+variable "configuration_file_path" {
+  description = "The name of the configuration file|hidden_configuration_file_path"
+  type        = string
+  default     = ""
+}
+
+variable "built_in_configurartion_file_name" {
+  description = "The name of the built-in configuration file|hidden"
+  type        = string
+  default     = "config.yaml"
+}
+
 variable "module_folder_path" {
   description = "The folder for the starter modules|hidden"
   type        = string

bootstrap/github/main.tf

@@ -8,12 +8,14 @@ module "resource_names" {
 }
 
 module "files" {
-  source                     = "./../modules/files"
-  starter_module_folder_path = local.starter_module_folder_path
-  pipeline_folder_path       = local.pipeline_folder_path
-  pipeline_files             = var.pipeline_files
-  pipeline_template_files    = var.pipeline_template_files
-  additional_files           = var.additional_files
+  source                            = "./../modules/files"
+  starter_module_folder_path        = local.starter_module_folder_path
+  pipeline_folder_path              = local.pipeline_folder_path
+  pipeline_files                    = var.pipeline_files
+  pipeline_template_files           = var.pipeline_template_files
+  additional_files                  = var.additional_files
+  configuration_file_path           = var.configuration_file_path
+  built_in_configurartion_file_name = var.built_in_configurartion_file_name
 }
 
 module "azure" {

bootstrap/github/variables.tf

@@ -81,7 +81,7 @@ variable "root_parent_management_group_display_name" {
 }
 
 variable "additional_files" {
-  description = "Additional files to upload to the repository. This must be specified as a comma-separated list of absolute file paths (e.g. c:\\config\\config.yaml or /home/user/config/config.yaml)|16"
+  description = "Additional files to upload to the repository. This must be specified as a comma-separated list of absolute file paths (e.g. c:\\config\\config.yaml or /home/user/config/config.yaml)|hidden"
   type        = list(string)
   default     = []
 }
@@ -91,6 +91,18 @@ variable "target_subscriptions" {
   type        = list(string)
 }
 
+variable "configuration_file_path" {
+  description = "The name of the configuration file to be generated|hidden_configuration_file_path"
+  type        = string
+  default     = ""
+}
+
+variable "built_in_configurartion_file_name" {
+  description = "The name of the built-in configuration file|hidden"
+  type        = string
+  default     = "config.yaml"
+}
+
 variable "module_folder_path" {
   description = "The folder for the starter modules|hidden"
   type        = string

bootstrap/local/main.tf

@@ -8,9 +8,11 @@ module "resource_names" {
 }
 
 module "files" {
-  source                     = "./../modules/files"
-  starter_module_folder_path = local.starter_module_folder_path
-  additional_files           = var.additional_files
+  source                            = "./../modules/files"
+  starter_module_folder_path        = local.starter_module_folder_path
+  additional_files                  = var.additional_files
+  configuration_file_path           = var.configuration_file_path
+  built_in_configurartion_file_name = var.built_in_configurartion_file_name
 }
 
 module "azure" {

bootstrap/local/variables.tf

@@ -53,7 +53,7 @@ variable "root_parent_management_group_display_name" {
 }
 
 variable "additional_files" {
-  description = "Additional files to upload to the repository. This must be specified as a comma-separated list of absolute file paths (e.g. c:\\config\\config.yaml or /home/user/config/config.yaml)|10"
+  description = "Additional files to upload to the repository. This must be specified as a comma-separated list of absolute file paths (e.g. c:\\config\\config.yaml or /home/user/config/config.yaml)|hidden"
   type        = list(string)
   default     = []
 }
@@ -63,6 +63,18 @@ variable "target_subscriptions" {
   type        = list(string)
 }
 
+variable "configuration_file_path" {
+  description = "The name of the configuration file to be generated|hidden_configuration_file_path"
+  type        = string
+  default     = ""
+}
+
+variable "built_in_configurartion_file_name" {
+  description = "The name of the built-in configuration file|hidden"
+  type        = string
+  default     = "config.yaml"
+}
+
 variable "module_folder_path" {
   description = "The folder for the starter modules|hidden"
   type        = string

bootstrap/modules/files/main.tf

@@ -7,12 +7,17 @@ locals {
   }
 }
 
+locals {
+  has_configuration_file = var.configuration_file_path != ""
+}
+
 locals {
   starter_module_files = { for file in fileset(var.starter_module_folder_path, "**") : file => {
     path = "${var.starter_module_folder_path}/${file}"
     flag = local.file_type_flags.module
-    }
+    } if !local.has_configuration_file || file != var.built_in_configurartion_file_name
   }
+
   pipeline_files = { for key, value in var.pipeline_files : value.target_path => {
     path = "${var.pipeline_folder_path}/${value.file_path}"
     flag = local.file_type_flags.pipeline
@@ -24,7 +29,8 @@ locals {
     }
   }
   starter_module_repo_files = merge(local.starter_module_files, local.pipeline_files, local.template_files)
-  additional_repo_files = { for file in var.additional_files : basename(file) => {
+  final_additional_files    = concat(var.additional_files, local.has_configuration_file ? [var.configuration_file_path] : [])
+  additional_repo_files = { for file in local.final_additional_files : basename(file) => {
     path = file
     flag = local.file_type_flags.additional
     }

bootstrap/modules/files/variables.tf

@@ -32,3 +32,15 @@ variable "additional_files" {
   type        = list(string)
   default     = []
 }
+
+variable "configuration_file_path" {
+  description = "Configuration file path"
+  type        = string
+  default     = ""
+}
+
+variable "built_in_configurartion_file_name" {
+  description = "Built-in configuration file name"
+  type        = string
+  default     = "config.yaml"
+}

docs/wiki/[User-Guide]-Quick-Start-Phase-2-Azure-DevOps.md

@@ -22,7 +22,6 @@
     1. `allow_storage_access_from_my_ip`: This controls whether to allow access to the storage account from your IP address. This is only needed for trouble shooting. This only applies if you have `use_private_networking` set to `true`. This defaults to `false`.
     1. `apply_approvers`: This is a list of service principal names (SPN) of people you wish to be in the group that approves apply of the Azure landing zone module. This is a comma-separated list like `[email protected],[email protected],[email protected]`. You may need to check what the SPN is prior to filling this out as it can vary based on identity provider.
     1. `root_parent_management_group_display_name`: This is the name of the root management group that you applied permissions to in a previous step. This defaults to `Tenant Root Group`, but if your organization has changed it or you want to use a different management group you'll need to enter the new display name.
-    1. `additional_files`: This is a method to supply additional files to your starter module. This is specifically used when using the `complete` starter module to supply the `config.yaml` file. This must be specified as a comma-separated list of absolute file paths (e.g. c:\\config\\config.yaml or /home/user/config/config.yaml). If you don't supply an absolute path, it will fail.
 1. You will now see a green message telling you that the next section is specific to the starter module you choose. Navigate to the documentation for the relevant starter module to get details of the specific inputs.
 1. Once you have entered the starter module input, you see that a Terraform `init` and `apply` happen.
 1. There will be a pause after the `plan` phase you allow you to validate what is going to be deployed.

docs/wiki/[User-Guide]-Quick-Start-Phase-2-GitHub.md

@@ -18,7 +18,6 @@
     1. `allow_storage_access_from_my_ip`: This controls whether to allow access to the storage account from your IP address. This is only needed for trouble shooting. This only applies if you have `use_private_networking` set to `true`. This defaults to `false`.
     1. `apply_approvers`: This is a list of service principal names (SPN) of people you wish to be in the group that approves apply of the Azure landing zone module. This is a comma-separated list like `[email protected],[email protected],[email protected]`. You may need to check what the SPN is prior to filling this out as it can vary based on identity provider.
     1. `root_parent_management_group_display_name`: This is the name of the root management group that you applied permissions to in a previous step. This defaults to `Tenant Root Group`, but if your organization has changed it or you want to use a different management group you'll need to enter the new display name.
-    1. `additional_files`: This is a method to supply additional files to your starter module. This is specifically used when using the `complete` starter module to supply the `config.yaml` file. This must be specified as a comma-separated list of absolute file paths (e.g. c:\\config\\config.yaml or /home/user/config/config.yaml). If you don't supply an absolute path, it will fail.
 1. You will now see a green message telling you that the next section is specific to the starter module you choose. Navigate to the documentation for the relevant starter module to get details of the specific inputs.
 1. Once you have entered the starter module input, you see that a Terraform `init` and `apply` happen.
 1. There will be a pause after the `plan` phase you allow you to validate what is going to be deployed.

docs/wiki/[User-Guide]-Quick-Start-Phase-2-Local.md

@@ -13,7 +13,6 @@
     1. `environment_name`: This is used to build up the names of your Azure and Azure DevOps resources, for example `rg-alz-<environment_name>-uksouth-001`. We recommend using `mgmt` for this.
     1. `postfix_number`: This is used to build up the names of your Azure and Azure DevOps resources, for example `rg-alz-mgmt-uksouth-<postfix_number>`. We recommend using `1` for this.
     1. `root_parent_management_group_display_name`: This is the name of the root management group that you applied permissions to in a previous step. This defaults to `Tenant Root Group`, but if your organization has changed it or you want to use a different management group you'll need to enter the new display name.
-    1. `additional_files`: This is a method to supply additional files to your starter module. This is specifically used when using the `complete` starter module to supply the `config.yaml` file. This must be specified as a comma-separated list of absolute file paths (e.g. c:\\config\\config.yaml or /home/user/config/config.yaml). If you don't supply an absolute path, it will fail.
 1. You will now see a green message telling you that the next section is specific to the starter module you choose. Navigate to the documentation for the relevant starter module to get details of the specific inputs.
 1. Once you have entered the starter module input, you see that a Terraform `init` and `apply` happen.
 1. There will be a pause after the `plan` phase you allow you to validate what is going to be deployed.