[WIP] Peer AKS VNet (mgmt) with VM based default templates

[nawazkh]

What type of PR is this?
/kind feature

What this PR does / why we need it:

• This PR updates the templates (based off of VMs) to use private IP of the API Server of the control plane of the workload cluster for inter-node communication.
• This PR also updates the Tiltfile with logic to peer AKS VNets with workload cluster's VNets.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes # #5261

Special notes for your reviewer:

• cherry-pick candidate

TODOs:

• squashed commits
• includes documentation
• adds unit tests

Release note:

Update VM based templates to use Private IP solution for workload cluster's inter-node communication. Also update Tiltfile to update AKS Vnet with workload cluster's VNet

[k8s-ci-robot]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from nawazkh. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 52.43%. Comparing base (736bfbc) to head (f0b219e).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5288      +/-   ##
==========================================
- Coverage   52.43%   52.43%   -0.01%     
==========================================
  Files         272      272              
  Lines       29401    29401              
==========================================
- Hits        15417    15415       -2     
- Misses      13178    13180       +2     
  Partials      806      806              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[nawazkh]

We have often struggled in getting more logs outputted while running in tilt.
With this change, if we add the below yaml fields to our tilt-settings.yaml, we will can play around with more logging

container_args:
  capz-controller-manager:
    - "--v=2"

[nawazkh]

/retest

[nawazkh]

this PR needs to be held for feature flag for ILB can be added.
/hold

[k8s-ci-robot]

@nawazkh: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-cluster-api-provider-azure-apiversion-upgrade	3025950	link	true	/test pull-cluster-api-provider-azure-apiversion-upgrade
pull-cluster-api-provider-azure-windows-with-ci-artifacts	f0b219e	link	false	/test pull-cluster-api-provider-azure-windows-with-ci-artifacts

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.