Lost tfstate

[vavallee]

I lost my tfstate file in a drive failure. Is there an easy way to import existing resources into terraform so that I can manage them again?

[mysticaltech]

@vavallee This is doable but will need some hacking. We have a scripts/cleanup.sh file. You need to have installed the hcloud cli (see readme), with your current project activated.

First, create a new state with terraform init.

Then modify the above cleanup script so that instead of deleting the resources, it runs the terraform import command for each resource, in the form of terraform import <resource_type>.<resource_name> <resource_ID>. See hcloud terraform provider docs for the right resource name to use.

It should work! After that, please do not hesitate to PR the import script, it would be really appreciated.

[vavallee]

Thank you! In fact I did the terraform import one at a time by hand. But oddly, even after doing that, if I run a terraform plan, it wants to recreate all the resources. Am I missing something?

[kldzj]

@mysticaltech Hey, I'm currently facing the same issue. IMHO it doesn't seem as straight forward as you suggest. Has anyone ever come forward with a recovery script that worked?

[vavallee]

DARKWAVE:~/new-tf-hetzner/terraform-hcloud-kube-hetzner/vconf$ terraform state list
hcloud_firewall.k3s
hcloud_load_balancer.k3s
hcloud_network.k3s
hcloud_placement_group.k3s-agent-1
hcloud_placement_group.k3s-control-plane-1
hcloud_primary_ip.primary_ip-22346049
hcloud_primary_ip.primary_ip-22346050
hcloud_primary_ip.primary_ip-22346053
hcloud_primary_ip.primary_ip-22346054
hcloud_primary_ip.primary_ip-22346055
hcloud_primary_ip.primary_ip-22346056
hcloud_server.k3s-agent-small-zic
hcloud_server.k3s-agent-small-zyp
hcloud_server.k3s-control-plane-sdg
DARKWAVE:~/new-tf-hetzner/terraform-hcloud-kube-hetzner/vconf$ terraform plan
...
...
Plan: 41 to add, 1 to change, 0 to destroy.

[mysticaltech]

You are missing some ressources IMHO (and the dependency create the need for recreation). Probably best to follow the steps above, it should be a quick modification of the provided script.

Then you can inspect with terraform plan, and see why it wants to change if it wants to change, it always provides a reason!

Between, all non-hetzner components are ok to be recreated, as they will not destroy, just apply the logic again and that should not disruptive.

If you want to backup, you can do so with velero by vmware, search this project for examples.

[vavallee]

Sadly, during the apply I am getting name collisions. Any ideas?

│ Error: SSH key name is already used (uniqueness_error)
│
│   with module.kube-hetzner.hcloud_ssh_key.k3s[0],
│   on .terraform/modules/kube-hetzner/main.tf line 6, in resource "hcloud_ssh_key" "k3s":
│    6: resource "hcloud_ssh_key" "k3s" {
│
╵
╷
│ Error: name is already used (uniqueness_error)
│
│   with module.kube-hetzner.hcloud_network.k3s,
│   on .terraform/modules/kube-hetzner/main.tf line 13, in resource "hcloud_network" "k3s":
│   13: resource "hcloud_network" "k3s" {
│
╵
╷
│ Error: name is already used (uniqueness_error)
│
│   with module.kube-hetzner.hcloud_firewall.k3s,
│   on .terraform/modules/kube-hetzner/main.tf line 38, in resource "hcloud_firewall" "k3s":
│   38: resource "hcloud_firewall" "k3s" {
│
╵
╷
│ Error: name is already used (uniqueness_error)
│
│   with module.kube-hetzner.hcloud_placement_group.control_plane[0],
│   on .terraform/modules/kube-hetzner/main.tf line 55, in resource "hcloud_placement_group" "control_plane":
│   55: resource "hcloud_placement_group" "control_plane" {
│
╵
╷
│ Error: name is already used (uniqueness_error)
│
│   with module.kube-hetzner.hcloud_placement_group.agent[0],
│   on .terraform/modules/kube-hetzner/main.tf line 62, in resource "hcloud_placement_group" "agent":
│   62: resource "hcloud_placement_group" "agent" {

[mysticaltech]

@vavalleem, I asked GPT-4 for help, here's what it proposes:

It looks like the name collisions are occurring because the resources already exist in Hetzner Cloud, but they haven't been imported into the Terraform state correctly. To resolve this issue, you can follow these steps:

Ensure all resources are imported into the Terraform state. Double-check the modified cleanup.sh script to make sure that it imports all the necessary resources as mentioned in your previous advice.

If the resources have been imported, but Terraform is still attempting to create new resources with the same names, you need to verify that the resource names in the Terraform configuration files match the names of the resources in Hetzner Cloud. You can use the hcloud CLI tool or the Hetzner Cloud Console to confirm the names of the existing resources.

If the resource names in Terraform configuration files are correct and match the names of the existing resources in Hetzner Cloud, but the error persists, you can try adding the -target flag while running terraform apply. This flag allows you to selectively apply changes to specific resources:

terraform apply -target=module.kube-hetzner.hcloud_ssh_key.k3s[0] \
                -target=module.kube-hetzner.hcloud_network.k3s \
                -target=module.kube-hetzner.hcloud_firewall.k3s \
                -target=module.kube-hetzner.hcloud_placement_group.control_plane[0] \
                -target=module.kube-hetzner.hcloud_placement_group.agent[0]

This command will apply changes only to the specified resources, so Terraform won't attempt to recreate any other resources that might cause name collisions.

If none of the above steps work, you might need to manually remove the resources from the Terraform configuration files that are causing the name collisions, import the existing resources with the correct names, and then add the resources back to the configuration files. This will ensure that Terraform recognizes the existing resources and doesn't try to create new ones with the same names.

After resolving the uniqueness issue, run terraform apply again to check if everything works as expected. If Terraform still tries to recreate resources or change configurations, inspect the output of terraform plan to determine the cause and address it accordingly.

[mysticaltech]

@vavallee @kldzj This may not be straight forward, but you could try OpenAI o1 to recreate the state for you for instance, based on import. Or give you a script that would do that.

[kldzj]

I tried it and didn't get close unfortunately. Ended up creating a new cluster with a S3 backend.

Thanks for the tip though. :)