Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Error using kubeconfig_server_address #1590

Open
janfrederik opened this issue Dec 12, 2024 · 2 comments
Open

[Bug]: Error using kubeconfig_server_address #1590

janfrederik opened this issue Dec 12, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@janfrederik
Copy link
Contributor

janfrederik commented Dec 12, 2024

Description

When I set kubeconfig_server_address in kube.tf, kubectl refuses to connect to the cluster with the generated kubeconfig because the certificate is not valid for that hostname:

E1212 17:02:47.374951  709663 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: 
Get \"https://my.domain-name.net:6443/api?timeout=32s\": tls: failed to verify certificate: x509: 
certificate is valid for ctrl1-kam, ctrl2-grm, ctrl3-zof, kubernetes, kubernetes.default, 
kubernetes.default.svc, kubernetes.default.svc.cluster.local, localhost, not my.domain-name.net"

Steps to reproduce:

  • Remove my.domain-name.net from DNS
  • Set kubeconfig_server_address = "my.domain-name.net" in kube.tf
  • tofu apply
  • Set domain in DNS to point to the ip4 of the first control plane
  • (test if DNS works: ok)
  • tofu output -raw kubeconfig > kubeconfig
  • KUBECONFIG=kubeconfig kubectl get nodes

--> Error.

Kube.tf file

provider "hcloud" {
  token = var.hcloud_token
}

module "kube-hetzner" {
  source = "kube-hetzner/kube-hetzner/hcloud"

  providers = {
    hcloud = hcloud
  }

  hcloud_token = var.hcloud_token

  ssh_public_key       = local.ssh_public_key
  ssh_private_key      = local.ssh_private_key

  control_plane_nodepools = [
    {
      name        = "ctrl1",
      server_type = "cax11",
      location    = "hel1",
      count       = 1,
      labels      = [],
      taints      = [],
    },
    {
      name        = "ctrl2",
      server_type = "cax11",
      location    = "fsn1",
      count       = 1,
      labels      = [],
      taints      = [],
    },
    {
      name        = "ctrl3",
      server_type = "cax11",
      location    = "nbg1",
      count       = 1,
      labels      = [],
      taints      = [],
    },
  ]

  allow_scheduling_on_control_plane = true

  agent_nodepools = [
    {
      name        = "agent-small",
      server_type = "cax11",
      location    = "nbg1",
      count       = 1,
      labels      = [],
      taints      = [],
    },
  ]

  initial_k3s_channel = "v1.30"

  # See https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases for the available versions.
  hetzner_ccm_version = "v1.20.0"
  kured_version = "1.16.2"
  ingress_controller = "nginx"
  nginx_version      = "4.11.3"
  enable_klipper_metal_lb = "true"

  automatically_upgrade_os = false
  sys_upgrade_controller_version = "v0.14.2"

  cluster_name = var.cluster_name
  use_cluster_name_in_node_name = false
  restrict_outbound_traffic = false

  base_domain = var.domain
  lb_hostname = var.domain

  kubeconfig_server_address = var.domain
  create_kubeconfig         = false
  export_values             = false
}

locals {
  ssh_private_key = fileexists("~/.ssh/id_ed25519") ? file("~/.ssh/id_ed25519") : (fileexists("~/.ssh/id_rsa") ? file("~/.ssh/id_rsa") : "")
  ssh_public_key  = fileexists("~/.ssh/id_ed25519.pub") ? file("~/.ssh/id_ed25519.pub") : (fileexists("~/.ssh/id_rsa.pub") ? file("~/.ssh/id_rsa.pub") : "")
}

variable "hcloud_token" {
  description = "Hetzner Cloud API Token."
  type        = string
  sensitive   = true
}

variable "cluster_name" {
  description = "Name of the cluster."
  type        = string
  default     = "k3s"
}

variable "domain" {
  description = "Domain name to use for the cluster."
  type        = string
}

Platform

linux

@janfrederik janfrederik added the bug Something isn't working label Dec 12, 2024
@janfrederik
Copy link
Contributor Author

This doesn't work either when the domain name is registered in the DNS before tofu apply.

@janfrederik
Copy link
Contributor Author

In #1395, I can't see anything that tells k3s to put this domain name in the cluster certificate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant