terraform-hcloud-kubernetes

Bare metal worker nodes can be added through Hetzner Robot. The module enables rescue mode, installs Talos, attaches the servers to a vSwitch, and applies the Talos worker configuration.

Example kubernetes.tf snippet:

hcloud_robot_user     = "<robot-user>"
hcloud_robot_password = "<robot-password>"

bare_metal_nodepools = [
  {
    name = "bare-metal"
    servers = [
      { number = 1111111, private_ipv4 = "10.0.88.2" },
      { number = 2222222, private_ipv4 = "10.0.88.3" }
    ]
  }
]

vSwitch Network

The module creates the vSwitch and connects it to the Hetzner Cloud Network. To use an existing vSwitch instead, set:

hcloud_vswitch_id = 12345

Each server needs a unique private_ipv4 from the vSwitch subnet. For the default 10.0.0.0/16 network, the bare metal subnet can also be calculated with:

echo "cidrsubnet(\"10.0.0.0/16\", 9, 176)" | tofu console
"10.0.88.0/25"

When bare metal servers are enabled for the first time, Cilium might need to be restarted to pick up the changed routing mode:

kubectl -n kube-system rollout restart ds/cilium ds/cilium-envoy
kubectl -n kube-system rollout restart deploy/cilium-operator

Robot Firewall

By default the module manages the Robot firewall of every bare metal server. Some servers have none, for example those with a 10G uplink, where the Robot API answers 404 FIREWALL_NOT_AVAILABLE and the apply fails. Opt out per server with robot_firewall_enabled = false:

bare_metal_nodepools = [
  {
    name = "bare-metal"
    servers = [
      { number = 1111111, private_ipv4 = "10.0.88.2" },                                  # Robot firewall managed
      { number = 2222222, private_ipv4 = "10.0.88.3", robot_firewall_enabled = false }  # no Robot firewall (10G)
    ]
  }
]

For such a server the module neither opens the Robot firewall for the rescue install nor applies the firewall rules afterwards. Switching an existing server to false leaves its current Robot firewall configuration in place; disable it in Robot if it should go as well.

A server without a Robot firewall is reachable from the internet on every port its Talos node listens on, so protect it otherwise, for example with Talos’ ingress firewall. bare_metal_config_patches applies to all bare metal servers, so servers that keep the Robot firewall get these rules as well:

bare_metal_config_patches = [
  { apiVersion = "v1alpha1", kind = "NetworkDefaultActionConfig", ingress = "block" },
  {
    apiVersion   = "v1alpha1"
    kind         = "NetworkRuleConfig"
    name         = "cluster-tcp"
    portSelector = { ports = ["1-65535"], protocol = "tcp" }
    ingress      = [{ subnet = "10.0.0.0/16" }, { subnet = "<pod-ipv4-cidr>" }]
  },
  {
    apiVersion   = "v1alpha1"
    kind         = "NetworkRuleConfig"
    name         = "cluster-udp"
    portSelector = { ports = ["1-65535"], protocol = "udp" }
    ingress      = [{ subnet = "10.0.0.0/16" }, { subnet = "<pod-ipv4-cidr>" }]
  },
  {
    apiVersion   = "v1alpha1"
    kind         = "NetworkRuleConfig"
    name         = "apid"
    portSelector = { ports = [50000], protocol = "tcp" }
    ingress      = [{ subnet = "<admin-ipv4>/32" }]
  }
]

Keep the Talos API reachable from wherever the module runs, and note that Cilium serves NodePorts in eBPF ahead of this firewall.

Install Disks and RAID

If install_disks and install_raid_level are omitted, the module automatically selects the first eligible non-removable disk. Explicit install_disks values must be disk IDs from /dev/disk/by-id:

servers = [
  {
    number        = 1234567
    private_ipv4  = "10.0.88.2"
    install_disks = ["ata-Samsung_SSD_870_ABC123"]
  }
]

Setting install_raid_level to raid1 creates a bootable Linux MD RAID1 array with metadata format 1.0. RAID1 requires at least two disks, and specifying multiple installation disks without install_raid_level is rejected.

List the RAID members explicitly:

servers = [
  {
    number             = 1234567
    private_ipv4       = "10.0.88.2"
    install_disks      = ["nvme-disk-1", "nvme-disk-2"]
    install_raid_level = "raid1"
  }
]

Alternatively, omit install_disks to include every eligible disk automatically:

servers = [
  {
    number             = 1234567
    private_ipv4       = "10.0.88.2"
    install_raid_level = "raid1"
  }
]

preserve_other_disks defaults to false. Set it to true on an individual server to preserve eligible disks that are not used for the Talos installation or RAID.

⚠️ Warning: Installing Talos is destructive. By default, every eligible disk is wiped to prevent the server from booting a previous installation, and all data on those disks is lost. Preserved disks can still contain bootable installations, so the firmware might boot from them. RAID configuration is used only during initial provisioning; changing it does not reprovision existing servers.

Public Cluster Access

By default, the cluster is accessible over the public internet. The firewall is automatically configured to use the IPv4 address and /64 IPv6 CIDR of the machine running this module. To disable this automatic configuration, set the following variables to false:

firewall_use_current_ipv4 = false
firewall_use_current_ipv6 = false

To manually specify source networks for the Talos API and Kube API, configure the firewall_api_source variable as follows:

firewall_api_source = [
  "1.2.3.0/32",
  "1:2:3::/64"
]

This allows explicit control over which networks can access your APIs, overriding the default behavior when set.

Internal Cluster Access

If your internal network is routed and accessible, you can directly access the cluster using internal IPs by setting:

cluster_access = "private"

For integrating Talos nodes with an internal network, configure a default route (0.0.0.0/0) in the Hetzner Network to point to your router or gateway. Additionally, add specific routes on the Talos nodes to encompass your entire network CIDR:

talos_extra_routes = ["10.0.0.0/8"]

# Optionally, disable NAT for your globally routed CIDR
network_native_routing_ipv4_cidr = "10.0.0.0/8"

# Optionally, use an existing Network
hcloud_network_id = 123456789

This setup ensures that the Talos nodes can route traffic appropriately across your internal network.

Access to Kubernetes API

Optionally, a hostname can be configured to direct access to the Kubernetes API through a node IP, load balancer, or Virtual IP (VIP):

kube_api_hostname = "kube-api.example.com"

Access from Public Internet

For accessing the Kubernetes API from the public internet, choose one of the following options based on your needs:

1. Use single Control Plane IP (default):
   By default the IP address of a single Control Plane node is used to access the Kube API.
2. Use a Load Balancer:
   Deploy a load balancer to manage API traffic, enhancing availability and load distribution.

   kube_api_load_balancer_enabled = true
   

3. Use a Virtual IP (Floating IP):
   A Floating IP is configured to automatically move between control plane nodes in case of an outage, ensuring continuous access to the Kubernetes API.

   control_plane_public_vip_ipv4_enabled = true
   
   # Optionally, specify an existing Floating IP
   control_plane_public_vip_ipv4_id = 123456789
   

Access from Internal Network

When accessing the Kubernetes API via an internal network, an internal Virtual IP (Alias IP) is utilized by default to route API requests within the network. This feature can be disabled with the following configuration:

control_plane_private_vip_ipv4_enabled = false

To enhance internal availability, a load balancer can be used:

kube_api_load_balancer_enabled = true

This setup ensures secure and flexible access to the Kubernetes API, accommodating different networking environments.

Example kubernetes.tf snippet:

# Configuration for cluster autoscaler node pools
cluster_autoscaler_enabled = true

cluster_autoscaler_nodepools = [
  {
    name     = "autoscaler"
    type     = "cpx22"
    location = "nbg1"
    min      = 0
    max      = 6
    labels   = { "autoscaler-node" = "true" }
    taints   = [ "autoscaler-node=true:NoExecute" ]
  }
]

Optionally, pass additional Helm values to the cluster autoscaler configuration:

cluster_autoscaler_helm_values = {
  extraArgs = {
    enforce-node-group-min-size   = true
    scale-down-delay-after-add    = "45m"
    scale-down-delay-after-delete = "4m"
    scale-down-unneeded-time      = "5m"
  }
}

Talos Upgrades and Configuration Changes

Cluster Autoscaler does not support upgrading nodes or changing their configuration, as its primary purpose is to manage short-lived nodes that handle load peaks. If you require long-lived autoscaled nodes, you can upgrade them manually using talosctl or use this Terraform module, which supports discovery of autoscaled nodes and manages their upgrades and configuration changes.

To enable this feature, add the following to your configuration:

cluster_autoscaler_discovery_enabled = true

Please note that errors may occur if a node pool has been scaled down recently, as Talos caches absent nodes for up to 30 minutes. You can pause automatic scaling by stopping the Cluster Autoscaler pods:

kubectl -n kube-system scale deployment cluster-autoscaler-hetzner-cluster-autoscaler --replicas=0

The module installs a curated set of Kubernetes components through Talos manifests and reconciles them during the normal module lifecycle.

Core Components

Optional Components

Custom Resource Definitions

Additional Manifests

Extra manifests can be added alongside the built-in components:

talos_extra_remote_manifests = [
  "https://example.com/extra-remote-manifest.yaml"
]

talos_extra_inline_manifests = [
  {
    name = "test-manifest"
    contents = <<-EOF
      ---
      apiVersion: v1
      kind: Secret
      metadata:
        name: test-secret
      data:
        secret: dGVzdA==
    EOF
  }
]

Cilium CIDR Policy Match Mode

By default, when using K8s NetworkPolicies CIDR-based selectors in Cilium do not match in-cluster entities (pods or nodes). This is however required when using NetworkPolicies that control traffic to the kube-apiserver pods as these use the Talos node IPs in our setup. Cilium’s policy engine can be directed to select nodes by CIDR / ipBlock. This requires you to add the following to your configuration:

cilium_policy_cidr_match_mode = "nodes"

It is safe to toggle this option on a running cluster, and toggling the option affects neither upgrades nor downgrades More information can be found in the Cilium docs.

Cilium Transparent Encryption

This module enables Cilium Transparent Encryption feature by default.

All pod network traffic is encrypted using WireGuard (Default) or protocols, includes automatic key rotation and efficient in-kernel encryption, covering all traffic types.

💡 Although WireGuard is the default option, Hetzner Cloud VMs supports AES-NI instruction set, making IPSec encryption more CPU-efficient compared to WireGuard. Consider enabling IPSec for CPU savings through hardware acceleration.

IPSec mode supports RFC4106 AES-GCM encryption with 128, 192 and 256 bits key sizes.

⚠️ IPSec encryption has the following limitations:

• No transparent encryption when chaining Cilium with other CNI plugins
• Host Policies not supported with IPSec
• Incompatible with BPF Host Routing (automatically disabled on switch)
• IPv6-only clusters not supported
• Maximum 65,535 nodes per cluster/clustermesh
• Single CPU core limitation per IPSec tunnel may affect high-throughput scenarios

Source: Cilium Documentation

Example kubernetes.tf configuration:

cilium_encryption_enabled = true                # Default true
cilium_encryption_type    = "wireguard"         # wireguard (Default) | ipsec
cilium_ipsec_algorithm    = "rfc4106(gcm(aes))" # IPSec AES key algorithm (Default rfc4106(gcm(aes)))
cilium_ipsec_key_size     = 256                 # IPSec AES key size (Default 256)
cilium_ipsec_key_id       = 1                   # IPSec key ID (Default 1)

IPSec Key Rotation

Keys automatically rotate when cilium_ipsec_key_id is incremented (1-15 range, resets to 1 after 15).

Cilium offers an Egress Gateway to ensure network compatibility with legacy systems and firewalls requiring fixed IPs. The use of Cilium Egress Gateway does not provide high availability and increases latency due to extra network hops and tunneling. Consider this configuration only as a last resort.

Example kubernetes.tf snippet:

# Enable Cilium Egress Gateway
cilium_egress_gateway_enabled = true

# Define worker nodepools including an egress-specific node pool
worker_nodepools = [
  # ... (other node pool configurations)
  {
    name     = "egress"
    type     = "cpx22"
    location = "nbg1"
    labels   = { "egress-node" = "true" }
    taints   = [ "egress-node=true:NoSchedule" ]
  }
]

Example Egress Gateway Policy:

apiVersion: cilium.io/v2
kind: CiliumEgressGatewayPolicy
metadata:
  name: sample-egress-policy
spec:
  selectors:
    - podSelector:
        matchLabels:
          io.kubernetes.pod.namespace: sample-namespace
          app: sample-app

  destinationCIDRs:
    - "0.0.0.0/0"

  egressGateway:
    nodeSelector:
      matchLabels:
        egress-node: "true"

Please visit the Cilium documentation for more details.

Each rule is defined with the following properties:

• description: A brief description of the rule.
• direction: The direction of traffic (in for inbound, out for outbound).
• source_ips: A list of source IP addresses for inbound rules.
• destination_ips: A list of destination IP addresses for outbound rules.
• protocol: The protocol used (valid options: tcp, udp, icmp, gre, esp).
• port: The port number (required for tcp and udp protocols, must not be specified for icmp, gre, and esp).

Example kubernetes.tf snippet:

firewall_extra_rules = [
  {
    description = "Custom UDP Rule"
    direction   = "in"
    source_ips  = ["0.0.0.0/0", "::/0"]
    protocol    = "udp"
    port        = "12345"
  },
  {
    description = "Custom TCP Rule"
    direction   = "in"
    source_ips  = ["1.2.3.4", "1:2:3:4::"]
    protocol    = "tcp"
    port        = "8080-9000"
  },
  {
    description = "Allow ICMP"
    direction   = "in"
    source_ips  = ["0.0.0.0/0", "::/0"]
    protocol    = "icmp"
  }
]

For access to Talos and the Kubernetes API, please refer to the Cluster Access configuration section.

Kubernetes Gateway API is the modern replacement for Kubernetes Ingress. It fixes many Ingress limitations by offering a richer, more consistent model for traffic management, and it’s designed to support multiple Gateway API implementations in parallel.

This module installs the Gateway API CRDs by default and deploys Cert Manager with Gateway API support enabled.

Example with Cilium Gateway API and Cert Manager TLS Certificate

To use Cilium’s Gateway API implementation, configure:

cilium_gateway_api_enabled = true

If Cert Manager and Cilium weren’t initially deployed/configured with Gateway API support enabled, you may need to restart their controllers to pick up the new configuration:

kubectl -n cert-manager rollout restart deployment
kubectl -n kube-system rollout restart deployment/cilium-operator

Create a Cert Manager Issuer (Let’s Encrypt HTTP-01 via Gateway API)

Issuer configures Cert Manager to request TLS certificates from Let’s Encrypt using the ACME HTTP-01 challenge. The important part is the gatewayHTTPRoute solver: when Cert Manager needs to prove domain ownership, it will temporarily create/attach an HTTPRoute under the referenced Gateway and serve the /.well-known/acme-challenge/... response there.

Replace the placeholder email address, and note that privateKeySecretRef is the secret where Cert Manager stores your ACME account key (not the issued certificate).

apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: letsencrypt-http01
  namespace: default
spec:
  acme:
    email: <[email protected]>
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-http01-key
    solvers:
      - http01:
          gatewayHTTPRoute:
            parentRefs:
              - name: cilium-gateway
                namespace: default
                kind: Gateway

Optional: Create a Cert Manager ClusterIssuer (Let’s Encrypt DNS-01 via Hetzner DNS)

To request wildcard certificates or certificates that do not depend on public HTTP routing, enable the Hetzner DNS webhook:

cert_manager_enabled                 = true
cert_manager_webhook_hetzner_enabled = true

Your DNS zone must be managed by Hetzner DNS. Create a Secret containing a Hetzner API token with permission to create and remove DNS records. For ClusterIssuer resources, place the Secret in the cert-manager cluster resource namespace, which defaults to cert-manager.

apiVersion: v1
kind: Secret
metadata:
  name: hetzner
  namespace: cert-manager
stringData:
  token: <hetzner-dns-token>

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-dns01
spec:
  acme:
    email: <[email protected]>
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-dns01-key
    solvers:
      - dns01:
          webhook:
            groupName: acme.hetzner.com
            solverName: hetzner
            config:
              tokenSecretKeyRef:
                name: hetzner
                key: token

When using this ClusterIssuer with the Gateway shim, replace cert-manager.io/issuer: letsencrypt-http01 with cert-manager.io/cluster-issuer: letsencrypt-dns01.

Create the Gateway (Cilium GatewayClass + Hetzner LB + Cert Manager integration)

A Gateway defines the external entry point for traffic. With gatewayClassName: cilium, the resource is reconciled by the Cilium Gateway controller. The infrastructure.annotations are passed through as Hetzner-specific load balancer settings (interpreted by the Hetzner Cloud Controller Manager) to control how the Hetzner Cloud LB is created and configured. See: Load Balancer Annotations

The cert-manager.io/issuer: letsencrypt-http01 annotation is used by Cert Manager’s Gateway shim so it knows which Issuer to use when populating the TLS secret referenced by the listener.

tls.certificateRefs points at the Kubernetes Secret that will contain the issued certificate and private key (example-com-tls). Cert Manager will keep that secret up to date, and the Gateway will use it to terminate TLS.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: cilium-gateway
  namespace: default
  annotations:
    cert-manager.io/issuer: letsencrypt-http01
spec:
  gatewayClassName: cilium
  infrastructure:
    annotations:
      load-balancer.hetzner.cloud/name: "cilium-gateway-nbg1"
      load-balancer.hetzner.cloud/location: "nbg1"
      load-balancer.hetzner.cloud/uses-proxyprotocol: "true"
  listeners:
    - name: https
      hostname: example.com
      port: 443
      protocol: HTTPS
      allowedRoutes:
        namespaces:
          from: All
      tls:
        mode: Terminate
        certificateRefs:
          - name: example-com-tls
            kind: Secret
            group: ""
    - name: http
      hostname: example.com
      port: 80
      protocol: HTTP
      allowedRoutes:
        namespaces:
          from: All

Create an HTTPRoute (bind hostname + route traffic to your service)

Finally, the HTTPRoute attaches to the Gateway and defines routing rules for example.com. In this example it forwards all matching requests to a Kubernetes Service called example-service on port 8080.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: example-app
  namespace: default
spec:
  hostnames:
    - example.com
  parentRefs:
    - name: cilium-gateway
      namespace: default
  rules:
    - backendRefs:
        - name: example-service
          port: 8080

The Hetzner Cloud Container Storage Interface (CSI) driver can be flexibly configured through the hcloud_csi_storage_classes variable. You can define multiple storage classes for your cluster:

• name: The name of the StorageClass (string, required).
• encrypted: Enable LUKS encryption for volumes (bool, required).
• defaultStorageClass: Set this class as the default (optional, bool, defaults to false).
• reclaimPolicy: The Kubernetes reclaim policy (Delete or Retain, optional, defaults to Delete).
• extraParameters: Additional parameters for the StorageClass (optional map).

Example:

hcloud_csi_storage_classes = [
  {
    name                = "hcloud-volumes"
    encrypted           = false
    defaultStorageClass = true
  },
  {
    name                = "hcloud-volumes-encrypted-xfs"
    encrypted           = true
    reclaimPolicy       = "Retain"
    extraParameters     = {
      "csi.storage.k8s.io/fstype" = "xfs"
      "fsFormatOption"            = "-i nrext64=1"
    }
  }
]

Other settings:

• hcloud_csi_encryption_passphrase: Optionally provide a custom encryption passphrase for LUKS-encrypted storage classes.

  hcloud_csi_encryption_passphrase = "<secret-passphrase>"
  

Storage Class Immutability: StorageClasses created by the Hcloud CSI driver are immutable. To change parameters after creation, you must either edit the StorageClass directly with kubectl, or delete it from both Terraform state and Kubernetes, then let this module recreate it.

For more details, see the HCloud CSI Driver documentation.

⚠️ Experimental: IPv6 Pod networking is opt-in. Hetzner Private Networks currently support only IPv4. When you enable IPv6 Pod networking with the CIDRs below and leave the routing settings unchanged, Cilium carries IPv6 Pod traffic between nodes through its IPv4 VXLAN overlay, and the module disables the Hetzner Cloud CCM Route Controller. For Internet egress, Cilium’s IPv6 eBPF masquerading translates each Pod’s IPv6 source address to its node’s public IPv6 address. This NAT feature is beta, so dual-stack Pod networking is not supported for production use by this module.

To try it on a new cluster, provide separate IPv6 Pod and Service CIDRs:

network_pod_ipv6_cidr     = "fd42:6:175::/56"
network_service_ipv6_cidr = "fd42:6:176::/112"

Choose unique IPv6 CIDRs for your cluster. Kubernetes allocates a /64 Pod CIDR per node from the Pod range. Cilium carries both IPv4 and IPv6 Pod traffic over its IPv4 VXLAN tunnel. Hetzner Cloud CCM still manages nodes and load balancers. IPv6 Internet egress uses each node’s public IPv6 address, so keep talos_public_ipv6_enabled = true.

⚠️ Existing clusters: Enabling IPv6 Pod networking on an existing IPv4-only cluster requires additional migration steps and may disrupt Pod networking. Only do this if you know exactly what you are doing.

Longhorn is a lightweight, reliable, and easy-to-use distributed block storage system for Kubernetes. It is fully independent from the Hetzner Cloud CSI driver.

You can enable Longhorn and configure it as the default StorageClass for your cluster via module variables:

• Enable Longhorn: Set longhorn_enabled to true to deploy Longhorn in your cluster.

• Default StorageClass: Set longhorn_default_storage_class to true if you want Longhorn to be the default StorageClass.

Example:

longhorn_enabled               = true
longhorn_default_storage_class = true

Migrating existing Longhorn disks from /var/lib/longhorn to /var/mnt/longhorn:

Starting with module version 6, this module follows the Longhorn guidance for Talos Linux v1.10 and later, which recommends UserVolumeConfig and /var/mnt/longhorn instead of the legacy /var/lib/longhorn data path. Existing disks are not migrated automatically, so version 6 keeps the legacy path available for a rolling migration. Clusters created with an earlier module version must complete this migration before upgrading to module version 7 or higher.

1. Back up your volumes, verify that all volumes and replicas are healthy, and ensure that other storage nodes have enough capacity for one node’s replicas.
2. Upgrade to module version 6. It creates /var/mnt/longhorn and uses it for new Longhorn disks while keeping existing /var/lib/longhorn disks available.
3. Migrate each storage node separately:
   1. In the Longhorn UI, open Node.
   2. Select the node and choose Edit Node and Disks from the action menu.
   3. Find the disk with the /var/lib/longhorn path. Note any custom storage reservation or tags that must be retained.
   4. Set Scheduling to Disable, set Eviction Requested to true, and save.
   5. Wait until the disk shows 0 replicas and no backing images. Do not remove the disk before eviction finishes.
   6. Open Edit Node and Disks again, remove the /var/lib/longhorn disk, and save.
   7. Add a filesystem disk with the /var/mnt/longhorn path, restore any custom storage reservation or tags, enable scheduling, and save.
   8. Wait until the new disk is ready and all volumes are healthy before migrating the next node.

Do not enable scheduling on both paths of the same node at once. They share the node’s EPHEMERAL filesystem, so Longhorn would count the same capacity twice. Complete the migration before upgrading to module version 7 or higher, which will remove the legacy /var/lib/longhorn mount.

For more information about Longhorn, see the Longhorn documentation.

By default, this module calculates optimal subnets based on the provided network CIDR (network_ipv4_cidr). The network is segmented automatically as follows:

• 1st Quarter: Reserved for other uses such as classic VMs.
• 2nd Quarter:
  • 1st Half: Allocated for Node Subnets (network_node_ipv4_cidr)
  • 2nd Half: Allocated for Service IPs (network_service_ipv4_cidr)
• 3rd and 4th Quarters:
  • Full Span: Allocated for Pod Subnets (network_pod_ipv4_cidr)

Each Kubernetes node requires a /24 subnet within network_pod_ipv4_cidr. To support this configuration, the optimal node subnet size (network_node_ipv4_subnet_mask_size) is calculated using the formula:
32 - (24 - subnet_mask_size(network_pod_ipv4_cidr)).

With the default 10.0.0.0/16 network CIDR (network_ipv4_cidr), the following values are calculated:

• Node Subnet Size: /25 (Max. 128 Nodes per Subnet)
• Node Subnets: 10.0.64.0/19 (Max. 64 Subnets, each with /25)
• Service IPs: 10.0.96.0/19 (Max. 8192 Services)
• Pod Subnet Size: /24 (Max. 256 Pods per Node)
• Pod Subnets: 10.0.128.0/17 (Max. 128 Nodes, each with /24)

The shared worker subnet and the shared Cluster Autoscaler subnet are created by default. Worker and Cluster Autoscaler node pools use their shared subnets unless they pin a legacy node subnet with subnet = "10.0.65.0/25". Treat pinned subnet values as immutable after the node pool has been created.

Please consider the following Hetzner Cloud limits:

• Up to 100 servers can be attached to a network.
• Up to 100 routes can be created per network.
• Up to 50 subnets can be created per network.
• A project can have up to 50 placement groups.

A /16 Network CIDR is sufficient to fully utilize Hetzner Cloud’s scaling capabilities. It supports:

• Up to 100 nodes, each with its own /24 Pod subnet route.
• Configuration of up to 50 nodepools, each with at least one placement group.

Here is a table with more example calculations:

This module natively supports Hcloud Object Storage. Below is an example of how to configure backups with MinIO Client (mc) and Hcloud Object Storage. While it’s possible to create the bucket through the Hcloud Console, this method does not allow for the configuration of automatic retention policies.

Create an alias for the endpoint using the following command:

mc alias set <alias> \
  https://<location>.your-objectstorage.com \
  <access-key> <secret-key> \
  --api "s3v4" \
  --path "off"

Create a bucket with automatic retention policies to protect your backups:

mc mb --with-lock --region <location> <alias>/<bucket>
mc retention set GOVERNANCE 14d --default <alias>/<bucket>

Configure your kubernetes.tf file:

talos_backup_s3_hcloud_url = "https://<bucket>.<location>.your-objectstorage.com"
talos_backup_s3_access_key = "<access-key>"
talos_backup_s3_secret_key = "<secret-key>"

# Optional: AGE X25519 Public Key for encryption
talos_backup_age_x25519_public_key = "<age-public-key>"

# Optional: Change schedule (cron syntax)
talos_backup_schedule = "0 * * * *"

For users of other object storage providers, configure kubernetes.tf as follows:

talos_backup_s3_region   = "<region>"
talos_backup_s3_endpoint = "<endpoint>"
talos_backup_s3_bucket   = "<bucket>"
talos_backup_s3_prefix   = "<prefix>"

# Use path-style URLs (set true if required by your provider)
talos_backup_s3_path_style = true

# Access credentials
talos_backup_s3_access_key = "<access-key>"
talos_backup_s3_secret_key = "<secret-key>"

# Optional: AGE X25519 Public Key for encryption
talos_backup_age_x25519_public_key = "<age-public-key>"

# Optional: Change schedule (cron syntax)
talos_backup_schedule = "0 * * * *"

To recover from a snapshot, please refer to the Talos Disaster Recovery section in the Documentation.

By default, the module builds missing AMD64 and ARM64 Talos snapshots with Packer and selects images whose labels exactly match the cluster name, Talos version, and schematic ID. Image building and selection can be configured independently for each architecture:

talos_cloud_amd64_image_build_enabled = true
talos_cloud_amd64_image_selector      = "exact"

talos_cloud_arm64_image_build_enabled = true
talos_cloud_arm64_image_selector      = "exact"

The supported selector values are:

• exact: Selects an image matching the cluster name, Talos version, and schematic ID.
• latest: Selects the newest image matching only the cluster name and architecture.

Disabling an image builder is useful when matching snapshots are created outside this module. For example:

talos_cloud_arm64_image_build_enabled = false
talos_cloud_arm64_image_selector      = "exact"

If no exactly matching snapshot exists, the lookup fails instead of silently selecting an older image.

To disable the ARM64 builder and explicitly select the newest existing cluster snapshot:

talos_cloud_arm64_image_build_enabled = false
talos_cloud_arm64_image_selector      = "latest"

⚠️ Warning: The latest selector is strongly discouraged and unsupported. It ignores the configured Talos version and schematic ID, so new nodes may boot with incompatible Talos versions, system extensions, or kernel arguments. This can cause bootstrap failures, unavailable nodes, or cluster disruption. It is not an officially supported recovery or emergency mechanism. Use it entirely at your own risk.

Talos uses the public discovery service operated by Sidero Labs to discover cluster members, including when Kubernetes is unavailable. Nodes require outbound access to TCP port 443 to use it.

The service is enabled by default and can be disabled with:

talos_discovery_service_enabled = false

For more details, refer to the official Talos discovery guide.

The module can install and configure the Talos Tailscale system extension on every control plane, worker, Cluster Autoscaler, and bare metal node:

talos_tailscale_enabled  = true
talos_tailscale_auth_key = var.tailscale_auth_key

The module leaves containerboot defaults unchanged: Tailscale does not accept tailnet DNS settings and reruns tailscale up on each start. Additional settings can be supplied as KEY=VALUE entries:

talos_tailscale_environment = [
  "TS_EXTRA_ARGS=--advertise-tags=tag:kubernetes",
]

TS_AUTH_ONCE=true avoids repeat login when state persists, but later changes to TS_EXTRA_ARGS are not reapplied. Use it only if that trade-off is acceptable.

The module leaves Cilium device selection unchanged. In testing, Cilium auto-detected tailscale0, used an MTU of 1280, and allowed pods to reach other nodes over Tailscale. Restricting Cilium to eth+ or excluding tailscale0 raised the MTU to 1450 but blocked that pod-to-tailnet traffic. If the lower MTU causes bandwidth problems, evaluate cilium_helm_values.devices for your network and test the traffic your workloads need.

If you also use the Tailscale Kubernetes Operator to expose Services or a Service CIDR while Cilium replaces kube-proxy, ensure socket load balancing is bypassed in pod namespaces. Set cilium_socket_lb_host_namespace_only_enabled = true unless Cilium Gateway API already enables this setting. The Talos node extension alone does not require it.

If talos_schematic_id is set, its custom schematic must already contain siderolabs/tailscale, because the module cannot modify a supplied schematic. The auth key is sensitive, but it is necessarily stored in Terraform state and the rendered Talos machine configuration.

This module allows you to create custom Kubernetes RBAC (Role-Based Access Control) roles and cluster roles that define specific permissions for users and groups. RBAC controls what actions users can perform on which Kubernetes resources. These custom roles can be used independently or combined with OIDC group mappings to automatically assign permissions based on user group membership from your identity provider.

Example Configuration

Cluster Roles (rbac_cluster_roles)

rbac_cluster_roles = [
  {
    name  = my-cluster-role                    # ClusterRole name
    rules = [
      {
        api_groups = [""]                      # Core API group (empty string for core resources)
        resources  = ["nodes"]                 # Cluster-wide resources this role can access
        verbs      = ["get", "list", "watch"]  # Actions allowed on these resources
      }
    ]
  }
]

Namespaced Roles (rbac_roles)

rbac_roles = [
  {
    name      = "my-role"                      # Role name
    namespace = "target-namespace"             # Namespace where the role will be created
    rules = [
      {
        api_groups = [""]                      # Core API group (empty string for core resources)
        resources  = ["pods", "services"]      # Resources this role can access
        verbs      = ["get", "list", "watch"]  # Actions allowed on these resources
      }
    ]
  }
]

The Kubernetes API server supports OIDC (OpenID Connect) authentication, allowing integration with one or more external identity providers like Keycloak, Auth0, Authentik, Zitadel, etc. When enabled, users can authenticate using their existing organizational credentials instead of managing separate Kubernetes certificates or tokens.

OIDC authentication works by validating JWT tokens issued by your identity provider, extracting user information and group memberships, and mapping them to Kubernetes RBAC roles. Each issuer must be unique. Configure multiple clients from the same issuer as audiences of a single provider. The module automatically configures MatchAny when a provider has multiple audiences.

Example Configuration

kube_api_oidc_providers = [
  {
    issuer_url     = "https://keycloak.example.com/realms/example"
    audiences      = ["kubectl", "headlamp"]
    username_claim = "preferred_username"
    groups_claim   = "groups"
    groups_prefix  = "oidc:"

    group_mappings = [
      {
        group         = "cluster-admins-group"
        cluster_roles = ["cluster-admin"]
      },
      {
        group         = "developers-group"
        cluster_roles = ["view"]
        roles = [
          {
            name      = "developer-role"
            namespace = "development"
          }
        ]
      }
    ]
  }
]

Client Configuration with kubelogin

Once OIDC is configured in your cluster, you’ll need to configure your local kubectl to authenticate using OIDC tokens. This requires the kubelogin plugin.

Install kubelogin

# Homebrew (macOS and Linux)
brew install kubelogin

# Krew (macOS, Linux, Windows and ARM)
kubectl krew install oidc-login

# Chocolatey (Windows)
choco install kubelogin

Test OIDC Authentication

First, verify that your OIDC provider is returning proper JWT tokens. Replace the placeholder values with your actual OIDC configuration:

kubectl oidc-login setup \
  --oidc-issuer-url=https://your-oidc-provider.com \
  --oidc-client-id=your-client-id \
  --oidc-client-secret=your-client-secret \
  --oidc-extra-scope=openid,email,profile             # Add or change the scopes according to your IDP

This will open your browser for authentication. After successful login, you should see a JWT token in your terminal that looks like:

{
  "aud": "your-client-id",
  "email": "[email protected]",
  "email_verified": true,
  "exp": 1749867571,
  "groups": [
    "developers",
    "kubernetes-users"
  ],
  "iat": 1749863971,
  "iss": "https://your-oidc-provider.com",
  "nonce": "random-nonce-string",
  "sub": "user-unique-identifier"
}

Verify that:

• The groups array contains your expected groups
• The email field matches your user email
• email_verified is true (required by K8s)

Configure kubectl

Add a new user to your ~/.kube/config file:

users:
- name: oidc-user
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      command: kubectl
      args:
        - oidc-login
        - get-token
        - --oidc-issuer-url=https://your-oidc-provider.com
        - --oidc-client-id=your-client-id
        - --oidc-client-secret=your-client-secret
        - --oidc-extra-scope=groups
        - --oidc-extra-scope=email
        - --oidc-extra-scope=name                           # Add or change the scopes according to your IDP

Update your context to use the new OIDC user:

contexts:
- context:
    cluster: your-cluster
    namespace: default
    user: oidc-user          # Changed from certificate-based user
  name: oidc@your-cluster    # Updated context name

Now you can switch to the OIDC context and authenticate using your identity provider:

kubectl config use-context your-cluster-oidc
kubectl get pods  # This will trigger OIDC authentication

Talos allows adding additional trusted root CA certificates to the system. This is useful for private image registries, OIDC providers with internal CAs, or other enterprise services.

The talos_certificates variable supports several input formats for maximum flexibility:

Single Certificate (Inline)

talos_certificates = {
  "my-custom-ca" = "-----BEGIN CERTIFICATE-----\n...\n-----END CERTIFICATE-----"
}

Multiple Certificates (List)

talos_certificates = {
  "enterprise-ca" = [
    file("root.crt"),
    file("intermediate.crt")
  ]
}

Multi-Cert PEM Block (String)

talos_certificates = {
  "legacy-ca" = <<-EOT
    -----BEGIN CERTIFICATE-----
    ...
    -----END CERTIFICATE-----
    -----BEGIN CERTIFICATE-----
    ...
    -----END CERTIFICATE-----
  EOT
}