OpenTofu - Création 1ère VM

1. Création d'une première VM avec OpenTofu

1.1. Arborescence du projet test

test/
├── cloud-init
│   ├── user-config.yml
│   └── vendor-config.yml
├── main.tf
├── provider.tf
├── secrets.tfvars
├── variables.tf
└── variables.tfvars

1.2. Créer le fichier provider.tf

provider.tf

# file: provider.tf
terraform {
  required_providers {
    proxmox = {
      source  = "bpg/proxmox"
      version = "0.77.1"
    }
  }
}

provider "proxmox" {
  endpoint  = var.pm_endpoint
  api_token = var.pm_api_token
  insecure  = false
  ssh {
    agent    = true
    username = "root"
  }
}

1.3. Créer le fichier variables.tf

• Ce fichier déclare les variables (avec en option, une valeur par défaut)

# file: variables.tf
variable "pm_endpoint" {
  type = string
  default = "https://pve1.home.lab:8006"
}

variable "pm_ssh_username" {
  type        = string
  default     = "root"
}

variable "pm_api_token" {
  description = "Token to connect Proxmox API"
  type        = string
  default     = ""
}

variable "target_node" {
  description = "Proxmox node"
  type        = string
  default     = "pve1.home.lab"
}

variable "vm_onboot" {
  description = "Auto start VM when node is start"
  type        = bool
  default     = false
}

variable "target_node_domain" {
  description = "Proxmox node domain"
  type        = string
  default     = ""
}

variable "vm_hostname" {
  description = "VM hostname"
  type        = string
  default     = "test"
}

variable "vm_domain" {
  description = "VM domain"
  type        = string
  default     = "home.local"
}

variable "vm_tags" {
  description = "VM tags"
  type        = list(string)
  default     = [ "opentofu", "almalinux9" ]
}

variable "template_id" {
  type        = number
  default     = 10001
}

variable "template_tag" {
  description = "Template tag"
  type        = string
  default     = "template"
}

variable "vm_sockets" {
  description = "Number of sockets"
  type        = number
  default     = 1
}

variable "vm_cores" {
  description = "Number of cores"
  type        = number
  default     = 1
}

variable "vm_cpu_type" {
  description = "CPU type"
  type        = string
  default     = "x86-64-v2-AES"
}

variable "vm_memory" {
  description = "Number of memory in MB"
  type        = number
  default     = 2048
}

1.4. Créer le fichier variables.tfvars

• Ce fichier change la valeur par défaut de certaines variables

variables.tfvars

# Provider Settings
pm_endpoint = "https://pve1.home.lab:8006/"
pm_ssh_username = "root"
pm_datastore_id = "gv0"

# Proxmox Settings
target_node = "pve1.home.lab"

# Template Clone id
template_id = 10001
template_tag = "almalinux9"

# VM Settings
vm_onboot = false
vm_hostname = "test-almalinux9"
vm_memory = 2048
vm_sockets = 1
vm_cores = 2
vm_domain = "home.lab"
vm_address = "192.168.50.66/24"
vm_gateway = "192.168.50.201"

1.5. Créer le fichier secrets.tfvars

• Remplacer <TOKEN> par la valeur du token généré précédemment

pm_api_token = "tofu@pve!tofu=<TOKEN>"

1.6. Créer le fichier main.tf

main.tf

# file: main.tf

# https://cloudinit.readthedocs.io/en/22.1_a/topics/examples.html
resource "proxmox_virtual_environment_file" "user_config" {
  content_type  = "snippets"
  datastore_id  = "gv0"
  overwrite     = true
  node_name     = var.target_node
  source_raw {
    data        = file("cloud-init/user-config.yml")
    file_name   = "user-config.yml"
  }
}

# https://cloudinit.readthedocs.io/en/22.1_a/topics/vendordata.html
resource "proxmox_virtual_environment_file" "vendor_config" {
  content_type  = "snippets"
  datastore_id  = "gv0"
  node_name     = var.target_node
  source_raw {
    data        = file("cloud-init/vendor-config.yml")
    file_name   = "vendor-config.yml"
  }
}

# https://registry.terraform.io/providers/bpg/proxmox/latest/docs/resources/virtual_environment_vm
resource "proxmox_virtual_environment_vm" "almalinux_vm" {
  # Dependencies
  depends_on = [ 
    proxmox_virtual_environment_file.user_config,
    proxmox_virtual_environment_file.vendor_config 
  ]
  
  # Proxmox node
  node_name = var.target_node

  # VM
  name = "${var.vm_hostname}.${var.vm_domain}"
  description = "Managed by OpenTofu"
  tags = ["opentofu", var.template_tag]
  clone {
    vm_id = var.template_id
    full = true
  }
  on_boot = var.vm_onboot
  #started = false
  #reboot = false
  #reboot_after_update = false
  
  # VM Options  
  operating_system {
    type = "l26"
  }
  agent {
    enabled = true
  }
  
  # VM Memory
  memory {
    dedicated = var.vm_memory
  }
  # VM Processors
  cpu {
    sockets = var.vm_sockets
    cores = var.vm_cores
    type = "x86-64-v2-AES"
    numa = false
  }
  # VM BIOS
  bios = "seabios"
  # VM Disk Settings
  scsi_hardware = "virtio-scsi-single"
  # VM Network Device Settings
  network_device {
    bridge = "vmbr0"
    model = "virtio"
    firewall = true
  }
  lifecycle {
    ignore_changes = [
      network_device,
    ]
  }
  # VM Customization with cloud-init
  # https://cloudbase-init.readthedocs.io/en/latest/config.html
  # https://cloudinit.readthedocs.io/en/latest/reference/examples.html
  initialization {
    # bug : replicated datastore GlusterFS does not work !
    #datastore_id = "gv0"
    dtastore = "local"
    ip_config {
      ipv4 {
        address = "${var.vm_address}"
        gateway = "${var.vm_gateway}"
      }
    }        
    # identifier for a file containing custom user data (conflicts with user_account).
    user_data_file_id   = proxmox_virtual_environment_file.user_config.id
    # identifier for a file containing all vendor data passed to the VM via cloud-init.
    vendor_data_file_id = proxmox_virtual_environment_file.vendor_config.id    
  }
}

2. Création des fichiers cloud-init

• De nombreux exemples sont disponibles sur le site de cloud-init
  • https://cloudinit.readthedocs.io/en/latest/reference/examples_library.html

2.1. Créer le fichier cloud-init/user-config.yml

• Par exemple, on peut ajouter la clé SSH d'ansible
  • https://cloudinit.readthedocs.io/en/latest/reference/yaml_examples/ansible_managed.html#cce-ansible-managed

cloud-init/user-config.yml

#cloud-config
hostname: test2-almalinux9
local-hostname: test2-almalinux9
fqdn: test2-almalinux9.home.lab
ssh_pwauth: true

users:
- name: ansible
  gecos: Ansible User
  groups: users,admin,wheel
  sudo: "ALL=(ALL) NOPASSWD:ALL"
  shell: /bin/bash
  lock_passwd: true
  ssh_authorized_keys:
    - "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRCJCQ1UD9QslWDSw5Pwsvba0Wsf1pO4how5BtNaZn0xLZpTq2nqFEJshUkd/zCWF7DWyhmNphQ8c+U+wcmdNVcg2pI1kPxq0VZzBfZ7cDwhjgeLsIvTXvU+HVRtsXh4c5FlUXpRjf/x+a3vqFRvNsRd1DE+5ZqQHbOVbnsStk3PZppaByMg+AZZMx56OUk2pZCgvpCwj6LIixqwuxNKPxmJf45RyOsPUXwCwkq9UD4me5jksTPPkt3oeUWw1ZSSF8F/141moWsGxSnd5NxCbPUWGoRfYcHc865E70nN4WrZkM7RFI/s5mvQtuj8dRL67JUEwvdvEDO0EBz21FV/iOracXd2omlTUSK+wYrWGtiwQwEgr4r5bimxDKy9L8UlaJZ+ONhLTP8ecTHYkaU1C75sLX9ZYd5YtqjiNGsNF+wdW6WrXrQiWeyrGK7ZwbA7lagSxIa7yeqnKDjdkcJvQXCYGLM9AMBKWeJaOpwqZ+dOunMDLd5VZrDCU2lpCSJ1M="

2.2. Créer le fichier cloud-init/vendor-config.yml

cloud-init/vendor-config.yml

#cloud-config

3. Provisionnement de la VM sur proxmox : tofu apply

3.1. Initialisation : tofu init

$ tofu init

Initializing the backend...

Initializing provider plugins...
- Finding bpg/proxmox versions matching "0.77.1"...
- Installing bpg/proxmox v0.77.1...
- Installed bpg/proxmox v0.77.1 (signed, key ID F0582AD6AE97C188)

3.2. Vérification de la planification : tofu plan

$ tofu plan -var-file=variables.tfvars

OpenTofu used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

OpenTofu will perform the following actions:

  # proxmox_virtual_environment_vm.almalinux_vm will be created
  + resource "proxmox_virtual_environment_vm" "almalinux_vm" {
      + acpi                    = true
      + bios                    = "seabios"
      + description             = "Managed by OpenTofu"
      + id                      = (known after apply)
      + ipv4_addresses          = (known after apply)
      + ipv6_addresses          = (known after apply)
      + keyboard_layout         = "en-us"
      + mac_addresses           = (known after apply)
      + machine                 = "q35"
      + migrate                 = false
      + name                    = "test.home.local"
      + network_interface_names = (known after apply)
      + node_name               = "pve1.home.lab"
      + on_boot                 = true
      + protection              = false
      + reboot                  = false
      + reboot_after_update     = true
      + scsi_hardware           = "virtio-scsi-single"
      + started                 = true
      + stop_on_destroy         = false
      + tablet_device           = true
      + tags                    = [
          + "opentofu",
          + "almalinux9",
        ]
      + template                = false
      + timeout_clone           = 1800
      + timeout_create          = 1800
      + timeout_migrate         = 1800
      + timeout_move_disk       = 1800
      + timeout_reboot          = 1800
      + timeout_shutdown_vm     = 1800
      + timeout_start_vm        = 1800
      + timeout_stop_vm         = 300
      + vm_id                   = (known after apply)

      + agent {
          + enabled = true
          + timeout = "15m"
          + trim    = false
          + type    = "virtio"
        }

      + clone {
          + full    = true
          + retries = 1
          + vm_id   = 10001
        }

      + cpu {
          + cores      = 2
          + hotplugged = 0
          + limit      = 0
          + numa       = true
          + sockets    = 1
          + type       = "x86-64-v2-AES"
          + units      = 1024
        }

      + disk {
          + aio               = "io_uring"
          + backup            = true
          + cache             = "none"
          + datastore_id      = "gv0"
          + discard           = "ignore"
          + file_format       = "raw"
          + interface         = "scsi0"
          + iothread          = true
          + path_in_datastore = (known after apply)
          + replicate         = true
          + size              = 10
          + ssd               = false
        }

      + initialization {
          + datastore_id         = "gv0"
          + meta_data_file_id    = (known after apply)
          + network_data_file_id = (known after apply)
          + type                 = (known after apply)
          + user_data_file_id    = (known after apply)
          + vendor_data_file_id  = (known after apply)

          + ip_config {
              + ipv4 {
                  + address = "192.168.50.66/24"
                  + gateway = "192.168.50.201"
                }
            }
        }

      + memory {
          + dedicated      = 2048
          + floating       = 0
          + keep_hugepages = false
          + shared         = 0
        }

      + network_device {
          + bridge      = "vmbr0"
          + enabled     = true
          + firewall    = false
          + mac_address = (known after apply)
          + model       = "virtio"
          + mtu         = 0
          + queues      = 0
          + rate_limit  = 0
          + vlan_id     = 0
        }

      + operating_system {
          + type = "l26"
        }

      + vga (known after apply)
    }

3.3. Déploiement de la planification sur Proxmox : tofu apply

• 1er test : il ne reste plus qu'à créer la VM de test AlmaLinux9... suspense !

$ tofu apply -var-file=variables.tfvars
[...]
Do you want to perform these actions?
  OpenTofu will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

proxmox_virtual_environment_file.vendor_config: Creating...
proxmox_virtual_environment_file.user_config: Creating...
proxmox_virtual_environment_file.user_config: Creation complete after 3s [id=gv0:snippets/user-config.yml]
proxmox_virtual_environment_file.vendor_config: Creation complete after 3s [id=gv0:snippets/vendor-config.yml]
proxmox_virtual_environment_vm.almalinux_vm: Creating...
proxmox_virtual_environment_vm.almalinux_vm: Still creating... [10s elapsed]
proxmox_virtual_environment_vm.almalinux_vm: Still creating... [20s elapsed]
proxmox_virtual_environment_vm.almalinux_vm: Still creating... [30s elapsed]
proxmox_virtual_environment_vm.almalinux_vm: Still creating... [40s elapsed]
proxmox_virtual_environment_vm.almalinux_vm: Still creating... [50s elapsed]
proxmox_virtual_environment_vm.almalinux_vm: Still creating... [1m0s elapsed]
proxmox_virtual_environment_vm.almalinux_vm: Still creating... [1m10s elapsed]
proxmox_virtual_environment_vm.almalinux_vm: Creation complete after 1m6s [id=118]
╷
│ Error: error updating VM: received an HTTP 596 response - Reason: Connection timed out
│
│   with proxmox_virtual_environment_vm.almalinux_vm,
│   on main.tf line 24, in resource "proxmox_virtual_environment_vm" "almalinux_vm":
│   24: resource "proxmox_virtual_environment_vm" "almalinux_vm" {

3.4. Erreur "HTTP 596 responses - Reason: Connection timed out"

• La commande précédente renvoie une erreur: _"HTTP 596 responses - Reason: Connection timed out"
  • Après avoir testé différentes configurations, j'ai trouvé un problème similaire dans un ticket sur Github (https://github.com/bpg/terraform-provider-proxmox/issues/1312).
  • En modifiant le stockage de l'initialisation sur le datastore local plutôt que mon datastore GlusterFS gv0, je n'ai plus l'erreur !

main.tf

  # VM Customization with cloud-init
  initialization {
-   datastore_id = "gv0"
+   datastore_id = "local"
    ip_config {

• 2ème test : provisionnement de la VM de test AlmaLinux9

$ tofu apply -var-file=variables.tfvars
[...]
Do you want to perform these actions?
  OpenTofu will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

proxmox_virtual_environment_vm.almalinux_vm: Creating...
proxmox_virtual_environment_vm.almalinux_vm: Still creating... [10s elapsed]
proxmox_virtual_environment_vm.almalinux_vm: Still creating... [20s elapsed]
proxmox_virtual_environment_vm.almalinux_vm: Still creating... [30s elapsed]
proxmox_virtual_environment_vm.almalinux_vm: Still creating... [40s elapsed]
proxmox_virtual_environment_vm.almalinux_vm: Still creating... [50s elapsed]
proxmox_virtual_environment_vm.almalinux_vm: Creation complete after 52s [id=120]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

4. Ressources

• https://blog.stephane-robert.info/docs/virtualiser/type1/proxmox/terraform/
• https://github.com/CultureLinux/opentofu/blob/main/proxmox-clone-ansible/main.tf
• https://xieme-art.org/post/instancier-une-machine-virtuelle-debian-dans-proxmox-avec-opentofu-et-cloud-init/
• https://registry.terraform.io/providers/bpg/proxmox/latest/docs#example-usage