Some months ago I attended to a talk in which it was explained how to provision infrastructure in AWS with Terraform, I was talking with @raularanda (the host of that talk) about the idea of making a lab similar but with KVM, this entry it’s the result of that conversation.
Terraform is a software which can help the user manage, build and change cloud infrastructure using code, it can interact with various API’s using providers (plugins), these peaces of software are a sort of intermediary between Terraform and those API’s. Terraform can be used to manage the Lifecycle of various created ‘resources’ of those API’s. A resource can be any kind of type of virtual device but also be just a file or a directory.
KVM (Kernel-based Virtual Machine) is an open-source virtualization system for Linux systems. It’s a hypervisor that can be used to create virtualized, hardware accelerated virtual machines. It can achieve this using the built-in QEMU hardware emulator to emulate processors. QEMU and KVM are both configured using XML. In general, KVM/QEMU are used together in combination with a virtualization library called libvirt, but they can be used without it.
Cloud-init is the industry standard multi-distribution method for cross-platform cloud instance initialization. It is supported across all major public cloud providers, provisioning systems for private cloud infrastructure, and bare-metal installations
So to provision a VM in KVM with Terraform is needed to describe some computing resources to support the VM and configure it using Terraform-Libvirt-provider plugin, the resources could be:
- CPUs (vpcu’s)
- Memory
- Users (SSH keys)
Setup libvirtd
In this article is assumed that you have installed KVM, it’s not your case you could check this entry, but for this lab it’s needed to turn off SElinux security driver for libvirt (enabled by default) because it will prevent from creating a domain:
# vim /etc/libvirt/qemu.conf
...
security_driver= [ "none" ]
# systemctl restart libvirtd
# systemctl status libvirtd
...
Active: active (running) since Sun 2020-02-02 17:40:18 CET; 13min ago
...
libvirtd[149803]: Configured security driver "none" disables default policy to create confined guests
Libvirt requires the host machine’s user permissions to be able to allocate computing resources.
# usermod -aG kvm,libvirt <USER>
$ su - <USER>
$ id -nG
<USER_GRP> ... kvm ... libvirt
Setup a workspace
To have all our work organized it’s strongly recommended to create a directory structure to keep all needed files in our project, images and downloads, in one place:
$ mkdir -p /mnt/libvirt/terraform/images/debian_pool /mnt/libvirt/terraform/downloads
Download image needed to this project, as we are going to base our VM on Debian stable we need to download it in our downloads directory:
$ cd /mnt/libvirt/terraform/downloads
$ wget https://cdimage.debian.org/debian-cd/10.2.0/amd64/iso-cd/debian-10.2.0-amd64-netinst.iso
Installing Terraform
# wget https://releases.hashicorp.com/terraform/0.12.20/terraform_0.12.20_linux_amd64.zip
$ unzip terraform_0.12.20_linux_amd64.zip
# mv terraform /usr/local/bin/
$ terraform -v
Terraform v0.12.20
+ provider.libvirt (unversioned)
+ provider.template v2.1.2
# mkdir /mnt/libvirt/terraform
# chown <USER>:<USER> /mnt/libvirt/terraform
$ cd /mnt/libvirt/terraform && terraform init
Terraform initialized in an empty directory!
Installing Libvirt provider
$ cd ~/.terraform.d
$ mkdir plugins
$ wget https://github.com/dmacvicar/terraform-provider-libvirt/releases/download/v0.6.0/terraform-provider-libvirt-0.6.0+git.1569597268.1c8597df.Ubuntu_18.04.amd64.tar.gz
$ tar xvf terraform-provider-libvirt-0.6.0+git.1569597268.1c8597df.Ubuntu_18.04.amd64.tar.gz
$ mv terraform-provider-libvirt ~/.terraform.d/plugins/
Setup VM disk
At this point we’re going to clone our already semi-configured image created following this approach, as a requirement for this step a VG vg-kvm should be created and also a LV called master, as described in .
# lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
...
master vg-kvm -wi-a----- 10.00g
# qemu-img convert -f raw -O qcow2 /dev/vg-kvm/master /mnt/libvirt/terraform/images/debian_pool/base.qcow2
# chown <USER>:<USER> /mnt/libvirt/terraform/images/debian_pool/base.qcow2
$ qemu-img info /mnt/libvirt/terraform/images/debian_pool/base.qcow2
image: /mnt/libvirt/terraform/images/debian_pool/base.qcow2
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 2.48 GiB
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
corrupt: false
$ ls -lh /mnt/libvirt/terraform/images/debian_pool/base.qcow2
-rw-r--r-- 1 <USER> <USER> 2.5G Feb 2 17:30 /mnt/libvirt/terraform/images/debian_pool/base.qcow2
Setup Terraform configuration (Libvirt Provider)
Let’s create libvirt.tf file for your VM deployment on KVM, Terraform uses ‘.tf’ files to define its configuration (using Hashicorp scripting language). Our target is to create a new VM and allocate the resources needed to support it. In ‘.tf’ files it can be found ${} to call functions and # to define comments. So let’s create a pool, volume, cloud_init config, and a domain:
- Resource definition:
resource "resource_type" "resource_name" { ... }- Resource type: resource we want to provide for a certain provider, libvirt_pool: the store to be used by VM images
- Resource name: user resource identifier (debian)
- Name: user pool identifier (debian-pool)
- Type: type dir provides the means to manage files within a directory, files here could have formats like qcow, vmdk or just raw
- Path: storage location
# add the provider, this code will connect to Hypervisor using libvirt
provider "libvirt" {
uri = "qemu:///system"
}
# create pool
resource "libvirt_pool" "debian" {
name = "debian-pool"
type = "dir"
path = "${path.module}/images/debian_pool"
}
# create image volume
resource "libvirt_volume" "image-raw" {
name = "debian-buster-amd64.qcow2"
##name = ${var.name}
pool = libvirt_pool.debian.name
source = "${path.module}/images/debian_pool/base.qcow2"
##source = "https://cdimage.debian.org/cdimage/openstack/current-10/debian-10-openstack-amd64.qcow2"
#format = "raw"
format = "qcow2"
}
# add cloudinit disk to pool
resource "libvirt_cloudinit_disk" "commoninit" {
name = "commoninit.iso"
pool = libvirt_pool.debian.name
user_data = data.template_file.user_data.rendered
}
# read the configuration
data "template_file" "user_data" {
template = file("${path.module}/cloud_init.cfg")
}
# Define KVM domain to create
resource "libvirt_domain" "master-k8s" {
name = "master-k8s"
memory = "4096"
vcpu = 2
network_interface {
network_name = "default"
}
disk {
volume_id = libvirt_volume.image-raw.id
}
console {
type = "pty"
target_type = "serial"
target_port = "0"
}
# type = "none" not allowed by the provider
graphics {
type = "spice"
listen_type = "address"
autoport = true
}
}
Setting up cloud-init for configuring the VM
Create a file called cloud_init.cfg which is going to use yaml syntax to create VM users
$ vim /mnt/libvirt/terrraform/cloud_init.cfg
#cloud-config
users:
- name: foo
sudo: ALL=(ALL) NOPASSWD:ALL
groups: users, admin
home: /home/foo
shell: /bin/bash
ssh_authorized_keys:
- ssh-rsa <PUB_KEY>
# install packages
packages:
- git
<PUB_KEY> has to be replaced by the content of this file ~/.ssh/id_rsa.pub
/mnt/libvirt/terraform] $ terraform init
Initializing the backend...
Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "template" (hashicorp/template) 2.1.2...
...
* provider.template: version = "~> 2.1"
Terraform has been successfully initialized!
...
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
$ terraform providers
.
├── provider.libvirt
└── provider.template
$ terraform plan
$ terraform apply
$ virsh pool-list
Name State Autostart
-----------------------------------
debian-pool active yes
$ virsh list --all
Id Name State
--------------------------------
1 master-k8s running
$ virsh console master-k8s
##Ctrl+ 5 to exit virsh console
$ terraform destroy
...
Destroy complete! Resources: 4 destroyed.
Reference links:
- https://medium.com/@niteshvganesh/instructions-on-how-to-use-terraform-to-create-a-small-scale-cloud-infrastructure-8c14cb8603a3#73cd
- https://computingforgeeks.com/how-to-provision-vms-on-kvm-with-terraform/
Happy Terraforming!
—
“There is only one corner of the universe you can be certain of improving, and that’s your own self.”
–Aldous Huxley
