Automated guest vm provisioning on KVM

There are multiple ways to set up guest virtual machines on KVM with virsh, each with their own strengths and drawbacks. Picking the best fit for your use case depends on many variables and understanding these tradeoffs.

Unattended installation

Most popular operating systems allow for unattended installation, and virt-install builds on this feature by streamlining the experience with libosinfo data. In practice, this means the unattended installation procedure for guest operating systems is the same no matter what tools each uses internally.

You simply add the --unattended flag to virt-install with a comma-separated list of options:

• profile= - either desktop or jeos ("just enough os", for servers). not all os support both profiles. default is "desktop"

• user-login= - the name of the user account to create

• admin-password-file/user-password-file - file containing the password for the admin/user account

• product-key= - license key to activate on the guest os after installation

Then you can create a vm from a local iso file:

virt-install --cdrom windows7.iso --unattended profile=desktop,user-login=sample_user,user-password-file=user_pass.txt

Or without an iso file for supported linux distributions:

virt-install --install fedora29 --unattended profile=jeos,admin-password-file=admin_pass.txt

Unattended installation with virt-install is the only reasonable option for vms with desktop environments or windows, but can be unreliable for other operating systems. Some like fedora and ubuntu LTS have decent support, while many other distros are not fully supported.

This mechanism is largely inefficient because it still needs to run the entire installation routine, wasting time and hardware resources, and also duplicates basic installation data for each vm running on the host. It has no support for creating multiple users, network/disk layouts or custom configurations and should only be used if the other methods describe below are not suitable.

cloud-init images

Many common linux distros now offer "cloud images", aka vm diskswith the os preinstalled and cloud-init set up to run on first boot.

There are some convenience options for virt-install to interact directly with cloud-init:

• root-password-generate=on/root-password-file= - set root password to a randomly generated string or read from a local file

• disable=on - disable cloud-init after first boot. always enable this, or your vm auth resets on reboot!

• root-ssh-key - authorize this public key for ssh access to the vms root user

• network-config=/user-data= - supply custom cloud-init configuration files

Once you downloaded a cloud image for your distro, you use --import and supply the cloud-init config, the simplest usage would be:

virt-install --import --os-variant debian12 \
  --disk path=guest.qcow2,format=qcow2,size=50,backing_store=debian-12-genericcloud-amd64.qcow2,backing_format=qcow2 \
  --cloud-init root-password-generate=on,disable=on

Notice the --disk not using the distro image directly, but instead automatically creating a new disk guest.qcow2, using the image as backing_store. This way, the base image is used as a readonly layer, overlayed with the newly created disk image. Writes are stored on the guest disk, making it safe to reuse the same base image across multiple vms, saving disk space on the kvm host.

You can also supply more advanced cloud-init configuration from files:

network.yaml

version: 2
ethernets:
  eth0:
    addresses: [192.168.1.100/24]
    gateway4: 192.168.1.1
    nameservers:
      addresses: [8.8.8.8]

user.yaml

ssh_pwauth: true
disable_root: false

chpasswd:
  expire: false
  list: |
    root:mypassword

Then pass them to --cloud-init:

virt-install --import --os-variant debian12 \
  --disk path=guest.qcow2,format=qcow2,size=50,backing_store=debian-12-genericcloud-amd64.qcow2,backing_format=qcow2 \
  --cloud-init user-data=user.yaml,network-config=network.yaml

Using cloud-init skips the os installation and only runs minimal necessary setup on first boot, and reuses base distro images across vms to save disk space. You have near complete control over guest customization, but some distros do not have cloud-init enabled images available, and managing external configuration files can be annoying at scale.

Customizing vm disk images

If the above options did not work for you, you can roll your own solution with virt-customize for linux guests.

You can take any os image (either downloaded from the official website or from the disk file after installing it locally), then customize it to your needs, and use that as a base image for new guests.

For example, we can use a non cloud-init enabled debian12 image and set the root password to 123, install the tree package and run a command to enable the openssh-server after first boot:

virt-customize -a debian12.qcow2 \
  --root-password password:123 \
  --install tree,openssh-server \
  --firstboot-command 'systemctl enable --now openssh-server'

The newly configured image can be used just like any other disk image for new guests:

virt-install --import \
  --disk path=guest.qcow2,format=qcow2,size=50,backing_store=debian12.qcow2,backing_format=qcow2

The virt-customize tool has many more options to set hostnames, network configuration, run scripts after first boot or create and manage users, directories and files. Have a look at man virt-customize for a complete list.

Using customized images is great if you intend to use images incompatible with cloud-init, or plan to reuse a specific base guest configuration for many vms in the future. The extra steps make it less appealing for one-time vms or automated setup of popular linux distros, which could be set up easier with cloud-init, although it offers even more control over customizations.