Adding shared iSCSI based storage to a Proxmox host and cluster

In this post we'll see how we can add external storage to a Proxmox Virtual Environment (PVE) host. We'll connect a PVE host to a Storage Array (SAN) through iSCSI and use the external storage as the storage backend for virtual machines.

First of let's take a look at our PVE host

The host is running on a physical server which has a couple of internal disks (this will not be used in this post)

The host also has a couple of network interface cards (NICs).

One of the network interfaces is used for the management traffic as well as the VM traffic (normally you'd separate these) where as the second NIC will be used to connect to the SAN with iSCSI.

On the SAN we have currently one iSCSI portal / target configured, and one LUN mapped to this target. The LUN is created from a RAID1 volume on the SAN backed by two HDDs of 1TB.

First of let's configure one of the NICs on the host to be able to connect to the SAN. We'll set an IP address on the same subnet as the SAN (this is not a requirement, the IPs can be on separate subnets)

Remember to apply the configuration

Now, let's verify that we can reach our SAN on the network

With that in place let's start configuring the iSCSI side of things on the PVE host

We'll start by adding the iSCSI target from the Datacenter->Storage menu

After filling the IP address in the portal field, the target should appear.

With this in place we should have the iSCSI target listed

If we check our newly added storage entity we can also see that we can see the actual LUN as available

Note that this is since we've let the default of "Use LUNs directly" be selected. This would let us add the disk directly to a virtual machine for it to use.

Also note that all LUNs available through this portal/target would have this setting. So if we present a second LUN from the same target it will appear in the VM Disks list

Instead of using these LUNs directly we would like to add them as storage pools so that they could be used by multiple VMs.

For making this happen we would go back to the Datacenter->Storage view and create a LVM volume

After creating a storage pool / datastore for both our mapped LUNs we should have them available on our host

It's time to put them to use by creating some virtual machines.

Ok, so after creating the VM and installing the OS we have workloads running on our iSCSI backed storage

Now, let's create a cluster and add a second host to the cluster, a host with a network interface able to reach our iSCSI storage.

As we can see we have the storage available also on our second host.

Now, let's try to migrate a virtual machine over to the second host

As we can see the migration is done, only migrating the state (memory) of the virtual machine

In the environment in use we have the migration done over a 1 Gb/s network so it will take some seconds based on the size of the state (memory)

With this in place we can make use of Proxmox HA functionality to get High availability for workloads. In case one of the hosts experiences an error which makes it and it's workloads unavailable the HA manager can start the workloads on the second host.

For this to work we'd need to add a third component as a witness component to prevent split-brain scenarios and to ensure quorum. Please check my previous post on how to configure this