When I started playing with LXD I just accepted the default storage configuration which creates an image file and uses that to initialize a ZFS pool. Since I’m using ZFS as my main file system this seemed silly as LXD can use an existing dataset as a source for a storage pool. So I wanted to migrate my existing containers to the new storage pool.
Although others seemed to to have the same problem there was no ready answer. Digging through the documentation I finally found out that the lxc move command had a -s option … I had an idea. 💡 Here’s what I came up with …
Preparation
First we create the dataset on the existing ZFS pool and add it to LXC.
1 2 | sudo zfs create -o mountpoint=none mypool/lxd lxc storage create pool2 source=mypool/lxd |
lxc storage list should show something like this now:
1 2 3 4 5 6 7 | +-------+-------------+--------+--------------------+---------+ | NAME | DESCRIPTION | DRIVER | SOURCE | USED BY | +-------+-------------+--------+--------------------+---------+ | pool1 | | zfs | /path/to/pool1.img | 2 | +-------+-------------+--------+--------------------+---------+ | pool2 | | zfs | mypool/lxd | 0 | +-------+-------------+--------+--------------------+---------+ |
pool1 is the old pool backed by the image file and is used by some containers at the moment as can be seen in the “Used By” column. pool2 is added by not used by any contaiers yet.
Moving
We now try to move our containers to pool2.
1 2 3 4 | # move container to pool2 lxc move some_container some_container-moved -s=pool2 # rename container back for sanity ;) lxc move some_container-moved some_container |
We can check with lxc storage list whether we succeeded.
1 2 3 4 5 6 7 | +-------+-------------+--------+--------------------+---------+ | NAME | DESCRIPTION | DRIVER | SOURCE | USED BY | +-------+-------------+--------+--------------------+---------+ | pool1 | | zfs | /path/to/pool1.img | 1 | +-------+-------------+--------+--------------------+---------+ | pool2 | | zfs | mypool/lxd | 1 | +-------+-------------+--------+--------------------+---------+ |
Indeed pool2 is beeing used now. 😀 Just to be sure we check that zfs list -r mypool/lxd also reflects this.
1 2 3 4 5 6 7 | NAME USED AVAIL REFER MOUNTPOINT mypool/lxd/containers 1,08G 92,9G 24K none mypool/lxd/containers/some_container 1,08G 92,9G 704M /var/snap/lxd/common/lxd/storage-pools/pool2/containers/some_container mypool/lxd/custom 24K 92,9G 24K none mypool/lxd/deleted 24K 92,9G 24K none mypool/lxd/images 24K 92,9G 24K none mypool/lxd/snapshots 24K 92,9G 24K none |
Awesome!
⚠ Note that this only moves the container, but not the LXC image it was cloned off of.
We can repeat this until all containers we care about are moved over to pool2.
Cleanup
To prevent new containers to use pool1 we have to edit the default profile.
1 2 | # change devices.root.pool to pool2 lxc profile edit default |
Finally …. when we’re happy with the migration and we’ve verified that everything works as expected we can now remove pool1.
1 | lxc storage rm pool1 |