LXD/Administration Tutorial

This page contains an LXD tutorial that can be used to learn more about profiles and other features of LXD.

Terminology

Container snapshots as the name states snapshots of the container in time and cannot be modified in any way. It is worth noting that because snapshots can store the container runtime state, which gives us ability of “stateful” snapshots. That is, the ability to rollback the container including its cpu and memory state at the time of the snapshot.

LXD is image based, all LXD containers come from an image. Images are typically clean Linux distribution images similar to what you would use for a virtual machine or cloud instance. It is possible to “publish” a container, making an image from it which can then be used by the local or remote LXD hosts.

Our first image

Let's get our hands even more dirty and create our first image. We will be using a generic 64 bit Funtoo Linux image.

   Note

The Funtoo's default build host is building only westmere stage for now.

Grab the image here: https://build.funtoo.org/1.4-release-std/x86-64bit/intel64-westmere/lxd-latest.tar.xz

Grab also the hash file: https://build.funtoo.org/1.4-release-std/x86-64bit/intel64-westmere/lxd-latest.tar.xz.hash.txt

   Tip

Check the hash of the downloaded file against the one from server. Proceed if they match.

Import the image

After we have successfully downloaded the archive we can now finally import it into LXD and start using it as our "seed" image for all our containers.

root # lxc image import lxd-latest.tar.xz --alias funtoo
Image imported with fingerprint: 6c2ca3af0222d503656f5a1838885f1b9b6aed2c1994f1d7ef94e2efcb7233c4
root # lxc image ls
+--------+--------------+--------+------------------------------------+--------+----------+-----------------------------+
| ALIAS  | FINGERPRINT  | PUBLIC |            DESCRIPTION             |  ARCH  |   SIZE   |         UPLOAD DATE         |
+--------+--------------+--------+------------------------------------+--------+----------+-----------------------------+
| funtoo | 6c2ca3af0222 | no     | Funtoo Current Generic Pure 64-bit | x86_64 |227.99MB  | Dec 13, 2017 at 11:01pm (UTC)  |
+--------+--------------+--------+------------------------------------+--------+----------+-----------------------------+

And there we have our very first Funtoo Linux image imported inside LXD. You can reference the image through the alias or through the fingerprint. Aliases can be added also later.

Let me show you some basic usage then.

Creating your first container

So now we can launch our first container. That is done using this command:

root # lxc launch funtoo fun-1
Creating fun-1
Starting fun-1
root #  lxc ls
+-------+---------+------+-----------------------------------------------+------------+-----------+
| NAME  |  STATE  | IPV4 |                     IPV6                      |    TYPE    | SNAPSHOTS |
+-------+---------+------+-----------------------------------------------+------------+-----------+
| fun-1 | RUNNING |      | fd42:156d:4593:a619:216:3eff:fef7:c1c2 (eth0) | PERSISTENT | 0         |
+-------+---------+------+-----------------------------------------------+------------+-----------+

   Tip

lxc launch is a shortcut for lxc init and lxc start, lxc init creates the container without starting it.

Profiles intermezzo

LXD has the ability to change quite a few container settings, including resource limitation, control of container startup and a variety of device pass-through options using what is called profiles. Multiple profiles can be applied to a single container, and the last profile overrides the other ones it the resources being configured is the same for multiple profiles. Let me show you how can this be used.

This is the default profile that gets inherited by all containers.

root # lxc profile list
+---------+---------+
|  NAME   | USED BY |
+---------+---------+
| default | 1       |
+---------+---------+

root #  lxc profile show default
config: {}
description: Default LXD profile
devices:
  eth0:
    nictype: bridged
    parent: lxdbr0
    type: nic
  root:
    path: /
    pool: default
    type: disk
name: default
used_by:
- /1.0/containers/fun-1

Now let's edit this profile for our funtoo containers. It will include some useful stuff.

root # lxc profile set default raw.lxc "lxc.mount.entry = none dev/shm tmpfs rw,nosuid,nodev,create=dir"
root # lxc profile set default environment.LANG "en_US.UTF-8"
root # lxc profile set default environment.LC_ALL "en_US.UTF-8"
root # lxc profile set default environment.LC_COLLATE "POSIX"

Profiles can store any configuration that a container can (key/value or devices) and any number of profiles can be applied to a container. Profiles are applied in the order they are specified so the last profile to specify a specific key wins. In any case, resource-specific configuration always overrides that coming from the profiles.

The default profile is set for any new container created which doesn't specify a different profiles list.

   Note

LXD supports simple instance types. Those are represented as a string which can be passed at container creation time. containers.md#instance-types

Using our first container

After we have done all these customizations we can now start using our container. The next command will give us shell inside the container.

root # lxc exec fun-1 bash

Now you should see a different prompt starting with

fun-1 ~ #

If we run top or ps for example we will see only the processes of the container.

fun-1 # ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0   4248   748 ?        Ss+  13:20   0:00 init [3]
root       266  0.0  0.0  30488   472 ?        Ss   13:20   0:00 /usr/sbin/sshd
root       312  0.2  0.0  17996  3416 ?        Ss   13:29   0:00 bash
root       317  0.0  0.0  19200  2260 ?        R+   13:29   0:00 ps aux

As you can see only the container's processes are shown. User running the processes is root here. What happens if we search for all sshd processes for example on the host box?

root # ps aux|grep ssh
root     14505  0.0  0.0  30564  1508 ?        Ss   Sep07   0:00 /usr/sbin/sshd   
100000   25863  0.0  0.0  30488   472 ?        Ss   15:20   0:00 /usr/sbin/sshd   
root     29487  0.0  0.0   8324   828 pts/2    S+   15:30   0:00 grep --colour=auto sshd
root #

So as you can see, the sshd process is running under user with uid 100000 on the host machine and has a different PID.

Basic actions with containers

Listing containers

root #  lxc ls
+-------+---------+-----------------------+------------------------------------------------+------------+-----------+
| NAME  |  STATE  |         IPV4          |                      IPV6                      |    TYPE    | SNAPSHOTS |
+-------+---------+-----------------------+------------------------------------------------+------------+-----------+
| fun-1 | RUNNING | 10.214.101.187 (eth0) | fd42:156d:4593:a619:a5ad:edaf:7270:e6c4 (eth0) | PERSISTENT | 0         |
|       |         |                       | fd42:156d:4593:a619:216:3eff:fef7:c1c2 (eth0)  |            |           |
+-------+---------+-----------------------+------------------------------------------------+------------+-----------+

lxc ls also accepts arguments as filters. For example lxc ls web will list all containers that have web in their name.

Container details

root # lxc info c1
Name: c1
Remote: unix://
Architecture: x86_64
Created: 2017/09/08 02:07 UTC
Status: Running
Type: persistent
Profiles: default, prf-funtoo
Pid: 6366
Ips:
  eth0: inet    10.214.101.79   vethFG4HXG
  eth0: inet6   fd42:156d:4593:a619:8619:546e:43f:2089  vethFG4HXG
  eth0: inet6   fd42:156d:4593:a619:216:3eff:fe4a:3d4f  vethFG4HXG
  eth0: inet6   fe80::216:3eff:fe4a:3d4f        vethFG4HXG
  lo:   inet    127.0.0.1
  lo:   inet6   ::1
Resources:
  Processes: 6
  CPU usage:
    CPU usage (in seconds): 25
  Memory usage:
    Memory (current): 69.01MB
    Memory (peak): 258.92MB
  Network usage:
    eth0:
      Bytes received: 83.65kB
      Bytes sent: 9.44kB
      Packets received: 188
      Packets sent: 93
    lo:
      Bytes received: 0B
      Bytes sent: 0B
      Packets received: 0
      Packets sent: 0

Container configuration

root #  lxc config edit c1
root ### This is a yaml representation of the configuration.
root ### Any line starting with a '# will be ignored.
root ###
root ### A sample configuration looks like:
root ### name: container1
root ### profiles:
root ### - default
root ### config:
root ###   volatile.eth0.hwaddr: 00:16:3e:e9:f8:7f
root ### devices:
root ###   homedir:
root ###     path: /extra
root ###     source: /home/user
root ###     type: disk
root ### ephemeral: false
root ###
root ### Note that the name is shown but cannot be changed

architecture: x86_64
config:
  image.architecture: x86_64
  image.description: Funtoo Current Generic Pure 64-bit
  image.name: funtoo-generic_64-pure64-funtoo-current-2016-12-10
  image.os: funtoo
  image.release: "1.0"
  image.variant: current
  volatile.base_image: e279c16d1a801b2bd1698df95e148e0a968846835f4769b24988f2eb3700100f
  volatile.eth0.hwaddr: 00:16:3e:4a:3d:4f
  volatile.eth0.name: eth0
  volatile.idmap.base: "0"
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'
  volatile.last_state.power: RUNNING
devices: {}
ephemeral: false
profiles:
- default
- prf-funtoo
stateful: false
description: ""

One can also add environment variables.

root # lxc config set <container> environment.LANG en_US.UTF-8
root # lxc config set <container> environment.LC_COLLATE POSIX

Resource control

LXD offers a variety of resource limits. Some of those are tied to the container itself, like memory quotas, CPU limits and I/O priorities. Some are tied to a particular device instead, like I/O bandwidth or disk usage limits.

As with all LXD configuration, resource limits can be dynamically changed while the container is running. Some may fail to apply, for example if setting a memory value smaller than the current memory usage, but LXD will try anyway and report back on failure.

All limits can also be inherited through profiles in which case each affected container will be constrained by that limit. That is, if you set limits.memory=256MB in the default profile, every container using the default profile (typically all of them) will have a memory limit of 256MB.

Disk

Setting a size limit on the container’s filesystem and have it enforced against the container. Right now LXD only supports disk limits if you’re using the ZFS or btrfs storage backend.

To set a disk limit (requires btrfs or ZFS):

root # lxc config device set c1 root size 20GB

CPU

To just limit a container to any 2 CPUs, do:

root # lxc config set c1 limits.cpu 2

To pin to specific CPU cores, say the second and fourth:

root # lxc config set c1 limits.cpu 1,3

More complex pinning ranges like this works too:

root # lxc config set c1 limits.cpu 0-3,7-11

Memory

To apply a straightforward memory limit run:

root # lxc config set c1 limits.memory 256MB

(The supported suffixes are kB, MB, GB, TB, PB and EB)

To turn swap off for the container (defaults to enabled):

root # lxc config set c1 limits.memory.swap false

To tell the kernel to swap this container’s memory first:

root # lxc config set c1 limits.memory.swap.priority 0

And finally if you don’t want hard memory limit enforcement:

root # lxc config set c1 limits.memory.enforce soft

Resource limits using profile - Funtoo Containers example

So I am going to create 3 profiles to mimic the resource limits for current Funtoo Containers.

Price	RAM	CPU Threads	Disk Space
$15/mo	4GB	6 CPU Threads	50GB
$30/mo	12GB	12 CPU Threads	100GB
$45/mo	48GB	24 CPU Threads	200GB

I am going to create one profile and copy/edit it for the remaining two options.

root # lxc profile create res-small
root # lxc profile edit res-small
config:
  limits.cpu: "6"
  limits.memory: 4GB
description: Small Variant of Funtoo Containers
devices:
  root:
    path: /
    pool: default
    size: 50GB
    type: disk
name: small
used_by: []
root # lxc profile copy res-small res-medium
root # lxc profile copy res-small res-large
root # lxc profile set res-medium limits.cpu 12
root # lxc profile set res-medium limits.memory 12GB
root # lxc profile device set res-medium root size 100GB
root # lxc profile set res-large limits.cpu 24
root # lxc profile set res-large limits.memory 48GB
root # lxc profile device set res-large root size 200GB

Now let's create a container and assign the res-small and funtoo profiles to it.

root # lxc init funtoo c-small
root # lxc profile assign c-small res-small
root # lxc profile add c-small funtoo