LXD & LXC

This article is part of a series of guides that describe NixNet's setup in excruciating detail. If you would like to follow them, please start at the Infrastructure page.

Caution: this guide is not finished yet; following it may leave you with an unuseable machine. To be notified of updates, please create an account and add it to your watchlist.

Custom Images

lxc publish <container>

LXD Networking

Ignore all of the IPv6-related steps; this leads to a broken IPv6 configuration! We're working on fixing our setup and will update these docs once we've figured out the issues.

Before setting any static IPs, you need to enable stateful DHCPv6 on LXD's virtual NIC. For more info about why, see a related thread on LXC's Discouse insance.

lxc network set lxdbr0 ipv6.dhcp.stateful true

To bind a container to a host port with NAT, you need a static IP on the container:

lxc network attach lxdbr0 <instance> eth0 eth0
lxc config device set <instance> eth0 ipv4.address=<container-ip> ipv6.address=<container-ip>

Then the proxy device needs to be added. Note the square brackets [] surrounding the IPv6 address in the second command.

lxc config device add <instance> port-forward_ipv4 proxy connect=tcp:<container-ip>:<port> listen=tcp:<host-public-ip>:<port> nat=true
lxc config device add <instance> port-forward_ipv6 proxy connect=tcp:[<container-ip>]:<port> listen=tcp:[<host-public-ip>]:<port> nat=true

<host-ip> needs to be the public IP address of your server.

LXD Storage Mounts

The following commands and paths are formatted for internal NixNet usage. You may need change the paths to fit your needs.

The LXC help gives this hint when adding a device:

lxc config device add [<remote>:]<container-name> <device-name> disk source=/opt/<service-name> path=opt

This adds a device of type disk to <container-name>. The source is the host's file path, and path is the path in the container.

Due to how permissioning works on unprivileged containers, the mount permissions have to be changed on the host system. Firstly you have to know what gid/uid offset LXD is using for it's containers. In the case of NixNet's setup, the containers are on an offset of 100000, which means the root user in the container has the permissions of user 100000 on the host system.

The next step is to get the uid/gid of the user that needs to access the mounted files in the container. To get this, simply run id <user> on the user.

For example, to get the uid of postgres:

# id postgres
8.0K    ./static
uid=70(postgres) gid=70(postgres) groups=70(postgres),70(postgres)

We then add these uid/gids to our offset (100000) and chown the files on the host.

Again following postgres as an example:

# chown -R 100070:100070 /opt/storage/<service-name>/postgresql/

The mounted files should now show up as being owned by the correct user inside of the container.