LXD/Laptop Network Setup

LXD can be really useful to use on a laptop, because you can spin up containers locally for development. A bit of manual setup is required but once set up, this configuration should give your containers Internet access regardless of what particular WiFi network you are connected to.

Special setup is required on a laptop when you are using WiFi. Without special steps, LXD will cause a conflict with the default route on your system which is managed by NetworkManager. This will result in your Internet connection failing. See https://bugs.funtoo.org/browse/FL-8295 for more information on the gory details if you are interested.

Fortunately, there is a relatively easy way to work around this.

LXD Setup

Please follow the steps in LXD, and set up /etc/subuid, /etc/subgid, and/etc/security/limits.conf as described in our official setup.

Using an Unmanaged Bridge

Rather than use LXD's managed bridge, we are going to set up an unmanaged bridge using Funtoo networking.

Create the following file on your laptop:

   /etc/conf.d/netif.brwan

template="bridge"
ipaddrs="10.0.30.1/24"

This will set up a 10.0.30.0/24 network and we are setting up your laptop to be the gateway for this network. We will get the containers to the Internet by connecting your containers to this bridge, and giving them an IP address in the 10.0.30.2 thru 10.0.30.254 range.

Now, let's enable this bridge:

root # cd /etc/init.d
root # ln -s netif.tmpl netif.brwan
root # rc-update add brwan default
root # rc

The bridge is now set up.

New Bridge In, Old Bridge Out

When you configured LXD, you probably told LXD to create a bridge for you. We want to get rid of this bridge if it exists. We also want LXD to see our new bridge. First, let's get LXD to see our new bridge. The best way for LXD to see our bridge is to restart it:

root # /etc/init.d/lxd restart

Now, LXD should display your bridge:

root # lxc network list
+--------+----------+---------+------+------+-------------+---------+
| NAME   |   TYPE   | MANAGED | IPV4 | IPV6 | DESCRIPTION | USED BY |
+--------+----------+---------+------+------+-------------+---------+
| brwan  | bridge   | NO      |      |      |             | 0       |
+--------+----------+---------+------+------+-------------+---------+
| lxdbr0 | bridge   | YES     |      |      |             | 1       |
+--------+----------+---------+------+------+-------------+---------+
| eth0   | physical | NO      |      |      |             | 0       |
+--------+----------+---------+------+------+-------------+---------+
| wlan0  | physical | NO      |      |      |             | 0       |
+--------+----------+---------+------+------+-------------+---------+

Great! LXD is now seeing our bridge. But we have a problem -- there is still the lxdbr0 managed bridge that LXD created. LXD will continually mess with the default route on this bridge and it will mess up your WiFi connection. So we need to completely remove it from the LXD configuration.

To do this, first delete any containers that are using this bridge.

Then run the following command and modify the reference to lxdbr0 to instead refer to our new bridge, brwan:

root # lxc profile edit default

This changes the default settings for any new containers so that they will use our new bridge. Now typing lxc network list should show nothing is referencing lxdbr0 so you should now be able to delete it:

root # lxc network delete lxdbr0

I recommend that at this point you reboot your system to make sure any routes referring to the old bridge that were inserted by LXD are cleaned out.

Container and Internet

We now have working setup, but your containers will not be able to access the Internet yet. Let's get a Funtoo-in-Funtoo environment set up:

root # wget https://build.funtoo.org/1.4-release-std/x86-64bit/intel64-skylake/2021-05-05/lxd-intel64-skylake-1.4-release-std-2021-05-05.tar.xz
root # lxc image import lxd-intel64-skylake-1.4-release-std-2021-05-05.tar.xz --alias funtoo
root # lxc launch funtoo test-image

Now, let's enter the test image:

root # lxc exec test-image -- /bin/bash
test-image # cd /etc/conf.d
test-image # nano netif.eth0

In the container network configuration, put the following information:

   /etc/conf.d/netif.eth0

template='interface'
ipaddrs='10.0.30.2/24'
gateway='10.0.30.1'
nameservers='1.1.1.1 1.0.0.1'

We are setting the container's IP address to 10.0.30.2. We're setting it's gateway to 10.0.30.1, which is the IP address of the bridge on our laptop. We are using Cloudflare for DNS.

Now, let's start this network:

test-image # cd /etc/init.d
test-image # ln -s netif.tmpl netif.eth0
test-image # rc-update add netif.eth0 default
test-image # rc

You should now be able to ping the gateway, the laptop bridge:

test-image # ping 10.0.30.1
PING 10.0.30.1 (10.0.30.1) 56(84) bytes of data.
64 bytes from 10.0.30.1: icmp_seq=1 ttl=64 time=0.120 ms
64 bytes from 10.0.30.1: icmp_seq=2 ttl=64 time=0.092 ms

So this part is working, but you won't be able to ping anything on the Internet -- yet. But make note -- this is the network configuration you will repeat for all your containers -- be sure to use different IP addresses for each container :)

IP Masquerade

We'll now set up IP Masquerading so that your laptop will use your active WiFi connection to provide Internet access to your containers. This is a one-time setup and will keep working after reboots.

Create the file /etc/local.d/masquerade.start:

   /etc/local.d/masquerade.start

/sbin/iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE
echo 1 > /proc/sys/net/ipv4/ip_forward

Make it executable -- and run it just this once. It will get automatically run when your system starts in the future:

root # chmod +x masquerade.start
root # ./masquerade.start

At this point, you should now be able to enter your container and ping things on the Internet:

root # lxc exec test-image -- /bin/bash
test-image # ping www.yahoo.com
PING new-fp-shed.wg1.b.yahoo.com (98.137.11.164) 56(84) bytes of data.
64 bytes from media-router-fp73.prod.media.vip.gq1.yahoo.com (98.137.11.164): icmp_seq=1 ttl=47 time=91.9 ms
64 bytes from media-router-fp73.prod.media.vip.gq1.yahoo.com (98.137.11.164): icmp_seq=2 ttl=47 time=110 ms
64 bytes from media-router-fp73.prod.media.vip.gq1.yahoo.com (98.137.11.164): icmp_seq=3 ttl=47 time=88.6 ms

ego sync and all the things you need to use Funtoo (like fetching distfiles) will now work.