LXD/Laptop Network Setup

LXD can be really useful to use on a laptop, because you can spin up containers locally for development. A bit of manual setup is required but once set up, this configuration should give your containers Internet access regardless of what particular WiFi network you are connected to.

Special setup is required on a laptop when you are using WiFi. Without special steps, LXD will cause a conflict with the default route on your system which is managed by NetworkManager. This will result in your Internet connection failing. See https://bugs.funtoo.org/browse/FL-8295 for more information on the gory details if you are interested.

Fortunately, there is a relatively easy way to work around this.

LXD Setup

Please follow the steps in LXD, and set up /etc/subuid, /etc/subgid, and/etc/security/limits.conf as described in our official setup.

Using an Unmanaged Bridge

Rather than use LXD's managed bridge, we are going to set up an unmanaged bridge using Funtoo networking.

Create the following file on your laptop:

   /etc/conf.d/netif.brwan

template="bridge"
ipaddrs="10.0.30.1/24"

This will set up a 10.0.30.0/24 network and we are setting up your laptop to be the gateway for this network. We will get the containers to the Internet by connecting your containers to this bridge, and giving them an IP address in the 10.0.30.2 thru 10.0.30.254 range.

Now, let's enable this bridge:

root # cd /etc/init.d
root # ln -s netif.tmpl netif.brwan
root # rc-update add netif.brwan default
 * service netif.brwan added to runlevel default
root # rc
 * Caching service dependencies ...
 * Network bridge brwan up ...
root #

The bridge is now set up.

New Bridge In, Old Bridge Out

When you configured LXD, you probably told LXD to create a bridge for you. We want to get rid of this bridge if it exists. We also want LXD to see our new bridge. First, let's get LXD to see our new bridge. The best way for LXD to see our bridge is to restart it:

root # /etc/init.d/lxd restart

Now, LXD should display your bridge:

root # lxc network list
+--------+----------+---------+------+------+-------------+---------+
| NAME   |   TYPE   | MANAGED | IPV4 | IPV6 | DESCRIPTION | USED BY |
+--------+----------+---------+------+------+-------------+---------+
| brwan  | bridge   | NO      |      |      |             | 0       |
+--------+----------+---------+------+------+-------------+---------+
| lxdbr0 | bridge   | YES     |      |      |             | 1       |
+--------+----------+---------+------+------+-------------+---------+
| eth0   | physical | NO      |      |      |             | 0       |
+--------+----------+---------+------+------+-------------+---------+
| wlan0  | physical | NO      |      |      |             | 0       |
+--------+----------+---------+------+------+-------------+---------+

Great! LXD is now seeing our bridge. But we have a problem -- there is still the lxdbr0 managed bridge that LXD created. LXD will continually mess with the default route on this bridge and it will mess up your WiFi connection. So we need to completely remove it from the LXD configuration.

To do this, first delete any containers that are using this bridge.

Then run the following command and modify the reference to lxdbr0 to instead refer to our new bridge, brwan:

root # lxc profile edit default

An editor will pop up. Make sure that you add brwan as the default bridge for the primary interface, so that "devices: eth0:" looks exactly like this. Remember that this is YAML which expects indents to consist of two spaces -- not tabs:

   

config: {}
description: Default LXD profile
devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: brwan
    type: nic
  root:
    path: /
    pool: default
    type: disk
name: default

Save your changes. Now, the default lxd profile will add a container to brwan by default.

This changes the default settings for any new containers so that they will use our new bridge. Now typing lxc network list should show nothing is referencing lxdbr0 so you should now be able to delete it:

root # lxc network delete lxdbr0

I recommend that at this point you reboot your system to make sure any routes referring to the old bridge that were inserted by LXD are cleaned out.

Container and Internet

We now have working setup, but your containers will not be able to access the Internet yet. Let's get a Funtoo-in-Funtoo environment set up:

root # wget https://build.funtoo.org/1.4-release-std/x86-64bit/intel64-skylake/2021-05-05/lxd-intel64-skylake-1.4-release-std-2021-05-05.tar.xz
root # lxc image import lxd-intel64-skylake-1.4-release-std-2021-05-05.tar.xz --alias funtoo
root # lxc launch funtoo test-image

Now, let's enter the test image:

root # lxc exec test-image -- /bin/bash
test-image # cd /etc/conf.d
test-image # nano netif.eth0

In the container network configuration, put the following information:

   /etc/conf.d/netif.eth0

template='interface'
ipaddrs='10.0.30.2/24'
gateway='10.0.30.1'
nameservers='1.1.1.1 1.0.0.1'

We are setting the container's IP address to 10.0.30.2. We're setting it's gateway to 10.0.30.1, which is the IP address of the bridge on our laptop. We are using Cloudflare for DNS.

Now, let's start this network:

test-image # cd /etc/init.d
test-image # ln -s netif.tmpl netif.eth0
test-image # rc-update add netif.eth0 default
test-image # rc

You should now be able to ping the gateway, the laptop bridge:

test-image # ping 10.0.30.1
PING 10.0.30.1 (10.0.30.1) 56(84) bytes of data.
64 bytes from 10.0.30.1: icmp_seq=1 ttl=64 time=0.120 ms
64 bytes from 10.0.30.1: icmp_seq=2 ttl=64 time=0.092 ms

So this part is working, but you won't be able to ping anything on the Internet -- yet. But make note -- this is the network configuration you will repeat for all your containers -- be sure to use different IP addresses for each container :)

IP Masquerade

   Note

This section will set up IP masquerading on your host (laptop) to provide transparent Internet access to devices connected to brwan -- like your containers.

We'll now set up IP Masquerading. This is done on the host -- not in the container. It will set up your system to use your active WiFi connection to provide Internet access to your containers. This is a one-time setup and will keep working after reboots.

Create the file /etc/local.d/masquerade.start:

   /etc/local.d/masquerade.start

/sbin/iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE
echo 1 > /proc/sys/net/ipv4/ip_forward

Make it executable -- and run it just this once. It will get automatically run when your system starts in the future:

root # chmod +x masquerade.start
root # ./masquerade.start

At this point, you should now be able to enter your container and ping things on the Internet:

root # lxc exec test-image -- /bin/bash
test-image # ping www.yahoo.com
PING new-fp-shed.wg1.b.yahoo.com (98.137.11.164) 56(84) bytes of data.
64 bytes from media-router-fp73.prod.media.vip.gq1.yahoo.com (98.137.11.164): icmp_seq=1 ttl=47 time=91.9 ms
64 bytes from media-router-fp73.prod.media.vip.gq1.yahoo.com (98.137.11.164): icmp_seq=2 ttl=47 time=110 ms
64 bytes from media-router-fp73.prod.media.vip.gq1.yahoo.com (98.137.11.164): icmp_seq=3 ttl=47 time=88.6 ms

ego sync and all the things you need to use Funtoo (like fetching distfiles) will now work.

DHCP Daemon

Now, you've probably noticed that we are statically assigning an IP address to our containers. This works, but having a DHCP daemon running to hand out IPv4 addresses is generally much nicer. To fix this, let's set up dnsmasq:

root # emerge dnsmasq

Put the following in your /etc/dnsmasq.conf:

   /etc/dnsmasq.conf

interface=brwan
dhcp-range=10.0.30.2,10.0.30.254,12h

Now, let's add dnsmasq to our default runlevel and start it:

root # rc-update add dnsmasq default
root # rc

dnsmasq should now be handing out IP addresses on your brwan bridge for your containers. Try it out and see how it's working.