Split shared networks with util-vserver:

What are we trying to manage here: vservers are pretty good and pretty secure wrt networking. But there comes a time when your iron is to big and can server vservers on security wise multiple networks. Multi-homes is always a pain especially if you have an external firewall. You will make vlans on your big iron, but how will you firewall between one vlan and another on that same iron? Will you duplicate functionality, and have extra firewalling on your big iron, while you have another big iron doing that dedicated already? That's where network namespaces come into the picture. Network namespaces give the ability to have a seperate ip-stack per namespace. That's right: that's including iptables, multiple routing tables and whatever. If we can have multiple ip-stacks, we can give each vserver a seperate ip-stack, right? Well, yeah, but why? An ip-stack usually means another l2 or l3 interfacing, especially if those vservers are supposed to be on the same network. So we group all vservers in that belong security wise into same network also into the same network namespace. This gives the benefit that we don't have to fool around with special virtual bridges, virtual ethernet devices or whatever.

So this page will describe how to setup multiples of (a group of vservers which share the same ipstack on a single vlan) on a single system with only using vlans as network virtualization.

Namespace ramblings

Playing with the latest and greatest ip route I discovered that ip supports the creation and naming of network namespaces, all be it with a trick. The current ip route utility manages network namespaces by creating a new namespace, and then bind-mounting /proc/self/ns/net (a file!) to /var/run/netns/<a file named after the namespace>.

Network namespaces always had a big problem of not being able to reach them, unless you know a pid in that namespace. There is another technique, and that is to open a file which is somehow located in that namespace. Enough of that.

Vserver to ip

How can we use that? First we must make sure that we are compatible: we can make vserver create the network namespace and let the ip utility know about that:

mkdir -p /var/run/netns
touch /var/run/netns/$VSERVER
vspace -e $VSERVER --net mount -o bind /proc/self/ns/net /var/run/netns/$VSERVER

From then on you can use the following incantations:

ip netns exec $VSERVER cmds
vspace -e $VSERVER --net cmds

Interface to vserver

To put a network device into that namespace we do something like this:

ip link set dev $IFACE netns $VSERVER

We can check that with:

ip netns exec $VSERVER ip a ls
vspace -e $VSERVER --net ip a ls

Useful interface creationism

What's a network namespace without an interface? Here are some create statements:

# Create an interface on a a "real" device
# A macvlan in bridge mode is as if we have a second network card
# in the same network. It already knows all mac-addresses, so the "bridging" code is lightweight
ip link add link $BASEDEVICE name $VSERVERDEVICE type macvlan mode bridge

# Create a virtual network tunnel
# This is basically a virtual cross cable type of interface
# You can use a veth like any ordinary device like
# plugging it into a bridge device
ip link add name $DEVICE_A type veth peer name $DEVICE_B

# Create a normal vlan interface on a device.
# Great when giving vservers a seperate vlan
ip link add link $BASEDEVICE name $VSERVERDEVICE type vlan id $VLANID

Remember though: all interfaces need valid mac-addresses, especially when going to the outside. I use this complex script to generate a pair of static mac-addresses for a veth:

stringasmac() {
        echo "$1"|sed 's/\(..\)\(..\)\(..\)\(..\)\(..\)\(..\)/\1:\2:\3:\4:\5:\6/'
}
read CONTEXTID < /etc/vservers/${VSERVER}/context
MACSTRING=$(printf "%04x" $VID)$(printf "%04x" $CONTEXTID)
MACADDRC=$(stringasmac 02${MACSTRING}00)
MACADDRH=$(stringasmac 02${MACSTRING}01)

Valid addresses are addresses that have bit0 unset and bit1 set in the first byte. It means a normal (bit0: 1 == multicast) and locally administrated (bit1: 1==Not official) address. You set them like:

ip link set address ${MACADDRC} dev ${CLIENTDEV}