Frequently Asked Questions

To talk about stuff, we need some naming. The physical machine is called 'Host' and the 'main' context running the Host Distro is called 'Host Context'. The virtual machine/distro is called 'Guest' and basically is a Distribution (Userspace) running inside a 'Guest Context'.

derjohn

A: With VServer you can only run Linux guests. The trick is that a guest does not run a kernel on its own (as XEN and UML do), it merely uses a virtualized host kernel-interface. VServer offers so called security contexts which make it possible to seperate one guest from each other, i.e. they cannot get data from each other. Imagine it as a chroot environment with much more security and features.

derjohn

A: Some. Check out the wiki for ready-made guest images. But you can easily build own guest images, e.g. with Debian's debootstrap. Checkout ((step-by-step Guide 2.6)) how to do that.

derjohn

A: Nope. XEN/UML/QEMU and VServer are just good friends. Because you ask, you probably know what XEN/UML/QEMU are. VServer in contrary to XEN/UML/QEMU not "emulate" any hardware you run a kernel on. You can run a VServer kernel in a XEN/UML/QEMU guest. This is confirmed to work at least with Linux 2.6/vs2.0.

derjohn

A: We hope so. It should be as least as secure as Linux is. We consider it much much more secure though.

derjohn

A: For a single guest, we basically have native performance. Some tests showed insignificant overhead (about 1-2%) others ran faster than on an unpatched kernel. This is IMVHO significantly less than other solutions waste, especially if you have more than a single guest (because of the resource sharing).

derjohn

A: Yes, on all SMP capable kernel architectures.

derjohn

A: Yes ....

• memory: Dynamically.
• CPU usage: Dynamically (token bucket)

derjohn

A: Yes, you can set maximum limits per guest, but you can only offer guaranteed resource availability with some ticks at the time. There is the possibility to ulimit and to rlimit. Rlimit is a new feature of kernel 2.6/vs2.0.

derjohn

A: Well, since vs2.1.1 linux-vserver supports a mechanism called 'I/O scheduling', which appeared in the 2.6 mainline some time ago. The mainline kernel offers several I/O schedulers:

# cat /sys/block/hdc/queue/scheduler
noop [anticipatory] deadline cfq

The default is anticipatory a.k.a. "AS". When running several guests on a host you probably want the I/O performance shared in a fair way among the different guests. The kernel comes with a "completely fair queueing" scheduler, CFQ, which can do that. (More on schedulers can be found at http://lwn.net/Articles/114770/)

This is how to set the scheduler to "cfq" manually:

root# echo "cfq" > /sys/block/hdc/queue/scheduler
root# cat /sys/block/hdc/queue/scheduler
noop anticipatory deadline [cfq]

Keep in mind that you have to do it on all physical discs. So if you run an md-softraid, do it to all physical /dev/hdXYZ discs!

If you run Debian there is a predefined way to set the /sys values at boot-time:

# apt-get install sysfsutils
[...]

# cat /etc/sysfs.conf  | grep cfq
block/sda/queue/scheduler = cfq
block/sdc/queue/scheduler = cfq

# /etc/init.d/sysfsutils restart

For non-vserver processes and CFQ you can set by which key the kernel decides about the fairness:

cat /sys/block/hdc/queue/iosched/key_type
pgid [tgid] uid gid

Hint: The 'key_type'-feature has been removed in the mainline kernel recently. Don't look for it any longer :(

The default is tgid, which means to share fairly among process groups. Think every guest is treated like a own process group. It's not possible to set a scheduler strategy within a guest. All processes belonging to the same guest are treated like "noop" within the guest. So: If you run apache and some ftp-server within the _same_ guest, there is no fair scheduling between them, but there is fair scheduling between the whole guest and all other guests.

And: It's possible to tune the scheduler parameters in several ways. Have a look at /sys/block/hdc/queue/....

You need a very recent Version of VS devel, e.g. the 2.1.1-rc18 can do it. Some older version have that feature too, then it got lost and was reinvented. So: Go and get a rc18 - only in 'devel', not stable!

derjohn

A: Device nodes allow Userspace to access hardware (or virtual resources). Creating a device node inside the guest's namespace will give access to that device, so for security reasons, the number of 'given' devices is small.

derjohn

A: Unification is Hard Links on Steroids. Guests can 'share' common files (usually binaries and libraries) in a secure way, by creating hard links with special properties (immutable but unlinkable (removable)). The tool to identify common files and to unify them is called vunify.

derjohn

A: The successor of vunify, a tool which does unification based on hash values (which allows to find common files in arbitrary paths.)

derjohn

A: For 'vhashify', just do these once:

mkdir /etc/vservers/.defaults/apps/vunify/hash /vservers/.hash
ln -0s /vservers/.hash /etc/vservers/.defaults/apps/vunify/hash/root

Then, do this one line per vserver:

mkdir /etc/vservers/<vservername>/apps/vunify   # vhashify reuses vunify configuration

The command 'ln' creates a link between two files. "ln -s" creates a symbolic link -- two files are linked by name. "ln -0s" uses a Vserver extention to create a unified link.

derjohn

A: If you are new to VServer I recommend to try 2.0.+. Take "alpha utils" Version 0.30.210. In Debian Sid there appeared well running version of it recently. (It's a .210 at the time of writing).

derjohn

A: Yes, but not on a shared partition for now. You need to put the guest on a separate partition, setup a vroot device (to make the quota access secure), copy that into the guest, and adjust the mtab line inside the guest.

derjohn

A: Context quotas are now called Disk Limits (so that we can tell them apart from the user/group quotas :). They are supported out of the box (with vs2.0) for all major filesystems (Ext2/3, ReiserFS, XFS, JFS)

derjohn

A: Currently not. Some developer has to move his ... to reimplement this functionality from the V4 code (I read that on the ML ;)). Will probably be superseded by the ngnet (next generation networking) soon. There is a Wiki page regarding this: http://linux-vserver.org/IPv6

derjohn

A: For now the networking is 'Host Business' -- the host is a router, and each guest is a server. You can set the capability ICMP_RAW in the context of the guest, or even the capability CAP_NET_RAW (which would even allow to sniff interfaces of other guests!). Likely to change with ngnet.

derjohn

A. [Update] Errrh, there is http://OpenVPS.org which is a set of scripts with a web-interface for webhosters/ISPs. A. [Update] Errrh, there is http://Openvcp.org which is a distributed system (agent!) with a web-interface, with which you can build/remove guests! cool stuff! beta, try out!

derjohn

A. Old-style config refers to a single text-file that contains all the configuration settings. With new-style config the configuration is split into several directories and files. You should probably go for new-style config if you are asking.

derjohn

A. Well, this page contains all configation options for vserver in version > 1.9 (I think .. I joined Linux-VServer in version 2, so I don't know for sure). The name of the page is derivived from the stylesheet(s) it contains: It displays background pictures of a very great flower, so regard it as highly optimized. It was designed by a non-designer, who asks us to create a better one. I played with the thought of creating a complete new theme for that page - but actually we all got used to the name "great flower page", so we stick to it. If you are unable to read it clearly, feel invited to join the IRC channel #vserver, we may tell you how to ;)

derjohn

A: First of all a single guest vserver only supports up to 16 IPs (There is a 64-IP patch available, which is in "derjohn's kernel", you need extra util-vserver anyway).

Here is a little helper-script that adds a list of IPs defined in a text file, one per line.

#!/bin/bash
j=1
for i in `cat myiplist`; do
        j=$(($j+1))
        mkdir $j
        echo $i > $j/ip
        echo $i > $j/ip-old
        echo "24" > $j/prefix
done

derjohn

A: Yes, use iptables with SNAT to masquerade it.

iptables -t nat -I POSTROUTING -s $VSERVER_NETZ  ! -d $VSERVER_NETZ -j SNAT --to $EXT_IP

See: HowtoPrivateNetworking and

http://www.tgunkel.de/it/software/doc/linux_server#h3-Vserver_Masquerading_SNAT (THX, [MUPPETS]Gonzo)

derjohn

A: When you shut down a guest (i.e. Template:Vserver foo stop), the IP is brought down on the host also. If this IP happens to be the primary IP of the host, the kernel will not only bring down the primary IP, but also all secondary IP addresses. But in very recent kernels, there is an option Template:Settable which prevents that nasty feature. It's called "alias promotion". You may set it via sysctl by adding Template:Net.ipv4.conf.all.promote secondaries=1 in /etc/sysctl.conf or via sysctl command line.

derjohn

A:

apt-get build-dep util-vserver

./configure --prefix=/usr/local/ --enable-release \
--mandir=/usr/local/share/man \
--infodir=/usr/local/share/info \
--sysconfdir=/etc --enable-dietlibc \
--localstatedir=/var \
--with-vrootdir=/var/lib/vservers

make

make install-distribution
(Which does a make install + setting a symlink ln -s /usr/local/lib/util-vserver/vshelper /sbin/vshelper )

To test which version you are running:

# which vserver
/usr/local/sbin/vserver

This should point to ..local...

If you dont want to build it yourself: On www.backports.org there are backported (for sarge) linux-images (2.6.16) with vserver-patch enabled and a updated util-vserver package as well.

derjohn

A: You need to patch util-vserver, too. So you obviously need to recompile util-vserver (see above). In the util-vserver directory there are header files in the ./kernel/ directory. Patch like this:

kernel/network.h:#define NB_IPV4ROOT    64

BTW: The initial patches can be found here: http://vserver.13thfloor.at/Experimental/VARIOUS/util-vserver-0.30.196-net64.diff.bz2 and http://vserver.13thfloor.at/Experimental/VARIOUS/delta-2.6.9-vs1.9.3-net64.diff

unknown

A: Simple ;) Assume you want to build a breezy guest on a sid host with IP 192.168.0.2 and hostname vubuntu, then do:

vserver vubuntu build --force -m debootstrap --hostname vubuntu.myvservers.net --netdev eth0 --interface 192.168.0.2/24 \
--context 42 -- -d breezy -m http://de.archive.ubuntu.com/ubuntu

[UPDATE] Currently there are problems in building breezy under unclear circumstances, which seems to have to do with udev. If the above didnt work, try:

vserver vubuntu build --force -m debootstrap --hostname vubuntu.myvservers.net --netdev eth0 --interface 192.168.0.2/24 \
--context 42 -- -d breezy -m http://de.archive.ubuntu.com/ubuntu -- --exclude=udev

In very recent versions of the utils, the problem should not occur anymore (it has to do with the 'secure-mount' if you look in the MLs)

Well, sid's debootstrap knows how to bootstrap Ubuntu linux. Make sure to have a current debootstrap package:

apt-get update
apt-get install debootstrap

The knowledge how to build ubuntu 'breezy badger' (which you probably want to be your guest at the time of writing) has been added recently.

derjohn

A: At least on Debian, I can tell you how to do it with the new-style config. If your guest is called "derjohn" and you want it to be started somewhere at the of your bootstrap process, then do:

echo "default" > /etc/vservers/derjohn/apps/init/mark

If you want to start it earlier, please read the init script "/etc/init.d/vserver-default" to find out how to do it. In most cases you don't need to change this. On Debian the vservers are started at "90", so after most other stuff is up (networking etc.).

Besides that I created a small helper script for managing the autostart foo: ((vserver-autostart))

derjohn

A: You are probably using util-vserver <= 0.30.209, which does use dynamic network contexts internally (With 0.30.210 this fact changed). So if you compiled your kernel without dynamic contexts, you may start guests, but you can't use the network context.The solution is either to switch to .210 util (or Hollow's toolset) or compile the kernel with dynamic network contexts. SE Keyword: invalid option `nid' testme.sh

derjohn

A: Look at /etc/ssh/sshd_config of the host:

Port 22
# Use these options to restrict which interfaces/protocols sshd will bind to
#ListenAddress ::

And now change the setting to

Port 22
# Use these options to restrict which interfaces/protocols sshd will bind to
ListenAddress your.hosts.ip.here  # not the guests IP! 

Then '/etc/init.d/ssh restart' on the host, after that on the guest (if you did apt-get install ssh on the guest already.)

Do I have to explain more? If the hosts sshd binds all available IP addresses on port 22 (The hosts 'sees' even all addresses of the guests!). So if the guest starts its sshd, it cant bind to port 22 any more. You need to change that setting only on the host.

(BTW: A similar approach has to be done for a lot of daemons, e.g. Apache. If the daemon does not support an explicit bind, you may use the chbind command to 'hide' IP addresses from the daemon before starting.)

derjohn

A: Before asking on the IRC channel, please check out the 'problematic programs' page:

Some programs do things that might work on a normal host but not inside a V-Server. This is often not a fault of V-Server itself, the programs are doing automagic things which fail and no proper error handling is done. Also sometimes the actions need special rights which are not permitted by default in V-Servers. Allowing CAPs is often not necessary since those special CAPs are only required once (e.g. when the program initializes the directories/settings/whatever).

OpenGroupware Apache Module

If your V-Server doesn't have access to localhost, then the connection to the OpenGroupware server will fail with a "Internal Server Error". The apache module for OpenGroupware called mod_ngobjweb uses a hardcoded "127.0.0.1" IP address in the source (handler.c line 339), this line you need to change to the IP address that should be used (the IP of the V-Server that runs the OpenGroupware? server)

Hylafax (with CAPI)

If you want to run hylafax in a V-Server, you will get a CAP and device problem which can be easily solved. First you need your capi20 devices in your V-Server, which can't be created by ./MAKEDEV (requires special CAPs) so copy the devices into the V-Server, like this (command run on the host):

cp -aR /dev/capi* /vservers/your_vserver/dev

Now hylafax can access your CAPI ISDN card but will exit after a few seconds, the problem is it tries to create a /dev/null nod in the hylafax chroot. This fails because of missing CAPs, so lets help hylafax again with copying the nod into the hylafax chroot in the V-Server. Like this (command run on the host):

cp -aR /dev/null /vservers/your_vserver/var/spool/hylafax/dev

Allright, now hylafax should have CAPI access and run properly.

Links inside screen inside a V-Server

Don't know why, but links crashes systematically being inside a screen session inside a V-Server started outside a V-Server. (please elaborate!)

screen inside a VServer

[root@ge root]# vserver zoe enter

zoe:/# screen
Cannot open your terminal '/dev/pts/5' - please check.

zoe:/# strace screen 
...
stat64("/dev/pts/5", {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 5), ...}) = 0
open("/dev/pts/5", O_RDWR|O_NONBLOCK)   = -1 EACCES (Permission denied)

is neither a bug nor an issue with screen, it just shows that a vserver context is not allowed to mess with host terminals. either use ssh/telnet to reach the 'guest' or start the screen session before you do the 'enter' (i.e. on the host)

util-vserver 0.30.211+ includes a workaround for the issue (vlogin), which allocates a tty inside the guest.

OpenLDAP Startup

slapd needs name resolution available in order to start up, otherwise it appears to hang. Make sure you have working DNS (or whatever) available to your vserver before starting one with slapd. This behavior is confirmed in my setup, no confirmation from others yet. My Setup: vservers all bind to an interface on a DMZ-like network segment, BIND runs on a vserver. slapd would hang at startup if the BIND vserver had not been started first.

rndc

Bind's rndc has a hardcoded 127.0.0.1 somewhere so any command to rndc will fail with connection refused. You should have a reachable localhost address defined in /etc/hosts and then you can use

rndc -s localhost

command. You can make a rndc.conf and set the default-server option, like that the '-s localhost' isn't necessary.

---

Please see the section below to get rndc working without switches.

-Paul Allen

---

Asterisk

Since some version of Asterisk (at least since 1.0.2), it will not run anymore. On start it fails with: "Unable to set high priority". This can be solved by allowing CAP_SYS_NICE for that V-Server. You can also not run Asterisk with the realtime priority - Just pass the '-p' command line argument to disable the read-time priority. Good doc on setting up Asterisk devices in the vserver: http://www.telephreak.org/papers/vpa/ (additional documentation for Asterisk 1.4 installation available here: https://customer.lylix.net/knowledgebase.php?action=displayarticle&catid=2&id=27)

Open/FreeSwan

No patch needed, openswan gives out some errors about writing to /proc, but those can be ignored.

Samba

Oplocks don't work as smbd insists on receiving break requests from 127.0.0.1 Just patch source/smbd/oplock.c (commenting paranoid code)

+++ oplock.c.orig       2005-02-14 14:27:51.000000000 +0200
--- oplock.c    2005-02-02 12:27:50.000000000 +0200
@@ -181,14 +181,12 @@
                return False;
        }

+#if 0
     /* Validate message from address (must be localhost). */
        if(from.sin_addr.s_addr != htonl(INADDR_LOOPBACK)) {
                DEBUG(0,("receive_local_message: invalid 'from' address \
 (was %lx should be 127.0.0.1)\n", (long)from.sin_addr.s_addr));
                return False;
        }
+#endif

        /* Setup the message header */
        SIVAL(buffer,OPBRK_CMD_LEN_OFFSET,msg_len);

or if you don't want to patch the samba source code you can disable oplock in Samba and it will work too!

Just put the following in your smb.conf:

kernel oplocks = no
oplocks = no

Note: The Vserver using Samba should also listen on the broadcast address. Thereby you will not be able to have two samba servers in the same net (on the same broadcast).

To assign the broadcast address to a vserver guest you need to create a new directory inside the /etc/vservers/<guest name>/interfaces/ directory. The directory is a number not used by this vserver guest. Inside the directory should be normal information and a file called nodev (see util-vserver for information). The contents of the dev file can be identical between directories.

Samba from Debian 3.1

The samba deb in sarge (3.1) provided file sharing. The only oddity observed is that the vserver guest running samba did not appear in a windows box's 'My Network Places'

Use a WINS server. The SMB browsing protocol relies heavily on broadcasts on the local net, which are problematic with vservers. WINS resolution on the other hand is unicast and works flawlessly under vserver.

Samba printer and file server with cups

Samba runs correctly in a Mandriva (Mdk) 10.1 Vserver, (Apart from the above oplock problem ?).First, edit your /etc/sysconfig/network file, and set networking to yes (This will solve problems for other services !):

# cat /etc/sysconfig/network
NETWORKING=yes

Some more tweaking is needeed in /etc/smb.conf

# cat /etc/smb.conf
...
  # YOUR VSERVER IP/MASK HERE
  interfaces = xxx.xxx.xxx.xxx/mask
...

But if you're using Samba + Cups to provide printing for Windows clients, AND if you want to use the Point and Print feature, there is more: In the [printers] section of your smb.conf, you should have the use client drivers directive set to no, or the driver upload procedure will fail !

# cat /etc/smb.conf
...
  use client driver = no
...

So, here is a full smb.conf file:

# cat /etc/samba/smb.conf | awk '!/^$/ && !/^\s*(#|;)/ {print $0}'
[global]
   workgroup = MYDOMAIN
   netbios name = MYHOSTNAME
   server string = MYCOMMENT (Samba %v)
   printcap name = cups
   load printers = yes
   printing = cups
   printer admin = @adm
   log file = /var/log/samba/log.%m
   max log size = 50
   map to guest = bad user
   security = domain
   password server = *
   encrypt passwords = yes
   smb passwd file = /etc/samba/smbpasswd
   username map = /etc/samba/smbusers
   idmap uid = 10000-20000
   idmap gid = 10000-20000
   socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=8192
   interfaces = 127. MYVSERVERIP/MYVSERVERMASK
   wins server = MYWINSIP
   dns proxy = no
   # for french users:
   dos charset = 850
   unix charset = ISO8859-1
[homes]
   comment = Home Directories
   browseable = no
   writable = no
[printers]
   comment = All Printers
   path = /var/spool/samba
   browseable = no
   guest ok = no
   writable = no
   printable = yes
   create mode = 0700
   print command = lpr-cups -P %p -o raw %s -r   # using client side printer drivers.
   use client driver = no
[print$]
   path = /var/lib/samba/printers
   browseable = yes
   write list = @adm root
   guest ok = yes
   inherit permissions = yes

...And a working smbusers:

# Unix_name = SMB_name1 SMB_name2 ...
root = administrator MYDOMAIN\administrator
nobody = guest pcguest smbguest

Cups print server

Symptoms: The Cups init script exits with:

Starting CUPS printing system: cupsd: Child exited with status 98!

And the logs (/var/log/cups/error_log) show:

 
E [date:hour...] StartListening: Unable to bind socket for address 0.0.0.0:631 - Address already in use.

...Or something like this.

With a correct "cupsd.conf file" (Tested version 1.1.21-0.rc1.7mdk, on Mandrake 10.1 - Now Mandriva), it works; All we need is to remove references to 127.0.0.1 or localhost from the file, as well as correctly unsetting the Listen directive:

 
LogLevel info
TempDir /var/spool/cups/tmp
# No 'Listen' directive !
Port 631
BrowseAddress @LOCAL
BrowseDeny All
BrowseAllow @LOCAL
BrowseOrder deny,allow
<Location />
  Order Deny,Allow
  Deny From All
  Allow From @LOCAL
</Location>
<Location /admin>
  AuthType Basic
  AuthClass System
  Order Deny,Allow
  Deny From All
  Allow From YOUR_NETWORK_ADDRESS/YOUR_NETMASK # Example: 172.16.0.0/24
# Or
  Allow From @LOCAL
</Location>

Then you'll need to modify the /etc/init.d/cups script, to comment any section referring to 127.0.0.1 lookup and configuration. This section exists at least on Mandrake 10.1, and is pretty long (Lines 35 to 55 and/or 79), and additionnaly four "else...if" lines must be commented far below (Lines 161 to 164) !

Remember to stop any Cupsd running in the host server, or to start it via a wrapper /etc/init.d/v_cups script:

 
#!/bin/sh
# chkconfig: 2345 15 60
# description: Wrapper to start cups bound to a single IP
USR_LIB_VSERVER=/usr/lib/util-vserver
exec $USR_LIB_VSERVER/vsysvwrapper cups $*

Do not forget to give a password to the root user, if you want to be able to manage your printers from the web interface (http://yourcupsvserver:631)!

 
# passwd root
...

If you use Mandriva 10.1 (And maybe some other distros), you’ll need to add the printers drivers for Cups, and reload it:

 
# urpmi --root /vservers/yourcupsvserver/ cups-drivers
# /etc/init.d/cups reload

…It added 67 Mb of packages for me.

Then use /etc/init.d/v_cups (re)start to launch Cups on the host server. You will now be able to make Cupsd start in the vserver , but more tweaking on the ACLs may be necessary to avoid authentification problems...

Bind9 on Debian GNU/Linux Woody (3.0), Sarge (3.1), Etch (4.0)

named provided by the bind9 binary packages fails to start because it is compiled with CAPs option.

The debian way is to build** your own package without CAPs:

su -
cd /usr/src
apt-get build-dep bind9
apt-get source bind9
cd bind9-x.x.x
vi debian/rules

Insert the following line after "./configure --prefix=/usr \":

--disable-linux-caps \ 

On a NPTL-enabled system you alse have to replace

 
--enable-threads \

with

 
--disable-threads \

or bind might refuse to run with an other user than root.

Save the file and go ahead with compiling/installing:

 
dpkg-buildpackage
dpkg -i ../bind9-x.x.x.deb
echo "bind9 hold" | dpkg --set-selections

or you may be better off to set the selections for all the packages that were generated by dpkg-buildpackage:

echo -e "bind9-doc hold\nbind9-host hold\nbind9utils hold\ndnsutils hold\nlibbind9-40 hold\nlibbind-dev hold\nlibdns43 hold\nlibisc44 hold\nlibisccc40 hold\nlibisccfg40 hold\nliblwres40 hold\nlwresd hold" | dpkg --set-selections

The last line is to set the package "on hold", so it is not touched by the update process. you have to take care of security holes by yourself now!

The Xs in "bind9-x.x.x" denote the version number of bind9. Alternatively you can allow the CAP_SYS_RESOURCE for that V-Server. The best way would be to fix bind, which is somehow broken when it comes to capabilities. Daniel Hokka Zakrisson repaired it. His patch is to be found here:

bind-9.3.2-caps-when-available.patch

So, if you recompile, it would be the cleanest way to apply that patch. Thanks Daniel! It would be also nice, if someone submits that patch to the bind people or maybe to your distribution's package maintainers in the first step.

Get my vserver-guest-ready Debian bind9 package for Debian Sid guests. Feedback welcome: aj@net-lab.net

---

AMD 64 Etch Vserver Guest WORKING!!! :o) 21Oct2008

Before I begin with all the grueling steps, if you want to be lazy, I wrote a Perl script to do all of this for you, and you can get it here.

• Note:

If you plan to use the script, you need to ensure you have Perl5, as well as have your locale set properly.

(On a brand-new Etch guest, I had to do those before my script would work)

I noticed after much googling that all links I found to any of the pre-compiled Bind9 (in the above bind9-capacheck directories) are DEAD. However, after many times of running into a brick wall, I caught something that isn't mentioned here...

Following the above steps to build Bind9 from source acquired with apt-get, if you look closely, on about line 499 or so after you execute the dpkg-buildpackage command, you'll see this:

dpkg-source: building bind9 using existing bind9_9.3.4.orig.tar.gz
dpkg-source: building bind9 in bind9_9.3.4-2etch3.diff.gz
dpkg-source: building bind9 in bind9_9.3.4-2etch3.dsc

...which is basically ignoring the debian/rules modifications you just made...

So, following you will find the steps I took to make Bind9 AND rndc run flawlessly on my vserver guest. Everything denoted with // in front is a comment:

su -
cd /usr/src
mkdir bind9-build
cd !$
apt-get build-dep bind9
apt-get source bind9
mkdir unused
mv bind9_* !$    //note the ones you don't need all have the underscore _
cd bind9-x.x.x     //whatever your version # is
nano debian/rules
beneath this line:     ./configure --prefix=/usr \
     -add    --disable-linux-caps \
     -change --enable-threads \     to:     --disable-threads \
dpkg-buildpackage
cd ../
dpkg -i *deb   //This will install all the debs that were created, but the actual bind9 deb will fail because of dependencies
//At this point, you can just run it again, or 'apt-get install -f' might fix it, but I'm unsure on that
dpkg -i *deb
echo "bind9 hold" | dpkg --set-selections

Here you will still see it fail to start, and even though it says that, it isn't named that failed, but only rndc.

So actually, you can stop here, but rndc will fail every time, and bind won't restart with the /etc/init.d/bind9 restart call

In fact, everytime you want to reload, you'll need to manually kill the named process ID and then issue:

/etc/init.d/bind9 start

But if you're a perfectionist like me, and you want a completely error-free starting Bind9 with working rndc, keep reading...

ps aux|grep named

That will show an output something like:

bind      9376  0.0  0.3  20844  3228 ?        Ss   15:52   0:00 /usr/sbin/named -u bind

Assuming the above was your actual output, your next command would be:

kill 9376     //Kill the process ID

Now here's where the trickery comes in...

For some reason, if you have the EXACT same key definitions in all the correct places, but the conf files aren't using the actual key file, rndc WILL fail...

Let me say that again... rndc WILL fail... I only know because it took me 3 days to find out why it was failing...

So first:

rndc-confgen -s your.eth0.ip.address -k your-custom-keyname

Assuming my eth0 IP is 11.22.33.44 and I used a keyname of my-vps-key, the following would be my output:

vps:~# rndc-confgen -s 11.22.33.44 -k my-vps-key
# Start of rndc.conf
key "my-vps-key" {
        algorithm hmac-md5;
        secret "D4fZPZPFpG037a/NHLedgg==";
};

options {
        default-key "my-vps-key";
        default-server 11.22.33.44;
        default-port 953;
};
# End of rndc.conf

# Use with the following in named.conf, adjusting the allow list as needed:
# key "my-vps-key" {
#       algorithm hmac-md5;
#       secret "D4fZPZPFpG037a/NHLedgg==";
# };
#
# controls {
#       inet 11.22.33.44 port 953
#               allow { 11.22.33.44; } keys { "my-vps-key"; };
# };
# End of named.conf

Now, this information is VERY important, and misleading at the same time...

Don't simply plop the 'named.conf' data into named.conf and the 'rndc.conf' data into rndc.conf... It will NOT work.

The 'key' portion is all you need for your keyfile, and your keyfile MUST be named the same as the name you put into rndc-confgen

There are several ways to do this, but the simplest is to highlight the key section, right click, click 'copy' and then:

nano /etc/bind/my-vps-key     //Don't forget, name must match the rdnc-confgen output!

Inside nano, hit shift+insert on your keyboard, and it *should* paste everything properly. Hit ctrl+x to exit nano (press y at the prompt to save).

Once complete, you can verify your key file with:

cat /etc/bind/my-vps-key

...and the output should read something like:

vps:~# cat /etc/bind/my-vps-key
key "my-vps-key" {
        algorithm hmac-md5;
        secret "D4fZPZPFpG037a/NHLedgg==";
};

Now, do the same thing for the 'options' section of the rndc-confgen output, except you'll be adding that to the rndc.conf file, along with an include statement for the key file you just made.

NOTE: Delete all lines in rndc.conf first so that it contains only what you paste. (ctrl+k will cut lines... cut all of them before pasting)

After pasting, your ENTIRE rndc.conf file should look thusly:

include "/etc/bind/my-vps-key";

options {
        default-key "my-vps-key";
        default-server 11.22.33.44;
        default-port 953;
};

Finally, you need to do the same process, but with the 'controls' section of the rndc-confgen output, and that goes in /etc/named.conf. Again, you will need to add the include statement for the key you created.

Also, while you're editing named.conf, you might as well go ahead and edit the IP addresses. The only one that really seems to matter is the 'localhost' zone

When finished, your named.conf should look something like this (only showing the relevant portions)

Be sure to remove the comment tags (#) from the beginning of the lines!!!

<snip>
// If you are just adding zones, please do that in /etc/bind/named.conf.local

include "/etc/bind/named.conf.options";
include "/etc/bind/my-vps-key";

controls {
inet 11.22.33.44 port 953
allow { 11.22.33.44; } keys { "my-vps-key"; };
};

zone "." {
        type hint;
        file "/etc/bind/db.root";
};

// be authoritative for the localhost forward and reverse zones, and for
// broadcast zones as per RFC 1912

zone "11.22.33.44" {
        type master;
        file "/etc/bind/db.local";
};

zone "127.in-addr.arpa" {
<snip>

I'm unsure if this next step is necessary, but I did it anyway...

Edit /etc/bind/db.local and modify all instances of 'localhost' and '127.0.0.1' with your IP address.

When complete, it should look something like this:

vps:~# cat /etc/bind/db.local
;
; BIND data file for local loopback interface
;
$TTL    604800
@       IN      SOA     11.22.33.44. root.11.22.33.44. (
                              1         ; Serial
                         604800         ; Refresh
                          86400         ; Retry
                        2419200         ; Expire
                         604800 )       ; Negative Cache TTL
;
@       IN      NS      11.22.33.44.
@       IN      A       11.22.33.44

All that's left is to start your new error-free Bind9 (we hope...)

/etc/init.d/bind9 start

If you find anything in this tutorial OR in the Perl script that's incorrect, or that would make it better, please let me know: paul@grandesigns.net

---

Update (11/04/08)...

After an apt-get dist-upgrade, bind9 failed to start. After poking around, I found out that somehow /etc/default/bind had the -u switch set to run Bind9 as the user 'named'. Weird, because with Bind9, there is no user 'named'. I set it back to: -u bind and everything fired right up. Also, my particular vserver doesn't log to dmesg, so if you want intuitive debug level info that isn't being printed in /var/log/syslog, then you can run named like this:

named -g -d 10

That tells named to start with the debug level globally set to 10.

I also tinkered a little more and found that you can remove all of the db. files in /etc/bind and in /etc/bind/named.conf except for db.local and db.root. In other words, you don't need to configure or keep db.0, db.127, db.255, db.empty, nor do you need them referenced in named.conf.

---

Problems getting this to work in an x86_64 vserver. Any hints? From what I can tell from stracing, turning off threads doesn't fix everything related to it. It also caused a juicy Ksymoops.

- Apparently this does work when running bind as user root. I Installed a bare Debian Etch vserver without granting any special capabilities on my x86_64 system, apt-get install bind9, kill any started named-processes and edit /etc/default/bind, changing the user (-u) from bind to root. Bind starts now starts when using /etc/init.d/bind9 start and resolving seems to work. Mind you, the usual warnings and rants about running stuff as root still applies! - 2007-08-06 MGS

Zimbra Mail

Zimbra is many applications (including Postfix and MySQL? and OpenLDAP? and more) which try to take over the interfaces, and depend a lot on binding from 127.0.0.1 - it is not hard to change, but there is a couple of tricks - it is documented here - http://wiki.zimbra.com/index.php?title=Install_VServer

xine

won't start with no error message.

"xine --verbose" shows this.

 
ERROR: Could not determine network interfaces, you must use a interfaces config line

This happens if you have the xineplug_inp_smb.so plugin. Delete it and everything is fine.

nagios

Compiling nagios-1.4 within a vserver requires an explicit lo with 127.0.0.1, otherwise it hangs during the configure with "checking for ICMP ping syntax..."

on older kernels, which do not support loopback isolation (separate 127.0.0.1) and use 'localhost' mapped to the first guest IP you can work around this by adding 127.0.0.1 to the interfaces (temporarily) but make sure to remove it afterwards.

Hula-project

Does not want to start. TODO: add more information.

Postfix

Postfix 2.1.5 (Debian Sarge)

On a vserver with two interfaces (lo and eth0), and a postfix 2.1.5 listening on lo, postfix can't send emails : "Invalid argument"... Setting smtp_bind_address (http://www.postfix.org/postconf.5.html#smtp_bind_address) to the external address solves the issue.

Postfix on Debian Etch (for local only mail)

See Postfix local only problem

Postfix Policy Daemon

Running a Debian 3.1 Sarge with Backports I have several issues with the postfix-policyd because it wants to set the rlimits.

Log returns:

 
cannot set rlimit: Operation not permitted

Strace tells us:

 
setrlimit(RLIMIT_NOFILE, {rlim_cur=4097, rlim_max=4097}) = -1 EPERM (Operation not permitted)

Output on the Host

 
# ulimit -Ha
...
-n: file descriptors           1024
...

Thats too little...

Solution:

The App has again a build in need to use CAP_SYS_RESOURCE (which is bad (tm)) so in the guest do:

# ulimit -HS -n 8192
# ulimit -Ha
.. shows us now the correct 8192 instead of 1024.

# vserver $yourVserverName restart
# vserver $yourVserverName enter

$yourVserverName # ulimit -Ha
...
-n: file descriptors           8192
...

Everything should be fine now !

DB2

DB2 needs at least the following capabilities:

CAP_NET_RAW
CAP_NET_ADMIN
CAP_SYS_ADMIN
CAP_IPC_LOCK
CAP_IPC_OWNER

ejabberd

ejabberd does not work properly in a Linux-VServer guest under Debian Etch. The server eventually stops accepting connections. Also, it's not possible to control ejabberd with ejabberdctl.

Newer versions of ejabberd (2.0.x) don't seem to have this problem. On Debian Etch a newer version can be installed from etch-backports (backports.org).

The Jabber server from the jabber package also works.

Other problem, which seems to happen with (at least) erlang R14B : epmd has been modified and accepts some commands only from 127/8, which causes problems. Details and solution can be found here : http://www.ejabberd.im/node/3709. In case this page becomes unavailable, the solution is to change network flags on the vserver, as follow (run these on the host) :

nattribute --set --nid vserver_name --flag ~single_ip
echo "~single_ip" > /etc/vservers/vserver_name/nflags

VMware Server

VMware Server needs the following capabilities to run correctly:

CAP_MKNOD
CAP_SYS_MODULE
CAP_SYS_ADMIN
CAP_SYS_NICE
CAP_NET_RAW
CAP_NET_ADMIN
CAP_IPC_OWNER

ntpd

The ntpd daemon needs to set the hardware time and can't do that from inside a vserver by default. Allow CAP_SYS_TIME and ntpd will be happy.

FAI server / fai-setup

FAI Server / fai-setup fai-setup wants to create some devices and needs to mount proc.

bcapabilities:

MKNOD

ccapabilities:

SECURE_MOUNT
SECURE_REMOUNT
BINARY_MOUNT

Only do it if you just want your FAIserver in a seperate tree from main server.

derjohn

A: Check out the 'problematic programs' page:

((ProblematicPrograms)) and/or get my [((ProblematicPrograms)) and/or get my vserver-guest-ready Debian package for Debian Sid guests from that URL: http://linux-vserver.derjohn.de/bind9-packages/bind9-capacheck_9.3.2-2_i386.deb and check out the readme. (Hint: This is fresh stuff. The give me Feedback)

[UPDATE] Since VServer Devel 2.1.1-rc18 you do not need to patch the userland tools anymore. The capabilities are masked.

derjohn

A: Template:Vserver-stat. Example output:

CTX   PROC    VSZ    RSS  userTIME   sysTIME    UPTIME NAME
0       77 965.1M 334.6M  14m14s18   2m28s69   1h33m46 root server
49152    7    14M   5.2M   0m00s40   0m00s30   1h30m15 chiffon

derjohn

A: It depends.

For vserver with legacy-interfaces support, you have to replace Template:/sbin/halt in guests with vreboot and start rebootmgr in host. You also need to have a dummy <guest>.conf file in /etc/vservers for each guest. Please have a look at /etc/init.d/rebootmgr.

Vserver with native interface utilizes /dev/initctl. No changes are needed in guests. Just make sure that REBOOT capability is adjusted in guests.

derjohn

A: Since vserver is an ongoing project, new features might replace old ones, some might still on development. Legacy-interfaces are available for backward compability (which might be removed someday). See Q: How can I reboot/halt guests?

derjohn

A: There are no known issues about running vserver on a preemption enabled kernel. I would like to add, that the vserver kernelhackers would probably exclude that option in 'make menuconfig' if there would be an incompatibility. Just my $.02 :)

derjohn

A: The first public occurance of linux-vserver was Oct 2001. The initial mail can be found here: http://www.cs.helsinki.fi/linux/linux-kernel/2001-40/1065.html So you can expect a mature software product wich does it's magic quite well (And hey, we have a version > 2.0 ! )

derjohn

A: Yes. I don't want to provide an in-depth OpenVPN tutorial, but want to show how I made OpenVPN work in a guest as server. I was not able to run it with a tun devive, due to a buglet in util-vserver and kernel when it comes to settings a an ip address a point to point link: If you add "ip addr add <ip> peer <mypeer> dev tun0" there is no way to map the tun0 interface into a guest, even not with a 'nodev' option. (bug confirmed to be reproducible by daniel_hoczac)

First of all you have to prepare the host with a persistent tuntap interface in tap-mode. The tools we need come from the uml-utilities. Then you need to create a device /dev/net/tun, which the OpenVPN userspace daemon reads. Well assume 10.10.10.100 is the server IP, and 10.10.10.101 is the client ip - to be cool be choose a /31 netmask (255.255.255.254), so we have a net without broadcast and don't waste IPs :)

On the host do:

# apt-get install uml-utilities
# cd /var/lib/vserver/<myopenvpnserver>/dev/
# ./MAKEDEV tun
  (creates the dev/net/tun device accessible by te guest - even a tap interface need /dev/net/tun !)
# tunctl -t tap0
  (creates the network device 'tap0' persistently)

Then add the ip to the guest:

# cat /etc/vservers/<myopenvpnserver>/interfaces/1/ip
10.10.10.100
# cat /etc/vservers/<myopenvpnserver>/interfaces/1/prefix
31
# cat /etc/vservers/<myopenvpnserver>/interfaces/1/dev
tap0
(This kind of config brings the ip when the vserver is started - only the tap0 interface has to exist already, see above!)

Here is a sample config for the guest (which is acting as a server):

Install OpenVPN package on server and client, in the Debian case:

# apt-get install openvpn

The server's conf looks like that:

# port and interface specs

# behave like a ssl-webserver
port 443
proto tcp-server

# tap device? (keep in mind you need /dev/net/tun !)
dev tap0

# now the ips we will use for the tunnel
ifconfig 10.10.10.100 255.255.255.254
ifconfig-noexec

# the server part

# Keep VPN connections, even if the client IP changes
float

# use compression (may also even obfuscate content filters)
comp-lzo

# use a static key - create it with 'openvpn --genkey --secret static.key'
secret static.key

# dont reload the key after a SIGUSR1
persist-key

# check alive all 10 secs
keepalive 10 60

# verbosity level (from 1 to 9, 9 is max log level)
verb 4
status openvpn-status.log

The client's conf may look like that (This example even makes the tunnel the clients default address):

# cat /etc/openvpn/client.conf
# port and interface specs

# the following is not necessary, if you bring up openvpn via Debian's init script:
daemon ovpn-my-clients-name

# behave like a ssl-webserver
port 443
proto tcp-client
remote %%%<insert-the-guest-primary-public-ip-here>%%%%
# what device tun ot tap?
dev tap

# now the ips we will use for the tunnel
ifconfig 10.10.10.101 255.255.255.254

# Keep VPN connections, even if the client IP changes
float
mssfix

# use compression (may also even obfuscate content filters)
comp-lzo

# use a static key
secret static.key

# dont reload the key after a SIGUSR1
persist-key

# check alive all 10 secs
keepalive 10 60

# verbosity level (from 1 to 9, 9 is max log level)
verb 4

# set the default route
route-gateway 10.10.10.100
redirect-gateway def1
# to add special routes you can do it wihtin the openvpn client conf:
# route <dest> <mask> <gateway>

# if you need to connect via proxy (like squid)
# http-proxy s p [up] [auth] : Connect to remote host through an HTTP proxy at
#                  address s and port p.  If proxy authentication is required,
#                  up is a file containing username/password on 2 lines, or
#                  'stdin' to prompt from console.  Add auth='ntlm' if
#                  the proxy requires NTLM authentication.

# http-proxy s p [up] [auth]


# http-proxy-option type [parm] : Set extended HTTP proxy options.
#                                  Repeat to set multiple options.
#                  VERSION version (default=1.0)
#                  AGENT user-agent

# http-proxy-option type [parm]

In the next lesson I will talk about OpenVPN's server mode, which can deal with with multiple clients connecting to one ip and one port (i.e. you only need one guest for tons or 'roadwarriros'), tls connections and pki.

Contributions welcome. :)

derjohn

A: If you have the choice make the host a 64 bit one. You can run a guest as 32 bit or as 64 bit on a 64 bit host. To run it as 32 bit, you need to compile the x86_64 (a.k.a. AMD64) with the following options:

[*] Kernel support for ELF binaries
<M> Kernel support for MISC binaries
[*] IA32 Emulation <---- without that, the entire 32bit API is not present
<M>   IA32 a.out support  

You can force the guest to behave like a 32 environment like this:

echo linux_32bit > /etc/vservers/$NAME/personality
echo i686 > /etc/vservers/$NAME/uts/machine

(thanks cehteh for the hint!)

But you can force debootstrap to but 32 bit binaries into the guest by 'export ARCH=i386';

export ARCH=i386 ; vserver build .... 

derjohn

A: I'll explain. I take as example your /tmp partition within the guest is too small, what will be likely the case if you stay with the 16MB default (vserver build mounts /tmp as 16 MB tempfs!).

# vnamespace -e XID  mount -t tmpfs -o remount,size=256m,mode=1777 none /var/lib/vservers/<guest>/tmp/

Be warned that the guest will not recognize the change, as the /etc/mtab file is not updated when you mount like this. To permanently change the mount, edit /etc/vserver/<guest>/fstab on the host.

derjohn

A: First you can read [1].

If you want a recipe, do that: 1. Check the size of memory pages. On x86 and x86_64 is usually 4 KB per page. 2. Create /etc/vserver/<guest>/rlimits/ 3. Check your physical memory size on the host, e.g. with "free -m". maxram = kilobytes/pagesize. 4. Limit the guests physical RAM to value smaller then maxram:

echo %%insertYourPagesHereSmallerThanMaxram%% > /etc/vserver/<guest>/rlimits/rss

5. Check your swapspace, e.g. with 'swapon -s'. maxswap = swapkilobytes/pagesize. 6. Limit the guest's maximum number of as pages to a value smaller than (maxram+maxswap):

echo %%desiredvalue%% > /etc/vserver/<guest>/rlimits/as

It should be clear this can still lead to OOM situations. Example: You have two guests and your as limit per guest is greater than 50% of (maxram+maxswap). If both guests request their maximum at the same point in time, there will be not enough mem .....

derjohn

A: Here you go: http://linux-vserver.derjohn.de/ . There is also some stuff on backports.org, but my kernels are always 'devel' branch.

derjohn

Yes but right now only on the host (rootserver). Please realize that all traffic is local and will not touch the forward chain.

BeginnerFAQ

 
sin_addr=inet_addr("xx.xx.xx.xx")}, yy) = -1 EINVAL (Invalid argument)

A: The vserver/master cannot communicate with another vserver on same host.

• check all netmasks on all interfaces (do they overlap) ?
• check policy routing (disable it temporary) ?
• check that lo is up (Networking within a host/vserver always uses lo interface)

CommonProblems

capabilities are not enabled in kernel-setup please check that CONFIG_SECURITY_CAPABILITIES is loaded or included in the kernel. ( check with "cat /path_to_kernel/.config

IrcQuestions

mount it via /etc/vservers/vserver-name/fstab, make sure to set the option 'dev' e.g.:

/dev/drbd0     /       xfs     rw,dev          0 0

util-vserver 210 won't be able to find some scripts for the reboot, add into /etc/vservers/vserver-name/apps/init/cmd.stop

/etc/init.d/rc
6

AdrianReyer