Jail building
From FBSD_tips
Back to "Admin building blocks"
DRAFT ARTICLE
... or "If you are going to shoot yourself in the foot, make sure it is a virtual foot"
Contents |
[edit] Background
Jails are about process isolation, not virtiualization in the hardware sense (see "Operating system-level virtualization" in the Wikipedia article). More Precisely, the FreeBSD jail system virtualizes the interfaces to the kernel any number of times and chroots groups of processes into each instance of this virtual interface, thus creating the individual jails. In theory the processes, once jailed will be unable to gain access to the host environment. This effect bears some resemblance to a virtual machine and can be administered largely as such but in fact, all the jails on a machine share the same kernel.
There are a number of resources available that detail how to create and run a jail (the jail man page being one of the best). The reason I include yet another one here is twofold.
- For completeness sake. There are several articles on this wiki that depend on building a jail and I wanted to have this basic building block covered here.
- There are some small specific details the usefulness of which will become apparent in the follow-on articles. This transgresses the pincipal of keeping the foundational articles generic, so I will note the specifics.
The *definitive* jail paper is here
[edit] Sources
Make sure you have the system sources installed (at /usr/src) that match the running system : problems can occur if they don't match. In which case you should update your OS sources (like this). Then 'make buildworld' will compile all the userland that the jail installation will use.
cd /usr/src make buildworld
[edit] Populating the jail directory
Now we will prepare the system tree in the jail directory. The following script populates the jail with all userland utilities. This method will become useful in follow-on articles. This is largely lifted from the jail man page.
[edit] SCRIPT : build_jail.sh
#!/bin/sh
args=`getopt D: $*`
if [ $? -ne 0 ]
then
echo 'Usage: ...'
echo '-D Directory to install jail to'
exit 2
fi
set -- $args
for i
do
case "$i"
in
-D)
D=$2
shift; shift;;
esac
done
if [ "${D}X" = "X" ]; then echo "No directory"; exit; fi
mkdir -p $D
LIVE=$D/live
mkdir $D/live
cd /usr/src && \
make hierarchy DESTDIR=${LIVE} && \
make installworld DESTDIR=${LIVE} && \
cd etc && \
make distribution DESTDIR=${LIVE} NO_MAKEDEV=yes && \
cd $D
ln -sf dev/null kernel
mkdir ${D}/admin
mtree -c -p ${LIVE} > ${D}/admin/installed.mtree
To download this script run :
curl --output build_jail.sh http://bsdtips.utcorp.net/mediawiki/index.php\?title=build_jail.sh\&action=raw
The two lines near the end provide for the creation of an 'admin' directory, into which we place an mtree (see also What changed) database that reflects the original state of the installed jail. As we make changes, this database will become useful
You can use it like this.
build_jail.sh -D /usr/jails/test1
[edit] Startup
Until now, we've focused on the concepts and activities nessesary for creating jails. At this point we will cover starting, stopping and managing the jail. If you want the jail(s) to become active at boot, the following is a reasonable set of system startup variables. You should however read and understand the handbook on it in any case.
[edit] Jail rc.conf variables
First, we set some variables that effect all jails in rc.conf.
# Jails jail_enable="YES" jail_list="test1 test2" jail_devfs_enable="YES" jail_devfs_ruleset="devfsrules_jail" jail_procfs_enable="YES"
This (in order) will enable the jail subsystem, set a list of jails to start, mount a devfs in each of the jails, apply the devfs rule(s) "devfsrules_jail" in each jail. Now we will add specifics for each jail.
# test1 jail_test1_rootdir=/usr/jails/test1/live/ jail_test1_hostname="test1.domain.com" jail_test1_ip="10.10.10.10" jail_test1_exec_start="/bin/sh /etc/rc" jail_test1_exec_stop="/bin/sh /etc/rc.shutdown" # test2 jail_test2_rootdir=/usr/jails/test2/live/ jail_test2_hostname="test2.domain.com" jail_test2_ip="10.10.10.11" jail_test2_exec_start="/bin/sh /etc/rc" jail_test2_exec_stop="/bin/sh /etc/rc.shutdown"
[edit] Network rc.conf variables
Now we must add the IP aliases the jail(s) will run on to /etc/rc.conf.
ifconfig_fxp0_alias0="inet 10.10.10.10 netmask 255.255.255.255" ifconfig_fxp0_alias1="inet 10.10.10.11 netmask 255.255.255.255"
Once you have saved rc.conf, you have to stop and start the network (this could cause problems if done logged in remotely).
/etc/rc.d/netif stop /etc/rc.d/netif start
You should be able to ping these aliases now. For further detail, man rc.conf(5), search for "network_interfaces".
[edit] Starting and stopping jails
Now you can either start and stop jails individually by naming them on the command line, or by running '/etc/rc.d/jail start' without any jail name argument.
> /etc/rc.d/jail start test
Configuring jails:.
Starting jails: test.khome.utcorp.net.
> jls
JID IP Address Hostname Path
1 10.10.10.10 test.domain.com /usr/jails/test1/live
> /etc/rc.d/jail stop test
Stopping jails: test.khome.utcorp.net.
[edit] Manage the jail
[edit] Package directory
If you have a ports builder jail, for instance in "/usr1/jails/ports/live", you would mount the ports tree read only by addingthis to /etc/fstab :
/usr1/jails/ports/live/usr/ports /usr1/jails/nxserver/live/usr/ports nullfs ro 0 0
Make sure the target directory exists and then mount
mkdir /usr1/jails/nxserver/live/usr/ports mount /usr1/jails/nxserver/live/usr/ports
[edit] From the host OS
As we have seen, we can list the running jails via jls :
> jls
JID IP Address Hostname Path
1 10.10.10.10 test.domain.com /usr/jails/test1/live
You can now use the JID (jail I.D.) to 'jump into' the jail to execute commands via jexec. For instance, you can run sysinstall to add whatever post install options you like (run sshd on start, etc.) :
jexec 1 sysinstall
[edit] From within the jail
The jexec command does not provide complte tty semantics, which some programs require. I therefore suggest running sshd inside the jail. First add the sshd startup to the jail and (re)start it.
> echo 'sshd_enable="YES"` >> /usr/jails/test1/live/etc/rc.conf > /etc/rc.d/jail stop test Stopping jails: test.khome.utcorp.net. > /etc/rc.d/jail start test Configuring jails:. Starting jails: test.khome.utcorp.net.
Now we will add a password to the root account. Then add a user to the system; assign that user into the group 'wheel' (so it can su to root). Now you can ssh into the jail and with minor exceptions treat it as a separate machine.
> jexec 1 passwd root Changing local password for root New Password: Retype New Password: > jexec 1 adduser [adduser script omitted, see Adding a user from the Handbook.
Use pw to add the user to the wheel group (see Dont edit /etc/passwd! Like so:
> jexec 1 pw groupmod wheel -M [USER]
[edit] OPTIONAL : Mounting ports tree
Building ports in a jail is a good, the basis for this and (one possible) method is discussed in Ports jail. Adding this line (substitute local paths) to /etc/fstab will put a read only copy of the ports tree into the jail :
/usr/jails/ports7/live/usr/ports /usr1/jails/tmp81/live/usr/ports nullfs ro 0 0
Then mkdir the mount point and mount it :
mkdir /usr1/jails/tmp81/live/usr/ports mount /usr1/jails/tmp81/live/usr/ports
Now you can pkg_add from the /usr/ports/packages/ALL/ or other package directories. See Ports building options for tips on how to build separate repositories with customized options.
[edit] Discussion
- Unpatched, FreeBSD only allows 1 IP per jail.
- Unless you enable raw sockets, ping will not work from jail. There is a reason it is disabled by default.
- Unless you enable SYS V IPC for jail, SYS V IPC will not work from jail. SYS V IPC is not virtualized like the rest of the jail kernel interfaces so, one copy is shared among the all the jails on a machine .... DANGER WILL ROBINSON!
- The build_jail.sh script has very little error checking built in to it, it could use some more.
- You might want to copy /etc/resolv.conf to the jail's etc. Depending on the network configuration.
- If you want to run sshd, you will likely want to add a ListenAddress [PRIMARY IP ADDRESS] statement to your /etc/ssh/sshd_config file in the host environment. Otherwise the host machine's sshd will bind port 22 on all addresses before the jail can. The add 'sshd_enable="YES"' to the jail's /etc/rc.conf.
Additional reading on jails
Gongo 22:05, 15 October 2007 (UTC)
