HSC
Network Security Consulting Agency Since 1989 - Specialized in Unix, Windows, TCP/IP and Internet
Text mode: access to the page content
Hervé Schauer Consultants
You are here: Home > Resources > Tips > The jail() function under FreeBSD
Go to: HSC Trainings
Télécharger le catalogue des formations
Search:  
Version française
   Services   
o Skills & Expertise
o Consulting
o ISO 27001 services
o Audit & Assessment
o Penetration tests
o Vunerability assessment (TSAR)
o Forensics
o ARJEL
o Training courses
o E-learning
   Conferences   
o Agenda
o Past events
o Tutorials
   Resources   
o Thematic index
o Tips
o Lectures
o Courses
o Articles
o Tools (download)
o Vulnerability watch
   Company   
o Hervé Schauer
o Team
o Job opportunities
o Credentials
o History
o Partnerships
o Associations
   Press and
 communication
 
 
o HSC Newsletter
o Bulletin juridique HSC
o Press review
o Press releases
o Publications
   Contacts   
o How to reach us
o Specific inquiries
o Directions to our office
o Hotels near our office
|>|The jail() function under FreeBSD  

by Yann Berthier (11/05/2001)



The jail(8) function under FreeBSD, part 2.

(yes, I know, there is no part 1, and I'm not sure there will be one either ;-)
In fact, parts 1 and 2 of the jail tip were merged into what we will call now
the part 2 - the actual tip)


0x00 Plan
0x01 Introduction - or, what is jail
0x02 A brief overview of jail
0x03 How to setup a jail
0x04 More applications in the jail
0x05 The future of jail
0x06 Conclusion


0x01 - What is jail

The traditional unix security model is simple, yet very effective: there is
the root account, and the others ;-) 

Several projects are trying to add more granularity to this model : support of
Posix 1.e ACLs, and notions derivated from what is called 'trusted operating
systems', such as MAC and DAC access (to name just a few).

See http://www.trustedbsd.org/ for an implementation for the FreeBSD project.

All of those technologies have one shortcoming, though: to dramatically
increase the administrative costs.

The jail implementation under FreeBSD is a -very effective in my opinion- way
to enhance the granularity of privileges while retaining the Unix security
model and its efficiency :)

Jail was initiated by Poul-Henning Kamp of the FreeBSD project. Many credits to
him for his wonderful work !

His original presentation on the subject is available here:
http://www.nluug.nl/events/sane2000/papers/kamp.pdf 

The jail() function appeared for the first time in FreeBSD 4.0


0x02 - A brief overview of jail

 . It's a virtual FreeBSD environment bound to an IP address. There can be as
   many jails as you set IP aliases 

 . Each jail is a fully functional environment, with it's own account database
   and configuration files.

 . A process in a jail is bound to its jail, and can't see processes outside
   its jail, nor interfere with.

 . You can delegate the root account of each jail while being able to enforce a
   system wide policy. And if one service in a jail is compromised, the
   attacker won't be able to escape the jail to leverage his gain to the rest
   of the system - even if he gains root access in the jail via a local
   exploit.


Let's take a look at an example:

You are an ISP doing multi hosting. 

You want to let your customer administrate the services he offers (update of
his web pages, creation of CGIs, creation of accounts for his POP3 server, and
so on). 

But you want to be sure your customer's settings can't affect your other
customers, and that, if his web site or whatever is compromised through a badly
written CGI script, or a new vulnerability is discovered in his FTP daemon, the
attacker won't be able to affect other customers, nor compromise the hosting
machine.

If you are in this case, jail is for you.

If you are not in this case, well, many chances are that jail is for you anyway
:)


0x03 - How to set up a basic jail

Convention: to distinguish between the guest environment (the jail) and the
host OS, the prompts of the commands contain the hostname:

ogoun for the host OS, whose main IP address is 10.0.0.75, and cell for the
jail.


# we create an IP alias on the ep0 network card ...
ifconfig ep0 alias 192.168.10.2 netmask 0xffffff00

# the directory of our jails will be /jail/$IP_ALIAS ...
mkdir -p /jail/192.168.10.2

# the show begins ... (we need the source tree of course)
cd /usr/src
export JAIL=/jail/192.168.10.2
make world DESTDIR=$JAIL
cd etc
make distribution DESTDIR=$JAIL NO_MAKEDEV=yes
cd $JAIL/dev
sh MAKEDEV jail
ln -sf null /boot/kernel/kernel
# to shut up the jail at boot time ...
touch ../etc/fstab


We use ipnat (from the ipfilter packet filtering tool) to redirect incoming
requests from the outside to the jail, and from the jail to the outside:

ogoun% cat /etc/filters/ipnat.rules
# 
map ep0 192.168.10.2/32 -> 0.0.0.0/32 portmap tcp/udp 10000:65000
map ep0 192.168.10.2/32 -> 0.0.0.0/32
# redirecting the request to the ftp port of our host to the ftp 
# port inside the jail
rdr ep0 10.0.0.75/32 port 21 -> 192.168.10.2 port 21

ogoun% ipnat -CF -f /etc/filters/ipnat.rules


To launch our jail :

ogoun% jail /jail/192.168.10.2/ cell.hsc.fr 192.168.10.2 /bin/tcsh


The jail seen from the inside ...

cell% ifconfig ep0
ep0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        inet 192.168.10.2 netmask 0xffffff00 broadcast 192.168.10.255
        ether 00:50:04:cc:3e:cc 
        media: 10baseT/UTP
        supported media: 10base2/BNC 10baseT/UTP 10base5/AUI

A line is added in the /etc/hosts file:

192.168.10.2            cell   cell.hsc.fr

The /etc/resolv.conf file contains the IP address of our DNS server:

cell% cat /etc/resolv.conf 
search hsc.fr
nameserver 1.2.3.4 


/etc/inetd.conf is edited, and the only service left uncommented is ftpd.
Remember, we are talking about the /etc/inetd.conf file of the jail !

If we launch inetd:

cell% inetd

cell% netstat -an
Active Internet connections (including servers)
Proto Recv-Q Send-Q  Local Address          Foreign Address        (state)
tcp4       0      0  192.168.10.2.21        *.*                    LISTEN


Populating the jail with users - well, one user in our case ;-)

The user monkeytest is created with the adduser(8) facility:

cell% cat /etc/passwd | grep monkeytest
monkeytest:*:1001:1001:monkeytest:/home/monkeytest:/bin/tcsh

cell% cat /etc/group | grep monkeytest
monkeytest:*:1001:


We can't see processes outside the jail ...

cell% ps aux
USER         PID %CPU %MEM   VSZ  RSS  TT  STAT STARTED      TIME COMMAND
root       48035  0.0  0.9  1488  290  p9  DWJ   2:17PM   0:00.14 /bin/csh
root       48921  0.0  0.7  1160  205  ??  DWsJ  2:17PM   0:00.01 inetd
root        8689  0.0  0.6  1196  200  p9  DWJ   2:31PM   0:00.01 su - monkeyte
monkeytest  8954  0.0  0.9  1428  268  p9  DWJ   2:31PM   0:00.03 -su (tcsh)
monkeytest  4108  0.0  0.3   576   78  p9  RW+J  3:29PM   0:00.00 ps aux


The flag 'J' in the STAT column let us know we are in a jail

In fact, many many other things let us know we are in a jail. 

Keep this in mind if you think as a jail for a good honeypot basis. 

However, a jail has several interesting properties as a honeypot:

 . the outgoing flow can be filtered and
   . the filtering rules can't be changed by the attacker,
   . the jail can't be used as a platform to attack other networks.
 . the administrator of the host OS can 'observe' the attacker without being
 seen
   . with a network sniffer (tcpdump)
   . with a tty sniffer (watch)
 . the log files in the jail can be looked at and kept safe outside of the jail
 (with no possibility for the attacker to tamper with)
 . the capability of FreeBSD to run native Linux binary is a plus, as most
 exploits found today are targeted toward the Linux platform

(no troll intended, it's only because of the vast number of Linux machines
installed 'as is' straight on the Internet)

No need to say, one should not rely only on ipfilter of the host box, in the
event of a vulnerability of the jail code which would permit to escape it !


Back to our jail ...:

We have access to the outside world: 

cell% ftp ftp.freebsd.org
Connected to ftp.freebsd.org.
220 usw3.freebsd.org FTP server (Version DG-4.1.73 983302105) ready.
Name (ftp.freebsd.org:yann): anonymous
331 Guest login ok, send your email address as password.
Password:
230 Guest login ok, access restrictions apply.
Remote system type is UNIX.
ftp> quit
221 Goodbye!


Except for raw sockets ...

cell% ping 1.2.3.4
ping: socket: Operation not permitted


And from outside the jail ...

ogoun% ifconfig ep0
ep0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        inet 10.0.0.75 netmask 0xffffff00 broadcast 192.70.106.255
        inet 192.168.10.2 netmask 0xffffff00 broadcast 192.168.10.255
        ether 00:50:04:cc:3e:cc 
        media: 10baseT/UTP
        supported media: 10base2/BNC 10baseT/UTP 10base5/AUI


ogoun% ps aux | grep J
root    48035  0.0  0.9  1488  290  p9  DWJ   2:17PM   0:00.14 /bin/csh
root    48921  0.0  0.7  1160  205  ??  DWsJ  2:17PM   0:00.01 inetd
root     8689  0.0  0.6  1196  200  p9  DWJ   2:31PM   0:00.01 su - monkeytest
titi     8954  0.0  0.9  1428  268  p9  DW+J  2:31PM   0:00.03 -su (tcsh)
yann     1617  0.0  0.5  1164  145  pa  DW+   3:29PM   0:00.01 grep J

The process 8954 is seen belonging to the user 'titi'. In fact, it's because
this user has the same uid (1001) as the user monkeytest inside the jail.

ogoun% netstat -an | fgrep '.21 '
tcp4       0      0  192.168.10.2.21        *.*                    LISTEN


One can now log via ftp to the ftpd server of the jail:

bubble:~$ ftp ogoun.hsc.fr 
Connected to 10.0.0.75 
220 cell.hsc.fr FTP server (Version 6.00LS) ready.
Name (10.0.0.75:yb): monkeytest
331 Password required for monkeytest.
Password:
230 User monkeytest logged in.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> ls
150 Opening ASCII mode data connection for '/bin/ls'.
total 193
drwx------  2 monkeytest  monkeytest     512 May  6 16:19 .ssh
-rw-r--r--  1 monkeytest  monkeytest  180935 Apr 19 20:32 thttpd-2.21b.tgz
226 Transfer complete.
ftp> quit
221 Goodbye.


ogoun% ipnat -l
List of active MAP/Redirect filters:
map ep0 192.168.10.2/32  -> 0.0.0.0/32  portmap tcp/udp 10000:65000
map ep0 192.168.10.2/32  -> 0.0.0.0/32 
rdr ep0 10.0.0.75/32 port 21 -> 192.168.10.2 port 21 tcp

List of active sessions:
MAP 192.168.10.2    1178  <- -> 192.70.107.75 10001 [1.2.3.4 53]
RDR 192.168.10.2    21    <- -> 192.70.107.75 21    [192.70.106.33 2821]


How to launch the jail at boot time

The $JAIL/etc/rc.conf file should be edited to fit the needs of the jail. 

cell% cat /etc/rc.conf
hostname="cell.hsc.fr"
syslogd_enable="YES"            # Run syslog daemon (or NO).
syslogd_flags="-s -s"
inetd_enable="YES"              # Run the network daemon dispatcher (or NO).  
inetd_flags="-wW -a 192.168.10.2"
sendmail_outbound_enable="YES"  # Dequeue stuck mail (YES/NO).
sendmail_outbound_flags="-q30m" # Flags to sendmail (outbound only)
sshd_enable="YES"


For the sshd daemon in the jail to be able to bind to the IP address of the
jail, you should say to your sshd daemon outside the jail _not_ to bind to this
IP address:

ogoun% cat /etc/ssh/sshd_config | grep -i listenaddress
ListenAddress 10.0.0.75
ListenAddress ::


And then ...

ogoun% cat /etc/rc.local
/usr/sbin/jail /jail/192.168.10.2/ cell.hsc.fr \
192.168.10.2 /bin/sh /etc/rc &


0x04 - More applications in the jail

An httpd server is compiled inside the jail (thttpd, see
http://www.acme.com/software/thttpd/).

The file /etc/rc.local is modified to launch the http server at boot time:

cell% cat /etc/rc.local
/usr/local/sbin/thttpd

A NAT entry should be added to redirect incoming connections to the port 80/tcp
of the host OS to the port 80/tcp of the jail:

ogoun% cat /etc/filters/ipnat.rules 
# 
map ep0 192.168.10.2/32 -> 0.0.0.0/32 portmap tcp/udp 10000:65000
map ep0 192.168.10.2/32 -> 0.0.0.0/32
# redirecting the request to the ftp port of our host to the ftp 
# port inside the jail
rdr ep0 10.0.0.75/32 port 21 -> 192.168.10.2 port 21
# redirecting requests for the sshd server
rdr ep0 10.0.0.75/32 port 22 -> 192.168.10.2 port 22
# redirecting requests for the httpd server
rdr ep0 10.0.0.75/32 port 80 -> 192.168.10.2 port 80


Our jail now begins to be a fully functional environment ...

cell% ps aux
USER         PID %CPU %MEM   VSZ  RSS  TT  STAT STARTED      TIME COMMAND
root       92416  0.0  0.3   580   92  pg  RW+J  3:04PM   0:00.00 ps aux
root       33099  0.0  0.5  1080  165  ??  DWsJ  2:55PM   0:00.01 cron
root       17139  0.0  0.5  1044  152  ??  DWsJ  2:56PM   0:00.01 syslogd -s -s
root       28713  0.0  0.6  1160  190  ??  DWsJ  2:56PM   0:00.00 inetd -wW -a 
root       30405  0.0  0.6  1080  178  ??  DWsJ  2:56PM   0:00.01 cron
root       32211  0.0  1.3  2312  394  ??  DWsJ  2:56PM   0:00.14 /usr/sbin/ssh
nobody     47140  0.0  0.9  1540  283  ??  DWsJ  2:56PM   0:00.01 /usr/local/sb
root       80634  0.0  1.0  2312  299  ??  DWsJ  6:07PM   0:00.71 sshd
root       87693  0.0  1.5  2404  481  ??  DWJ   3:04PM   0:00.15 sshd: monkeyt
monkeytest 89426  0.0  0.8  1428  259  pg  DWsJ  3:04PM   0:00.05 -tcsh (tcsh)
root       90233  0.0  0.7  1200  215  pg  DWJ   3:04PM   0:00.01 su -
root       90434  0.0  0.9  1448  276  pg  DWJ   3:04PM   0:00.04 -su (csh)

cell% netstat -an
Active Internet connections (including servers)
Proto Recv-Q Send-Q  Local Address          Foreign Address        (state)
tcp4       0      0  192.168.10.2.22        *.*                    LISTEN
tcp4       0      0  192.168.10.2.80        *.*                    LISTEN
tcp4       0      0  192.168.10.2.21        *.*                    LISTEN
Active UNIX domain sockets
Address  Type   Recv-Q Send-Q    Inode     Conn     Refs  Nextref Addr
c93d6440 dgram       0      0        0 c933c180        0        0
c933c180 dgram       0      0 c9558ac0        0 c93d6440        0 /var/run/log
c93d61c0 dgram       0      0        0 c933c080        0        0
c933c080 dgram       0      0 c94b1900        0 c93d61c0        0 /var/run/log


0x05 - The future of jail: the near -current implementation ;-)

For now, several sysctl exist to control the behavior of jails:

ogoun% sysctl -a | grep jail
jail.set_hostname_allowed: 0
jail.socket_unixiproute_only: 1
jail.sysvipc_allowed: 0

Anyway, this is somewhat limited.

Here is a mail from Robert Watson in a FreeBSD mailing list regarding the
future of jail ...

This weekend I was spending some time tweaking the jail(8) code to improve
it's SMPng-happiness as well as manageability.  Unfortunately, I ended up
rewriting it in the process :-).  I changed the model somewhat so that
jails are now persistently configred, joined, et al, and broke out the
chroot() from the creation/joining process, as with increased namespaces
(such as System V IPC) creating a nice clean failure was increasingly
difficult.  Aspects of individual jails may now be managed using sysctl's,
which appears to work reasonably well.  Clearly there's a lot of work left
to do, but I'd appreciate comments if people are interested:

  http://www.watson.org/~robert/jailng/
  
Simple example:

dev# ./jailctl 
usage:
  jailctl create [jailname]
  jailctl destroy [jailname]
  jailctl join [jailname] [-c chrootpath] [path] [cmd] [args...]
dev# ./jailctl create test
dev# sysctl -a | grep jail
jail.instance.test.sysvipc_permitted: 0
jail.instance.test.set_hostname_permitted: 1
jail.instance.test.socket_ipv4_permitted: 1
jail.instance.test.socket_unix_permitted: 1
jail.instance.test.socket_route_permitted: 1
jail.instance.test.socket_other_permitted: 0
jail.instance.test.ipv4addr: 0
dev# ./jailctl join test -c /tmp /bin/sh   
# ps ax
  PID  TT  STAT      TIME COMMAND
  907  d0  DWJ    0:00.02 /bin/sh
  908  d0  RW+J   0:00.00 ps ax
# exit
dev# ./jailctl destroy test
dev#

I also have a jailinit(8) in the works which would allow improved
startup/shutdown in the style of init(8) (sans the whole sigchild thing).
Another feature I'd like to add is a jail signal call that allows a signal
to be delivered to all processes inside a jail from outside, allowing an  
easier forceable shutdown.


0x06 - Conclusion

As it's clear with the mail from Robert Watson, jail is a work in progress, and
that many improvements will appear in a near future, notably regarding the
manageability of jails.

Anyway, jails are for now ready for production use, and are in fact used in the
real life. Give it a try, it's a powerful tool to compartment your FreeBSD
server.

To test a working example and see a jail in action, see the OpenRoot project:
http://sektor7.ath.cx/openroot/ where the initiator of the project give to
everyone the root access.



Last modified on 12 November 2003 at 13:55:00 CET - webmaster@hsc.fr
Mentions légales - Information on this server - © 1989-2013 Hervé Schauer Consultants