view README @ 2763:4b524192e62b

bitkeeper revision 1.1159.138.1 (41811be9-M5W9ujjnrgAr5BvIUxzIQ)

Doc fixes. Definitely more still to do.
author kaf24@freefall.cl.cam.ac.uk
date Thu Oct 28 16:18:49 2004 +0000 (2004-10-28)
parents 11be1dfb262b
children b6703e4c6e16
line source
1 ###############################
2 __ __ ____ ___
3 \ \/ /___ _ __ |___ \ / _ \
4 \ // _ \ '_ \ __) || | | |
5 / \ __/ | | | / __/ | |_| |
6 /_/\_\___|_| |_| |_____(_)___/
8 ###############################
10 University of Cambridge Computer Laboratory
11 28 October 2004
13 http://www.cl.cam.ac.uk/netos/xen
15 About the Xen Virtual Machine Monitor
16 =====================================
18 "Xen" is a Virtual Machine Monitor (VMM) originally developed by the
19 Systems Research Group of the University of Cambridge Computer
20 Laboratory, as part of the UK-EPSRC funded XenoServers project.
22 The XenoServers project aims to provide a "public infrastructure for
23 global distributed computing", and Xen plays a key part in that,
24 allowing us to efficiently partition a single machine to enable
25 multiple independent clients to run their operating systems and
26 applications in an environment providing protection, resource
27 isolation and accounting. The project web page contains further
28 information along with pointers to papers and technical reports:
29 http://www.cl.cam.ac.uk/xeno
31 Xen has since grown into a project in its own right, enabling us to
32 investigate interesting research issues regarding the best techniques
33 for virtualizing resources such as the CPU, memory, disk and network.
34 The project has been bolstered by support from Intel Research
35 Cambridge, and HP Labs, who are now working closely with us. We're
36 also in receipt of support from Microsoft Research Cambridge to port
37 Windows XP to run on Xen.
39 Xen enables multiple operating system images to execute concurrently
40 on the same hardware with very low performance overhead --- much lower
41 than commercial offerings for the same x86 platform.
43 This is achieved by requiring OSs to be specifically ported to run on
44 Xen, rather than allowing unmodified OS images to be used. Crucially,
45 only the OS needs to be changed -- all of the user-level application
46 binaries, libraries etc can run unmodified. Hence the modified OS
47 kernel can typically just be dropped into any existing OS distribution
48 or installation.
50 Xen currently runs on the x86 architecture, but could in principle be
51 ported to others. In fact, it would have been rather easier to write
52 Xen for pretty much any other architecture as x86 is particularly
53 tricky to handle. A good description of Xen's design, implementation
54 and performance is contained in our October 2003 SOSP paper, available
55 at http://www.cl.cam.ac.uk/netos/papers/2003-xensosp.pdf
56 [update: work to port Xen to x86_64 and IA64 is underway]
58 Five different operating systems have been ported to run on Xen:
59 Linux 2.4/2.6, Windows XP, NetBSD, FreeBSD and Plan 9.
61 The Linux 2.4 port (currently Linux 2.4.26) works very well -- we
62 regularly use it to host complex applications such as PostgreSQL,
63 Apache, BK servers etc. It runs every user-space applications we've
64 tried. We refer to our version of Linux ported to run on Xen as
65 "XenLinux", although really it's just standard Linux ported to a new
66 virtual CPU architecture that we call xen-x86.
68 NetBSD has been ported to Xen by Christian Limpach, and will hopefully
69 soon become part of the standard release. Work on a FreeBSD port has
70 been started by Kip Macy, and we hope to see this complete for the 2.0
71 release. Ron Minnich has been working on Plan 9.
73 The Windows XP port is nearly finished. It's running user space
74 applications and is generally in pretty good shape thanks to some hard
75 work by a team over the summer. Of course, there are issues with
76 releasing this code to others. We should be able to release the
77 source and binaries to anyone that has signed the Microsoft academic
78 source license, which these days has very reasonable terms. We are in
79 discussions with Microsoft about the possibility of being able to make
80 binary releases to a larger user community. Obviously, there are
81 issues with product activation in this environment which need to be
82 thought through.
84 So, for the moment, you only get to run Linux 2.4/2.6 and NetBSD on
85 Xen, but we hope this will change before too long. Even running
86 multiple copies of the same OS can be very useful, as it provides a
87 means of containing faults to one OS image, and also for providing
88 performance isolation between the various OS, enabling you to either
89 restrict, or reserve resources for, particular VM instances.
91 It's also useful for development -- each version of Linux can have
92 different patches applied, enabling different kernels to be tried
93 out. For example, the "vservers" patch used by PlanetLab applies
94 cleanly to our ported version of Linux.
96 We've successfully booted over 128 copies of Linux on the same machine
97 (a dual CPU hyperthreaded Xeon box) but we imagine that it would be
98 more normal to use some smaller number, perhaps 10-20.
100 A common question is "how many virtual machines can I run on hardware
101 xyz?". The answer is very application dependent, but the rule of thumb
102 is that you should expect to be able to run the same workload under
103 multiple guest OSs that you could run under a single Linux instance,
104 with an additional overhead of a few MB per OS instance.
106 One key feature in this new release of Xen is `live migration'. This
107 enables virtual machines instances to be dynamically moved between
108 physical Xen machines, with typical downtimes of just a few tens of
109 milliseconds. This is really useful for admins that want to take a
110 node down for maintenance, or to load balance a large number of
111 virtual machines across a cluster.
114 Hardware support
115 ================
117 Xen is intended to be run on server-class machines, and the current
118 list of supported hardware very much reflects this, avoiding the need
119 for us to write drivers for "legacy" hardware. It is likely that some
120 desktop chipsets will fail to work properly with the default Xen
121 configuration: specifying 'noacpi' or 'ignorebiostables' when booting
122 Xen may help in these cases.
124 Xen requires a "P6" or newer processor (e.g. Pentium Pro, Celeron,
125 Pentium II, Pentium III, Pentium IV, Xeon, AMD Athlon, AMD Duron).
126 Multiprocessor machines are supported, and we also have basic support
127 for HyperThreading (SMT), although this remains a topic for ongoing
128 research. We're also working on an AMD x86_64 port (though Xen should
129 run on Opterons in 32-bit mode just fine).
131 Xen can currently use up to 4GB of memory. It's possible for x86
132 machines to address more than that (64GB), but it requires using a
133 different page table format (3-level rather than 2-level) that we
134 don't currently support, as we are concentrating on an x86_64 port
135 that will more easily support large-memory configurations.
137 In contrast to previous Xen versions, in Xen 2.0 device drivers run
138 within a privileged guest OS rather than within Xen itself. This means
139 that we should be compatible with the full set of device hardware
140 supported by Linux. The default XenLinux build contains support for
141 relatively modern server-class network and disk hardware, but you can
142 add suppport for other hardware by configuring your XenLinux kernel in
143 the normal way (e.g. "make xconfig").
146 Building Xen and XenLinux
147 =========================
149 The public master BK repository for the 2.0 release lives at:
150 bk://xen.bkbits.net/xen-2.0.bk
152 To fetch a local copy, install the BitKeeper tools, then run:
153 'bk clone bk://xen.bkbits.net/xen-2.0.bk'
155 You can do a complete build of Xen, the control tools, and the
156 XenLinux kernel images with "make world". This can take 10 minutes
157 even on a fast machine. If you're on an SMP machine you may wish to
158 give the '-j4' argument to make to get a parallel build. All of the
159 files that are built are placed under the ./install directory. You
160 can then install everything to the standard system directories
161 (e.g. /boot, /usr/bin, /usr/lib/python/ etc) by typing "make install".
163 Take a look in install/boot/:
164 install/boot/xen.gz The Xen 'kernel' (formerly image.gz)
165 install/boot/vmlinuz-2.4.27-xen0 Domain 0 XenLinux kernel (xenolinux.gz)
166 install/boot/vmlinuz-2.4.27-xenU Unprivileged XenLinux kernel
168 The difference between the two Linux kernels that are built is
169 due to the configuration file used for each. The "U" suffixed
170 unprivileged version doesn't contain any of the physical hardware
171 device drivers, so is 30% smaller and hence may be preferred for
172 your non-privileged domains.
174 The install/boot directory will also contain the config files
175 used for building the XenLinux kernels, and also versions of Xen
176 and XenLinux kernels that contain debug symbols (xen-syms and
177 vmlinux-syms-2.4.27-xen0) which are essential for interpreting crash
178 dumps.
180 Inspect the Makefile if you want to see what goes on during a
181 build. Building Xen and the tools is straightforward, but XenLinux is
182 more complicated. The makefile needs a 'pristine' linux kernel tree
183 which it will then add the Xen architecture files to. You can tell the
184 makefile the location of the appropriate linux compressed tar file by
185 setting the LINUX_SRC environment variable
186 (e.g. "LINUX_SRC=/tmp/linux-2.4.27.tar.gz make world") or by placing
187 the tar file somewhere in the search path of LINUX_SRC_PATH which
188 defaults to ".:..". If the makefile can't find a suitable kernel tar
189 file it attempts to download it from kernel.org, but this won't work
190 if you're behind a firewall.
192 After untaring the pristine kernel tree, the makefile uses the
193 'mkbuildtree' script to add the Xen patches the kernel. "make world"
194 then build two different XenLinux images, one with a "-xen0" extension
195 which contains hardware device drivers and is intended to be used in
196 the first virtual machine ("domain 0"), and one with a "-xenU"
197 extension that just contains virtual-device drivers. The latter can be
198 used for all non hardware privileged domains, and is substantially
199 smaller than the other kernel with its selection of hardware drivers.
201 If you don't want to use bitkeeper to download the source, you can
202 download prebuilt binaries and src tar balls from the project
203 downloads page: http://www.cl.cam.ac.uk/netos/xen/downloads/
206 Using the domain control tools
207 ==============================
209 Before starting domains you'll need to start the node management
210 daemon: "xend start".
211 The primary tool for starting and controlling domains is "xm".
212 "xm help <cmd>" will tell you how to use it.
214 Further documentation is in the docs/ directory. Postscript, PDF and
215 HTML versions of the user manual can be found in the ps/, pdf/ and
216 html/ subdirectories.