view README.CD @ 717:0b04d8a47390

bitkeeper revision 1.425 (3f606cc1o_4A4klDuXS7zt2NJS08Ow)

Merge labyrinth.cl.cam.ac.uk:/auto/groups/xeno/BK/xeno.bk
into labyrinth.cl.cam.ac.uk:/auto/anfs/scratch/labyrinth/iap10/xeno-clone/xeno.bk
author iap10@labyrinth.cl.cam.ac.uk
date Thu Sep 11 12:38:25 2003 +0000 (2003-09-11)
parents f6f8ca2fa8d4
children afe5451d9f97
line source
1 #############################
2 __ __ _ ___
3 \ \/ /___ _ __ / | / _ \
4 \ // _ \ '_ \ | || | | |
5 / \ __/ | | | | || |_| |
6 /_/\_\___|_| |_| |_(_)___/
8 #############################
10 XenDemoCD 1.0 rc1
11 University of Cambridge Computer Laboratory
12 31 Aug 2003
14 http://www.cl.cam.ac.uk/netos/xen
16 Welcome to the Xen Demo CD!
18 Executive Summary
19 =================
21 This CD is a standalone demo of the Xen Virtual Machine Monitor (VMM)
22 and Linux-2.4 OS port (xenolinux). It runs entirely off the CD,
23 without requiring hard disk installation. This is acheived using a ram
24 disk to store mutable file system data while using the CD for
25 everything else. The CD can also be used for installing Xen/xenolinux
26 to disk, and includes a source code snapshot along with all of the
27 tools required to build it.
29 Booting the CD
30 ==============
32 The Xen VMM is currently fairly h/w specific, but porting new device
33 drivers is relatively straight forward thanks to Xen's Linux driver
34 compatibility layer. The current snapshot supports the following
35 hardware:
37 CPU: Pentium Pro/II/III/IV/Xeon, Athlon (i.e. P6 or newer) SMP supported
38 IDE: Intel PIIX chipset, others will be PIO only (slow)
39 SCSI: Adaptec / Dell PERC Raid (aacraid), megaraid, Adaptec aic7xxx
40 Net: Recommended: Intel e1000, Broadcom BCM57xx (tg3), 3c905 (3c59x)
41 Tested but requires extra copies : pcnet32
42 Untested and also requires extra copies : Intel e100, tulip
44 Because of the demo CD's use of ram disks, make sure you have plenty
45 of RAM (256MB+).
47 To try out the Demo, boot from CD (you may need to change your BIOS
48 configuration to do this), hit a key on either the keyboard or serial
49 line to pull up the Grub boot menu, then select one of the four boot
50 options:
52 Xen / linux-2.4.22 X using DHCP
53 Xen / linux-2.4.22 X using cmdline IP config
54 Xen / linux-2.4.22 text using DHCP
55 Xen / linux-2.4.22 text using cmdline IP config
56 linux-2.4.22
57 linux-2.4.22 single
58 linux-2.4.20-rc1 single
60 The last three options are plain linux kernels that run on the bare
61 machine, and are included simply to help diagnose driver compatibility
62 problems. If you are going for a command line IP config, hit "e" at
63 the grub menu, then edit the "ip=" parameters to reflect your setup
64 e.g. "ip=<ipaddr>::<gateway>:<netmask>::eth0:off". It shouldn't be
65 necessary to set either the nfs server or hostname
66 parameters. Alternatively, once xenolinux has booted you can login and
67 setup networking with ifconfig and route in the normal way.
69 To make things easier for yourself, its worth trying to arrange for an
70 IP address which is the first in a sequential range of free IP
71 addresses. Its useful to give each VM instance its own IP address
72 (though it is possible to do NAT or use private addresses etc), and
73 the configuration files on the CD allocate IP addresses sequentially
74 for subsequent domains unless told otherwise.
76 After selecting the kernel to boot, stand back and watch Xen boot,
77 closely followed by "domain 0" running the xenolinux kernel. The boot
78 messages are also sent to the serial line (the baud rate can be set on
79 the Xen cmdline), which can be very useful for debugging should
80 anything important scroll off the screen. Xen's startup messages will
81 look quite familiar as much of the hardware initialisation (SMP boot,
82 apic setup) and device drivers are derived from Linux.
84 If everything is well, you should see the linux rc scripts start a
85 bunch of standard services including sshd. Login on the console or
86 via ssh::
87 username: user root
88 password: xendemo xendemo
90 Once logged in, it should look just like any regular linux box. All
91 the usual tools and commands should work as per usual. You can start
92 an xserver with 'startx' if you elected not to start one at boot. The
93 current rc scripts also starts an Apache web server, which you should
94 be able to issue requests to on port 80. If you want to browse the
95 Xen / Xenolinux source, it's all located under /local, complete with
96 BitKeeper repository.
98 Because CD's aren't exactly known for their high performance, the
99 machine will likely feel rather sluggish. You may wish to go ahead and
100 install Xen/XenoLinux on your hard drive, either dropping Xen and the
101 XenoLinux kernel down onto a pre-existing Linux distribution, or using
102 the file systems from the CD (which are based on RH7.2). See the
103 installation instructions later in this document.
106 Starting other domains
107 ======================
109 There's a web interface for starting and managing other domains (VMs),
110 but since we generally use the command line tools they're probably
111 rather better debugged at present. The key command is 'xenctl' which
112 lives in /local/bin and uses /etc/xenctl.xml for its default
113 configuration. Run 'xenctl' without any arguments to get a help
114 message.
116 To create a new domain, using the same xenolinux kernel image as used
117 for domain0, the next consecutive IP address, and the same CD-based
118 file system, type:
120 xenctl new -n give_this_domain_a_name
122 domctl will return printing the domain id that has been allocated to
123 the new domain (probably '1' if this is the first domain to be fired
124 up). If you're running off the CD this will take a while, as there's
125 huge piles of Java goop grinding away... Then, fire up the domain:
127 xenctl start -n<domid>
129 You should see your domain boot and be able to ping and ssh into it as
130 before.
132 "xenctl list" provides status information about running domains,
133 though is currently only allowed to be run by domain 0. It accesses
134 /proc/xeno/domains to read this information from Xen. You can also use
135 xenctl to 'stop' (pause) a domain, or 'kill' a domain. You can either
136 kill it nicely by sending a shutdown event and waiting for it to
137 terminate, or blow the sucker away with extreme prejudice.
139 If you want to configure the new domain differently, type 'xenctl' to
140 get a list of arguments, e.g. use the "-4" option to set a diffrent
141 IPv4 address. If you haven't any spare IP addresses on your network,
142 you can configure other domains with link-local addresses
143 (169.254/16), but then you'll only be able to access domains other
144 than domain0 from within the machine (they won't be externally
145 routeable). To automate this, there's an /etc/xenctl-linklocal.xml
146 which you can copy in place of /etc/xenctl.xml
148 xenctl can be used to set the new kernel's command line, and hence
149 determine what it uses as a root file system etc. Although the default
150 is to boot in the same manner that domain0 did (using the RAM-based
151 file system for root and the CD for usr) it's possible to configure any
152 of the following possibilities, for example:
154 * initrd=/boot/initrd init=/linuxrc
155 boot using an initial ram disk, executing /linuxrc (as per this CD)
157 * root=/dev/hda3 ro
158 boot using a standard hard disk partition as root
160 * root=/dev/xvda1 ro
161 boot using a pre-configured 'virtual block device' that will be
162 attached to a virtual disk that will previously had a file system
163 installed on it.
165 * root=/dev/nfs nfsroot=/path/on/server ip=<blah_including server_IP>
166 Boot using an NFS mounted root file system. This could be from a
167 remote NFS server, or from an NFS server running in another
168 domain. The latter is rather a useful option.
171 A typical setup might be to allocate a standard disk partition for
172 each domain and populate it with files. To save space, having a shared
173 read-only usr partition might make sense.
175 Alternatively, you can use 'virtual disks', which are stored as files
176 within a custom file system. "xenctl partitions add" can be used to
177 'format' a partition with the file system, then virtual disks can be
178 created with "xenctl vd create". Virtual disks can then be attached to
179 a running domain as a 'virtual block device' using "xenctl vdb
180 create". The virtual disk can then optionally be partitioned
181 (e.g. "fdisk /dev/xvda") or have a file system created on it directly
182 (e.g. "mkfs -t ext3 /dev/xvda"). The virtual disk can then accessed
183 by a virtual block device associated with another domain, and even
184 used as a boot device.
186 Both virtual disks and real partitions should only be shared domains
187 in a read-only fashion otherwise the linux kernels will obviously get
188 very confused if the file system structure changes underneath them!
189 If you want read-write sharing, export the directory to other domains
190 via NFS.
193 About The Xen Demo CD
194 =====================
196 The purpose of the Demo CD is to distribute a snapshot of Xen's source,
197 and simultaneously provide a convenient means for enabling people to
198 get experience playing with Xen without needing to install it on their
199 hard drive. If you decide to install Xen/XenoLinux you can do so
200 simply by folling the INSTALL instructions and copying the contents of
201 the CD on to a suitably formated disk partition and install the Grub
202 bootloader.
204 This is a bootable CD that loads Xen, and then a Linux 2.4.22 OS image
205 ported to run on Xen. The CD contains a copy of a file system based on
206 the RedHat 7.2 distribution that is able to run directly off the CD
207 ("live ISO"), using a "tmpfs" RAM-based file system for root (/etc
208 /var etc). Changes you make to the tmpfs will obviously not be
209 persistent across reboots!
211 Because of the use of a ram-based file system for root, you'll need
212 plenty of memory to run this CD -- something like 96MB per VM. This is
213 not a restriction of Xen : once you've installed Xen, XenoLinux and
214 the file system images on your hard drive you'll find you can boot VMs
215 in just a few MBs.
217 The CD contains a snapshot of the Xen and XenoLinux code base that we
218 believe to be pretty stable, but lacks some of the features that are
219 currently still work in progress e.g. OS suspend/resume to file, and
220 various memory management enhancements to provide fast inter-OS
221 communication and sharing of memory pages between OSs. We'll release
222 newer snapshots as required, in the form of a BitKeeper repository
223 hosted on http://xen.bkbits.net (follow instructions from the project
224 home page). We're obviously grateful to receive any
225 bug fixes or other code you can contribute.
228 Installing from the CD
229 ----------------------
231 If you're installing Xen/XenoLinux onto an existing linux file system
232 distribution, its typically necessary to copy the Xen VMM
233 (/boot/image.gz) and XenoLinux kernels (/boot/xenolinux.gz) then
234 modify the Grub config (/boot/grub/menu.lst or /boot/grub/grub.conf)
235 on the target system.
237 Xen is a "multiboot" standard boot image. Despite being a 'standard',
238 few boot loaders actually support it. The only two we know of are
239 Grub, and our modified version of linux kexec (for booting off a
240 XenoBoot CD -- PlanetLab have adopted the same boot CD approach).
242 If you need to install grub on your system, you can do so either by
243 building the Grub source tree /usr/local/grub-0.93-iso9660-splashimage
244 or by copying over all the files in /boot/grub and then running
245 /sbin/grub and following the usual grub documentation. You'll then
246 need to configure the Grub config file.
248 A typical Grub menu option might look like:
250 title Xen / XenoLinux 2.4.22
251 kernel /boot/image.gz dom0_mem=131072 ser_baud=115200 noht
252 module /boot/xenolinux.gz root=/dev/sda4 ro console=tty0
254 The first line specifies which xen image to use, and what command line
255 arguments to pass to Xen. In this case, we set the maximum amount of
256 memory to allocate to domain0, and the serial baud rate (the default
257 is 9600 baud). We could also disable smp support (nosmp) or disable
258 hyper-threading support (noht). If you have multiple network interface
259 there are various options to select which ones to use.
261 The second line specifies which xenolinux image to use, and the
262 standard linux command line arguments to pass to the kernel. In this
263 case, we're configuring the root partition and stating that it should
264 be mounted read-only (normal practise). If the file system isn't
265 configured for DHCP then we'd probably want to configure that on the
266 kernel command line too.
268 If we were booting with an initial ram disk (initrd), then this would
269 require a second "module" line, with no arguments.
272 Installing the file systems from the CD
273 ---------------------------------------
275 If you haven't an existing Linux installation onto which you can just
276 drop down the Xen and XenoLinux images, then the file systems on the
277 CD provide a quick way of doing an install.
279 Choose one or two partitions, depending on whether you want a separate
280 /usr or not. Make file systems on it/them e.g.:
281 mkfs -t ext3 /dev/hda3
282 [or mkfs -t ext2 /dev/hda3 && tune2fs -j /dev/hda3 if using an old
283 version of mkfs]
285 Next, mount the file system(s) e.g.:
286 mkdir /mnt/root && mount /dev/hda3 /mnt/root
287 [mkdir /mnt/usr && mount /dev/hda4 /mnt/usr]
289 To install the root file system, simply untar /usr/XenDemoCD/root.tar.gz
290 cd /mnt/root && tar -zxpf /usr/XenDemoCD/root.tar.gz
292 You'll need to edit /mnt/root/etc/fstab to reflect your file system
293 configuration. Changing the password file (etc/shadow) is probably a
294 good idea too.
296 To install the usr file system, copy the file system from CD on /usr,
297 though leaving out the "XenDemoCD" and "boot" directories:
298 cd /usr && cp -a doc games include lib local root share tmp X11R6 bin dict etc html kerberos libexec man sbin src /mnt/usr/
300 If you intend to boot off these file systems (i.e. use them for
301 domain0), then you probably want to copy the /usr/boot directory on
302 the cd over the top of the current symlink to /boot on your root
303 filesystem (after deleting the current symlink) i.e.:
304 cd /mnt/root ; rm boot ; cp -a /usr/boot .
306 The XenDemoCD directory is only useful if you want to build your own
307 version of the XenDemoCD (see below).
309 Debugging
310 ---------
312 Xen has a set of debugging features that can be useful to try and
313 figure out what's going on. Hit 'h' on the serial line or ScrollLock-h
314 on the keyboard to get a list of supported commands.
316 If you have a crash you'll likely get a crash dump containing and EIP
317 (PC), which along with and 'objdump -d image' can be useful in
318 figuring out what's happened.
321 Description of how the XenDemoCD boots
322 --------------------------------------
324 1. Grub is used to load Xen, a xenolinux kernel, and an initrd (initial
325 ram disk). [The source of the version of Grub used is in /usr/local/]
327 2. the init=/linuxrc command line causes linux to execute /linuxrc in
328 the initrd.
330 3. the /linuxrc file attempts to mount the CD by trying the likely
331 locations /dev/hd[abcd].
333 4. it then creates a 'tmpfs' file system and untars the
334 'XenDemoCD/root.tar.gz' file into the tmpfs. This contains hopefully
335 all the files that need to be mutable. (this would be so much easier
336 if Linux supported 'stacked' or union file systems...)
338 5. Next, /linuxrc uses the pivot_root call to change the root file
339 system to the tmpfs, with the CD mounted as /usr.
341 6. It then invokes /sbin/init in the tmpfs and the boot proceeds
342 normally.
345 Building your own version of the XenDemoCD
346 ------------------------------------------
348 The filesystems on the CD are based heavily on Peter Anvin's
349 SuperRescue CD version 2.1.2, which takes its content from RedHat
350 7.2. Since Xen uses a "multiboot" image format, it was necessary to
351 change the bootloader from isolinux to Grub0.93 with Leonid
352 Lisovskiy's <lly@pisem.net> grub.0.93-iso9660.patch
354 The Xen Demo CD contains all of the build scripts that were used to
355 create it, so its possible to 'unpack' the current iso, modifiy it,
356 then build a new iso. The procedure for doing so is as follows:
358 First, mount either the CD, or the iso image of the CD:
360 mount /dev/cdrom /mnt/cdrom
361 or:
362 mount -o loop xendemo-1.0beta.iso /mnt/cdrom
364 cd to the directory you want to 'unpack' the iso into then run the
365 unpack script:
367 cd /local/xendemocd
368 /mnt/cdrom/XenDemoCD/unpack-iso.sh
370 The result is a 'build' directory containing the file system tree
371 under the 'root' directory. e.g. /local/xendemocd/build/root
373 To add or remove rpms, its possible to use 'rpm' with the --root
374 option to set the path. For more complex changes, it easiest to boot a
375 machine using using the tree via NFS root. Before doing this, you'll
376 need to edit fstab to comment out the seperate mount of /usr.
378 One thing to watch out for: as part of the CD build process, the
379 contents of the 'rootpatch' tree gets copied over the existing 'root'
380 tree replacing various files. The intention of the rootpatch tree is
381 to contain the files that have been modified from the original RH
382 distribution (e.g. various /etc files). This was done to make it
383 easier to upgrade to newer RH versions in the future. The downside of
384 this is that if you edit an existing file in the root tree you should
385 check that you don't also need to propagate the change to the
386 rootpatch tree to avoid it being overwritten.
388 Once you've made the changes and want to build a new iso, here's the
389 procedure:
391 cd /local/xendemocd/build
392 echo '<put_your_name_here>' > Builder
393 ./make.sh put_your_version_id_here >../buildlog 2>&1
395 This process can take 30 mins even on a fast machine, but you should
396 eventually end up with an iso image in the build directory.
398 notes:
400 root - the root of the file system heirarchy as presented to the running system
402 rootpatch - contains files that have been modified from the standard RH, and copied over the root tree as part of the build procedure.
404 irtree - the filie system tree that will go into the initrd (initial ram disk)
406 work - a working directory used in the build process
408 usr - this should really be in 'work' as its created as part of the
409 build process. It contains the 'immutable' files that will be served
410 from the CD rather than the tmpfs containing the contents of root.tar.gz
411 Some files that are normally in /etc or /var that are large and
412 actually unlikely to need changing have been moved into /usr/root
413 and replaced with links.
415 Ian Pratt
416 9 Sep 2003