direct-io.hg

view README.CD @ 2299:8be25c10fa1e

bitkeeper revision 1.1159.42.4 (4124c9a2Di3cas2RmMeElljc94T5_A)

Merge scramble.cl.cam.ac.uk:/auto/groups/xeno/BK/xeno.bk
into scramble.cl.cam.ac.uk:/local/scratch/kaf24/xeno
author kaf24@scramble.cl.cam.ac.uk
date Thu Aug 19 15:39:14 2004 +0000 (2004-08-19)
parents b6006dac0b1a
children ce1a57f8737a bcc898397e06 631de0f7529b 11be1dfb262b
line source
1 #############################
2 __ __ _ _____
3 \ \/ /___ _ __ / | |___ /
4 \ // _ \ '_ \ | | |_ \
5 / \ __/ | | | | |_ ___) |
6 /_/\_\___|_| |_| |_(_)____/
8 #############################
10 XenDemoCD 1.3
11 University of Cambridge Computer Laboratory
12 24 Jan 2004
14 http://www.cl.cam.ac.uk/netos/xen
16 Welcome to the Xen Demo CD!
18 Executive Summary
19 =================
21 This CD is a standalone demo of the Xen Virtual Machine Monitor (VMM)
22 and Linux-2.4 OS port (Xenlinux). It runs entirely off the CD,
23 without requiring hard disk installation. This is achieved using a RAM
24 disk to store mutable file system data while using the CD for
25 everything else. The CD can also be used for installing Xen/Xenlinux
26 to disk, and includes a source code snapshot along with all of the
27 tools required to build it.
29 Booting the CD
30 ==============
32 It should be possible to get Xen working with any relatively modern
33 hardware supported by standard Linux. However, the version of XenLinux
34 built for the DemoCD is fairly h/w specific. If you need other
35 hardware, you'll have to configure and build your own xenlinux kernel.
36 Xen does require an 'i686'-class CPU or newer, so won't work on 486's
37 or plain Pentiums.
39 We have compiled in drivers for the following hardware:
41 CPU: Pentium Pro/II/III/IV/Xeon, Athlon (i.e. P6 or newer) SMP supported
42 IDE: Intel PIIX chipset, others will be PIO only (slow)
43 SCSI: Adaptec / Dell PERC Raid (aacraid), fusion MPT, megaraid, Adaptec aic7xxx
44 Net: Recommended: Intel e1000, Broadcom BCM57xx (tg3), 3c905 (3c59x)
45 Also supported: pcnet32, Intel e100, tulip
47 Because of the demo CD's use of RAM disks, make sure you have plenty
48 of RAM (256MB+).
50 To try out the Demo, boot from CD (you may need to change your BIOS
51 configuration to do this), then select one of the four boot options
52 from the Grub menu:
54 Xen / linux-2.4.26
55 Xen / linux-2.4.26 using cmdline IP configuration
56 Xen / linux-2.4.26 in "safe mode"
57 linux-2.4.22
59 The last option is a plain linux kernel that runs on the bare machine,
60 and is included simply to help diagnose driver compatibility
61 problems. The "safe mode" boot option might be useful if you're having
62 problems getting Xen to work with your hardware, as it disables various
63 features such as SMP, and enables some debugging.
65 If you are going for a command line IP config, hit "e" at
66 the grub menu, then edit the "ip=" parameters to reflect your setup
67 e.g. "ip=<ipaddr>::<gateway>:<netmask>::eth0:off". It shouldn't be
68 necessary to set either the nfs server or hostname
69 parameters. Alternatively, once Xenlinux has booted you can login and
70 setup networking with 'dhclient' or 'ifconfig' and 'route' in the
71 normal way.
73 To make things easier for yourself, it's worth trying to arrange for an
74 IP address which is the first in a sequential range of free IP
75 addresses. It's useful to give each VM instance its own public IP
76 address (though it is possible to do NAT or use private addresses),
77 and the configuration files on the CD allocate IP addresses
78 sequentially for subsequent domains unless told otherwise.
80 After selecting the kernel to boot, stand back and watch Xen boot,
81 closely followed by "domain 0" running the Xenlinux kernel. The boot
82 messages can also sent to the serial line by specifying the baud rate
83 on the Xen cmdline (e.g., 'com1=9600,8n1'); this can be very useful
84 for debugging should anything important scroll off the screen. Xen's
85 startup messages will look quite familiar as much of the hardware
86 initialisation (SMP boot, apic setup) and device drivers are derived
87 from Linux.
89 If everything is well, you should see the linux rc scripts start a
90 bunch of standard services including sshd. Login on the console or
91 via ssh::
92 username: user root
93 password: xendemo xendemo
95 Once logged in, it should look just like any regular linux box. All
96 the usual tools and commands should work as per usual. However,
97 because of the poor random access performance of CD drives, the
98 machine will feel very slugish, and you may run out of memory if you
99 make significant modifications to the ramfs filesystem -- for the full
100 experience, install a Xen and Xenlinux image on you hard drive :-)
102 You can configure networking, either with 'dhclient' or manually via
103 'ifconfig' and 'route', remembering to edit /etc/resolv.conf if you
104 want DNS to work.
106 You can start an X server with 'startx'. It defaults to a conservative
107 1024x768, but you can edit the script for higher resoloutions. The CD
108 contains a load of standard software. You should be able to start
109 Apache, PostgreSQL, Mozilla etc in the normal way, but because
110 everything is running off CD the performance will be very sluggish and
111 you may run out of memory for the 'tmpfs' file system. You may wish
112 to go ahead and install Xen/Xenlinux on your hard drive, either
113 dropping Xen and the Xenlinux kernel down onto a pre-existing Linux
114 distribution, or using the file systems from the CD (which are based
115 on RH9). See the installation instructions later in this document.
117 If your video card requires 'agpgart' then it unfortunately won't yet
118 work with Xen, and you'll only be able to configure a VGA X
119 server. We're working on a fix for this for the next release.
121 If you want to browse the Xen / Xenlinux source, it's all located
122 under /usr/local/src/xeno-1.3, complete with BitKeeper
123 repository. We've also included source code and configuration
124 information for the various benchmarks we used in the SOSP paper.
127 Starting other domains
128 ======================
130 The first thing you need to do is to start the "xend" control daemon
131 with "xend start". You may wish to add an appropriate link to xend in
132 you /etc/rcX.d directory e.g. "ln -sf ../init.d/xend S97xend"
134 If you're not intending to configure the new domain with an IP address
135 on your LAN, then you'll probably want to use NAT. The
136 'xen_nat_enable' installs a few useful iptables rules into domain0 to
137 enable NAT. [NB: We plan to support RSIP in future]
139 Xen has a management interface that can be manipulated from domain0 to
140 create new domains, control their CPU, network and memory resource
141 allocations, allocate IP addresses, grant access to disk partitions,
142 and suspend/resume domains to files, etc. The management interface is
143 implemented as a set of library functions (implemented in C) for which
144 there are Python language bindings.
146 We have developed a simple set of example python tools for
147 manipulating the interface, with the intention that more sophisticated
148 high-level management tools will be developed in due course. Within
149 the source repository the tools live in tools/examples/ but are
150 installed in /usr/local/bin/ on the CD.
152 Starting a new domain is achieved using xc_dom_create.py which
153 allocates resources to a new domain, populates it with a kernel image
154 (and optionally a ramdisk) and then starts it.
156 It parses a configuration file written in the Python language, the
157 default location of which is "/etc/xc/defaults", but this may be
158 overridden with the "-f" option. For the Demo CD, the defaults file
159 will cause domains to be created with ram-based root file systems, and
160 mount their /usr partition from the CD, just like domain0. (If you are
161 writing your own config file, the "example" script may be a better
162 starting point)
164 Variables can be initialised and passed into configuration files. Some
165 of these may be compulsory, others optional.
167 The 'defaults' file on the CD requires the 'ip' variable to be set to
168 tell Xen what IP address(es) should be routed to this domain. Xen
169 will route packets to the domain if they bear one of these addresses
170 as a destination address, and will also ensure that packets sent from
171 the domain contain one of the addresses as a source address (to
172 prevent spoofing). If multiple IP addresses are to be assigned to a
173 domain they can be listed in a comma separated list (with no
174 whitespace).
176 The 'mem' variable can be used to change the default memory allocation
177 of 64MB. For example to start a domain with two IP addresses and
178 72MB:
180 xc_dom_create.py -Dip=128.23.45.34,169.254.1.1 -Dmem=72
182 [multiple variables may also be set with a single '-D' flag by
183 separating them with ':'. Also, it's possible to use DNS hostnames
184 rather than IP addresses.]
186 When invoked with the '-n' option xc_dom_create.py will do a dry run
187 and just print out what resources and configuration the domain will
188 have e.g.:
190 [root@xendemo]# xc_dom_create.py -D ip=commando-1.xeno,169.254.2.3 -Dmem=100
191 Parsing config file 'defaults'
193 VM image : "/boot/xenlinux.gz"
194 VM ramdisk : "/boot/initrd.gz"
195 VM memory (MB) : "100"
196 VM IP address(es) : "128.232.38.51:169.254.2.3"
197 VM block device(s) : "phy:cdrom,hdd,r"
198 VM cmdline : "ip=128.232.38.51:169.254.1.0:128.232.32.1:255.255.240.0::eth0:off root=/dev/ram0 rw init=/linuxrc 4 LOCALIP=169.254.2.3"
200 xc_dom_create.py will print the local TCP port to which you should
201 connect to perform console I/O. A suitable console client is provided
202 by the Python module xenctl.console_client: running this module from
203 the command line with <host> and <port> parameters will start a
204 terminal session. This module is also installed as /usr/bin/xencons,
205 from a copy in tools/misc/xencons. An alternative to manually running
206 a terminal client is to specify '-c' to xc_dom_create.py, or add
207 'auto_console=True' to the defaults file. This will cause
208 xc_dom_create.py to automatically become the console terminal after
209 starting the domain.
211 The 169.254.x.x network is special in that it is the 'link local'
212 subnet, and is isolated from the external network and hence can only
213 be used for communication between virtual machines. By convention, we
214 usually give each domain a link local address. The startup scripts on
215 the CD have been modified to accept a LINKLOCAL= parameter on the
216 kernel command line and initialise an IP alias accordingly (see
217 /etc/sysinit/network-scripts/ifcfg-eth0).
219 Linux only allows one IP address to be specified on the kernel command
220 line, so if you specify multiple IP addresses you'll need to configure
221 the new Linux VM with the other addresses manually (using ifconfig)
222 having logged in.
224 If you inspect the 'defaults' config script you'll see that the new
225 domain was started with a '4' on the kernel command line to tell
226 'init' to go to runlevel 4 rather than the default of 3 used by
227 domain0. This is done simply to suppress a bunch of harmless error
228 messages that would otherwise occur when the new (unprivileged) domain
229 tried to access physical hardware resources to try setting the
230 hwclock, system font, run gpm etc.
232 After it's booted, you should be able to ssh into your new domain from
233 domain0 using the link local 19.254.x.x address you assigned. If you
234 assigned a further IP address you should be able to ssh in using that
235 address too. If you ran the xen_enable_nat script, a bunch of port
236 redirects have been installed to enable you to ssh in to other domains
237 remotely even if you didn't assign an externally routeable address.
238 To access the new virtual machine remotely, use:
240 ssh -p2201 root@IP.address.Of.Domain0 # use 2202 for domain 2 etc.
242 You can manipulate running domains using the xc_dom_control.py tool.
243 Invoking it without arguments prints some usage information.
245 To see what domains are running, run 'xc_dom_control.py list'. Using the
246 tool you can change scheduling parameters, pause a domain, send it a
247 shutdown request, or blow it away with the 'destroy' command. You can
248 even suspend it to disk (but you probably won't have enough memory to
249 do the latter if you're running off the demo CD).
251 To find usage information for xc_dom_control.py, run the script with
252 no arguments.
255 Troubleshooting Problems
256 ========================
258 If you have problems booting Xen, there are a number of boot parameters
259 that may be able to help diagnose problems:
261 ignorebiostables Disable parsing of BIOS-supplied tables. This may
262 help with some chipsets that aren't fully supported
263 by Xen. If you specify this option then ACPI tables are
264 also ignored, and SMP support is disabled.
266 noreboot Don't reboot the machine automatically on errors.
267 This is useful to catch debug output if you aren't
268 catching console messages via the serial line.
270 nosmp Disable SMP support.
271 This option is implied by 'ignorebiostables'.
273 noacpi Disable ACPI tables, which confuse Xen on some chipsets.
274 This option is implied by 'ignorebiostables'.
276 watchdog Enable NMI watchdog which can report certain failures.
278 noht Disable Hyperthreading.
280 badpage=<page number>[,<page number>]*
281 Specify a list of pages not to be allocated for use
282 because they contain bad bytes. For example, if your
283 memory tester says that byte 0x12345678 is bad, you would
284 place 'badpage=0x12345' on Xen's command line (i.e., the
285 last three digits of the byte address are not included!).
287 com1=<baud>,DPS[,<io_base>,<irq>]
288 com2=<baud>,DPS[,<io_base>,<irq>]
289 Xen supports up to two 16550-compatible serial ports.
290 For example: 'com1=9600,8n1,0x408,5' maps COM1 to a
291 9600-baud port, 8 data bits, no parity, 1 stop bit,
292 I/O port base 0x408, IRQ 5.
293 If the I/O base and IRQ are standard (com1:0x3f8,4;
294 com2:0x2f8,3) then they need not be specified.
296 console=<specifier list>
297 Specify the destination for Xen console I/O.
298 This is a comma-separated list of, for example:
299 vga: use VGA console and allow keyboard input
300 com1: use serial port com1
301 com2H: use serial port com2. Transmitted chars will
302 have the MSB set. Received chars must have
303 MSB set.
304 com2L: use serial port com2. Transmitted chars will
305 have the MSB cleared. Received chars must
306 have MSB cleared.
307 The latter two examples allow a single port to be
308 shared by two subsystems (eg. console and
309 debugger). Sharing is controlled by MSB of each
310 transmitted/received character.
311 [NB. Default for this option is 'com1,vga']
313 conswitch=<switch-char><auto-switch-char>
314 Specify how to switch serial-console input between
315 Xen and DOM0. The required sequence is CTRL-<switch_char>
316 pressed three times. Specifying '`' disables switching.
317 The <auto-switch-char> specifies whether Xen should
318 auto-switch input to DOM0 when it boots -- if it is 'x'
319 then auto-switching is disabled. Any other value, or
320 omitting the character, enables auto-switching.
321 [NB. Default for this option is 'a']
323 nmi=<nmi-error-behaviour>
324 Specify what to do with an NMI parity or I/O error.
325 'nmi=fatal': Xen prints a diagnostic and then hangs.
326 'nmi=dom0': Inform DOM0 of the NMI.
327 'nmi=ignore': Ignore the NMI.
328 [NB. Default is 'dom0' ('fatal' for debug builds).]
330 dom0_mem=xxx Set the initial amount of memory for domain0.
332 pdb=xxx Enable the pervasive debugger. See docs/pdb.txt
333 xxx defines how the gdb stub will communicate:
334 com1 use com1
335 com1H use com1 (with high bit set)
336 com2 use on com2
337 com2H use com2 (with high bit set)
339 It's probably a good idea to join the Xen developer's mailing list on
340 Sourceforge: http://lists.sourceforge.net/lists/listinfo/xen-devel
343 About The Xen Demo CD
344 =====================
346 The purpose of the Demo CD is to distribute a snapshot of Xen's
347 source, and simultaneously provide a convenient means for enabling
348 people to get experience playing with Xen without needing to install
349 it on their hard drive. If you decide to install Xen/Xenlinux you can
350 do so simply by following the installation instructions below -- which
351 essentially involves copying the contents of the CD on to a suitably
352 formated disk partition, and then installing or updating the Grub
353 bootloader.
355 This is a bootable CD that loads Xen, and then a Linux 2.4.22 OS image
356 ported to run on Xen. The CD contains a copy of a file system based on
357 the RedHat 9 distribution that is able to run directly off the CD
358 ("live ISO"), using a "tmpfs" RAM-based file system for root (/etc
359 /var etc). Changes you make to the tmpfs will obviously not be
360 persistent across reboots!
362 Because of the use of a RAM-based file system for root, you'll need
363 plenty of memory to run this CD -- something like 96MB per VM. This is
364 not a restriction of Xen : once you've installed Xen, Xenlinux and
365 the file system images on your hard drive you'll find you can boot VMs
366 in just a few MBs.
368 The CD contains a snapshot of the Xen and Xenlinux code base that we
369 believe to be pretty stable, but lacks some of the features that are
370 currently still work in progress e.g. OS suspend/resume to disk, and
371 various memory management enhancements to provide fast inter-OS
372 communication and sharing of memory pages between OSs. We'll release
373 newer snapshots as required, making use of a BitKeeper repository
374 hosted on http://xen.bkbits.net (follow instructions from the project
375 home page). We're obviously grateful to receive any bug fixes or
376 other code you can contribute. We suggest you join the
377 xen-devel@lists.sourceforge.net mailing list.
380 Installing from the CD
381 ======================
383 If you're installing Xen/Xenlinux onto an existing linux file system
384 distribution, just copy the Xen VMM (/boot/image.gz) and Xenlinux
385 kernels (/boot/xenlinux.gz), then modify the Grub config
386 (/boot/grub/menu.lst or /boot/grub/grub.conf) on the target system.
387 It should work on pretty much any distribution.
389 Xen is a "multiboot" standard boot image. Despite being a 'standard',
390 few boot loaders actually support it. The only two we know of are
391 Grub, and our modified version of linux kexec (for booting off a
392 XenoBoot CD -- PlanetLab have adopted the same boot CD approach).
394 If you need to install grub on your system, you can do so either by
395 building the Grub source tree
396 /usr/local/src/grub-0.93-iso9660-splashimage or by copying over all
397 the files in /boot/grub and then running /sbin/grub and following the
398 usual grub documentation. You'll then need to edit the Grub
399 config file.
401 A typical Grub menu option might look like:
403 title Xen / Xenlinux 2.4.26
404 kernel /boot/xen.gz dom0_mem=131072 com1=115200 noht watchdog
405 module /boot/vmlinuz-2.4.26-xen0 root=/dev/sda4 ro
407 The first line specifies which Xen image to use, and what command line
408 arguments to pass to Xen. In this case we set the maximum amount of
409 memory to allocate to domain0, and enable serial I/O on COM1 at 115200
410 baud. We could also disable smp support (nosmp) or disable
411 hyper-threading support (noht).
413 The second line specifies which xenlinux image to use, and the
414 standard linux command line arguments to pass to the kernel. In this
415 case, we're configuring the root partition and stating that it should
416 initially be mounted read-only (normal practice).
418 If we were booting with an initial ram disk (initrd), then this would
419 require a second "module" line.
421 Installing the Xen tools and source
422 ===================================
424 The tools and source live in the /usr/local/src/xen-2.0 directory on
425 the CD (and may also be downloaded from the project downloads
426 page). You'll need to copy them to some mutable storage before using
427 them.
429 If you have the BitKeeper BK tools installed you can check the
430 repository is up to date by cd'ing into the xeno-2.0.bk directory and
431 typing 'bk pull' (assuming you have an Internet connection).
433 You can rebuild Xen, the tools and XenLinux by typing 'make
434 world'. You can install them to the standard directories with 'make
435 install', or into the ./install subtree with 'make dist'.
438 Modifying xc_mycreatelinuxdom1.py
439 =================================
441 xc_mycreatelinuxdom1.py.py can be used to set the new kernel's command line,
442 and hence determine what it uses as a root file system, etc. Although
443 the default is to boot in the same manner that domain0 did (using the
444 RAM-based file system for root and the CD for /usr) it's possible to
445 configure any of the following possibilities, for example:
447 * initrd=/boot/initrd init=/linuxrc
448 boot using an initial ram disk, executing /linuxrc (as per this CD)
450 * root=/dev/hda3 ro
451 boot using a standard hard disk partition as root
452 !!! remember to grant access in createlinuxdom.py.
454 * root=/dev/xvda1 ro
455 boot using a pre-configured 'virtual block device' that will be
456 attached to a virtual disk that previously has had a file system
457 installed on it.
459 * root=/dev/nfs nfsroot=/path/on/server ip=<blah_including server_IP>
460 Boot using an NFS mounted root file system. This could be from a
461 remote NFS server, or from an NFS server running in another
462 domain. The latter is rather a useful option.
464 A typical setup might be to allocate a standard disk partition for
465 each domain and populate it with files. To save space, having a shared
466 read-only usr partition might make sense.
468 Block devices should only be shared between domains in a read-only
469 fashion otherwise the linux kernels will obviously get very confused
470 as the file system structure may change underneath them (having the
471 same partition mounted rw twice is a sure fire way to cause
472 irreparable damage)! If you want read-write sharing, export the
473 directory to other domains via NFS from domain0.
478 Installing the file systems from the CD
479 =======================================
481 If you haven't got an existing Linux installation onto which you can
482 just drop down the Xen and Xenlinux images, then the file systems on
483 the CD provide a quick way of doing an install. However, you're
484 probably better off in the long run doing a proper Redhat, Fedora,
485 Debian etc install rather than just doing the hack described below:
487 Choose one or two partitions, depending on whether you want a separate
488 /usr or not. Make file systems on it/them e.g.:
489 mkfs -t ext3 /dev/hda3
490 [or mkfs -t ext2 /dev/hda3 && tune2fs -j /dev/hda3 if using an old
491 version of mkfs]
493 Next, mount the file system(s) e.g.:
494 mkdir /mnt/root && mount /dev/hda3 /mnt/root
495 [mkdir /mnt/usr && mount /dev/hda4 /mnt/usr]
497 To install the root file system, simply untar /usr/XenDemoCD/root.tar.gz:
498 cd /mnt/root && tar -zxpf /usr/XenDemoCD/root.tar.gz
500 You'll need to edit /mnt/root/etc/fstab to reflect your file system
501 configuration. Changing the password file (etc/shadow) is probably a
502 good idea too.
504 To install the usr file system, copy the file system from CD on /usr,
505 though leaving out the "XenDemoCD" and "boot" directories:
506 cd /usr && cp -a X11R6 etc java libexec root src bin dict kerberos local sbin tmp doc include lib man share /mnt/usr
508 If you intend to boot off these file systems (i.e. use them for
509 domain 0), then you probably want to copy the /usr/boot directory on
510 the cd over the top of the current symlink to /boot on your root
511 filesystem (after deleting the current symlink) i.e.:
512 cd /mnt/root ; rm boot ; cp -a /usr/boot .
514 The XenDemoCD directory is only useful if you want to build your own
515 version of the XenDemoCD (see below).
518 Debugging
519 =========
521 Xen has a set of debugging features that can be useful to try and
522 figure out what's going on. Hit 'h' on the serial line (if you
523 specified a baud rate on the Xen command line) or ScrollLock-h on the
524 keyboard to get a list of supported commands.
526 If you have a crash you'll likely get a crash dump containing an EIP
527 (PC) which, along with an 'objdump -d image', can be useful in
528 figuring out what's happened. Debug a Xenlinux image just as you
529 would any other Linux kernel.
531 We supply a handy debug terminal program which you can find in
532 /usr/local/src/xen-1.0/xeno-1.0.bk/tools/misc/miniterm/
533 This should be built and executed on another machine that is connected
534 via a null modem cable. Documentation is included.
535 Alternatively, if the Xen machine is connected to a serial-port server
536 then we supply a dumb TCP terminal client:
537 'tools/xenctl/lib/console_client.py <server host> <server port>'
540 Installing Xen / Xenlinux on a RedHat distribution
541 ===================================================
543 When using Xen / Xenlinux on a standard Linux distribution there are
544 a couple of things to watch out for:
546 The first Linux VM that is started when Xen boots start (Domain 0) is
547 given direct access to the graphics card, so it may use it as a
548 console. Other domains don't have ttyN consoles, so attempts to run a
549 'mingetty' against them will fail, generating periodic warning
550 messages from 'init' about services respawning too fast. They should
551 work for domain0 just fine.
552 IMPORTANT: To prevent warning messages when running RH9 you'll need to
553 remove ttyN from /etc/inittab for domains>0. Due to a bug in the RH9
554 /etc/rc.sysinit script #'ing the lines out of /etc/inittab won't work
555 as it ignores the '#' and tries to access them anyway.
557 Every Xenlinux instance owns a bidirectional 'virtual console'.
558 The device node to which this console is attached can be configured
559 by specifying 'xencons=' on the OS command line:
560 'xencons=off' --> disable virtual console
561 'xencons=tty' --> attach console to /dev/tty1 (tty0 at boot-time)
562 'xencons=ttyS' --> attach console to /dev/ttyS0
563 The default is to attach to /dev/tty1, and also to create dummy
564 devices for /dev/tty2-63 to avoid warnings from many standard distro
565 startup scripts. The exception is domain 0, which by default attaches
566 to /dev/ttyS0.
568 Note that, because domains>0 don't have any privileged access at all,
569 certain commands in the default boot sequence will fail e.g. attempts
570 to update the hwclock, change the console font, update the keytable
571 map, start apmd (power management), or gpm (mouse cursor). Either
572 ignore the errors, or remove them from the startup scripts. Deleting
573 the following links are a good start: S24pcmcia S09isdn S17keytable
574 S26apmd S85gpm
576 If you want to use a single root file system that works cleanly for
577 domain0 and domains>0, one trick is to use different 'init' run
578 levels. For example, on the Xen Demo CD we use run level 3 for domain
579 0, and run level 4 for domains>0. This enables different startup
580 scripts to be run in depending on the run level number passed on the
581 kernel command line.
583 Xenlinux kernels can be built to use runtime loadable modules just
584 like normal linux kernels. Modules should be installed under
585 /lib/modules in the normal way.
587 If there's some kernel feature that hasn't been built into our default
588 kernel, there's a pretty good change that if its a non-hardware
589 related option you'll just be able to enable it and rebuild. If its
590 not on the xconfig menu, hack the arch/xen/config.in to put the menu
591 back in.
593 If you're going to use the link local 169.254.1.x addresses to
594 communicate between VMs, there are a couple of other issues to watch
595 out for. RH9 appears to have a bug where by default it configures the
596 loopback interface with a 169.254 address, which stops it working
597 properly on eth0 for communicating with other domains.
599 This utterly daft RH9 behaviour can be stopped by appending
600 "NOZEROCONF=yes" to /etc/sysconfig/networking-scripts/ifcfg-lo
602 If you're going to use NFS root files systems mounted either from an
603 external server or from domain0 there are a couple of other gotchas.
604 The default /etc/sysconfig/iptables rules block NFS, so part way
605 through the boot sequence things will suddenly go dead.
607 If you're planning on having a separate NFS /usr partition, the RH9
608 boot scripts don't make life easy, as they attempt to mount NFS file
609 systems way to late in the boot process. The easiest way I found to do
610 this was to have a '/linuxrc' script run ahead of /sbin/init that
611 mounts /usr:
612 #!/bin/bash
613 /sbin/ipconfig lo 127.0.0.1
614 /sbin/portmap
615 /bin/mount /usr
616 exec /sbin/init "$@" <>/dev/console 2>&1
618 The one slight complication with the above is that /sbib/portmap is
619 dynamically linked against /usr/lib/libwrap.so.0 Since this is in
620 /usr, it won't work. I solved this by copying the file (and link)
621 below the /usr mount point, and just let the file be 'covered' when
622 the mount happens.
624 In some installations, where a shared read-only /usr is being used, it
625 may be desirable to move other large directories over into the
626 read-only /usr. For example, on the XenDemoCD we replace /bin /lib and
627 /sbin with links into /usr/root/bin /usr/root/lib and /usr/root/sbin
628 respectively. This creates other problems for running the /linuxrc
629 script, requiring bash, portmap, mount, ifconfig, and a handful of
630 other shared libraries to be copied below the mount point. I guess I
631 should have written a little statically linked C program...
635 Description of how the XenDemoCD boots
636 ======================================
638 1. Grub is used to load Xen, a Xenlinux kernel, and an initrd (initial
639 ram disk). [The source of the version of Grub used is in /usr/local/src]
641 2. the init=/linuxrc command line causes linux to execute /linuxrc in
642 the initrd.
644 3. the /linuxrc file attempts to mount the CD by trying the likely
645 locations : /dev/hd[abcd].
647 4. it then creates a 'tmpfs' file system and untars the
648 'XenDemoCD/root.tar.gz' file into the tmpfs. This contains hopefully
649 all the files that need to be mutable (this would be so much easier
650 if Linux supported 'stacked' or union file systems...)
652 5. Next, /linuxrc uses the pivot_root call to change the root file
653 system to the tmpfs, with the CD mounted as /usr.
655 6. It then invokes /sbin/init in the tmpfs and the boot proceeds
656 normally.
659 Building your own version of the XenDemoCD
660 ==========================================
662 The 'live ISO' version of RedHat is based heavily on Peter Anvin's
663 SuperRescue CD version 2.1.2 and J. McDaniel's Plan-B:
665 http://www.kernel.org/pub/dist/superrescue/v2/
666 http://projectplanb.org/
668 Since Xen uses a "multiboot" image format, it was necessary to change
669 the bootloader from isolinux to Grub0.93 with Leonid Lisovskiy's
670 <lly@pisem.net> grub.0.93-iso9660.patch
672 The Xen Demo CD contains all of the build scripts that were used to
673 create it, so it is possible to 'unpack' the current iso, modifiy it,
674 then build a new iso. The procedure for doing so is as follows:
676 First, mount either the CD, or the iso image of the CD:
678 mount /dev/cdrom /mnt/cdrom
679 or:
680 mount -o loop xendemo-1.0.iso /mnt/cdrom
682 cd to the directory you want to 'unpack' the iso into then run the
683 unpack script:
685 cd /local/xendemocd
686 /mnt/cdrom/XenDemoCD/unpack-iso.sh
688 The result is a 'build' directory containing the file system tree
689 under the 'root' directory. e.g. /local/xendemocd/build/root
691 To add or remove rpms, its possible to use 'rpm' with the --root
692 option to set the path. For more complex changes, it easiest to boot a
693 machine using using the tree via NFS root. Before doing this, you'll
694 need to edit fstab to comment out the seperate mount of /usr.
696 One thing to watch out for: as part of the CD build process, the
697 contents of the 'rootpatch' tree gets copied over the existing 'root'
698 tree replacing various files. The intention of the rootpatch tree is
699 to contain the files that have been modified from the original RH
700 distribution (e.g. various /etc files). This was done to make it
701 easier to upgrade to newer RH versions in the future. The downside of
702 this is that if you edit an existing file in the root tree you should
703 check that you don't also need to propagate the change to the
704 rootpatch tree to avoid it being overwritten.
706 Once you've made the changes and want to build a new iso, here's the
707 procedure:
709 cd /local/xendemocd/build
710 echo '<put_your_name_here>' > Builder
711 ./make.sh put_your_version_id_here >../buildlog 2>&1
713 This process can take 30 mins even on a fast machine, but you should
714 eventually end up with an iso image in the build directory.
716 Notes:
718 root - the root of the file system heirarchy as presented to the
719 running system
721 rootpatch - contains files that have been modified from the standard
722 RH, and copied over the root tree as part of the build
723 procedure.
725 irtree - the file system tree that will go into the initrd (initial
726 ram disk)
728 work - a working directory used in the build process
730 usr - this should really be in 'work' as its created as part of the
731 build process. It contains the 'immutable' files that will
732 be served from the CD rather than the tmpfs containing the
733 contents of root.tar.gz. Some files that are normally in /etc
734 or /var that are large and actually unlikely to need changing
735 have been moved into /usr/root and replaced with links.
738 Ian Pratt
739 9 Sep 2003