1.1 --- a/.rootkeys	Tue Sep 09 21:51:39 2003 +0000
     1.2 +++ b/.rootkeys	Wed Sep 10 09:57:56 2003 +0000
     1.3 @@ -3,6 +3,8 @@ 3eba336c4BUxP3H1DjQl8sInORf1Yw BitKeeper
     1.4  3ddb6b0buTaC5zg1_a8FoAR9FWi_mw BitKeeper/etc/ignore
     1.5  3ddb79c9_hgSp-gsQm8HqWM_9W3B_A BitKeeper/etc/logging_ok
     1.6  3eb788d6Kleck_Cut0ouGneviGzliQ Makefile
     1.7 +3f5ef5a24IaQasQE2tyMxrfxskMmvw README
     1.8 +3f5ef5a2l4kfBYSQTUaOyyD76WROZQ README.CD
     1.9  3e6377b24eQqYMsDi9XrFkIgTzZ47A tools/balloon/Makefile
    1.10  3e6377d6eiFjF1hHIS6JEIOFk62xSA tools/balloon/README
    1.11  3e6377dbGcgnisKw16DPCaND7oGO3Q tools/balloon/balloon.c
    1.12 @@ -142,7 +144,10 @@ 3f13d81eQ9Vz-h-6RDGFkNR9CRP95g tools/mis
    1.13  3f13d81e6Z6806ihYYUw8GVKNkYnuw tools/misc/enable_nat.README
    1.14  3f1668d4-FUY6Enc7MB3GcwUtfJ5HA tools/misc/mkdevnodes
    1.15  3f1668d4F29Jsw0aC0bJEIkOBiagiQ tools/misc/read_console_udp.c
    1.16 +3f5ef5a2ir1kVAthS14Dc5QIRCEFWg tools/misc/xen-clone
    1.17 +3f5ef5a2dTZP0nnsFoeq2jRf3mWDDg tools/misc/xen-clone.README
    1.18  3ddb79bcbOVHh38VJzc97-JEGD4dJQ xen/Makefile
    1.19 +3f5ef5a2Qtt8AshYs-KXFFNhKALeIg xen/README
    1.20  3ddb79bcWnTwYsQRWl_PaneJfa6p0w xen/Rules.mk
    1.21  3e74d2be6ELqhaY1sW0yyHRKhpOvDQ xen/TODO
    1.22  3ddb79bcZbRBzT3elFWSX7u6NtMagQ xen/arch/i386/Makefile

     2.1 --- /dev/null	Thu Jan 01 00:00:00 1970 +0000
     2.2 +++ b/README	Wed Sep 10 09:57:56 2003 +0000
     2.3 @@ -0,0 +1,211 @@
     2.4 +#############################
     2.5 + __  __            _   ___  
     2.6 + \ \/ /___ _ __   / | / _ \ 
     2.7 +  \  // _ \ '_ \  | || | | |
     2.8 +  /  \  __/ | | | | || |_| |
     2.9 + /_/\_\___|_| |_| |_(_)___/ 
    2.10 +                                                   
    2.11 +#############################
    2.12 +
    2.13 +University of Cambridge Computer Laboratory
    2.14 +31 Aug 2003
    2.15 +
    2.16 +http://www.cl.cam.ac.uk/netos/xen
    2.17 +
    2.18 +About the Xen Virtual Machine Monitor
    2.19 +=====================================
    2.20 +
    2.21 +"Xen" is a Virtual Machine Monitor (VMM) developed by the Systems
    2.22 +Research Group of the University of Cambridge Computer Laboratory, as
    2.23 +part of the UK-EPSRC funded XenoServers project.
    2.24 +
    2.25 +The XenoServers project aims to provide a "public infrastructure for
    2.26 +global distributed computing", and Xen plays a key part in that,
    2.27 +allowing us to efficiently partition a single machine to enable
    2.28 +multiple independent clients to run their operating systems and
    2.29 +applications in an environment providing protection, resource
    2.30 +isolation and accounting.  The project web page contains further
    2.31 +information along with pointers to papers and technical reports:
    2.32 +http://www.cl.cam.ac.uk/xeno
    2.33 +
    2.34 +Xen has since grown into a project in its own right, enabling us to
    2.35 +investigate interesting research issues regarding the best techniques
    2.36 +for virtualizing resources such as the CPU, memory, disk and network.
    2.37 +The project has been bolstered by support from Intel Research
    2.38 +Cambridge, who are now working closely with us. We've now also
    2.39 +received support from Microsoft Research Cambridge to port Windows XP
    2.40 +to run on Xen.
    2.41 +
    2.42 +Xen enables multiple operating system images to be run simultaneously
    2.43 +on the same hardware with very low performance overhead --- much lower
    2.44 +than commercial offerings on the same x86 platform.
    2.45 +
    2.46 +This is achieved by requiring OSs to be specifically ported to run on
    2.47 +Xen, rather than allowing unmodified OS images to be used. Crucially,
    2.48 +only the OS needs to be changed -- all of the user-level application
    2.49 +binaries, libraries etc can run unmodified. Hence, the modified OS
    2.50 +kernel can typically just be dropped into any existing OS distribution
    2.51 +or installation.
    2.52 +
    2.53 +Xen currently runs on the x86 architecture, but could in principle be
    2.54 +ported to other CPUs. In fact, it would have been rather easier to
    2.55 +write Xen for pretty much any other architecture as x86 doesn't do us
    2.56 +any favours at all. The best description of Xen's deign,
    2.57 +implementation and performance is contained in our October 2003 SOSP
    2.58 +paper: http://www.cl.cam.ac.uk/netos/papers/2003-xensosp.pdf
    2.59 +
    2.60 +We have been working on porting 3 different operating systems to run
    2.61 +on Xen: Linux 2.4, Windows XP, and NetBSD.
    2.62 +
    2.63 +The Linux 2.4 port (currently Linux 2.4.22) works very well -- we
    2.64 +regularly use it to host complex applications such as PostgreSQL,
    2.65 +Apache, BK servers etc. It runs all applications we've tried.  We
    2.66 +refer to our version of Linux ported to run on Xen as "XenoLinux",
    2.67 +through really it's just standard Linux ported to a new virtual CPU
    2.68 +architecture that we call xeno-x86 (abbreviated to just "xeno").
    2.69 +
    2.70 +Unfortunately, the NetBSD port has stalled due to lack of man
    2.71 +power. We believe most of the hard stuff has already been done, and
    2.72 +are hoping to get the ball rolling again soon. In hindsight, a FreeBSD
    2.73 +4 port might have been more useful to the community. 
    2.74 +
    2.75 +The Windows XP port is nearly finished. It's running user space
    2.76 +applications and is generally in pretty good shape thanks to some hard
    2.77 +work by the team over the summer.  Of course, there are issues with
    2.78 +releasing this code to others.  We should be able to release the
    2.79 +source and binaries to anyone else that's signed the Microsoft
    2.80 +academic source license, which these days has very reasonable
    2.81 +terms. We are in discussions with Microsoft about the possibility of
    2.82 +being able to make binary releases to a larger user
    2.83 +community. Obviously, there are issues with product activation in this
    2.84 +environment and such like, which need to be thought through.
    2.85 +
    2.86 +So, for the moment, you only get to run multiple copies of Linux on
    2.87 +Xen, but we hope this will change before too long.  Even running
    2.88 +multiple copies of the same OS can be very useful, as it provides a
    2.89 +means of containing faults to one OS image, and also for providing
    2.90 +performance isolation between the various OS, enabling you to either
    2.91 +restrict, or reserve resources for, particular VM instances.
    2.92 +
    2.93 +Its also useful for development -- each version of Linux can have
    2.94 +different patches applied, enabling different kernels to be tried
    2.95 +out. For example, the "vservers" patch used by PlanetLab applies
    2.96 +cleanly to our ported version of Linux.
    2.97 +
    2.98 +We've successfully booted over 128 copies of Linux on the same machine
    2.99 +(a dual CPU hyperthreaded Xeon box) but we imagine that it would be
   2.100 +more normal to use some smaller number, perhaps 10-20.
   2.101 +
   2.102 +Known limitations and work in progress
   2.103 +======================================
   2.104 +
   2.105 +The "xenctl" tool is still rather clunky and not very user
   2.106 +friendly. In particular, it should have an option to create and start
   2.107 +a domain with all the necessary parameters set from a named xml file.
   2.108 +
   2.109 +The java xenctl tool is really just a frontend for a bunch of C tools
   2.110 +named xi_* that do the actual work of talking to Xen and setting stuff
   2.111 +up. Some local users prefer to drive the xi_ tools directly, typically
   2.112 +from simple shell scripts. These tools are even less user friendly
   2.113 +than xenctl but its arguably clearer what's going on.
   2.114 +
   2.115 +There's also a web based interface for controlling domains that uses
   2.116 +apache/tomcat, but it has fallen out of sync with respect to the
   2.117 +underlying tools, so doesn't always work as expected and needs to be
   2.118 +fixed.
   2.119 +
   2.120 +The current Virtual Firewall Router (VFR) implementation in the
   2.121 +snapshot tree is very rudimentary, and in particular, lacks the IP
   2.122 +port-space sharing across domains that we've proposed that promises to
   2.123 +provide a better alternative to NAT.  There's a complete new
   2.124 +implementation under development which also supports much better
   2.125 +logging and auditing support.  The current network scheduler is just
   2.126 +simple round-robin between domains, without any rate limiting or rate
   2.127 +guarantees. Dropping in a new scheduler should be straightforward, and
   2.128 +is planned as part of the VFRv2 work package.
   2.129 +
   2.130 +Another area that needs further work is the interface between Xen and
   2.131 +domain0 user space where the various XenoServer control daemons run.
   2.132 +The current interface is somewhat ad-hoc, making use of various
   2.133 +/proc/xeno entries that take a random assortment of arguments. We
   2.134 +intend to reimplement this to provide a consistent means of feeding
   2.135 +back accounting and logging information to the control daemon.
   2.136 +
   2.137 +There's also a number of memory management hacks that didn't make this
   2.138 +release: We have plans for a "universal buffer cache" that enables
   2.139 +otherwise unused system memory to be used by domains in a read-only
   2.140 +fashion. We also have plans for inter-domain shared-memory to enable
   2.141 +high-performance bulk transport for cases where the usual internal
   2.142 +networking performance isn't good enough (e.g. communication with a
   2.143 +internal file server on another domain).
   2.144 +
   2.145 +We also have plans to implement domain suspend/resume-to-file. This is
   2.146 +basically an extension to the current domain building process to
   2.147 +enable domain0 to read out all of the domain's state and store it in a
   2.148 +file. There are complications here due to Xen's para-virtualised
   2.149 +design, whereby since the physical machine memory pages available to
   2.150 +the guest OS are likely to be different when the OS is resumed, we
   2.151 +need to re-write the page tables appropriately. 
   2.152 +
   2.153 +We have the equivalent of balloon driver functionality to control
   2.154 +domain's memory usage, enabling a domain to give back unused pages to
   2.155 +Xen. This needs properly documenting, and perhaps a way of domain0
   2.156 +signalling to a domain that it requires it to reduce its memory
   2.157 +footprint, rather than just the domain volunteering.
   2.158 +
   2.159 +The current disk scheduler is rather simplistic (batch round robin),
   2.160 +and could be replaced by e.g. Cello if we have QoS isolation
   2.161 +problems. For most things it seems to work OK, but there's currently
   2.162 +no service differentiation or weighting.
   2.163 +
   2.164 +Currently, although Xen runs on SMP and SMT (hyperthreaded) machines,
   2.165 +the scheduling is far from smart -- domains are currently statically
   2.166 +assigned to a CPU when they are created (in a round robin fashion).
   2.167 +The scheduler needs to be modified such that before going idle a
   2.168 +logical CPU looks for work on other run queues (particularly on the
   2.169 +same physical CPU). 
   2.170 +
   2.171 +Xen currently only supports uniprocessor guest OSes. We have designed
   2.172 +the Xen interface with MP guests in mind, and plan to build an MP
   2.173 +Linux guest in due course. Basically, an MP guest would consist of
   2.174 +multiple scheduling domains (one per CPU) sharing a single memory
   2.175 +protection domain. The only extra complexity for the Xen VM system is
   2.176 +ensuring that when a page transitions from holding a page table or
   2.177 +page directory to a write-able page, we must ensure that no other CPU
   2.178 +still has the page in its TLB to ensure memory system integrity.  One
   2.179 +other issue for supporting MP guests is that we'll need some sort of
   2.180 +CPU gang scheduler, which will require some research.
   2.181 +
   2.182 +
   2.183 +Hardware support
   2.184 +================
   2.185 +
   2.186 +Xen is intended to be run on server-class machines, and the current
   2.187 +list of supported hardware very much reflects this, avoiding the need
   2.188 +for us to write drivers for "legacy" hardware.
   2.189 +
   2.190 +Xen requires a "P6" or newer processor (e.g. Pentium Pro, Celeron,
   2.191 +Pentium II, Pentium III, Pentium IV, Xeon, AMD Athlon, AMD Duron).
   2.192 +Multiprocessor machines are supported, and we also have basic support
   2.193 +for HyperThreading (SMT), though this remains a topic for ongoing
   2.194 +research. We're also looking at an AMD x86_64 port (though it should
   2.195 +run on Opterons in 32 bit mode just fine).
   2.196 +
   2.197 +Xen can currently use up to 4GB of memory. Its possible for x86
   2.198 +machines to address more than that (64GB), but it requires using a
   2.199 +different page table format (3-level rather than 2-level) that we
   2.200 +currently don't support. Adding 3-level PAE support wouldn't be
   2.201 +difficult, but we'd also need to add support to all the guest
   2.202 +OSs. Volunteers welcome!
   2.203 +
   2.204 +We currently support a relative modern set of network cards: Intel
   2.205 +e1000, Broadcom BCM 57xx (tg3), 3COM 3c905 (3c59x). Adding support for
   2.206 +other NICs that support hardware DMA scatter/gather from half-word
   2.207 +aligned addresses is relatively straight forward, by porting the
   2.208 +equivalent Linux driver. Drivers for a number of other older cards
   2.209 +have recently been added [pcnet32, e100, tulip], but are untested and
   2.210 +not recommended.
   2.211 +
   2.212 +
   2.213 +Ian Pratt
   2.214 +9 Sep 2003
   2.215 \ No newline at end of file

     3.1 --- /dev/null	Thu Jan 01 00:00:00 1970 +0000
     3.2 +++ b/README.CD	Wed Sep 10 09:57:56 2003 +0000
     3.3 @@ -0,0 +1,416 @@
     3.4 +#############################
     3.5 + __  __            _   ___  
     3.6 + \ \/ /___ _ __   / | / _ \ 
     3.7 +  \  // _ \ '_ \  | || | | |
     3.8 +  /  \  __/ | | | | || |_| |
     3.9 + /_/\_\___|_| |_| |_(_)___/ 
    3.10 +                                                   
    3.11 +#############################
    3.12 +
    3.13 + XenDemoCD 1.0 rc1 
    3.14 + University of Cambridge Computer Laboratory
    3.15 + 31 Aug 2003
    3.16 +
    3.17 + http://www.cl.cam.ac.uk/netos/xen
    3.18 +
    3.19 +Welcome to the Xen Demo CD! 
    3.20 +
    3.21 +Executive Summary
    3.22 +=================
    3.23 +
    3.24 +This CD is a standalone demo of the Xen Virtual Machine Monitor (VMM)
    3.25 +and Linux-2.4 OS port (xenolinux). It runs entirely off the CD,
    3.26 +without requiring hard disk installation. This is acheived using a ram
    3.27 +disk to store mutable file system data while using the CD for
    3.28 +everything else. The CD can also be used for installing Xen/xenolinux
    3.29 +to disk, and includes a source code snapshot along with all of the
    3.30 +tools required to build it.
    3.31 +
    3.32 +Booting the CD
    3.33 +==============
    3.34 +
    3.35 +The Xen VMM is currently fairly h/w specific, but porting new device
    3.36 +drivers is relatively straight forward thanks to Xen's Linux driver
    3.37 +compatibility layer. The current snapshot supports the following
    3.38 +hardware:
    3.39 +
    3.40 +CPU:  Pentium Pro/II/III/IV/Xeon, Athlon (i.e. P6 or newer) SMP supported
    3.41 +IDE:  Intel PIIX chipset, others will be PIO only (slow)
    3.42 +SCSI: Adaptec / Dell PERC Raid (aacraid), megaraid, Adaptec aic7xxx
    3.43 +Net:  Recommended: Intel e1000, Broadcom BCM57xx (tg3), 3c905 (3c59x)
    3.44 +      Tested but requires extra copies : pcnet32
    3.45 +      Untested and also requires extra copies : Intel e100, tulip
    3.46 +
    3.47 +Because of the demo CD's use of ram disks, make sure you have plenty
    3.48 +of RAM (256MB+).
    3.49 +
    3.50 +To try out the Demo, boot from CD (you may need to change your BIOS
    3.51 +configuration to do this), hit a key on either the keyboard or serial
    3.52 +line to pull up the Grub boot menu, then select one of the four boot
    3.53 +options:
    3.54 +
    3.55 + Xen / linux-2.4.22 X using DHCP
    3.56 + Xen / linux-2.4.22 X using cmdline IP config
    3.57 + Xen / linux-2.4.22 text using DHCP
    3.58 + Xen / linux-2.4.22 text using cmdline IP config
    3.59 + linux-2.4.22
    3.60 + linux-2.4.22 single
    3.61 + linux-2.4.20-rc1 single
    3.62 +
    3.63 +The last three options are plain linux kernels that run on the bare
    3.64 +machine, and are included simply to help diagnose driver compatibility
    3.65 +problems. If you are going for a command line IP config, hit "e" at
    3.66 +the grub menu, then edit the "ip=" parameters to reflect your setup
    3.67 +e.g. "ip=<ipaddr>::<gateway>:<netmask>::eth0:off". It shouldn't be
    3.68 +necessary to set either the nfs server or hostname
    3.69 +parameters. Alternatively, once xenolinux has booted you can login and
    3.70 +setup networking with ifconfig and route in the normal way.
    3.71 +
    3.72 +To make things easier for yourself, its worth trying to arrange for an
    3.73 +IP address which is the first in a sequential range of free IP
    3.74 +addresses.  Its useful to give each VM instance its own IP address
    3.75 +(though it is possible to do NAT or use private addresses etc), and
    3.76 +the configuration files on the CD allocate IP addresses sequentially
    3.77 +for subsequent domains unless told otherwise.
    3.78 +
    3.79 +After selecting the kernel to boot, stand back and watch Xen boot,
    3.80 +closely followed by "domain 0" running the xenolinux kernel. The boot
    3.81 +messages are also sent to the serial line (the baud rate can be set on
    3.82 +the Xen cmdline), which can be very useful for debugging should
    3.83 +anything important scroll off the screen. Xen's startup messages will
    3.84 +look quite familiar as much of the hardware initialisation (SMP boot,
    3.85 +apic setup) and device drivers are derived from Linux.
    3.86 +
    3.87 +If everything is well, you should see the linux rc scripts start a
    3.88 +bunch of standard services including sshd.  Login on the console or
    3.89 +via ssh::
    3.90 + username: user         root
    3.91 + password: xendemo      xendemo
    3.92 +
    3.93 +Once logged in, it should look just like any regular linux box. All
    3.94 +the usual tools and commands should work as per usual.  You can start
    3.95 +an xserver with 'startx' if you elected not to start one at boot.  The
    3.96 +current rc scripts also starts an Apache web server, which you should
    3.97 +be able to issue requests to on port 80.  If you want to browse the
    3.98 +Xen / Xenolinux source, it's all located under /local, complete with
    3.99 +BitKeeper repository.
   3.100 +
   3.101 +Because CD's aren't exactly known for their high performance, the
   3.102 +machine will likely feel rather sluggish. You may wish to go ahead and
   3.103 +install Xen/XenoLinux on your hard drive, either dropping Xen and the
   3.104 +XenoLinux kernel down onto a pre-existing Linux distribution, or using
   3.105 +the file systems from the CD (which are based on RH7.2). See the
   3.106 +installation instructions later in this document.
   3.107 +
   3.108 +
   3.109 +Starting other domains
   3.110 +======================
   3.111 +
   3.112 +There's a web interface for starting and managing other domains (VMs),
   3.113 +but since we generally use the command line tools they're probably
   3.114 +rather better debugged at present. The key command is 'xenctl' which
   3.115 +lives in /local/bin and uses /etc/xenctl.xml for its default
   3.116 +configuration. Run 'xenctl' without any arguments to get a help
   3.117 +message.
   3.118 +
   3.119 +To create a new domain, using the same xenolinux kernel image as used
   3.120 +for domain0, the next consecutive IP address, and the same CD-based
   3.121 +file system, type:
   3.122 +
   3.123 + xenctl new -n give_this_domain_a_name
   3.124 +
   3.125 +domctl will return printing the domain id that has been allocated to
   3.126 +the new domain (probably '1' if this is the first domain to be fired
   3.127 +up). If you're running off the CD this will take a while, as there's
   3.128 +huge piles of Java goop grinding away...  Then, fire up the domain:
   3.129 +
   3.130 + xenctl start -n<domid>                       
   3.131 +
   3.132 +You should see your domain boot and be able to ping and ssh into it as
   3.133 +before.
   3.134 +
   3.135 +"xenctl list" provides status information about running domains,
   3.136 +though is currently only allowed to be run by domain 0. It accesses
   3.137 +/proc/xeno/domains to read this information from Xen. You can also use
   3.138 +xenctl to 'stop' (pause) a domain, or 'kill' a domain. You can either
   3.139 +kill it nicely by sending a shutdown event and waiting for it to
   3.140 +terminate, or blow the sucker away with extreme prejudice. 
   3.141 +
   3.142 +If you want to configure the new domain differently, type 'xenctl' to
   3.143 +get a list of arguments, e.g. use the "-4" option to set a diffrent
   3.144 +IPv4 address. If you haven't any spare IP addresses on your network,
   3.145 +you can configure other domains with link-local addresses
   3.146 +(169.254/16), but then you'll only be able to access domains other
   3.147 +than domain0 from within the machine (they won't be externally
   3.148 +routeable). To automate this, there's an /etc/xenctl-linklocal.xml
   3.149 +which you can copy in place of /etc/xenctl.xml
   3.150 +
   3.151 +xenctl can be used to set the new kernel's command line, and hence
   3.152 +determine what it uses as a root file system etc. Although the default
   3.153 +is to boot in the same manner that domain0 did (using the RAM-based
   3.154 +file system for root and the CD for usr) it's possible to configure any
   3.155 +of the following possibilities, for example:
   3.156 +
   3.157 + * initrd=/boot/initrd init=/linuxrc
   3.158 +     boot using an initial ram disk, executing /linuxrc (as per this CD)     
   3.159 +
   3.160 + * root=/dev/hda3 ro
   3.161 +     boot using a standard hard disk partition as root
   3.162 +
   3.163 + * root=/dev/xvda1 ro
   3.164 +     boot using a pre-configured 'virtual block device' that will be
   3.165 +     attached to a virtual disk that will previously had a file system
   3.166 +     installed on it.
   3.167 +
   3.168 + * root=/dev/nfs nfsroot=/path/on/server ip=<blah_including server_IP>
   3.169 +     Boot using an NFS mounted root file system. This could be from a
   3.170 +     remote NFS server, or from an NFS server running in another
   3.171 +     domain. The latter is rather a useful option.
   3.172 +
   3.173 +
   3.174 +A typical setup might be to allocate a standard disk partition for
   3.175 +each domain and populate it with files. To save space, having a shared
   3.176 +read-only usr partition might make sense.
   3.177 +
   3.178 +Alternatively, you can use 'virtual disks', which are stored as files
   3.179 +within a custom file system. "xenctl partitions add" can be used to
   3.180 +'format' a partition with the file system, then virtual disks can be
   3.181 +created with "xenctl vd create". Virtual disks can then be attached to
   3.182 +a running domain as a 'virtual block device' using "xenctl vdb
   3.183 +create". The virtual disk can then optionally be partitioned
   3.184 +(e.g. "fdisk /dev/xvda") or have a file system created on it directly
   3.185 +(e.g. "mkfs -t ext3 /dev/xvda").  The virtual disk can then accessed
   3.186 +by a virtual block device associated with another domain, and even
   3.187 +used as a boot device.
   3.188 +
   3.189 +Both virtual disks and real partitions should only be shared domains
   3.190 +in a read-only fashion otherwise the linux kernels will obviously get
   3.191 +very confused if the file system structure changes underneath them!
   3.192 +If you want read-write sharing, export the directory to other domains
   3.193 +via NFS. 
   3.194 +
   3.195 +
   3.196 +About The Xen Demo CD
   3.197 +=====================
   3.198 +
   3.199 +The purpose of the Demo CD is to distribute a snapshot of Xen's source,
   3.200 +and simultaneously provide a convenient means for enabling people to
   3.201 +get experience playing with Xen without needing to install it on their
   3.202 +hard drive. If you decide to install Xen/XenoLinux you can do so
   3.203 +simply by folling the INSTALL instructions and copying the contents of
   3.204 +the CD on to a suitably formated disk partition and install the Grub
   3.205 +bootloader.
   3.206 +
   3.207 +This is a bootable CD that loads Xen, and then a Linux 2.4.22 OS image
   3.208 +ported to run on Xen. The CD contains a copy of a file system based on
   3.209 +the RedHat 7.2 distribution that is able to run directly off the CD
   3.210 +("live ISO"), using a "tmpfs" RAM-based file system for root (/etc
   3.211 +/var etc). Changes you make to the tmpfs will obviously not be
   3.212 +persistent across reboots!
   3.213 +
   3.214 +Because of the use of a ram-based file system for root, you'll need
   3.215 +plenty of memory to run this CD -- something like 96MB per VM. This is
   3.216 +not a restriction of Xen : once you've installed Xen, XenoLinux and
   3.217 +the file system images on your hard drive you'll find you can boot VMs
   3.218 +in just a few MBs.
   3.219 +
   3.220 +The CD contains a snapshot of the Xen and XenoLinux code base that we
   3.221 +believe to be pretty stable, but lacks some of the features that are
   3.222 +currently still work in progress e.g. OS suspend/resume to file, and
   3.223 +various memory management enhancements to provide fast inter-OS
   3.224 +communication and sharing of memory pages between OSs. We'll release
   3.225 +newer snapshots as required, in the form of a BitKeeper repository
   3.226 +hosted on http://xen.bkbits.net (follow instructions from the project
   3.227 +home page).  We're obviously grateful to receive any
   3.228 +bug fixes or other code you can contribute.
   3.229 +
   3.230 +
   3.231 +Installing from the CD
   3.232 +----------------------
   3.233 +
   3.234 +If you're installing Xen/XenoLinux onto an existing linux file system
   3.235 +distribution, its typically necessary to copy the Xen VMM
   3.236 +(/boot/image.gz) and XenoLinux kernels (/boot/xenolinux.gz) then
   3.237 +modify the Grub config (/boot/grub/menu.lst or /boot/grub/grub.conf)
   3.238 +on the target system.
   3.239 +
   3.240 +Xen is a "multiboot" standard boot image. Despite being a 'standard',
   3.241 +few boot loaders actually support it. The only two we know of are
   3.242 +Grub, and our modified version of linux kexec (for booting off a
   3.243 +XenoBoot CD -- PlanetLab have adopted the same boot CD approach).
   3.244 +
   3.245 +If you need to install grub on your system, you can do so either by
   3.246 +building the Grub source tree /usr/local/grub-0.93-iso9660-splashimage
   3.247 +or by copying over all the files in /boot/grub and then running
   3.248 +/sbin/grub and following the usual grub documentation. You'll then
   3.249 +need to configure the Grub config file.
   3.250 +
   3.251 +A typical Grub menu option might look like:
   3.252 +
   3.253 +title Xen / XenoLinux 2.4.22
   3.254 +        kernel /boot/image.gz dom0_mem=131072 ser_baud=115200 noht
   3.255 +        module /boot/xenolinux.gz root=/dev/sda4 ro console=tty0
   3.256 +
   3.257 +The first line specifies which xen image to use, and what command line
   3.258 +arguments to pass to Xen. In this case, we set the maximum amount of
   3.259 +memory to allocate to domain0, and the serial baud rate (the default
   3.260 +is 9600 baud). We could also disable smp support (nosmp) or disable
   3.261 +hyper-threading support (noht). If you have multiple network interface
   3.262 +there are various options to select which ones to use.
   3.263 +
   3.264 +The second line specifies which xenolinux image to use, and the
   3.265 +standard linux command line arguments to pass to the kernel. In this
   3.266 +case, we're configuring the root partition and stating that it should
   3.267 +be mounted read-only (normal practise). If the file system isn't
   3.268 +configured for DHCP then we'd probably want to configure that on the
   3.269 +kernel command line too.
   3.270 +
   3.271 +If we were booting with an initial ram disk (initrd), then this would
   3.272 +require a second "module" line, with no arguments.
   3.273 +
   3.274 +
   3.275 +Installing the file systems from the CD
   3.276 +---------------------------------------
   3.277 +
   3.278 +If you haven't an existing Linux installation onto which you can just
   3.279 +drop down the Xen and XenoLinux images, then the file systems on the
   3.280 +CD provide a quick way of doing an install.
   3.281 +
   3.282 +Choose one or two partitions, depending on whether you want a separate
   3.283 +/usr or not. Make file systems on it/them e.g.: 
   3.284 +  mkfs -t ext3 /dev/hda3
   3.285 +  [or mkfs -t ext2 /dev/hda3 && tune2fs -j /dev/hda3 if using an old
   3.286 +version of mkfs]
   3.287 +
   3.288 +Next, mount the file system(s) e.g.:
   3.289 +  mkdir /mnt/root && mount /dev/hda3 /mnt/root
   3.290 +  [mkdir /mnt/usr && mount /dev/hda4 /mnt/usr]
   3.291 +  
   3.292 +To install the root file system, simply untar /usr/XenDemoCD/root.tar.gz
   3.293 +  cd /mnt/root && tar -zxpf /usr/XenDemoCD/root.tar.gz
   3.294 +
   3.295 +You'll need to edit /mnt/root/etc/fstab to reflect your file system
   3.296 +configuration. Changing the password file (etc/shadow) is probably a
   3.297 +good idea too.
   3.298 +
   3.299 +To install the usr file system, copy the file system from CD on /usr,
   3.300 +though leaving out the "XenDemoCD" and "boot" directories:
   3.301 +  cd /usr && cp -a doc games include lib local root share tmp X11R6 bin dict etc html kerberos libexec man sbin src /mnt/usr/
   3.302 +  
   3.303 +If you intend to boot off these file systems (i.e. use them for
   3.304 +domain0), then you probably want to copy the /usr/boot directory on
   3.305 +the cd over the top of the current symlink to /boot on your root
   3.306 +filesystem (after deleting the current symlink) i.e.:
   3.307 +  cd /mnt/root ; rm boot ; cp -a /usr/boot .
   3.308 +
   3.309 +The XenDemoCD directory is only useful if you want to build your own
   3.310 +version of the XenDemoCD (see below).
   3.311 +
   3.312 +Debugging
   3.313 +---------
   3.314 +
   3.315 +Xen has a set of debugging features that can be useful to try and
   3.316 +figure out what's going on. Hit 'h' on the serial line or ScrollLock-h
   3.317 +on the keyboard to get a list of supported commands. 
   3.318 +
   3.319 +If you have a crash you'll likely get a crash dump containing and EIP
   3.320 +(PC), which along with and 'objdump -d image' can be useful in
   3.321 +figuring out what's happened. 
   3.322 +
   3.323 +
   3.324 +Description of how the XenDemoCD boots
   3.325 +--------------------------------------
   3.326 +
   3.327 +1. Grub is used to load Xen, a xenolinux kernel, and an initrd (initial
   3.328 +ram disk). [The source of the version of Grub used is in /usr/local/]
   3.329 +
   3.330 +2. the init=/linuxrc command line causes linux to execute /linuxrc in
   3.331 +the initrd. 
   3.332 +
   3.333 +3. the /linuxrc file attempts to mount the CD by trying the likely
   3.334 +locations /dev/hd[abcd].  
   3.335 +
   3.336 +4. it then creates a 'tmpfs' file system and untars the
   3.337 +'XenDemoCD/root.tar.gz' file into the tmpfs. This contains hopefully
   3.338 +all the files that need to be mutable. (this would be so much easier
   3.339 +if Linux supported 'stacked' or union file systems...)
   3.340 +
   3.341 +5. Next, /linuxrc uses the pivot_root call to change the root file
   3.342 +system to the tmpfs, with the CD mounted as /usr.
   3.343 +
   3.344 +6. It then invokes /sbin/init in the tmpfs and the boot proceeds
   3.345 +normally.
   3.346 +
   3.347 +
   3.348 +Building your own version of the XenDemoCD
   3.349 +------------------------------------------
   3.350 +
   3.351 +The filesystems on the CD are based heavily on Peter Anvin's
   3.352 +SuperRescue CD version 2.1.2, which takes its content from RedHat
   3.353 +7.2. Since Xen uses a "multiboot" image format, it was necessary to
   3.354 +change the bootloader from isolinux to Grub0.93 with Leonid
   3.355 +Lisovskiy's <lly@pisem.net> grub.0.93-iso9660.patch
   3.356 +
   3.357 +The Xen Demo CD contains all of the build scripts that were used to
   3.358 +create it, so its possible to 'unpack' the current iso, modifiy it,
   3.359 +then build a new iso. The procedure for doing so is as follows:
   3.360 +
   3.361 +First, mount either the CD, or the iso image of the CD:
   3.362 + 
   3.363 +  mount /dev/cdrom /mnt/cdrom 
   3.364 +or:
   3.365 +  mount -o loop xendemo-1.0beta.iso  /mnt/cdrom
   3.366 +
   3.367 +cd to the directory you want to 'unpack' the iso into then run the
   3.368 +unpack script:
   3.369 +
   3.370 +  cd /local/xendemocd
   3.371 +  /mnt/cdrom/XenDemoCD/unpack-iso.sh
   3.372 +
   3.373 +The result is a 'build' directory containing the file system tree
   3.374 +under the 'root' directory. e.g. /local/xendemocd/build/root
   3.375 +
   3.376 +To add or remove rpms, its possible to use 'rpm' with the --root
   3.377 +option to set the path. For more complex changes, it easiest to boot a
   3.378 +machine using using the tree via NFS root. Before doing this, you'll
   3.379 +need to edit fstab to comment out the seperate mount of /usr.
   3.380 +
   3.381 +One thing to watch out for: as part of the CD build process, the
   3.382 +contents of the 'rootpatch' tree gets copied over the existing 'root'
   3.383 +tree replacing various files. The intention of the rootpatch tree is
   3.384 +to contain the files that have been modified from the original RH
   3.385 +distribution (e.g. various /etc files). This was done to make it
   3.386 +easier to upgrade to newer RH versions in the future. The downside of
   3.387 +this is that if you edit an existing file in the root tree you should
   3.388 +check that you don't also need to propagate the change to the
   3.389 +rootpatch tree to avoid it being overwritten.
   3.390 +
   3.391 +Once you've made the changes and want to build a new iso, here's the
   3.392 +procedure:
   3.393 +
   3.394 +cd /local/xendemocd/build  
   3.395 +echo '<put_your_name_here>' > Builder
   3.396 +./make.sh put_your_version_id_here >../buildlog 2>&1            
   3.397 +
   3.398 +This process can take 30 mins even on a fast machine, but you should
   3.399 +eventually end up with an iso image in the build directory.
   3.400 +
   3.401 +notes:
   3.402 +
   3.403 +root - the root of the file system heirarchy as presented to the running system
   3.404 +
   3.405 +rootpatch - contains files that have been modified from the standard RH, and copied over the root tree as part of the build procedure.
   3.406 +
   3.407 +irtree - the filie system tree that will go into the initrd (initial ram disk)
   3.408 +
   3.409 +work - a working directory used in the build process
   3.410 +
   3.411 +usr - this should really be in 'work' as its created as part of the
   3.412 +build process. It contains the 'immutable' files that will be served
   3.413 +from the CD rather than the tmpfs containing the contents of root.tar.gz
   3.414 +Some files that are normally in /etc or /var that are large and
   3.415 +actually unlikely to need changing have been moved into /usr/root
   3.416 +and replaced with links.
   3.417 +
   3.418 +Ian Pratt
   3.419 +9 Sep 2003
   3.420 \ No newline at end of file

     4.1 --- /dev/null	Thu Jan 01 00:00:00 1970 +0000
     4.2 +++ b/tools/misc/xen-clone	Wed Sep 10 09:57:56 2003 +0000
     4.3 @@ -0,0 +1,78 @@
     4.4 +#!/bin/sh -x
     4.5 +
     4.6 +# usage: xen-clone bk_repository dest_dir orig_linux_dir
     4.7 +# 
     4.8 +# this script contains some CL site specific details, but can easily be adapted
     4.9 +#
    4.10 +
    4.11 +# test which site we're on
    4.12 +[ -d /usr/groups/xeno/ -a -d /usr/groups/srgboot ] && SITE=UCCL
    4.13 +
    4.14 +case "$SITE" in
    4.15 +UCCL)
    4.16 +	BK_REP=${1:-/usr/groups/xeno/BK/xeno.bk}
    4.17 +	# BK_REP=${1:-xeno-master/xeno.bk}
    4.18 +	LINUX_DIR=${3:-/usr/groups/xeno/archive/}
    4.19 +	;;
    4.20 +*)
    4.21 +	BK_REP=${1:-bk://xen.bkbits.net/xeno.bk}
    4.22 +	# BK_REP=${1:-ssh://xen@xen.bkbits.net/xeno.bk}
    4.23 +	LINUX_DIR=${3:-..}
    4.24 +;;
    4.25 +esac
    4.26 +
    4.27 +DEST_DIR=${2:-xeno-clone}
    4.28 +DEST_BK_REP=`basename ${BK_REP}`
    4.29 +
    4.30 +echo usage: xen-clone bk_repository dest_dir orig_linux_dir
    4.31 +echo Source BK Repository : ${BK_REP}
    4.32 +echo Destination Dir/Repository : ${DEST_DIR}/${DEST_BK_REP}
    4.33 +echo Pristine Linux Source directory : ${LINUX_DIR}
    4.34 +
    4.35 +mkdir -p ${DEST_DIR}
    4.36 +cd ${DEST_DIR}
    4.37 +TOP=`/bin/pwd`
    4.38 +
    4.39 +# site-specific set up of installation directories
    4.40 +case "$SITE" in
    4.41 +CL)
    4.42 +	PATH=$PATH:/usr/groups/xeno/build_tools/bin
    4.43 +	mkdir -p install/boot
    4.44 +	cd install/boot
    4.45 +	ln -sf ../../xeno-roots/roots .
    4.46 +	ln -sf ../../xeno-roots/usr .
    4.47 +	ln -sf /usr/groups/srgboot/${USER}/xenoboot.sh .
    4.48 +	ln -sf `pwd` /usr/groups/srgboot/${USER}/${DEST_DIR}
    4.49 +	cd ../..
    4.50 +	;;
    4.51 +esac
    4.52 +
    4.53 +# clone the master repository (now checked-out by default)
    4.54 +bk clone ${BK_REP} ${DEST_BK_REP}
    4.55 +
    4.56 +# identify this version of linux
    4.57 +LINUX_VER=`/bin/ls -ld ${DEST_BK_REP}/xenolinux-sparse | sed -e 's!^.*xenolinux-\([0-9.]\+\)-sparse!\1!'`
    4.58 +
    4.59 +# copy in the master Linux tree for this kernel
    4.60 +tar -zxf ${LINUX_DIR}/linux-${LINUX_VER}.tar.gz || tar -zxf ${LINUX_DIR}/linux-${LINUX_VER}.tgz || cp -a ${LINUX_DIR}/linux-${LINUX_VER} . ||  wget ftp://ftp.kernel.org/pub/linux/kernel/v2.4/linux-${LINUX_VER}.tar.gz -O- | tar -zxf - || exit -1
    4.61 +
    4.62 +# build and install Xen and tools
    4.63 +cd ${DEST_BK_REP}
    4.64 +make install
    4.65 +
    4.66 +# Turn linux into xenolinux then build it
    4.67 +cd xenolinux-${LINUX_VER}-sparse
    4.68 +./mkbuildtree ../../linux-${LINUX_VER}
    4.69 +cd ../..
    4.70 +mv linux-${LINUX_VER} xenolinux-${LINUX_VER}
    4.71 +cd xenolinux-${LINUX_VER}
    4.72 +export ARCH=xeno
    4.73 +export INSTALL_MOD_PATH=${TOP}/install
    4.74 +make oldconfig
    4.75 +make dep
    4.76 +make bzImage
    4.77 +make install
    4.78 +make modules
    4.79 +make modules_install
    4.80 +cd ..
    4.81 +

     5.1 --- /dev/null	Thu Jan 01 00:00:00 1970 +0000
     5.2 +++ b/tools/misc/xen-clone.README	Wed Sep 10 09:57:56 2003 +0000
     5.3 @@ -0,0 +1,23 @@
     5.4 +
     5.5 +xen-clone
     5.6 +
     5.7 +usage: xen-clone <bk_repository> <dest_dir> <orig_linux_dir>
     5.8 +
     5.9 +This script can be used to 'bk clone' and build a xen and xenolinux image 
    5.10 +from the master BK repository, either from a local copy, or from the
    5.11 +public repository bk://xen.bkbits.net/xeno.bk
    5.12 +
    5.13 +In many circumstances, it can be invoked without any arguments and
    5.14 +just `does the right thing'.
    5.15 +
    5.16 +The default dest_dir is 'xeno-clone', relative to the current directory.
    5.17 +
    5.18 +To build xenolinux, the script needs a pristine copy of the equivalent 
    5.19 +linux tree. The script looks in a couple of places on the local filesystem, 
    5.20 +then tries a download from from ftp://ftp.kernel.org/pub/linux/kernel/v2.4/
    5.21 +
    5.22 +The script also tries a number of optional UCCL site-specific operations
    5.23 +that configure the test machine booting infrastructure to boot the 
    5.24 +resultant image.
    5.25 +
    5.26 +

     6.1 --- /dev/null	Thu Jan 01 00:00:00 1970 +0000
     6.2 +++ b/xen/README	Wed Sep 10 09:57:56 2003 +0000
     6.3 @@ -0,0 +1,37 @@
     6.4 +
     6.5 +*****************************************************
     6.6 +   Xeno Hypervisor (16/3/03)
     6.7 +
     6.8 +'make': Builds ELF executable called 'image' in base directory
     6.9 +'make clean': removes *all* build and target files
    6.10 +
    6.11 +
    6.12 +
    6.13 +Booting secondary processors
    6.14 +----------------------------
    6.15 +
    6.16 +It's twisty and turny, so this is (roughly) the code path:
    6.17 +
    6.18 +start_of_day (i386/setup.c)
    6.19 +smp_boot_cpus (i386/smpboot.c)
    6.20 + * initialises boot CPU data
    6.21 + * parses APIC tables
    6.22 + * for each cpu:
    6.23 +   do_boot_cpu (i386/smpboot.c)
    6.24 +    * forks a new idle process
    6.25 +    * points initial stack inside new task struct
    6.26 +    * points initial EIP at a trampoline in very low memory
    6.27 +    * frobs remote APIC....
    6.28 +
    6.29 +On other processor:
    6.30 + * trampoline sets GDT and IDT
    6.31 + * jumps at main boot address with magic register value
    6.32 + * after setting proper page and descriptor tables, jumps at...
    6.33 +   initialize_secondary (i386/smpboot.c)
    6.34 +    * simply reads ESP/EIP out of the (new) idle task
    6.35 +    * this causes a jump to...
    6.36 +      start_secondary (i386/smpboot.c)
    6.37 +       * reset all processor state
    6.38 +       * barrier, then write bitmasks to signal back to boot cpu
    6.39 +       * then barrel into...
    6.40 +         cpu_idle (i386/process.c)