annotate docs/src/user.tex @ 9799:1d42739fee3b

Fix user manual regarding trace buffers.
1. debug building is not needed for tracing buffer...
2. ...but trace buffer default size is 0

Signed-off-by: Atsushi SAKAI <sakaia@jp.fujitsu.com>
author kaf24@firebug.cl.cam.ac.uk
date Fri Apr 21 09:09:29 2006 +0100 (2006-04-21)
parents 14659382edd3
children 6d476981e3a5
rev   line source
kaf24@2911 1 \documentclass[11pt,twoside,final,openright]{report}
smh22@8242 2 \usepackage{a4,graphicx,html,parskip,setspace,times,xspace,url}
smh22@2882 3 \setstretch{1.15}
kaf24@2743 4
smh22@8242 5 \renewcommand{\ttdefault}{pcr}
smh22@2848 6
smh22@2848 7 \def\Xend{{Xend}\xspace}
smh22@2848 8 \def\xend{{xend}\xspace}
smh22@2848 9
smh22@8242 10 \latexhtml{\renewcommand{\path}[1]{{\small {\tt #1}}}}{\renewcommand{\path}[1]{{\tt #1}}}
smh22@2889 11
smh22@2889 12
kaf24@2743 13 \begin{document}
kaf24@2743 14
kaf24@2743 15 % TITLE PAGE
kaf24@2743 16 \pagestyle{empty}
kaf24@2743 17 \begin{center}
kaf24@2743 18 \vspace*{\fill}
kaf24@2743 19 \includegraphics{figs/xenlogo.eps}
kaf24@2743 20 \vfill
kaf24@2743 21 \vfill
kaf24@2743 22 \vfill
kaf24@2743 23 \begin{tabular}{l}
FMJ@7771 24 {\Huge \bf Users' Manual} \\[4mm]
FMJ@7771 25 {\huge Xen v3.0} \\[80mm]
kaf24@2743 26 \end{tabular}
kaf24@2777 27 \end{center}
kaf24@2777 28
smh22@8242 29 {\bf DISCLAIMER: This documentation is always under active development
smh22@8242 30 and as such there may be mistakes and omissions --- watch out for
smh22@8242 31 these and please report any you find to the developers' mailing list,
smh22@8242 32 xen-devel@lists.xensource.com. The latest version is always available
smh22@8242 33 on-line. Contributions of material, suggestions and corrections are
smh22@8242 34 welcome.}
kaf24@2777 35
kaf24@2743 36 \vfill
emellor@8244 37 \clearpage
emellor@8244 38
emellor@8244 39
emellor@8244 40 % COPYRIGHT NOTICE
emellor@8244 41 \pagestyle{empty}
emellor@8244 42
emellor@8244 43 \vspace*{\fill}
emellor@8244 44
emellor@8244 45 Xen is Copyright \copyright 2002-2005, University of Cambridge, UK, XenSource
emellor@8244 46 Inc., IBM Corp., Hewlett-Packard Co., Intel Corp., AMD Inc., and others. All
emellor@8244 47 rights reserved.
emellor@8244 48
emellor@8244 49 Xen is an open-source project. Most portions of Xen are licensed for copying
emellor@8244 50 under the terms of the GNU General Public License, version 2. Other portions
emellor@8244 51 are licensed under the terms of the GNU Lesser General Public License, the
emellor@8244 52 Zope Public License 2.0, or under ``BSD-style'' licenses. Please refer to the
emellor@8244 53 COPYING file for details.
emellor@8244 54
emellor@9433 55 Xen includes software by Christopher Clark. This software is covered by the
emellor@9433 56 following licence:
emellor@9433 57
emellor@9433 58 \begin{quote}
emellor@9433 59 Copyright (c) 2002, Christopher Clark. All rights reserved.
emellor@9433 60
emellor@9433 61 Redistribution and use in source and binary forms, with or without
emellor@9433 62 modification, are permitted provided that the following conditions are met:
emellor@9433 63
emellor@9433 64 \begin{itemize}
emellor@9433 65 \item Redistributions of source code must retain the above copyright notice,
emellor@9433 66 this list of conditions and the following disclaimer.
emellor@9433 67
emellor@9433 68 \item Redistributions in binary form must reproduce the above copyright
emellor@9433 69 notice, this list of conditions and the following disclaimer in the
emellor@9433 70 documentation and/or other materials provided with the distribution.
emellor@9433 71
emellor@9433 72 \item Neither the name of the original author; nor the names of any
emellor@9433 73 contributors may be used to endorse or promote products derived from this
emellor@9433 74 software without specific prior written permission.
emellor@9433 75 \end{itemize}
emellor@9433 76
emellor@9433 87 \end{quote}
emellor@9433 88
kaf24@2743 89 \cleardoublepage
kaf24@2743 90
FMJ@7771 91
kaf24@2743 92 % TABLE OF CONTENTS
kaf24@2743 93 \pagestyle{plain}
kaf24@2743 94 \pagenumbering{roman}
kaf24@2743 95 { \parskip 0pt plus 1pt
kaf24@2743 96 \tableofcontents }
kaf24@2743 97 \cleardoublepage
kaf24@2743 98
FMJ@7771 99
kaf24@2743 100 % PREPARE FOR MAIN TEXT
kaf24@2743 101 \pagenumbering{arabic}
kaf24@2743 102 \raggedbottom
kaf24@2743 103 \widowpenalty=10000
kaf24@2743 104 \clubpenalty=10000
kaf24@2743 105 \parindent=0pt
kaf24@2763 106 \parskip=5pt
kaf24@2743 107 \renewcommand{\topfraction}{.8}
kaf24@2743 108 \renewcommand{\bottomfraction}{.8}
kaf24@2743 109 \renewcommand{\textfraction}{.2}
kaf24@2743 110 \renewcommand{\floatpagefraction}{.8}
smh22@2793 111 \setstretch{1.1}
kaf24@2743 112
kaf24@2743 113
kaf24@6977 114 %% Chapter Introduction moved to introduction.tex
smh22@8242 115 \chapter{Introduction}
smh22@8242 116
smh22@8242 117
smh22@8242 118 Xen is an open-source \emph{para-virtualizing} virtual machine monitor
smh22@8242 119 (VMM), or ``hypervisor'', for the x86 processor architecture. Xen can
smh22@8242 120 securely execute multiple virtual machines on a single physical system
smh22@8242 121 with close-to-native performance. Xen facilitates enterprise-grade
smh22@8242 122 functionality, including:
smh22@8242 123
smh22@8242 124 \begin{itemize}
smh22@8242 125 \item Virtual machines with performance close to native hardware.
smh22@8242 126 \item Live migration of running virtual machines between physical hosts.
smh22@8242 127 \item Up to 32 virtual CPUs per guest virtual machine, with VCPU hotplug.
smh22@8242 128 \item x86/32, x86/32 with PAE, and x86/64 platform support.
smh22@8242 129 \item Intel Virtualization Technology (VT-x) for unmodified guest operating systems (including Microsoft Windows).
smh22@8242 130 \item Excellent hardware support (supports almost all Linux device
smh22@8242 131 drivers).
smh22@8242 132 \end{itemize}
smh22@8242 133
smh22@8242 134
smh22@8242 135 \section{Usage Scenarios}
smh22@8242 136
smh22@8242 137 Usage scenarios for Xen include:
smh22@8242 138
smh22@8242 139 \begin{description}
smh22@8242 140 \item [Server Consolidation.] Move multiple servers onto a single
smh22@8242 141 physical host with performance and fault isolation provided at the
smh22@8242 142 virtual machine boundaries.
smh22@8242 143 \item [Hardware Independence.] Allow legacy applications and operating
smh22@8242 144 systems to exploit new hardware.
smh22@8242 145 \item [Multiple OS configurations.] Run multiple operating systems
smh22@8242 146 simultaneously, for development or testing purposes.
smh22@8242 147 \item [Kernel Development.] Test and debug kernel modifications in a
smh22@8242 148 sand-boxed virtual machine --- no need for a separate test machine.
smh22@8242 149 \item [Cluster Computing.] Management at VM granularity provides more
smh22@8242 150 flexibility than separately managing each physical host, but better
smh22@8242 151 control and isolation than single-system image solutions,
smh22@8242 152 particularly by using live migration for load balancing.
smh22@8242 153 \item [Hardware support for custom OSes.] Allow development of new
smh22@8242 154 OSes while benefiting from the wide-ranging hardware support of
smh22@8242 155 existing OSes such as Linux.
smh22@8242 156 \end{description}
smh22@8242 157
smh22@8242 158
smh22@8242 159 \section{Operating System Support}
smh22@8242 160
smh22@8242 161 Para-virtualization permits very high performance virtualization, even
smh22@8242 162 on architectures like x86 that are traditionally very hard to
smh22@8242 163 virtualize.
smh22@8242 164
smh22@8242 165 This approach requires operating systems to be \emph{ported} to run on
smh22@8242 166 Xen. Porting an OS to run on Xen is similar to supporting a new
smh22@8242 167 hardware platform, however the process is simplified because the
smh22@8242 168 para-virtual machine architecture is very similar to the underlying
smh22@8242 169 native hardware. Even though operating system kernels must explicitly
smh22@8242 170 support Xen, a key feature is that user space applications and
smh22@8242 171 libraries \emph{do not} require modification.
smh22@8242 172
smh22@8242 173 With hardware CPU virtualization as provided by Intel VT and AMD
kaf24@8708 174 SVM technology, the ability to run an unmodified guest OS kernel
smh22@8242 175 is available. No porting of the OS is required, although some
smh22@8242 176 additional driver support is necessary within Xen itself. Unlike
smh22@8242 177 traditional full virtualization hypervisors, which suffer a tremendous
smh22@8242 178 performance overhead, the combination of Xen and VT or Xen and
smh22@8242 179 Pacifica technology complement one another to offer superb performance
smh22@8242 180 for para-virtualized guest operating systems and full support for
smh22@8242 181 unmodified guests running natively on the processor. Full support for
smh22@8242 182 VT and Pacifica chipsets will appear in early 2006.
smh22@8242 183
smh22@8242 184 Paravirtualized Xen support is available for increasingly many
smh22@8242 185 operating systems: currently, mature Linux support is available and
smh22@8242 186 included in the standard distribution. Other OS ports---including
smh22@8242 187 NetBSD, FreeBSD and Solaris x86 v10---are nearing completion.
smh22@8242 188
smh22@8242 189
smh22@8242 190 \section{Hardware Support}
smh22@8242 191
smh22@8242 192 Xen currently runs on the x86 architecture, requiring a ``P6'' or
smh22@8242 193 newer processor (e.g.\ Pentium Pro, Celeron, Pentium~II, Pentium~III,
smh22@8242 194 Pentium~IV, Xeon, AMD~Athlon, AMD~Duron). Multiprocessor machines are
smh22@8242 195 supported, and there is support for HyperThreading (SMT). In
smh22@8242 196 addition, ports to IA64 and Power architectures are in progress.
smh22@8242 197
smh22@8242 198 The default 32-bit Xen supports up to 4GB of memory. However Xen 3.0
smh22@8242 199 adds support for Intel's Physical Addressing Extensions (PAE), which
smh22@8242 200 enable x86/32 machines to address up to 64 GB of physical memory. Xen
smh22@8242 201 3.0 also supports x86/64 platforms such as Intel EM64T and AMD Opteron
smh22@8242 202 which can currently address up to 1TB of physical memory.
smh22@8242 203
smh22@8242 204 Xen offloads most of the hardware support issues to the guest OS
smh22@8242 205 running in the \emph{Domain~0} management virtual machine. Xen itself
smh22@8242 206 contains only the code required to detect and start secondary
smh22@8242 207 processors, set up interrupt routing, and perform PCI bus
smh22@8242 208 enumeration. Device drivers run within a privileged guest OS rather
smh22@8242 209 than within Xen itself. This approach provides compatibility with the
smh22@8242 210 majority of device hardware supported by Linux. The default XenLinux
smh22@8242 211 build contains support for most server-class network and disk
smh22@8242 212 hardware, but you can add support for other hardware by configuring
smh22@8242 213 your XenLinux kernel in the normal way.
smh22@8242 214
smh22@8242 215
smh22@8242 216 \section{Structure of a Xen-Based System}
smh22@8242 217
smh22@8242 218 A Xen system has multiple layers, the lowest and most privileged of
smh22@8242 219 which is Xen itself.
smh22@8242 220
smh22@8242 221 Xen may host multiple \emph{guest} operating systems, each of which is
smh22@8242 222 executed within a secure virtual machine. In Xen terminology, a
smh22@8242 223 \emph{domain}. Domains are scheduled by Xen to make effective use of the
smh22@8242 224 available physical CPUs. Each guest OS manages its own applications.
smh22@8242 225 This management includes the responsibility of scheduling each
smh22@8242 226 application within the time allotted to the VM by Xen.
smh22@8242 227
smh22@8242 228 The first domain, \emph{domain~0}, is created automatically when the
smh22@8242 229 system boots and has special management privileges. Domain~0 builds
smh22@8242 230 other domains and manages their virtual devices. It also performs
smh22@8242 231 administrative tasks such as suspending, resuming and migrating other
smh22@8242 232 virtual machines.
smh22@8242 233
smh22@8242 234 Within domain~0, a process called \emph{xend} runs to manage the system.
smh22@8242 235 \Xend\ is responsible for managing virtual machines and providing access
smh22@8242 236 to their consoles. Commands are issued to \xend\ over an HTTP interface,
smh22@8242 237 via a command-line tool.
smh22@8242 238
smh22@8242 239
smh22@8242 240 \section{History}
smh22@8242 241
smh22@8242 242 Xen was originally developed by the Systems Research Group at the
smh22@8242 243 University of Cambridge Computer Laboratory as part of the XenoServers
smh22@8242 244 project, funded by the UK-EPSRC\@.
smh22@8242 245
smh22@8242 246 XenoServers aim to provide a ``public infrastructure for global
smh22@8242 247 distributed computing''. Xen plays a key part in that, allowing one to
smh22@8242 248 efficiently partition a single machine to enable multiple independent
smh22@8242 249 clients to run their operating systems and applications in an
smh22@8242 250 environment. This environment provides protection, resource isolation
smh22@8242 251 and accounting. The project web page contains further information along
smh22@8242 252 with pointers to papers and technical reports:
smh22@8242 253 \path{http://www.cl.cam.ac.uk/xeno}
smh22@8242 254
smh22@8242 255 Xen has grown into a fully-fledged project in its own right, enabling us
smh22@8242 256 to investigate interesting research issues regarding the best techniques
smh22@8242 257 for virtualizing resources such as the CPU, memory, disk and network.
smh22@8242 258 Project contributors now include XenSource, Intel, IBM, HP, AMD, Novell,
smh22@8242 259 RedHat.
smh22@8242 260
smh22@8242 261 Xen was first described in a paper presented at SOSP in
smh22@8242 262 2003\footnote{\tt
smh22@8242 263 http://www.cl.cam.ac.uk/netos/papers/2003-xensosp.pdf}, and the first
smh22@8242 264 public release (1.0) was made that October. Since then, Xen has
smh22@8242 265 significantly matured and is now used in production scenarios on many
smh22@8242 266 sites.
smh22@8242 267
smh22@8242 268 \section{What's New}
smh22@8242 269
smh22@8242 270 Xen 3.0.0 offers:
smh22@8242 271
smh22@8242 272 \begin{itemize}
smh22@8242 273 \item Support for up to 32-way SMP guest operating systems
smh22@8242 274 \item Intel (Physical Addressing Extensions) PAE to support 32-bit
smh22@8242 275 servers with more than 4GB physical memory
smh22@8242 276 \item x86/64 support (Intel EM64T, AMD Opteron)
smh22@8242 277 \item Intel VT-x support to enable the running of unmodified guest
smh22@8242 278 operating systems (Windows XP/2003, Legacy Linux)
smh22@8242 279 \item Enhanced control tools
smh22@8242 280 \item Improved ACPI support
smh22@8242 281 \item AGP/DRM graphics
smh22@8242 282 \end{itemize}
smh22@8242 283
smh22@8242 284
smh22@8242 285 Xen 3.0 features greatly enhanced hardware support, configuration
smh22@8242 286 flexibility, usability and a larger complement of supported operating
smh22@8242 287 systems. This latest release takes Xen a step closer to being the
smh22@8242 288 definitive open source solution for virtualization.
smh22@8242 289
kaf24@2743 290
FMJ@8225 291
FMJ@8225 292 \part{Installation}
FMJ@8225 293
FMJ@8225 294 %% Chapter Basic Installation
smh22@8242 295 \chapter{Basic Installation}
FMJ@8225 296
smh22@8242 297 The Xen distribution includes three main components: Xen itself, ports
smh22@8242 298 of Linux and NetBSD to run on Xen, and the userspace tools required to
smh22@8242 299 manage a Xen-based system. This chapter describes how to install the
smh22@8242 300 Xen~3.0 distribution from source. Alternatively, there may be pre-built
smh22@8242 301 packages available as part of your operating system distribution.
FMJ@8225 302
FMJ@8226 303
smh22@8242 304 \section{Prerequisites}
smh22@8242 305 \label{sec:prerequisites}
FMJ@8226 306
smh22@8242 307 The following is a full list of prerequisites. Items marked `$\dag$' are
smh22@8242 308 required by the \xend\ control tools, and hence required if you want to
smh22@8242 309 run more than one virtual machine; items marked `$*$' are only required
smh22@8242 310 if you wish to build from source.
smh22@8242 311 \begin{itemize}
smh22@8242 312 \item A working Linux distribution using the GRUB bootloader and running
smh22@8242 313 on a P6-class or newer CPU\@.
smh22@8242 314 \item [$\dag$] The \path{iproute2} package.
smh22@8242 315 \item [$\dag$] The Linux bridge-utils\footnote{Available from {\tt
smh22@8242 316 http://bridge.sourceforge.net}} (e.g., \path{/sbin/brctl})
smh22@8242 317 \item [$\dag$] The Linux hotplug system\footnote{Available from {\tt
smh22@8242 318 http://linux-hotplug.sourceforge.net/}} (e.g.,
emellor@8244 319 \path{/sbin/hotplug} and related scripts). On newer distributions,
emellor@8244 320 this is included alongside the Linux udev system\footnote{See {\tt
emellor@8244 321 http://www.kernel.org/pub/linux/utils/kernel/hotplug/udev.html/}}.
smh22@8242 322 \item [$*$] Build tools (gcc v3.2.x or v3.3.x, binutils, GNU make).
smh22@8242 323 \item [$*$] Development installation of zlib (e.g.,\ zlib-dev).
smh22@8242 324 \item [$*$] Development installation of Python v2.2 or later (e.g.,\
smh22@8242 325 python-dev).
smh22@8242 326 \item [$*$] \LaTeX\ and transfig are required to build the
smh22@8242 327 documentation.
smh22@8242 328 \end{itemize}
smh22@8242 329
smh22@8242 330 Once you have satisfied these prerequisites, you can now install either
smh22@8242 331 a binary or source distribution of Xen.
smh22@8242 332
smh22@8242 333 \section{Installing from Binary Tarball}
smh22@8242 334
smh22@8242 335 Pre-built tarballs are available for download from the XenSource downloads
smh22@8242 336 page:
smh22@8242 337 \begin{quote} {\tt http://www.xensource.com/downloads/}
smh22@8242 338 \end{quote}
smh22@8242 339
smh22@8242 340 Once you've downloaded the tarball, simply unpack and install:
smh22@8242 341 \begin{verbatim}
smh22@8242 342 # tar zxvf xen-3.0-install.tgz
smh22@8242 343 # cd xen-3.0-install
smh22@8242 344 # sh ./install.sh
smh22@8242 345 \end{verbatim}
smh22@8242 346
smh22@8242 347 Once you've installed the binaries you need to configure your system as
smh22@8242 348 described in Section~\ref{s:configure}.
smh22@8242 349
smh22@8242 350 \section{Installing from RPMs}
smh22@8242 351 Pre-built RPMs are available for download from the XenSource downloads
smh22@8242 352 page:
smh22@8242 353 \begin{quote} {\tt http://www.xensource.com/downloads/}
smh22@8242 354 \end{quote}
smh22@8242 355
smh22@8242 356 Once you've downloaded the RPMs, you typically install them via the
smh22@8242 357 RPM commands:
smh22@8242 358
smh22@8242 359 \verb|# rpm -iv rpmname|
smh22@8242 360
smh22@8242 361 See the instructions and the Release Notes for each RPM set referenced at:
smh22@8242 362 \begin{quote}
smh22@8242 363 {\tt http://www.xensource.com/downloads/}.
smh22@8242 364 \end{quote}
smh22@8242 365
smh22@8242 366 \section{Installing from Source}
smh22@8242 367
smh22@8242 368 This section describes how to obtain, build and install Xen from source.
smh22@8242 369
smh22@8242 370 \subsection{Obtaining the Source}
smh22@8242 371
smh22@8242 372 The Xen source tree is available as either a compressed source tarball
smh22@8242 373 or as a clone of our master Mercurial repository.
smh22@8242 374
smh22@8242 375 \begin{description}
smh22@8242 376 \item[Obtaining the Source Tarball]\mbox{} \\
smh22@8242 377 Stable versions and daily snapshots of the Xen source tree are
smh22@8242 378 available from the Xen download page:
smh22@8242 379 \begin{quote} {\tt \tt http://www.xensource.com/downloads/}
smh22@8242 380 \end{quote}
smh22@8242 381 \item[Obtaining the source via Mercurial]\mbox{} \\
smh22@8242 382 The source tree may also be obtained via the public Mercurial
smh22@8242 383 repository at:
smh22@8242 384 \begin{quote}{\tt http://xenbits.xensource.com}
smh22@8242 385 \end{quote} See the instructions and the Getting Started Guide
smh22@8242 386 referenced at:
smh22@8242 387 \begin{quote}
smh22@8242 388 {\tt http://www.xensource.com/downloads/}
smh22@8242 389 \end{quote}
smh22@8242 390 \end{description}
smh22@8242 391
smh22@8242 392 % \section{The distribution}
smh22@8242 393 %
smh22@8242 394 % The Xen source code repository is structured as follows:
smh22@8242 395 %
smh22@8242 396 % \begin{description}
smh22@8242 397 % \item[\path{tools/}] Xen node controller daemon (Xend), command line
smh22@8242 398 % tools, control libraries
smh22@8242 399 % \item[\path{xen/}] The Xen VMM.
smh22@8242 400 % \item[\path{buildconfigs/}] Build configuration files
smh22@8242 401 % \item[\path{linux-*-xen-sparse/}] Xen support for Linux.
smh22@8242 402 % \item[\path{patches/}] Experimental patches for Linux.
smh22@8242 403 % \item[\path{docs/}] Various documentation files for users and
smh22@8242 404 % developers.
smh22@8242 405 % \item[\path{extras/}] Bonus extras.
smh22@8242 406 % \end{description}
smh22@8242 407
smh22@8242 408 \subsection{Building from Source}
smh22@8242 409
smh22@8242 410 The top-level Xen Makefile includes a target ``world'' that will do the
smh22@8242 411 following:
smh22@8242 412
smh22@8242 413 \begin{itemize}
smh22@8242 414 \item Build Xen.
smh22@8242 415 \item Build the control tools, including \xend.
smh22@8242 416 \item Download (if necessary) and unpack the Linux 2.6 source code, and
smh22@8242 417 patch it for use with Xen.
smh22@8242 418 \item Build a Linux kernel to use in domain~0 and a smaller unprivileged
smh22@8242 419 kernel, which can be used for unprivileged virtual machines.
smh22@8242 420 \end{itemize}
smh22@8242 421
smh22@8242 422 After the build has completed you should have a top-level directory
smh22@8242 423 called \path{dist/} in which all resulting targets will be placed. Of
smh22@8242 424 particular interest are the two XenLinux kernel images, one with a
smh22@8242 425 ``-xen0'' extension which contains hardware device drivers and drivers
smh22@8242 426 for Xen's virtual devices, and one with a ``-xenU'' extension that
smh22@8242 427 just contains the virtual ones. These are found in
smh22@8242 428 \path{dist/install/boot/} along with the image for Xen itself and the
smh22@8242 429 configuration files used during the build.
smh22@8242 430
smh22@8242 431 %The NetBSD port can be built using:
smh22@8242 432 %\begin{quote}
smh22@8242 433 %\begin{verbatim}
smh22@8242 434 %# make netbsd20
smh22@8242 435 %\end{verbatim}\end{quote}
smh22@8242 436 %NetBSD port is built using a snapshot of the netbsd-2-0 cvs branch.
smh22@8242 437 %The snapshot is downloaded as part of the build process if it is not
smh22@8242 438 %yet present in the \path{NETBSD\_SRC\_PATH} search path. The build
smh22@8242 439 %process also downloads a toolchain which includes all of the tools
smh22@8242 440 %necessary to build the NetBSD kernel under Linux.
smh22@8242 441
smh22@8242 442 To customize the set of kernels built you need to edit the top-level
smh22@8242 443 Makefile. Look for the line:
smh22@8242 444 \begin{quote}
smh22@8242 445 \begin{verbatim}
smh22@8242 446 KERNELS ?= linux-2.6-xen0 linux-2.6-xenU
smh22@8242 447 \end{verbatim}
smh22@8242 448 \end{quote}
smh22@8242 449
smh22@8242 450 You can edit this line to include any set of operating system kernels
smh22@8242 451 which have configurations in the top-level \path{buildconfigs/}
smh22@8242 452 directory.
smh22@8242 453
smh22@8242 454 %% Inspect the Makefile if you want to see what goes on during a
smh22@8242 455 %% build. Building Xen and the tools is straightforward, but XenLinux
smh22@8242 456 %% is more complicated. The makefile needs a `pristine' Linux kernel
smh22@8242 457 %% tree to which it will then add the Xen architecture files. You can
smh22@8242 458 %% tell the makefile the location of the appropriate Linux compressed
smh22@8242 459 %% tar file by
smh22@8242 460 %% setting the LINUX\_SRC environment variable, e.g. \\
smh22@8242 461 %% \verb!# LINUX_SRC=/tmp/linux-2.6.11.tar.bz2 make world! \\ or by
smh22@8242 462 %% placing the tar file somewhere in the search path of {\tt
smh22@8242 463 %% LINUX\_SRC\_PATH} which defaults to `{\tt .:..}'. If the
smh22@8242 464 %% makefile can't find a suitable kernel tar file it attempts to
smh22@8242 465 %% download it from kernel.org (this won't work if you're behind a
smh22@8242 466 %% firewall).
smh22@8242 467
smh22@8242 468 %% After untaring the pristine kernel tree, the makefile uses the {\tt
smh22@8242 469 %% mkbuildtree} script to add the Xen patches to the kernel.
smh22@8242 470
smh22@8242 471 %% \framebox{\parbox{5in}{
smh22@8242 472 %% {\bf Distro specific:} \\
smh22@8242 473 %% {\it Gentoo} --- if not using udev (most installations,
smh22@8242 474 %% currently), you'll need to enable devfs and devfs mount at boot
smh22@8242 475 %% time in the xen0 config. }}
smh22@8242 476
smh22@8242 477 \subsection{Custom Kernels}
smh22@8242 478
smh22@8242 479 % If you have an SMP machine you may wish to give the {\tt '-j4'}
smh22@8242 480 % argument to make to get a parallel build.
smh22@8242 481
smh22@8242 482 If you wish to build a customized XenLinux kernel (e.g.\ to support
smh22@8242 483 additional devices or enable distribution-required features), you can
smh22@8242 484 use the standard Linux configuration mechanisms, specifying that the
smh22@8242 485 architecture being built for is \path{xen}, e.g:
smh22@8242 486 \begin{quote}
smh22@8242 487 \begin{verbatim}
smh22@8242 488 # cd linux-2.6.12-xen0
smh22@8242 489 # make ARCH=xen xconfig
smh22@8242 490 # cd ..
smh22@8242 491 # make
smh22@8242 492 \end{verbatim}
smh22@8242 493 \end{quote}
smh22@8242 494
smh22@8242 495 You can also copy an existing Linux configuration (\path{.config}) into
smh22@8242 496 e.g.\ \path{linux-2.6.12-xen0} and execute:
smh22@8242 497 \begin{quote}
smh22@8242 498 \begin{verbatim}
smh22@8242 499 # make ARCH=xen oldconfig
smh22@8242 500 \end{verbatim}
smh22@8242 501 \end{quote}
smh22@8242 502
smh22@8242 503 You may be prompted with some Xen-specific options. We advise accepting
smh22@8242 504 the defaults for these options.
smh22@8242 505
smh22@8242 506 Note that the only difference between the two types of Linux kernels
smh22@8242 507 that are built is the configuration file used for each. The ``U''
smh22@8242 508 suffixed (unprivileged) versions don't contain any of the physical
smh22@8242 509 hardware device drivers, leading to a 30\% reduction in size; hence you
smh22@8242 510 may prefer these for your non-privileged domains. The ``0'' suffixed
smh22@8242 511 privileged versions can be used to boot the system, as well as in driver
smh22@8242 512 domains and unprivileged domains.
smh22@8242 513
smh22@8242 514 \subsection{Installing Generated Binaries}
smh22@8242 515
smh22@8242 516 The files produced by the build process are stored under the
smh22@8242 517 \path{dist/install/} directory. To install them in their default
smh22@8242 518 locations, do:
smh22@8242 519 \begin{quote}
smh22@8242 520 \begin{verbatim}
smh22@8242 521 # make install
smh22@8242 522 \end{verbatim}
smh22@8242 523 \end{quote}
smh22@8242 524
smh22@8242 525 Alternatively, users with special installation requirements may wish to
smh22@8242 526 install them manually by copying the files to their appropriate
smh22@8242 527 destinations.
smh22@8242 528
smh22@8242 529 %% Files in \path{install/boot/} include:
smh22@8242 530 %% \begin{itemize}
smh22@8242 531 %% \item \path{install/boot/xen-3.0.gz} Link to the Xen 'kernel'
smh22@8242 532 %% \item \path{install/boot/vmlinuz-2.6-xen0} Link to domain 0
smh22@8242 533 %% XenLinux kernel
smh22@8242 534 %% \item \path{install/boot/vmlinuz-2.6-xenU} Link to unprivileged
smh22@8242 535 %% XenLinux kernel
smh22@8242 536 %% \end{itemize}
smh22@8242 537
smh22@8242 538 The \path{dist/install/boot} directory will also contain the config
smh22@8242 539 files used for building the XenLinux kernels, and also versions of Xen
smh22@8242 540 and XenLinux kernels that contain debug symbols such as
smh22@8242 541 (\path{xen-syms-3.0.0} and \path{vmlinux-syms-}) which are
smh22@8242 542 essential for interpreting crash dumps. Retain these files as the
smh22@8242 543 developers may wish to see them if you post on the mailing list.
smh22@8242 544
smh22@8242 545
smh22@8242 546 \section{Configuration}
smh22@8242 547 \label{s:configure}
smh22@8242 548
smh22@8242 549 Once you have built and installed the Xen distribution, it is simple to
smh22@8242 550 prepare the machine for booting and running Xen.
smh22@8242 551
smh22@8242 552 \subsection{GRUB Configuration}
smh22@8242 553
smh22@8242 554 An entry should be added to \path{grub.conf} (often found under
smh22@8242 555 \path{/boot/} or \path{/boot/grub/}) to allow Xen / XenLinux to boot.
smh22@8242 556 This file is sometimes called \path{menu.lst}, depending on your
smh22@8242 557 distribution. The entry should look something like the following:
smh22@8242 558
smh22@8242 559 %% KMSelf Thu Dec 1 19:06:13 PST 2005 262144 is useful for RHEL/RH and
smh22@8242 560 %% related Dom0s.
smh22@8242 561 {\small
smh22@8242 562 \begin{verbatim}
smh22@8242 563 title Xen 3.0 / XenLinux 2.6
smh22@8242 564 kernel /boot/xen-3.0.gz dom0_mem=262144
smh22@8242 565 module /boot/vmlinuz-2.6-xen0 root=/dev/sda4 ro console=tty0
smh22@8242 566 \end{verbatim}
smh22@8242 567 }
smh22@8242 568
smh22@8242 569 The kernel line tells GRUB where to find Xen itself and what boot
smh22@8242 570 parameters should be passed to it (in this case, setting the domain~0
smh22@8242 571 memory allocation in kilobytes and the settings for the serial port).
smh22@8242 572 For more details on the various Xen boot parameters see
smh22@8242 573 Section~\ref{s:xboot}.
smh22@8242 574
smh22@8242 575 The module line of the configuration describes the location of the
smh22@8242 576 XenLinux kernel that Xen should start and the parameters that should be
smh22@8242 577 passed to it. These are standard Linux parameters, identifying the root
smh22@8242 578 device and specifying it be initially mounted read only and instructing
smh22@8242 579 that console output be sent to the screen. Some distributions such as
smh22@8242 580 SuSE do not require the \path{ro} parameter.
smh22@8242 581
smh22@8242 582 %% \framebox{\parbox{5in}{
smh22@8242 583 %% {\bf Distro specific:} \\
smh22@8242 584 %% {\it SuSE} --- Omit the {\tt ro} option from the XenLinux
smh22@8242 585 %% kernel command line, since the partition won't be remounted rw
smh22@8242 586 %% during boot. }}
smh22@8242 587
smh22@8242 588 To use an initrd, add another \path{module} line to the configuration,
smh22@8242 589 like: {\small
smh22@8242 590 \begin{verbatim}
smh22@8242 591 module /boot/my_initrd.gz
smh22@8242 592 \end{verbatim}
smh22@8242 593 }
smh22@8242 594
smh22@8242 595 %% KMSelf Thu Dec 1 19:05:30 PST 2005 Other configs as an appendix?
smh22@8242 596
smh22@8242 597 When installing a new kernel, it is recommended that you do not delete
smh22@8242 598 existing menu options from \path{menu.lst}, as you may wish to boot your
smh22@8242 599 old Linux kernel in future, particularly if you have problems.
smh22@8242 600
smh22@8242 601 \subsection{Serial Console (optional)}
smh22@8242 602
smh22@8242 603 Serial console access allows you to manage, monitor, and interact with
smh22@8242 604 your system over a serial console. This can allow access from another
smh22@8242 605 nearby system via a null-modem (``LapLink'') cable or remotely via a serial
smh22@8242 606 concentrator.
smh22@8242 607
smh22@8242 608 You system's BIOS, bootloader (GRUB), Xen, Linux, and login access must
smh22@8242 609 each be individually configured for serial console access. It is
smh22@8242 610 \emph{not} strictly necessary to have each component fully functional,
smh22@8242 611 but it can be quite useful.
smh22@8242 612
smh22@8242 613 For general information on serial console configuration under Linux,
smh22@8242 614 refer to the ``Remote Serial Console HOWTO'' at The Linux Documentation
smh22@8242 615 Project: \url{http://www.tldp.org}
smh22@8242 616
smh22@8242 617 \subsubsection{Serial Console BIOS configuration}
smh22@8242 618
smh22@8242 619 Enabling system serial console output neither enables nor disables
smh22@8242 620 serial capabilities in GRUB, Xen, or Linux, but may make remote
smh22@8242 621 management of your system more convenient by displaying POST and other
smh22@8242 622 boot messages over serial port and allowing remote BIOS configuration.
smh22@8242 623
smh22@8242 624 Refer to your hardware vendor's documentation for capabilities and
smh22@8242 625 procedures to enable BIOS serial redirection.
smh22@8242 626
smh22@8242 627
smh22@8242 628 \subsubsection{Serial Console GRUB configuration}
smh22@8242 629
smh22@8242 630 Enabling GRUB serial console output neither enables nor disables Xen or
smh22@8242 631 Linux serial capabilities, but may made remote management of your system
smh22@8242 632 more convenient by displaying GRUB prompts, menus, and actions over
smh22@8242 633 serial port and allowing remote GRUB management.
smh22@8242 634
smh22@8242 635 Adding the following two lines to your GRUB configuration file,
smh22@8242 636 typically either \path{/boot/grub/menu.lst} or \path{/boot/grub/grub.conf}
smh22@8242 637 depending on your distro, will enable GRUB serial output.
smh22@8242 638
smh22@8242 639 \begin{quote}
smh22@8242 640 {\small \begin{verbatim}
smh22@8242 641 serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
smh22@8242 642 terminal --timeout=10 serial console
smh22@8242 643 \end{verbatim}}
smh22@8242 644 \end{quote}
smh22@8242 645
smh22@8242 646 Note that when both the serial port and the local monitor and keyboard
smh22@8242 647 are enabled, the text ``\emph{Press any key to continue}'' will appear
smh22@8242 648 at both. Pressing a key on one device will cause GRUB to display to
smh22@8242 649 that device. The other device will see no output. If no key is
smh22@8242 650 pressed before the timeout period expires, the system will boot to the
smh22@8242 651 default GRUB boot entry.
smh22@8242 652
smh22@8242 653 Please refer to the GRUB documentation for further information.
smh22@8242 654
smh22@8242 655
smh22@8242 656 \subsubsection{Serial Console Xen configuration}
smh22@8242 657
smh22@8242 658 Enabling Xen serial console output neither enables nor disables Linux
smh22@8242 659 kernel output or logging in to Linux over serial port. It does however
smh22@8242 660 allow you to monitor and log the Xen boot process via serial console and
smh22@8242 661 can be very useful in debugging.
smh22@8242 662
kaf24@9027 663 %% kernel /boot/xen-2.0.gz dom0_mem=131072 console=com1,vga com1=115200,8n1
smh22@8242 664 %% module /boot/vmlinuz-2.6-xen0 root=/dev/sda4 ro
smh22@8242 665
smh22@8242 666 In order to configure Xen serial console output, it is necessary to
smh22@8242 667 add a boot option to your GRUB config; e.g.\ replace the previous
smh22@8242 668 example kernel line with:
smh22@8242 669 \begin{quote} {\small \begin{verbatim}
smh22@8242 670 kernel /boot/xen.gz dom0_mem=131072 com1=115200,8n1
smh22@8242 671 \end{verbatim}}
smh22@8242 672 \end{quote}
smh22@8242 673
kaf24@9027 674 This configures Xen to output on COM1 at 115,200 baud, 8 data bits, no
kaf24@9027 675 parity and 1 stop bit. Modify these parameters for your environment.
kaf24@9027 676 See Section~\ref{s:xboot} for an explanation of all boot parameters.
smh22@8242 677
smh22@8242 678 One can also configure XenLinux to share the serial console; to achieve
smh22@8242 679 this append ``\path{console=ttyS0}'' to your module line.
smh22@8242 680
smh22@8242 681
smh22@8242 682 \subsubsection{Serial Console Linux configuration}
smh22@8242 683
smh22@8242 684 Enabling Linux serial console output at boot neither enables nor
smh22@8242 685 disables logging in to Linux over serial port. It does however allow
smh22@8242 686 you to monitor and log the Linux boot process via serial console and can be
smh22@8242 687 very useful in debugging.
smh22@8242 688
smh22@8242 689 To enable Linux output at boot time, add the parameter
smh22@8242 690 \path{console=ttyS0} (or ttyS1, ttyS2, etc.) to your kernel GRUB line.
smh22@8242 691 Under Xen, this might be:
smh22@8242 692 \begin{quote}
smh22@8242 693 {\footnotesize \begin{verbatim}
smh22@8242 694 module /vmlinuz-2.6-xen0 ro root=/dev/VolGroup00/LogVol00 \
smh22@8242 695 console=ttyS0, 115200
smh22@8242 696 \end{verbatim}}
smh22@8242 697 \end{quote}
smh22@8242 698 to enable output over ttyS0 at 115200 baud.
smh22@8242 699
smh22@8242 700
smh22@8242 701
smh22@8242 702 \subsubsection{Serial Console Login configuration}
smh22@8242 703
smh22@8242 704 Logging in to Linux via serial console, under Xen or otherwise, requires
smh22@8242 705 specifying a login prompt be started on the serial port. To permit root
smh22@8242 706 logins over serial console, the serial port must be added to
smh22@8242 707 \path{/etc/securetty}.
smh22@8242 708
smh22@8242 709 \newpage
smh22@8242 710 To automatically start a login prompt over the serial port,
smh22@8242 711 add the line: \begin{quote} {\small {\tt c:2345:respawn:/sbin/mingetty
smh22@8242 712 ttyS0}} \end{quote} to \path{/etc/inittab}. Run \path{init q} to force
smh22@8242 713 a reload of your inttab and start getty.
smh22@8242 714
smh22@8242 715 To enable root logins, add \path{ttyS0} to \path{/etc/securetty} if not
smh22@8242 716 already present.
smh22@8242 717
smh22@8242 718 Your distribution may use an alternate getty; options include getty,
smh22@8242 719 mgetty and agetty. Consult your distribution's documentation
smh22@8242 720 for further information.
smh22@8242 721
smh22@8242 722
smh22@8242 723 \subsection{TLS Libraries}
smh22@8242 724
smh22@8242 725 Users of the XenLinux 2.6 kernel should disable Thread Local Storage
smh22@8242 726 (TLS) (e.g.\ by doing a \path{mv /lib/tls /lib/tls.disabled}) before
smh22@8242 727 attempting to boot a XenLinux kernel\footnote{If you boot without first
smh22@8242 728 disabling TLS, you will get a warning message during the boot process.
smh22@8242 729 In this case, simply perform the rename after the machine is up and
smh22@8242 730 then run \path{/sbin/ldconfig} to make it take effect.}. You can
smh22@8242 731 always reenable TLS by restoring the directory to its original location
smh22@8242 732 (i.e.\ \path{mv /lib/tls.disabled /lib/tls}).
smh22@8242 733
smh22@8242 734 The reason for this is that the current TLS implementation uses
smh22@8242 735 segmentation in a way that is not permissible under Xen. If TLS is not
smh22@8242 736 disabled, an emulation mode is used within Xen which reduces performance
smh22@8242 737 substantially. To ensure full performance you should install a
smh22@8242 738 `Xen-friendly' (nosegneg) version of the library.
smh22@8242 739
smh22@8242 740
smh22@8242 741 \section{Booting Xen}
smh22@8242 742
smh22@8242 743 It should now be possible to restart the system and use Xen. Reboot and
smh22@8242 744 choose the new Xen option when the Grub screen appears.
smh22@8242 745
smh22@8242 746 What follows should look much like a conventional Linux boot. The first
smh22@8242 747 portion of the output comes from Xen itself, supplying low level
smh22@8242 748 information about itself and the underlying hardware. The last portion
smh22@8242 749 of the output comes from XenLinux.
smh22@8242 750
smh22@8242 751 You may see some error messages during the XenLinux boot. These are not
smh22@8242 752 necessarily anything to worry about---they may result from kernel
smh22@8242 753 configuration differences between your XenLinux kernel and the one you
smh22@8242 754 usually use.
smh22@8242 755
smh22@8242 756 When the boot completes, you should be able to log into your system as
smh22@8242 757 usual. If you are unable to log in, you should still be able to reboot
smh22@8242 758 with your normal Linux kernel by selecting it at the GRUB prompt.
smh22@8242 759
FMJ@8226 760
FMJ@8226 761 % Booting Xen
smh22@8242 762 \chapter{Booting a Xen System}
smh22@8242 763
smh22@8242 764 Booting the system into Xen will bring you up into the privileged
smh22@8242 765 management domain, Domain0. At that point you are ready to create
smh22@8242 766 guest domains and ``boot'' them using the \texttt{xm create} command.
smh22@8242 767
smh22@8242 768 \section{Booting Domain0}
smh22@8242 769
smh22@8242 770 After installation and configuration is complete, reboot the system
smh22@8242 771 and and choose the new Xen option when the Grub screen appears.
smh22@8242 772
smh22@8242 773 What follows should look much like a conventional Linux boot. The
smh22@8242 774 first portion of the output comes from Xen itself, supplying low level
smh22@8242 775 information about itself and the underlying hardware. The last
smh22@8242 776 portion of the output comes from XenLinux.
smh22@8242 777
smh22@8242 778 %% KMSelf Wed Nov 30 18:09:37 PST 2005: We should specify what these are.
smh22@8242 779
smh22@8242 780 When the boot completes, you should be able to log into your system as
smh22@8242 781 usual. If you are unable to log in, you should still be able to
smh22@8242 782 reboot with your normal Linux kernel by selecting it at the GRUB prompt.
smh22@8242 783
smh22@8242 784 The first step in creating a new domain is to prepare a root
smh22@8242 785 filesystem for it to boot. Typically, this might be stored in a normal
smh22@8242 786 partition, an LVM or other volume manager partition, a disk file or on
smh22@8242 787 an NFS server. A simple way to do this is simply to boot from your
smh22@8242 788 standard OS install CD and install the distribution into another
smh22@8242 789 partition on your hard drive.
smh22@8242 790
smh22@8242 791 To start the \xend\ control daemon, type
smh22@8242 792 \begin{quote}
smh22@8242 793 \verb!# xend start!
smh22@8242 794 \end{quote}
smh22@8242 795
smh22@8242 796 If you wish the daemon to start automatically, see the instructions in
smh22@8242 797 Section~\ref{s:xend}. Once the daemon is running, you can use the
smh22@8242 798 \path{xm} tool to monitor and maintain the domains running on your
smh22@8242 799 system. This chapter provides only a brief tutorial. We provide full
smh22@8242 800 details of the \path{xm} tool in the next chapter.
smh22@8242 801
smh22@8242 802 % \section{From the web interface}
smh22@8242 803 %
smh22@8242 804 % Boot the Xen machine and start Xensv (see Chapter~\ref{cha:xensv}
smh22@8242 805 % for more details) using the command: \\
smh22@8242 806 % \verb_# xensv start_ \\
smh22@8242 807 % This will also start Xend (see Chapter~\ref{cha:xend} for more
smh22@8242 808 % information).
smh22@8242 809 %
smh22@8242 810 % The domain management interface will then be available at {\tt
smh22@8242 811 % http://your\_machine:8080/}. This provides a user friendly wizard
smh22@8242 812 % for starting domains and functions for managing running domains.
smh22@8242 813 %
smh22@8242 814 % \section{From the command line}
smh22@8242 815 \section{Booting Guest Domains}
smh22@8242 816
smh22@8242 817 \subsection{Creating a Domain Configuration File}
smh22@8242 818
smh22@8242 819 Before you can start an additional domain, you must create a
smh22@8242 820 configuration file. We provide two example files which you can use as
smh22@8242 821 a starting point:
smh22@8242 822 \begin{itemize}
smh22@8242 823 \item \path{/etc/xen/xmexample1} is a simple template configuration
smh22@8242 824 file for describing a single VM\@.
smh22@8242 825 \item \path{/etc/xen/xmexample2} file is a template description that
smh22@8242 826 is intended to be reused for multiple virtual machines. Setting the
smh22@8242 827 value of the \path{vmid} variable on the \path{xm} command line
smh22@8242 828 fills in parts of this template.
smh22@8242 829 \end{itemize}
smh22@8242 830
smh22@8242 831 There are also a number of other examples which you may find useful.
smh22@8242 832 Copy one of these files and edit it as appropriate. Typical values
smh22@8242 833 you may wish to edit include:
smh22@8242 834
smh22@8242 835 \begin{quote}
smh22@8242 836 \begin{description}
smh22@8242 837 \item[kernel] Set this to the path of the kernel you compiled for use
smh22@8242 838 with Xen (e.g.\ \path{kernel = ``/boot/vmlinuz-2.6-xenU''})
smh22@8242 839 \item[memory] Set this to the size of the domain's memory in megabytes
smh22@8242 840 (e.g.\ \path{memory = 64})
smh22@8242 841 \item[disk] Set the first entry in this list to calculate the offset
smh22@8242 842 of the domain's root partition, based on the domain ID\@. Set the
smh22@8242 843 second to the location of \path{/usr} if you are sharing it between
smh22@8242 844 domains (e.g.\ \path{disk = ['phy:your\_hard\_drive\%d,sda1,w' \%
smh22@8242 845 (base\_partition\_number + vmid),
smh22@8242 846 'phy:your\_usr\_partition,sda6,r' ]}
smh22@8242 847 \item[dhcp] Uncomment the dhcp variable, so that the domain will
smh22@8242 848 receive its IP address from a DHCP server (e.g.\ \path{dhcp=``dhcp''})
smh22@8242 849 \end{description}
smh22@8242 850 \end{quote}
smh22@8242 851
smh22@8242 852 You may also want to edit the {\bf vif} variable in order to choose
smh22@8242 853 the MAC address of the virtual ethernet interface yourself. For
smh22@8242 854 example:
smh22@8242 855
smh22@8242 856 \begin{quote}
smh22@8242 857 \verb_vif = ['mac=00:16:3E:F6:BB:B3']_
smh22@8242 858 \end{quote}
smh22@8242 859 If you do not set this variable, \xend\ will automatically generate a
smh22@8242 860 random MAC address from the range 00:16:3E:xx:xx:xx, assigned by IEEE to
smh22@8242 861 XenSource as an OUI (organizationally unique identifier). XenSource
smh22@8242 862 Inc. gives permission for anyone to use addresses randomly allocated
smh22@8242 863 from this range for use by their Xen domains.
smh22@8242 864
smh22@8242 865 For a list of IEEE OUI assignments, see
smh22@8242 866 \url{http://standards.ieee.org/regauth/oui/oui.txt}
smh22@8242 867
smh22@8242 868
smh22@8242 869 \subsection{Booting the Guest Domain}
smh22@8242 870
smh22@8242 871 The \path{xm} tool provides a variety of commands for managing
smh22@8242 872 domains. Use the \path{create} command to start new domains. Assuming
smh22@8242 873 you've created a configuration file \path{myvmconf} based around
smh22@8242 874 \path{/etc/xen/xmexample2}, to start a domain with virtual machine
smh22@8242 875 ID~1 you should type:
smh22@8242 876
smh22@8242 877 \begin{quote}
smh22@8242 878 \begin{verbatim}
smh22@8242 879 # xm create -c myvmconf vmid=1
smh22@8242 880 \end{verbatim}
smh22@8242 881 \end{quote}
smh22@8242 882
smh22@8242 883 The \path{-c} switch causes \path{xm} to turn into the domain's
smh22@8242 884 console after creation. The \path{vmid=1} sets the \path{vmid}
smh22@8242 885 variable used in the \path{myvmconf} file.
smh22@8242 886
smh22@8242 887 You should see the console boot messages from the new domain appearing
smh22@8242 888 in the terminal in which you typed the command, culminating in a login
smh22@8242 889 prompt.
smh22@8242 890
smh22@8242 891
smh22@8242 892 \section{Starting / Stopping Domains Automatically}
smh22@8242 893
smh22@8242 894 It is possible to have certain domains start automatically at boot
smh22@8242 895 time and to have dom0 wait for all running domains to shutdown before
smh22@8242 896 it shuts down the system.
smh22@8242 897
smh22@8242 898 To specify a domain is to start at boot-time, place its configuration
smh22@8242 899 file (or a link to it) under \path{/etc/xen/auto/}.
smh22@8242 900
smh22@8242 901 A Sys-V style init script for Red Hat and LSB-compliant systems is
smh22@8242 902 provided and will be automatically copied to \path{/etc/init.d/}
smh22@8242 903 during install. You can then enable it in the appropriate way for
smh22@8242 904 your distribution.
smh22@8242 905
smh22@8242 906 For instance, on Red Hat:
smh22@8242 907
smh22@8242 908 \begin{quote}
smh22@8242 909 \verb_# chkconfig --add xendomains_
smh22@8242 910 \end{quote}
smh22@8242 911
smh22@8242 912 By default, this will start the boot-time domains in runlevels 3, 4
smh22@8242 913 and 5.
smh22@8242 914
smh22@8242 915 You can also use the \path{service} command to run this script
smh22@8242 916 manually, e.g:
smh22@8242 917
smh22@8242 918 \begin{quote}
smh22@8242 919 \verb_# service xendomains start_
smh22@8242 920
smh22@8242 921 Starts all the domains with config files under /etc/xen/auto/.
smh22@8242 922 \end{quote}
smh22@8242 923
smh22@8242 924 \begin{quote}
smh22@8242 925 \verb_# service xendomains stop_
smh22@8242 926
smh22@8242 927 Shuts down all running Xen domains.
smh22@8242 928 \end{quote}
smh22@8242 929
FMJ@8225 930
FMJ@8225 931
FMJ@8225 932 \part{Configuration and Management}
FMJ@8225 933
FMJ@8226 934 %% Chapter Domain Management Tools and Daemons
smh22@8242 935 \chapter{Domain Management Tools}
FMJ@8226 936
smh22@8242 937 This chapter summarizes the management software and tools available.
FMJ@7772 938
kaf24@2743 939
smh22@8242 940 \section{\Xend\ }
smh22@8242 941 \label{s:xend}
FMJ@8226 942
smh22@2882 943
smh22@8273 944 The \Xend\ node control daemon performs system management functions
smh22@8273 945 related to virtual machines. It forms a central point of control of
smh22@8273 946 virtualized resources, and must be running in order to start and manage
smh22@8273 947 virtual machines. \Xend\ must be run as root because it needs access to
smh22@8273 948 privileged system management functions.
smh22@8273 949
smh22@8273 950 An initialization script named \texttt{/etc/init.d/xend} is provided to
smh22@8273 951 start \Xend\ at boot time. Use the tool appropriate (i.e. chkconfig) for
smh22@8273 952 your Linux distribution to specify the runlevels at which this script
smh22@8273 953 should be executed, or manually create symbolic links in the correct
smh22@8273 954 runlevel directories.
smh22@8273 955
smh22@8273 956 \Xend\ can be started on the command line as well, and supports the
smh22@8273 957 following set of parameters:
FMJ@8226 958
smh22@8242 959 \begin{tabular}{ll}
smh22@8242 960 \verb!# xend start! & start \xend, if not already running \\
smh22@8242 961 \verb!# xend stop! & stop \xend\ if already running \\
smh22@8242 962 \verb!# xend restart! & restart \xend\ if running, otherwise start it \\
smh22@8242 963 % \verb!# xend trace_start! & start \xend, with very detailed debug logging \\
smh22@8242 964 \verb!# xend status! & indicates \xend\ status by its return code
smh22@8242 965 \end{tabular}
FMJ@8225 966
smh22@8242 967 A SysV init script called {\tt xend} is provided to start \xend\ at
smh22@8242 968 boot time. {\tt make install} installs this script in
smh22@8242 969 \path{/etc/init.d}. To enable it, you have to make symbolic links in
smh22@8242 970 the appropriate runlevel directories or use the {\tt chkconfig} tool,
smh22@8242 971 where available. Once \xend\ is running, administration can be done
smh22@8242 972 using the \texttt{xm} tool.
smh22@8242 973
smh22@8273 974 \subsection{Logging}
smh22@8273 975
smh22@8273 976 As \xend\ runs, events will be logged to \path{/var/log/xend.log} and
smh22@8273 977 (less frequently) to \path{/var/log/xend-debug.log}. These, along with
smh22@8273 978 the standard syslog files, are useful when troubleshooting problems.
smh22@8273 979
smh22@8273 980 \subsection{Configuring \Xend\ }
smh22@8273 981
smh22@8273 982 \Xend\ is written in Python. At startup, it reads its configuration
smh22@8273 983 information from the file \path{/etc/xen/xend-config.sxp}. The Xen
smh22@8273 984 installation places an example \texttt{xend-config.sxp} file in the
smh22@8273 985 \texttt{/etc/xen} subdirectory which should work for most installations.
smh22@8273 986
smh22@8273 987 See the example configuration file \texttt{xend-debug.sxp} and the
smh22@8273 988 section 5 man page \texttt{xend-config.sxp} for a full list of
smh22@8273 989 parameters and more detailed information. Some of the most important
smh22@8273 990 parameters are discussed below.
smh22@8273 991
smh22@8273 992 An HTTP interface and a Unix domain socket API are available to
smh22@8273 993 communicate with \Xend. This allows remote users to pass commands to the
smh22@8273 994 daemon. By default, \Xend does not start an HTTP server. It does start a
smh22@8273 995 Unix domain socket management server, as the low level utility
smh22@8273 996 \texttt{xm} requires it. For support of cross-machine migration, \Xend\
smh22@8273 997 can start a relocation server. This support is not enabled by default
smh22@8273 998 for security reasons.
smh22@8273 999
smh22@8273 1000 Note: the example \texttt{xend} configuration file modifies the defaults and
smh22@8273 1001 starts up \Xend\ as an HTTP server as well as a relocation server.
smh22@8273 1002
smh22@8273 1003 From the file:
smh22@8273 1004
smh22@8273 1005 \begin{verbatim}
smh22@8273 1006 #(xend-http-server no)
smh22@8273 1007 (xend-http-server yes)
smh22@8273 1008 #(xend-unix-server yes)
smh22@8273 1009 #(xend-relocation-server no)
smh22@8273 1010 (xend-relocation-server yes)
smh22@8273 1011 \end{verbatim}
smh22@8273 1012
smh22@8273 1013 Comment or uncomment lines in that file to disable or enable features
smh22@8273 1014 that you require.
smh22@8273 1015
smh22@8273 1016 Connections from remote hosts are disabled by default:
smh22@8273 1017
smh22@8273 1018 \begin{verbatim}
smh22@8273 1019 # Address xend should listen on for HTTP connections, if xend-http-server is
smh22@8273 1020 # set.
smh22@8273 1021 # Specifying 'localhost' prevents remote connections.
smh22@8273 1022 # Specifying the empty string '' (the default) allows all connections.
smh22@8273 1023 #(xend-address '')
smh22@8273 1024 (xend-address localhost)
smh22@8273 1025 \end{verbatim}
smh22@8273 1026
smh22@8273 1027 It is recommended that if migration support is not needed, the
smh22@8273 1028 \texttt{xend-relocation-server} parameter value be changed to
smh22@8273 1029 ``\texttt{no}'' or commented out.
smh22@8242 1030
smh22@8242 1031 \section{Xm}
smh22@8242 1032 \label{s:xm}
smh22@8242 1033
smh22@8242 1034 The xm tool is the primary tool for managing Xen from the console. The
smh22@8242 1035 general format of an xm command line is:
smh22@8242 1036
smh22@8242 1037 \begin{verbatim}
smh22@8242 1038 # xm command [switches] [arguments] [variables]
smh22@8242 1039 \end{verbatim}
smh22@8242 1040
smh22@8242 1041 The available \emph{switches} and \emph{arguments} are dependent on the
smh22@8242 1042 \emph{command} chosen. The \emph{variables} may be set using
smh22@8242 1043 declarations of the form {\tt variable=value} and command line
smh22@8242 1044 declarations override any of the values in the configuration file being
smh22@8242 1045 used, including the standard variables described above and any custom
smh22@8242 1046 variables (for instance, the \path{xmdefconfig} file uses a {\tt vmid}
smh22@8242 1047 variable).
smh22@8242 1048
emellor@8244 1049 For online help for the commands available, type:
emellor@8244 1050
emellor@8244 1051 \begin{quote}
emellor@8244 1052 \begin{verbatim}
emellor@8244 1053 # xm help
emellor@8244 1054 \end{verbatim}
emellor@8244 1055 \end{quote}
emellor@8244 1056
emellor@8244 1057 This will list the most commonly used commands. The full list can be obtained
emellor@8244 1058 using \verb_xm help --long_. You can also type \path{xm help $<$command$>$}
emellor@8244 1059 for more information on a given command.
emellor@8244 1060
smh22@8242 1061 \subsection{Basic Management Commands}
smh22@8242 1062
emellor@8244 1063 One useful command is \verb_# xm list_ which lists all domains running in rows
emellor@8244 1064 of the following format:
smh22@8242 1065 \begin{center} {\tt name domid memory vcpus state cputime}
smh22@8242 1066 \end{center}
smh22@8242 1067
smh22@8242 1068 The meaning of each field is as follows:
smh22@8242 1069 \begin{quote}
smh22@8242 1070 \begin{description}
smh22@8242 1071 \item[name] The descriptive name of the virtual machine.
smh22@8242 1072 \item[domid] The number of the domain ID this virtual machine is
smh22@8242 1073 running in.
smh22@8242 1074 \item[memory] Memory size in megabytes.
smh22@8242 1075 \item[vcpus] The number of virtual CPUs this domain has.
smh22@8242 1076 \item[state] Domain state consists of 5 fields:
smh22@8242 1077 \begin{description}
smh22@8242 1078 \item[r] running
smh22@8242 1079 \item[b] blocked
smh22@8242 1080 \item[p] paused
smh22@8242 1081 \item[s] shutdown
smh22@8242 1082 \item[c] crashed
smh22@8242 1083 \end{description}
smh22@8242 1084 \item[cputime] How much CPU time (in seconds) the domain has used so
smh22@8242 1085 far.
smh22@8242 1086 \end{description}
smh22@8242 1087 \end{quote}
smh22@8242 1088
smh22@8242 1089 The \path{xm list} command also supports a long output format when the
smh22@8273 1090 \path{-l} switch is used. This outputs the full details of the
smh22@8242 1091 running domains in \xend's SXP configuration format.
kaf24@2743 1092
kaf24@6977 1093
smh22@8242 1094 You can get access to the console of a particular domain using
smh22@8242 1095 the \verb_# xm console_ command (e.g.\ \verb_# xm console myVM_).
kaf24@2743 1096
smh22@8242 1097
smh22@8242 1098
smh22@8242 1099 %% Chapter Domain Configuration
smh22@8242 1100 \chapter{Domain Configuration}
smh22@8242 1101 \label{cha:config}
smh22@8242 1102
smh22@8242 1103 The following contains the syntax of the domain configuration files
smh22@8242 1104 and description of how to further specify networking, driver domain
smh22@8242 1105 and general scheduling behavior.
smh22@8242 1106
smh22@8242 1107
smh22@8242 1108 \section{Configuration Files}
smh22@8242 1109 \label{s:cfiles}
smh22@8242 1110
smh22@8242 1111 Xen configuration files contain the following standard variables.
smh22@8242 1112 Unless otherwise stated, configuration items should be enclosed in
smh22@8242 1113 quotes: see the configuration scripts in \path{/etc/xen/}
smh22@8242 1114 for concrete examples.
smh22@8242 1115
smh22@8242 1116 \begin{description}
smh22@8242 1117 \item[kernel] Path to the kernel image.
smh22@8242 1118 \item[ramdisk] Path to a ramdisk image (optional).
smh22@8242 1119 % \item[builder] The name of the domain build function (e.g.
smh22@8242 1120 % {\tt'linux'} or {\tt'netbsd'}.
smh22@8242 1121 \item[memory] Memory size in megabytes.
smh22@8242 1122 \item[vcpus] The number of virtual CPUs.
smh22@8242 1123 \item[console] Port to export the domain console on (default 9600 +
smh22@8242 1124 domain ID).
emellor@8330 1125 \item[vif] Network interface configuration. This may simply contain
emellor@8330 1126 an empty string for each desired interface, or may override various
emellor@8330 1127 settings, e.g.\
smh22@8242 1128 \begin{verbatim}
emellor@8244 1129 vif = [ 'mac=00:16:3E:00:00:11, bridge=xen-br0',
smh22@8242 1130 'bridge=xen-br1' ]
smh22@8242 1131 \end{verbatim}
smh22@8242 1132 to assign a MAC address and bridge to the first interface and assign
smh22@8242 1133 a different bridge to the second interface, leaving \xend\ to choose
emellor@8330 1134 the MAC address. The settings that may be overridden in this way are
emellor@8330 1135 type, mac, bridge, ip, script, backend, and vifname.
smh22@8242 1136 \item[disk] List of block devices to export to the domain e.g.
smh22@8242 1137 \verb_disk = [ 'phy:hda1,sda1,r' ]_
smh22@8242 1138 exports physical device \path{/dev/hda1} to the domain as
smh22@8242 1139 \path{/dev/sda1} with read-only access. Exporting a disk read-write
smh22@8242 1140 which is currently mounted is dangerous -- if you are \emph{certain}
smh22@8242 1141 you wish to do this, you can specify \path{w!} as the mode.
smh22@8242 1142 \item[dhcp] Set to {\tt `dhcp'} if you want to use DHCP to configure
smh22@8242 1143 networking.
smh22@8242 1144 \item[netmask] Manually configured IP netmask.
smh22@8242 1145 \item[gateway] Manually configured IP gateway.
smh22@8242 1146 \item[hostname] Set the hostname for the virtual machine.
smh22@8242 1147 \item[root] Specify the root device parameter on the kernel command
smh22@8242 1148 line.
smh22@8242 1149 \item[nfs\_server] IP address for the NFS server (if any).
smh22@8242 1150 \item[nfs\_root] Path of the root filesystem on the NFS server (if
smh22@8242 1151 any).
smh22@8242 1152 \item[extra] Extra string to append to the kernel command line (if
smh22@8242 1153 any)
smh22@8242 1154 \end{description}
smh22@8242 1155
smh22@8242 1156 Additional fields are documented in the example configuration files
smh22@8242 1157 (e.g. to configure virtual TPM functionality).
smh22@8242 1158
smh22@8242 1159 For additional flexibility, it is also possible to include Python
smh22@8242 1160 scripting commands in configuration files. An example of this is the
smh22@8242 1161 \path{xmexample2} file, which uses Python code to handle the
smh22@8242 1162 \path{vmid} variable.
smh22@8242 1163
smh22@8242 1164
smh22@8242 1165 %\part{Advanced Topics}
smh22@8242 1166
smh22@8242 1167
smh22@8242 1168 \section{Network Configuration}
smh22@8242 1169
smh22@8242 1170 For many users, the default installation should work ``out of the
smh22@8242 1171 box''. More complicated network setups, for instance with multiple
smh22@8242 1172 Ethernet interfaces and/or existing bridging setups will require some
smh22@8242 1173 special configuration.
smh22@8242 1174
smh22@8242 1175 The purpose of this section is to describe the mechanisms provided by
smh22@8242 1176 \xend\ to allow a flexible configuration for Xen's virtual networking.
smh22@8242 1177
smh22@8242 1178 \subsection{Xen virtual network topology}
smh22@8242 1179
smh22@8242 1180 Each domain network interface is connected to a virtual network
smh22@8242 1181 interface in dom0 by a point to point link (effectively a ``virtual
smh22@8242 1182 crossover cable''). These devices are named {\tt
smh22@8242 1183 vif$<$domid$>$.$<$vifid$>$} (e.g.\ {\tt vif1.0} for the first
smh22@8242 1184 interface in domain~1, {\tt vif3.1} for the second interface in
smh22@8242 1185 domain~3).
smh22@8242 1186
smh22@8242 1187 Traffic on these virtual interfaces is handled in domain~0 using
smh22@8242 1188 standard Linux mechanisms for bridging, routing, rate limiting, etc.
smh22@8242 1189 Xend calls on two shell scripts to perform initial configuration of
smh22@8242 1190 the network and configuration of new virtual interfaces. By default,
smh22@8242 1191 these scripts configure a single bridge for all the virtual
smh22@8242 1192 interfaces. Arbitrary routing / bridging configurations can be
smh22@8242 1193 configured by customizing the scripts, as described in the following
smh22@8242 1194 section.
smh22@8242 1195
smh22@8242 1196 \subsection{Xen networking scripts}
smh22@8242 1197
smh22@8242 1198 Xen's virtual networking is configured by two shell scripts (by
emellor@8244 1199 default \path{network-bridge} and \path{vif-bridge}). These are called
smh22@8242 1200 automatically by \xend\ when certain events occur, with arguments to
smh22@8242 1201 the scripts providing further contextual information. These scripts
smh22@8242 1202 are found by default in \path{/etc/xen/scripts}. The names and
smh22@8242 1203 locations of the scripts can be configured in
smh22@8242 1204 \path{/etc/xen/xend-config.sxp}.
smh22@8242 1205
smh22@8242 1206 \begin{description}
emellor@8244 1207 \item[network-bridge:] This script is called whenever \xend\ is started or
smh22@8242 1208 stopped to respectively initialize or tear down the Xen virtual
smh22@8242 1209 network. In the default configuration initialization creates the
smh22@8242 1210 bridge `xen-br0' and moves eth0 onto that bridge, modifying the
smh22@8242 1211 routing accordingly. When \xend\ exits, it deletes the Xen bridge
smh22@8242 1212 and removes eth0, restoring the normal IP and routing configuration.
smh22@8242 1213
smh22@8242 1214 %% In configurations where the bridge already exists, this script
smh22@8242 1215 %% could be replaced with a link to \path{/bin/true} (for instance).
smh22@8242 1216
smh22@8242 1217 \item[vif-bridge:] This script is called for every domain virtual
smh22@8242 1218 interface and can configure firewalling rules and add the vif to the
smh22@8242 1219 appropriate bridge. By default, this adds and removes VIFs on the
smh22@8242 1220 default Xen bridge.
smh22@8242 1221 \end{description}
smh22@8242 1222
emellor@8244 1223 Other example scripts are available (\path{network-route} and
emellor@8244 1224 \path{vif-route}, \path{network-nat} and \path{vif-nat}).
smh22@8242 1225 For more complex network setups (e.g.\ where routing is required or
smh22@8242 1226 integrate with existing bridges) these scripts may be replaced with
smh22@8242 1227 customized variants for your site's preferred configuration.
smh22@8242 1228
kaf24@8881 1229 \section{Driver Domain Configuration}
kaf24@8881 1230 \label{s:ddconf}
kaf24@8881 1231
kaf24@8881 1232 \subsection{PCI}
kaf24@8881 1233 \label{ss:pcidd}
kaf24@8881 1234
kaf24@9732 1235 Individual PCI devices can be assigned to a given domain (a PCI driver domain)
kaf24@9732 1236 to allow that domain direct access to the PCI hardware.
kaf24@9732 1237
kaf24@9732 1238 While PCI Driver Domains can increase the stability and security of a system
kaf24@9732 1239 by addressing a number of security concerns, there are some security issues
kaf24@9732 1240 that remain that you can read about in Section~\ref{s:ddsecurity}.
kaf24@9732 1241
kaf24@9732 1242 \subsubsection{Compile-Time Setup}
kaf24@9732 1243 To use this functionality, ensure
kaf24@8881 1244 that the PCI Backend is compiled in to a privileged domain (e.g. domain 0)
kaf24@8881 1245 and that the domains which will be assigned PCI devices have the PCI Frontend
kaf24@8881 1246 compiled in. In XenLinux, the PCI Backend is available under the Xen
kaf24@8881 1247 configuration section while the PCI Frontend is under the
kaf24@8881 1248 architecture-specific "Bus Options" section. You may compile both the backend
kaf24@8881 1249 and the frontend into the same kernel; they will not affect each other.
kaf24@8881 1250
kaf24@9732 1251 \subsubsection{PCI Backend Configuration - Binding at Boot}
kaf24@8881 1252 The PCI devices you wish to assign to unprivileged domains must be "hidden"
kaf24@8881 1253 from your backend domain (usually domain 0) so that it does not load a driver
kaf24@8881 1254 for them. Use the \path{pciback.hide} kernel parameter which is specified on
kaf24@8881 1255 the kernel command-line and is configurable through GRUB (see
kaf24@8881 1256 Section~\ref{s:configure}). Note that devices are not really hidden from the
kaf24@9732 1257 backend domain. The PCI Backend appears to the Linux kernel as a regular PCI
kaf24@9732 1258 device driver. The PCI Backend ensures that no other device driver loads
kaf24@9732 1259 for the devices by binding itself as the device driver for those devices.
kaf24@9732 1260 PCI devices are identified by hexadecimal slot/funciton numbers (on Linux,
kaf24@9732 1261 use \path{lspci} to determine slot/funciton numbers of your devices) and
kaf24@9732 1262 can be specified with or without the PCI domain: \\
kaf24@8881 1263 \centerline{ {\tt ({\em bus}:{\em slot}.{\em func})} example {\tt (02:1d.3)}} \\
kaf24@8881 1264 \centerline{ {\tt ({\em domain}:{\em bus}:{\em slot}.{\em func})} example {\tt (0000:02:1d.3)}} \\
kaf24@8881 1265
kaf24@8881 1266 An example kernel command-line which hides two PCI devices might be: \\
kaf24@8881 1267 \centerline{ {\tt root=/dev/sda4 ro console=tty0 pciback.hide=(02:01.f)(0000:04:1d.0) } } \\
kaf24@8881 1268
kaf24@9732 1269 \subsubsection{PCI Backend Configuration - Late Binding}
kaf24@9732 1270 PCI devices can also be bound to the PCI Backend after boot through the manual
kaf24@9732 1271 binding/unbinding facilities provided by the Linux kernel in sysfs (allowing
kaf24@9732 1272 for a Xen user to give PCI devices to driver domains that were not specified
kaf24@9732 1273 on the kernel command-line). There are several attributes with the PCI
kaf24@9732 1274 Backend's sysfs directory (\path{/sys/bus/pci/drivers/pciback}) that can be
kaf24@9732 1275 used to bind/unbind devices:
kaf24@9732 1276
kaf24@9732 1277 \begin{description}
kaf24@9732 1278 \item[slots] lists all of the PCI slots that the PCI Backend will try to seize
kaf24@9732 1279 (or "hide" from Domain 0). A PCI slot must appear in this list before it can
kaf24@9732 1280 be bound to the PCI Backend through the \path{bind} attribute.
kaf24@9732 1281 \item[new\_slot] write the name of a slot here (in 0000:00:00.0 format) to
kaf24@9732 1282 have the PCI Backend seize the device in this slot.
kaf24@9732 1283 \item[remove\_slot] write the name of a slot here (same format as
kaf24@9732 1284 \path{new\_slot}) to have the PCI Backend no longer try to seize devices in
kaf24@9732 1285 this slot. Note that this does not unbind the driver from a device it has
kaf24@9732 1286 already seized.
kaf24@9732 1287 \item[bind] write the name of a slot here (in 0000:00:00.0 format) to have
kaf24@9732 1288 the Linux kernel attempt to bind the device in that slot to the PCI Backend
kaf24@9732 1289 driver.
kaf24@9732 1290 \item[unbind] write the name of a skit here (same format as \path{bind}) to have
kaf24@9732 1291 the Linux kernel unbind the device from the PCI Backend. DO NOT unbind a
kaf24@9732 1292 device while it is currently given to a PCI driver domain!
kaf24@9732 1293 \end{description}
kaf24@9732 1294
kaf24@9732 1295 Some examples:
kaf24@9732 1296
kaf24@9732 1297 Bind a device to the PCI Backend which is not bound to any other driver.
kaf24@9732 1298 \begin{verbatim}
kaf24@9732 1299 # # Add a new slot to the PCI Backend's list
kaf24@9732 1300 # echo -n 0000:01:04.d > /sys/bus/pci/drivers/pciback/new_slot
kaf24@9732 1301 # # Now that the backend is watching for the slot, bind to it
kaf24@9732 1302 # echo -n 0000:01:04.d > /sys/bus/pci/drivers/pciback/bind
kaf24@9732 1303 \end{verbatim}
kaf24@9732 1304
kaf24@9732 1305 Unbind a device from its driver and bind to the PCI Backend.
kaf24@9732 1306 \begin{verbatim}
kaf24@9732 1307 # # Unbind a PCI network card from its network driver
kaf24@9732 1308 # echo -n 0000:05:02.0 > /sys/bus/pci/drivers/3c905/unbind
kaf24@9732 1309 # # And now bind it to the PCI Backend
kaf24@9732 1310 # echo -n 0000:05:02.0 > /sys/bus/pci/drivers/pciback/new_slot
kaf24@9732 1311 # echo -n 0000:05:02.0 > /sys/bus/pci/drivers/pciback/bind
kaf24@9732 1312 \end{verbatim}
kaf24@9732 1313
kaf24@9732 1314 Note that the "-n" option in the example is important as it causes echo to not
kaf24@9732 1315 output a new-line.
kaf24@9732 1316
kaf24@9732 1317 \subsubsection{PCI Frontend Configuration}
kaf24@8881 1318 To configure a domU to receive a PCI device:
kaf24@8881 1319
kaf24@8881 1320 \begin{description}
kaf24@8881 1321 \item[Command-line:]
kaf24@8881 1322 Use the {\em pci} command-line flag. For multiple devices, use the option
kaf24@8881 1323 multiple times. \\
kaf24@8881 1324 \centerline{ {\tt xm create netcard-dd pci=01:00.0 pci=02:03.0 }} \\
kaf24@8881 1325
kaf24@8881 1326 \item[Flat Format configuration file:]
kaf24@8881 1327 Specify all of your PCI devices in a python list named {\em pci}. \\
kaf24@8881 1328 \centerline{ {\tt pci=['01:00.0','02:03.0'] }} \\
kaf24@8881 1329
kaf24@8881 1330 \item[SXP Format configuration file:]
kaf24@8881 1331 Use a single PCI device section for all of your devices (specify the numbers
kaf24@8881 1332 in hexadecimal with the preceding '0x'). Note that {\em domain} here refers
kaf24@8881 1333 to the PCI domain, not a virtual machine within Xen.
kaf24@8881 1334 {\small
kaf24@8881 1335 \begin{verbatim}
kaf24@8881 1336 (device (pci
kaf24@8881 1337 (dev (domain 0x0)(bus 0x3)(slot 0x1a)(func 0x1)
kaf24@8881 1338 (dev (domain 0x0)(bus 0x1)(slot 0x5)(func 0x0)
kaf24@8881 1339 )
kaf24@8881 1340 \end{verbatim}
kaf24@8881 1341 }
kaf24@8881 1342 \end{description}
kaf24@8881 1343
smh22@8242 1344 %% There are two possible types of privileges: IO privileges and
smh22@8242 1345 %% administration privileges.
smh22@8242 1346
smh22@8242 1347
smh22@8242 1348
smh22@8242 1349
smh22@8242 1350 % Chapter Storage and FileSytem Management
smh22@8242 1351 \chapter{Storage and File System Management}
smh22@8242 1352
smh22@8242 1353 Storage can be made available to virtual machines in a number of
smh22@8242 1354 different ways. This chapter covers some possible configurations.
smh22@8242 1355
smh22@8242 1356 The most straightforward method is to export a physical block device (a
smh22@8242 1357 hard drive or partition) from dom0 directly to the guest domain as a
smh22@8242 1358 virtual block device (VBD).
smh22@8242 1359
smh22@8242 1360 Storage may also be exported from a filesystem image or a partitioned
smh22@8242 1361 filesystem image as a \emph{file-backed VBD}.
smh22@8242 1362
smh22@8242 1363 Finally, standard network storage protocols such as NBD, iSCSI, NFS,
smh22@8242 1364 etc., can be used to provide storage to virtual machines.
smh22@8242 1365
smh22@8242 1366
smh22@8242 1367 \section{Exporting Physical Devices as VBDs}
smh22@8242 1368 \label{s:exporting-physical-devices-as-vbds}
smh22@8242 1369
smh22@8242 1370 One of the simplest configurations is to directly export individual
smh22@8242 1371 partitions from domain~0 to other domains. To achieve this use the
smh22@8242 1372 \path{phy:} specifier in your domain configuration file. For example a
smh22@8242 1373 line like
smh22@8242 1374 \begin{quote}
smh22@8242 1375 \verb_disk = ['phy:hda3,sda1,w']_
smh22@8242 1376 \end{quote}
smh22@8242 1377 specifies that the partition \path{/dev/hda3} in domain~0 should be
smh22@8242 1378 exported read-write to the new domain as \path{/dev/sda1}; one could
smh22@8242 1379 equally well export it as \path{/dev/hda} or \path{/dev/sdb5} should
smh22@8242 1380 one wish.
smh22@8242 1381
smh22@8242 1382 In addition to local disks and partitions, it is possible to export
smh22@8242 1383 any device that Linux considers to be ``a disk'' in the same manner.
smh22@8242 1384 For example, if you have iSCSI disks or GNBD volumes imported into
smh22@8242 1385 domain~0 you can export these to other domains using the \path{phy:}
smh22@8242 1386 disk syntax. E.g.:
smh22@8242 1387 \begin{quote}
smh22@8242 1388 \verb_disk = ['phy:vg/lvm1,sda2,w']_
smh22@8242 1389 \end{quote}
smh22@8242 1390
smh22@8242 1391 \begin{center}
smh22@8242 1392 \framebox{\bf Warning: Block device sharing}
smh22@8242 1393 \end{center}
smh22@8242 1394 \begin{quote}
smh22@8242 1395 Block devices should typically only be shared between domains in a
smh22@8242 1396 read-only fashion otherwise the Linux kernel's file systems will get
smh22@8242 1397 very confused as the file system structure may change underneath
smh22@8242 1398 them (having the same ext3 partition mounted \path{rw} twice is a
smh22@8242 1399 sure fire way to cause irreparable damage)! \Xend\ will attempt to
smh22@8242 1400 prevent you from doing this by checking that the device is not
smh22@8242 1401 mounted read-write in domain~0, and hasn't already been exported
smh22@8242 1402 read-write to another domain. If you want read-write sharing,
smh22@8242 1403 export the directory to other domains via NFS from domain~0 (or use
smh22@8242 1404 a cluster file system such as GFS or ocfs2).
smh22@8242 1405 \end{quote}
smh22@8242 1406
smh22@8242 1407
smh22@8242 1408 \section{Using File-backed VBDs}
smh22@8242 1409
smh22@8242 1410 It is also possible to use a file in Domain~0 as the primary storage
smh22@8242 1411 for a virtual machine. As well as being convenient, this also has the
smh22@8242 1412 advantage that the virtual block device will be \emph{sparse} ---
smh22@8242 1413 space will only really be allocated as parts of the file are used. So
smh22@8242 1414 if a virtual machine uses only half of its disk space then the file
smh22@8242 1415 really takes up half of the size allocated.
smh22@8242 1416
smh22@8242 1417 For example, to create a 2GB sparse file-backed virtual block device
smh22@8242 1418 (actually only consumes 1KB of disk):
smh22@8242 1419 \begin{quote}
smh22@8242 1420 \verb_# dd if=/dev/zero of=vm1disk bs=1k seek=2048k count=1_
smh22@8242 1421 \end{quote}
smh22@8242 1422
smh22@8242 1423 Make a file system in the disk file:
smh22@8242 1424 \begin{quote}
smh22@8242 1425 \verb_# mkfs -t ext3 vm1disk_
smh22@8242 1426 \end{quote}
smh22@8242 1427
smh22@8242 1428 (when the tool asks for confirmation, answer `y')
smh22@8242 1429
smh22@8242 1430 Populate the file system e.g.\ by copying from the current root:
smh22@8242 1431 \begin{quote}
smh22@8242 1432 \begin{verbatim}
smh22@8242 1433 # mount -o loop vm1disk /mnt
smh22@8242 1434 # cp -ax /{root,dev,var,etc,usr,bin,sbin,lib} /mnt
smh22@8242 1435 # mkdir /mnt/{proc,sys,home,tmp}
smh22@8242 1436 \end{verbatim}
smh22@8242 1437 \end{quote}
smh22@8242 1438
smh22@8242 1439 Tailor the file system by editing \path{/etc/fstab},
smh22@8242 1440 \path{/etc/hostname}, etc.\ Don't forget to edit the files in the
smh22@8242 1441 mounted file system, instead of your domain~0 filesystem, e.g.\ you
smh22@8242 1442 would edit \path{/mnt/etc/fstab} instead of \path{/etc/fstab}. For
smh22@8242 1443 this example put \path{/dev/sda1} to root in fstab.
smh22@8242 1444
smh22@8242 1445 Now unmount (this is important!):
smh22@8242 1446 \begin{quote}
smh22@8242 1447 \verb_# umount /mnt_
smh22@8242 1448 \end{quote}
smh22@8242 1449
smh22@8242 1450 In the configuration file set:
smh22@8242 1451 \begin{quote}
smh22@8242 1452 \verb_disk = ['file:/full/path/to/vm1disk,sda1,w']_
smh22@8242 1453 \end{quote}
smh22@8242 1454
smh22@8242 1455 As the virtual machine writes to its `disk', the sparse file will be
smh22@8242 1456 filled in and consume more space up to the original 2GB.
smh22@8242 1457
smh22@8242 1458 {\bf Note that file-backed VBDs may not be appropriate for backing
smh22@8242 1459 I/O-intensive domains.} File-backed VBDs are known to experience
smh22@8242 1460 substantial slowdowns under heavy I/O workloads, due to the I/O
smh22@8242 1461 handling by the loopback block device used to support file-backed VBDs
smh22@8242 1462 in dom0. Better I/O performance can be achieved by using either
smh22@8242 1463 LVM-backed VBDs (Section~\ref{s:using-lvm-backed-vbds}) or physical
smh22@8242 1464 devices as VBDs (Section~\ref{s:exporting-physical-devices-as-vbds}).
smh22@8242 1465
smh22@8242 1466 Linux supports a maximum of eight file-backed VBDs across all domains
smh22@8242 1467 by default. This limit can be statically increased by using the
smh22@8242 1468 \emph{max\_loop} module parameter if CONFIG\_BLK\_DEV\_LOOP is
smh22@8242 1469 compiled as a module in the dom0 kernel, or by using the
smh22@8242 1470 \emph{max\_loop=n} boot option if CONFIG\_BLK\_DEV\_LOOP is compiled
smh22@8242 1471 directly into the dom0 kernel.
smh22@8242 1472
smh22@8242 1473
smh22@8242 1474 \section{Using LVM-backed VBDs}
smh22@8242 1475 \label{s:using-lvm-backed-vbds}
smh22@8242 1476
smh22@8242 1477 A particularly appealing solution is to use LVM volumes as backing for
smh22@8242 1478 domain file-systems since this allows dynamic growing/shrinking of
smh22@8242 1479 volumes as well as snapshot and other features.
smh22@8242 1480
smh22@8242 1481 To initialize a partition to support LVM volumes:
smh22@8242 1482 \begin{quote}
smh22@8242 1483 \begin{verbatim}
smh22@8242 1484 # pvcreate /dev/sda10
smh22@8242 1485 \end{verbatim}
smh22@8242 1486 \end{quote}
smh22@8242 1487
smh22@8242 1488 Create a volume group named `vg' on the physical partition:
smh22@8242 1489 \begin{quote}
smh22@8242 1490 \begin{verbatim}
smh22@8242 1491 # vgcreate vg /dev/sda10
smh22@8242 1492 \end{verbatim}
smh22@8242 1493 \end{quote}
smh22@8242 1494
smh22@8242 1495 Create a logical volume of size 4GB named `myvmdisk1':
smh22@8242 1496 \begin{quote}
smh22@8242 1497 \begin{verbatim}
smh22@8242 1498 # lvcreate -L4096M -n myvmdisk1 vg
smh22@8242 1499 \end{verbatim}
smh22@8242 1500 \end{quote}
smh22@8242 1501
smh22@8242 1502 You should now see that you have a \path{/dev/vg/myvmdisk1} Make a
smh22@8242 1503 filesystem, mount it and populate it, e.g.:
smh22@8242 1504 \begin{quote}
smh22@8242 1505 \begin{verbatim}
smh22@8242 1506 # mkfs -t ext3 /dev/vg/myvmdisk1
smh22@8242 1507 # mount /dev/vg/myvmdisk1 /mnt
smh22@8242 1508 # cp -ax / /mnt
smh22@8242 1509 # umount /mnt
smh22@8242 1510 \end{verbatim}
smh22@8242 1511 \end{quote}
smh22@8242 1512
smh22@8242 1513 Now configure your VM with the following disk configuration:
smh22@8242 1514 \begin{quote}
smh22@8242 1515 \begin{verbatim}
smh22@8242 1516 disk = [ 'phy:vg/myvmdisk1,sda1,w' ]
smh22@8242 1517 \end{verbatim}
smh22@8242 1518 \end{quote}
smh22@8242 1519
smh22@8242 1520 LVM enables you to grow the size of logical volumes, but you'll need
smh22@8242 1521 to resize the corresponding file system to make use of the new space.
smh22@8242 1522 Some file systems (e.g.\ ext3) now support online resize. See the LVM
smh22@8242 1523 manuals for more details.
smh22@8242 1524
smh22@8242 1525 You can also use LVM for creating copy-on-write (CoW) clones of LVM
smh22@8242 1526 volumes (known as writable persistent snapshots in LVM terminology).
smh22@8242 1527 This facility is new in Linux 2.6.8, so isn't as stable as one might
smh22@8242 1528 hope. In particular, using lots of CoW LVM disks consumes a lot of
smh22@8242 1529 dom0 memory, and error conditions such as running out of disk space
smh22@8242 1530 are not handled well. Hopefully this will improve in future.
smh22@8242 1531
emellor@8244 1532 To create two copy-on-write clones of the above file system you would
smh22@8242 1533 use the following commands:
smh22@8242 1534
smh22@8242 1535 \begin{quote}
smh22@8242 1536 \begin{verbatim}
smh22@8242 1537 # lvcreate -s -L1024M -n myclonedisk1 /dev/vg/myvmdisk1
smh22@8242 1538 # lvcreate -s -L1024M -n myclonedisk2 /dev/vg/myvmdisk1
smh22@8242 1539 \end{verbatim}
smh22@8242 1540 \end{quote}
smh22@8242 1541
smh22@8242 1542 Each of these can grow to have 1GB of differences from the master
smh22@8242 1543 volume. You can grow the amount of space for storing the differences
smh22@8242 1544 using the lvextend command, e.g.:
smh22@8242 1545 \begin{quote}
smh22@8242 1546 \begin{verbatim}
smh22@8242 1547 # lvextend +100M /dev/vg/myclonedisk1
smh22@8242 1548 \end{verbatim}
smh22@8242 1549 \end{quote}
smh22@8242 1550
smh22@8242 1551 Don't let the `differences volume' ever fill up otherwise LVM gets
smh22@8242 1552 rather confused. It may be possible to automate the growing process by
smh22@8242 1553 using \path{dmsetup wait} to spot the volume getting full and then
smh22@8242 1554 issue an \path{lvextend}.
smh22@8242 1555
smh22@8242 1556 In principle, it is possible to continue writing to the volume that
smh22@8242 1557 has been cloned (the changes will not be visible to the clones), but
smh22@8242 1558 we wouldn't recommend this: have the cloned volume as a `pristine'
smh22@8242 1559 file system install that isn't mounted directly by any of the virtual
smh22@8242 1560 machines.
smh22@8242 1561
smh22@8242 1562
smh22@8242 1563 \section{Using NFS Root}
smh22@8242 1564
smh22@8242 1565 First, populate a root filesystem in a directory on the server
smh22@8242 1566 machine. This can be on a distinct physical machine, or simply run
smh22@8242 1567 within a virtual machine on the same node.
smh22@8242 1568
smh22@8242 1569 Now configure the NFS server to export this filesystem over the
smh22@8242 1570 network by adding a line to \path{/etc/exports}, for instance:
smh22@8242 1571
smh22@8242 1572 \begin{quote}
smh22@8242 1573 \begin{small}
smh22@8242 1574 \begin{verbatim}
smh22@8242 1575 /export/vm1root (rw,sync,no_root_squash)
smh22@8242 1576 \end{verbatim}
smh22@8242 1577 \end{small}
smh22@8242 1578 \end{quote}
smh22@8242 1579
smh22@8242 1580 Finally, configure the domain to use NFS root. In addition to the
smh22@8242 1581 normal variables, you should make sure to set the following values in
smh22@8242 1582 the domain's configuration file:
smh22@8242 1583
smh22@8242 1584 \begin{quote}
smh22@8242 1585 \begin{small}
smh22@8242 1586 \begin{verbatim}
smh22@8242 1587 root = '/dev/nfs'
smh22@8242 1588 nfs_server = '' # substitute IP address of server
smh22@8242 1589 nfs_root = '/path/to/root' # path to root FS on the server
smh22@8242 1590 \end{verbatim}
smh22@8242 1591 \end{small}
smh22@8242 1592 \end{quote}
smh22@8242 1593
smh22@8242 1594 The domain will need network access at boot time, so either statically
smh22@8242 1595 configure an IP address using the config variables \path{ip},
smh22@8242 1596 \path{netmask}, \path{gateway}, \path{hostname}; or enable DHCP
smh22@8242 1597 (\path{dhcp='dhcp'}).
smh22@8242 1598
smh22@8242 1599 Note that the Linux NFS root implementation is known to have stability
smh22@8242 1600 problems under high load (this is not a Xen-specific problem), so this
smh22@8242 1601 configuration may not be appropriate for critical servers.
smh22@8242 1602
smh22@8242 1603
smh22@8242 1604 \chapter{CPU Management}
smh22@8242 1605
smh22@8242 1606 %% KMS Something sage about CPU / processor management.
smh22@8242 1607
smh22@8242 1608 Xen allows a domain's virtual CPU(s) to be associated with one or more
smh22@8242 1609 host CPUs. This can be used to allocate real resources among one or
smh22@8242 1610 more guests, or to make optimal use of processor resources when
smh22@8242 1611 utilizing dual-core, hyperthreading, or other advanced CPU technologies.
smh22@8242 1612
smh22@8242 1613 Xen enumerates physical CPUs in a `depth first' fashion. For a system
smh22@8242 1614 with both hyperthreading and multiple cores, this would be all the
smh22@8242 1615 hyperthreads on a given core, then all the cores on a given socket,
smh22@8242 1616 and then all sockets. I.e. if you had a two socket, dual core,
smh22@8242 1617 hyperthreaded Xeon the CPU order would be:
smh22@8242 1618
smh22@8242 1619
smh22@8242 1620 \begin{center}
smh22@8242 1621 \begin{tabular}{l|l|l|l|l|l|l|r}
smh22@8242 1622 \multicolumn{4}{c|}{socket0} & \multicolumn{4}{c}{socket1} \\ \hline
smh22@8242 1623 \multicolumn{2}{c|}{core0} & \multicolumn{2}{c|}{core1} &
smh22@8242 1624 \multicolumn{2}{c|}{core0} & \multicolumn{2}{c}{core1} \\ \hline
smh22@8242 1625 ht0 & ht1 & ht0 & ht1 & ht0 & ht1 & ht0 & ht1 \\
smh22@8242 1626 \#0 & \#1 & \#2 & \#3 & \#4 & \#5 & \#6 & \#7 \\
smh22@8242 1627 \end{tabular}
smh22@8242 1628 \end{center}
smh22@8242 1629
smh22@8242 1630
smh22@8242 1631 Having multiple vcpus belonging to the same domain mapped to the same
smh22@8242 1632 physical CPU is very likely to lead to poor performance. It's better to
smh22@8242 1633 use `vcpus-set' to hot-unplug one of the vcpus and ensure the others are
smh22@8242 1634 pinned on different CPUs.
smh22@8242 1635
smh22@8242 1636 If you are running IO intensive tasks, its typically better to dedicate
smh22@8242 1637 either a hyperthread or whole core to running domain 0, and hence pin
smh22@8242 1638 other domains so that they can't use CPU 0. If your workload is mostly
smh22@8242 1639 compute intensive, you may want to pin vcpus such that all physical CPU
smh22@8242 1640 threads are available for guest domains.
smh22@8242 1641
smh22@8242 1642 \chapter{Migrating Domains}
smh22@8242 1643
smh22@8242 1644 \section{Domain Save and Restore}
smh22@8242 1645
smh22@8242 1646 The administrator of a Xen system may suspend a virtual machine's
smh22@8242 1647 current state into a disk file in domain~0, allowing it to be resumed at
smh22@8242 1648 a later time.
smh22@8242 1649
smh22@8242 1650 For example you can suspend a domain called ``VM1'' to disk using the
smh22@8242 1651 command:
smh22@8242 1652 \begin{verbatim}
smh22@8242 1653 # xm save VM1 VM1.chk
smh22@8242 1654 \end{verbatim}
smh22@8242 1655
smh22@8242 1656 This will stop the domain named ``VM1'' and save its current state
smh22@8242 1657 into a file called \path{VM1.chk}.
smh22@8242 1658
smh22@8242 1659 To resume execution of this domain, use the \path{xm restore} command:
smh22@8242 1660 \begin{verbatim}
smh22@8242 1661 # xm restore VM1.chk
smh22@8242 1662 \end{verbatim}
smh22@8242 1663
smh22@8242 1664 This will restore the state of the domain and resume its execution.
smh22@8242 1665 The domain will carry on as before and the console may be reconnected
smh22@8242 1666 using the \path{xm console} command, as described earlier.
smh22@8242 1667
smh22@8242 1668 \section{Migration and Live Migration}
smh22@8242 1669
smh22@8242 1670 Migration is used to transfer a domain between physical hosts. There
smh22@8242 1671 are two varieties: regular and live migration. The former moves a
smh22@8242 1672 virtual machine from one host to another by pausing it, copying its
smh22@8242 1673 memory contents, and then resuming it on the destination. The latter
smh22@8242 1674 performs the same logical functionality but without needing to pause
smh22@8242 1675 the domain for the duration. In general when performing live migration
smh22@8242 1676 the domain continues its usual activities and---from the user's
smh22@8242 1677 perspective---the migration should be imperceptible.
smh22@8242 1678
smh22@8242 1679 To perform a live migration, both hosts must be running Xen / \xend\ and
smh22@8242 1680 the destination host must have sufficient resources (e.g.\ memory
smh22@8242 1681 capacity) to accommodate the domain after the move. Furthermore we
smh22@8242 1682 currently require both source and destination machines to be on the same
smh22@8242 1683 L2 subnet.
smh22@8242 1684
smh22@8242 1685 Currently, there is no support for providing automatic remote access
smh22@8242 1686 to filesystems stored on local disk when a domain is migrated.
smh22@8242 1687 Administrators should choose an appropriate storage solution (i.e.\
smh22@8242 1688 SAN, NAS, etc.) to ensure that domain filesystems are also available
smh22@8242 1689 on their destination node. GNBD is a good method for exporting a
smh22@8242 1690 volume from one machine to another. iSCSI can do a similar job, but is
smh22@8242 1691 more complex to set up.
smh22@8242 1692
smh22@8242 1693 When a domain migrates, it's MAC and IP address move with it, thus it is
smh22@8242 1694 only possible to migrate VMs within the same layer-2 network and IP
smh22@8242 1695 subnet. If the destination node is on a different subnet, the
smh22@8242 1696 administrator would need to manually configure a suitable etherip or IP
smh22@8242 1697 tunnel in the domain~0 of the remote node.
smh22@8242 1698
smh22@8242 1699 A domain may be migrated using the \path{xm migrate} command. To live
smh22@8242 1700 migrate a domain to another machine, we would use the command:
smh22@8242 1701
smh22@8242 1702 \begin{verbatim}
smh22@8242 1703 # xm migrate --live mydomain destination.ournetwork.com
smh22@8242 1704 \end{verbatim}
smh22@8242 1705
smh22@8242 1706 Without the \path{--live} flag, \xend\ simply stops the domain and
smh22@8242 1707 copies the memory image over to the new node and restarts it. Since
smh22@8242 1708 domains can have large allocations this can be quite time consuming,
smh22@8242 1709 even on a Gigabit network. With the \path{--live} flag \xend\ attempts
smh22@8242 1710 to keep the domain running while the migration is in progress, resulting
smh22@8242 1711 in typical down times of just 60--300ms.
smh22@8242 1712
smh22@8242 1713 For now it will be necessary to reconnect to the domain's console on the
smh22@8242 1714 new machine using the \path{xm console} command. If a migrated domain
smh22@8242 1715 has any open network connections then they will be preserved, so SSH
smh22@8242 1716 connections do not have this limitation.
smh22@8242 1717
smh22@8242 1718
smh22@8242 1719 %% Chapter Securing Xen
smh22@8242 1720 \chapter{Securing Xen}
smh22@8242 1721
smh22@8242 1722 This chapter describes how to secure a Xen system. It describes a number
smh22@8242 1723 of scenarios and provides a corresponding set of best practices. It
smh22@8242 1724 begins with a section devoted to understanding the security implications
smh22@8242 1725 of a Xen system.
smh22@8242 1726
smh22@8242 1727
smh22@8242 1728 \section{Xen Security Considerations}
smh22@8242 1729
smh22@8242 1730 When deploying a Xen system, one must be sure to secure the management
smh22@8242 1731 domain (Domain-0) as much as possible. If the management domain is
smh22@8242 1732 compromised, all other domains are also vulnerable. The following are a
smh22@8242 1733 set of best practices for Domain-0:
smh22@8242 1734
smh22@8242 1735 \begin{enumerate}
smh22@8242 1736 \item \textbf{Run the smallest number of necessary services.} The less
smh22@8242 1737 things that are present in a management partition, the better.
smh22@8242 1738 Remember, a service running as root in the management domain has full
smh22@8242 1739 access to all other domains on the system.
smh22@8242 1740 \item \textbf{Use a firewall to restrict the traffic to the management
smh22@8242 1741 domain.} A firewall with default-reject rules will help prevent
smh22@8242 1742 attacks on the management domain.
smh22@8242 1743 \item \textbf{Do not allow users to access Domain-0.} The Linux kernel
smh22@8242 1744 has been known to have local-user root exploits. If you allow normal
smh22@8242 1745 users to access Domain-0 (even as unprivileged users) you run the risk
smh22@8242 1746 of a kernel exploit making all of your domains vulnerable.
smh22@8242 1747 \end{enumerate}
smh22@8242 1748
kaf24@8881 1749 \section{Driver Domain Security Considerations}
kaf24@8881 1750 \label{s:ddsecurity}
kaf24@8881 1751
kaf24@8881 1752 Driver domains address a range of security problems that exist regarding
kaf24@8881 1753 the use of device drivers and hardware. On many operating systems in common
kaf24@8881 1754 use today, device drivers run within the kernel with the same privileges as
kaf24@8881 1755 the kernel. Few or no mechanisms exist to protect the integrity of the kernel
kaf24@8881 1756 from a misbehaving (read "buggy") or malicious device driver. Driver
kaf24@8881 1757 domains exist to aid in isolating a device driver within its own virtual
kaf24@8881 1758 machine where it cannot affect the stability and integrity of other
kaf24@8881 1759 domains. If a driver crashes, the driver domain can be restarted rather than
kaf24@8881 1760 have the entire machine crash (and restart) with it. Drivers written by
kaf24@8881 1761 unknown or untrusted third-parties can be confined to an isolated space.
kaf24@8881 1762 Driver domains thus address a number of security and stability issues with
kaf24@8881 1763 device drivers.
kaf24@8881 1764
kaf24@8881 1765 However, due to limitations in current hardware, a number of security
kaf24@8881 1766 concerns remain that need to be considered when setting up driver domains (it
kaf24@8881 1767 should be noted that the following list is not intended to be exhaustive).
kaf24@8881 1768
kaf24@8881 1769 \begin{enumerate}
kaf24@8881 1770 \item \textbf{Without an IOMMU, a hardware device can DMA to memory regions
kaf24@8881 1771 outside of its controlling domain.} Architectures which do not have an
kaf24@8881 1772 IOMMU (e.g. most x86-based platforms) to restrict DMA usage by hardware
kaf24@8881 1773 are vulnerable. A hardware device which can perform arbitrary memory reads
kaf24@8881 1774 and writes can read/write outside of the memory of its controlling domain.
kaf24@8881 1775 A malicious or misbehaving domain could use a hardware device it controls
kaf24@8881 1776 to send data overwriting memory in another domain or to read arbitrary
kaf24@8881 1777 regions of memory in another domain.
kaf24@8881 1778 \item \textbf{Shared buses are vulnerable to sniffing.} Devices that share
kaf24@8881 1779 a data bus can sniff (and possible spoof) each others' data. Device A that
kaf24@8881 1780 is assigned to Domain A could eavesdrop on data being transmitted by
kaf24@8881 1781 Domain B to Device B and then relay that data back to Domain A.
kaf24@8881 1782 \item \textbf{Devices which share interrupt lines can either prevent the
kaf24@8881 1783 reception of that interrupt by the driver domain or can trigger the
kaf24@8881 1784 interrupt service routine of that guest needlessly.} A devices which shares
kaf24@8881 1785 a level-triggered interrupt (e.g. PCI devices) with another device can
kaf24@8881 1786 raise an interrupt and never clear it. This effectively blocks other devices
kaf24@8881 1787 which share that interrupt line from notifying their controlling driver
kaf24@8881 1788 domains that they need to be serviced. A device which shares an
kaf24@8881 1789 any type of interrupt line can trigger its interrupt continually which
kaf24@8881 1790 forces execution time to be spent (in multiple guests) in the interrupt
kaf24@8881 1791 service routine (potentially denying time to other processes within that
kaf24@8881 1792 guest). System architectures which allow each device to have its own
kaf24@8881 1793 interrupt line (e.g. PCI's Message Signaled Interrupts) are less
kaf24@8881 1794 vulnerable to this denial-of-service problem.
kaf24@8881 1795 \item \textbf{Devices may share the use of I/O memory address space.} Xen can
kaf24@8881 1796 only restrict access to a device's physical I/O resources at a certain
kaf24@8881 1797 granularity. For interrupt lines and I/O port address space, that
kaf24@8881 1798 granularity is very fine (per interrupt line and per I/O port). However,
kaf24@8881 1799 Xen can only restrict access to I/O memory address space on a page size
kaf24@8881 1800 basis. If more than one device shares use of a page in I/O memory address
kaf24@8881 1801 space, the domains to which those devices are assigned will be able to
kaf24@8881 1802 access the I/O memory address space of each other's devices.
kaf24@8881 1803 \end{enumerate}
kaf24@8881 1804
kaf24@8881 1805
smh22@8242 1806 \section{Security Scenarios}
smh22@8242 1807
smh22@8242 1808
smh22@8242 1809 \subsection{The Isolated Management Network}
smh22@8242 1810
smh22@8242 1811 In this scenario, each node has two network cards in the cluster. One
smh22@8242 1812 network card is connected to the outside world and one network card is a
smh22@8242 1813 physically isolated management network specifically for Xen instances to
smh22@8242 1814 use.
smh22@8242 1815
smh22@8242 1816 As long as all of the management partitions are trusted equally, this is
smh22@8242 1817 the most secure scenario. No additional configuration is needed other
smh22@8242 1818 than forcing Xend to bind to the management interface for relocation.
smh22@8242 1819
smh22@8242 1820
smh22@8242 1821 \subsection{A Subnet Behind a Firewall}
smh22@8242 1822
smh22@8242 1823 In this scenario, each node has only one network card but the entire
smh22@8242 1824 cluster sits behind a firewall. This firewall should do at least the
smh22@8242 1825 following:
smh22@8242 1826
smh22@8242 1827 \begin{enumerate}
smh22@8242 1828 \item Prevent IP spoofing from outside of the subnet.
smh22@8242 1829 \item Prevent access to the relocation port of any of the nodes in the
smh22@8242 1830 cluster except from within the cluster.
smh22@8242 1831 \end{enumerate}
smh22@8242 1832
smh22@8242 1833 The following iptables rules can be used on each node to prevent
smh22@8242 1834 migrations to that node from outside the subnet assuming the main
smh22@8242 1835 firewall does not do this for you:
smh22@8242 1836
smh22@8242 1837 \begin{verbatim}
smh22@8242 1838 # this command disables all access to the Xen relocation
smh22@8242 1839 # port:
smh22@8242 1840 iptables -A INPUT -p tcp --destination-port 8002 -j REJECT
smh22@8242 1841
smh22@8242 1842 # this command enables Xen relocations only from the specific
smh22@8242 1843 # subnet:
smh22@8242 1844 iptables -I INPUT -p tcp -{}-source \
smh22@8242 1845 --destination-port 8002 -j ACCEPT
smh22@8242 1846 \end{verbatim}
smh22@8242 1847
smh22@8242 1848 \subsection{Nodes on an Untrusted Subnet}
smh22@8242 1849
smh22@8242 1850 Migration on an untrusted subnet is not safe in current versions of Xen.
smh22@8242 1851 It may be possible to perform migrations through a secure tunnel via an
smh22@8242 1852 VPN or SSH. The only safe option in the absence of a secure tunnel is to
smh22@8242 1853 disable migration completely. The easiest way to do this is with
smh22@8242 1854 iptables:
smh22@8242 1855
smh22@8242 1856 \begin{verbatim}
smh22@8242 1857 # this command disables all access to the Xen relocation port
smh22@8242 1858 iptables -A INPUT -p tcp -{}-destination-port 8002 -j REJECT
smh22@8242 1859 \end{verbatim}
smh22@8242 1860
smh22@8242 1861 \part{Reference}
kaf24@2743 1862
FMJ@8225 1863 %% Chapter Build and Boot Options
smh22@8242 1864 \chapter{Build and Boot Options}
smh22@8242 1865
smh22@8242 1866 This chapter describes the build- and boot-time options which may be
smh22@8242 1867 used to tailor your Xen system.
smh22@8242 1868
smh22@8242 1869 \section{Top-level Configuration Options}
smh22@8242 1870
smh22@8242 1871 Top-level configuration is achieved by editing one of two
smh22@8242 1872 files: \path{Config.mk} and \path{Makefile}.
smh22@8242 1873
smh22@8242 1874 The former allows the overall build target architecture to be
smh22@8242 1875 specified. You will typically not need to modify this unless
smh22@8242 1876 you are cross-compiling or if you wish to build a PAE-enabled
smh22@8242 1877 Xen system. Additional configuration options are documented
smh22@8242 1878 in the \path{Config.mk} file.
smh22@8242 1879
smh22@8242 1880 The top-level \path{Makefile} is chiefly used to customize the set of
smh22@8242 1881 kernels built. Look for the line:
smh22@8242 1882 \begin{quote}
smh22@8242 1883 \begin{verbatim}
smh22@8242 1884 KERNELS ?= linux-2.6-xen0 linux-2.6-xenU
smh22@8242 1885 \end{verbatim}
smh22@8242 1886 \end{quote}
smh22@8242 1887
smh22@8242 1888 Allowable options here are any kernels which have a corresponding
smh22@8242 1889 build configuration file in the \path{buildconfigs/} directory.
smh22@8242 1890
smh22@8242 1891
smh22@8242 1892
smh22@8242 1893 \section{Xen Build Options}
smh22@8242 1894
smh22@8242 1895 Xen provides a number of build-time options which should be set as
smh22@8242 1896 environment variables or passed on make's command-line.
smh22@8242 1897
smh22@8242 1898 \begin{description}
smh22@8242 1899 \item[verbose=y] Enable debugging messages when Xen detects an
smh22@8242 1900 unexpected condition. Also enables console output from all domains.
smh22@8242 1901 \item[debug=y] Enable debug assertions. Implies {\bf verbose=y}.
smh22@8242 1902 (Primarily useful for tracing bugs in Xen).
smh22@8242 1903 \item[debugger=y] Enable the in-Xen debugger. This can be used to
smh22@8242 1904 debug Xen, guest OSes, and applications.
smh22@8242 1905 \item[perfc=y] Enable performance counters for significant events
smh22@8242 1906 within Xen. The counts can be reset or displayed on Xen's console
smh22@8242 1907 via console control keys.
smh22@8242 1908 \end{description}
smh22@8242 1909
smh22@8242 1910
smh22@8242 1911 \section{Xen Boot Options}
smh22@8242 1912 \label{s:xboot}
smh22@8242 1913
smh22@8242 1914 These options are used to configure Xen's behaviour at runtime. They
smh22@8242 1915 should be appended to Xen's command line, either manually or by
smh22@8242 1916 editing \path{grub.conf}.
smh22@8242 1917
smh22@8242 1918 \begin{description}
smh22@8242 1919 \item [ noreboot ] Don't reboot the machine automatically on errors.
smh22@8242 1920 This is useful to catch debug output if you aren't catching console
smh22@8242 1921 messages via the serial line.
smh22@8242 1922 \item [ nosmp ] Disable SMP support. This option is implied by
smh22@8242 1923 `ignorebiostables'.
smh22@8242 1924 \item [ watchdog ] Enable NMI watchdog which can report certain
smh22@8242 1925 failures.
smh22@8242 1926 \item [ noirqbalance ] Disable software IRQ balancing and affinity.
smh22@8242 1927 This can be used on systems such as Dell 1850/2850 that have
smh22@8242 1928 workarounds in hardware for IRQ-routing issues.
smh22@8242 1929 \item [ badpage=$<$page number$>$,$<$page number$>$, \ldots ] Specify
smh22@8242 1930 a list of pages not to be allocated for use because they contain bad
smh22@8242 1931 bytes. For example, if your memory tester says that byte 0x12345678
smh22@8242 1932 is bad, you would place `badpage=0x12345' on Xen's command line.
smh22@8242 1933 \item [ com1=$<$baud$>$,DPS,$<$io\_base$>$,$<$irq$>$
smh22@8242 1934 com2=$<$baud$>$,DPS,$<$io\_base$>$,$<$irq$>$ ] \mbox{}\\
smh22@8242 1935 Xen supports up to two 16550-compatible serial ports. For example:
smh22@8242 1936 `com1=9600, 8n1, 0x408, 5' maps COM1 to a 9600-baud port, 8 data
smh22@8242 1937 bits, no parity, 1 stop bit, I/O port base 0x408, IRQ 5. If some
smh22@8242 1938 configuration options are standard (e.g., I/O base and IRQ), then
smh22@8242 1939 only a prefix of the full configuration string need be specified. If
smh22@8242 1940 the baud rate is pre-configured (e.g., by the bootloader) then you
smh22@8242 1941 can specify `auto' in place of a numeric baud rate.
smh22@8242 1942 \item [ console=$<$specifier list$>$ ] Specify the destination for Xen
smh22@8242 1943 console I/O. This is a comma-separated list of, for example:
smh22@8242 1944 \begin{description}
smh22@8242 1945 \item[ vga ] Use VGA console and allow keyboard input.
smh22@8242 1946 \item[ com1 ] Use serial port com1.
smh22@8242 1947 \item[ com2H ] Use serial port com2. Transmitted chars will have the
smh22@8242 1948 MSB set. Received chars must have MSB set.
smh22@8242 1949 \item[ com2L] Use serial port com2. Transmitted chars will have the
smh22@8242 1950 MSB cleared. Received chars must have MSB cleared.
smh22@8242 1951 \end{description}
smh22@8242 1952 The latter two examples allow a single port to be shared by two
smh22@8242 1953 subsystems (e.g.\ console and debugger). Sharing is controlled by
smh22@8242 1954 MSB of each transmitted/received character. [NB. Default for this
smh22@8242 1955 option is `com1,vga']
smh22@8242 1956 \item [ sync\_console ] Force synchronous console output. This is
smh22@8242 1957 useful if you system fails unexpectedly before it has sent all
smh22@8242 1958 available output to the console. In most cases Xen will
smh22@8242 1959 automatically enter synchronous mode when an exceptional event
smh22@8242 1960 occurs, but this option provides a manual fallback.
smh22@8242 1961 \item [ conswitch=$<$switch-char$><$auto-switch-char$>$ ] Specify how
smh22@8242 1962 to switch serial-console input between Xen and DOM0. The required
smh22@8242 1963 sequence is CTRL-$<$switch-char$>$ pressed three times. Specifying
smh22@8242 1964 the backtick character disables switching. The
smh22@8242 1965 $<$auto-switch-char$>$ specifies whether Xen should auto-switch
smh22@8242 1966 input to DOM0 when it boots --- if it is `x' then auto-switching is
smh22@8242 1967 disabled. Any other value, or omitting the character, enables
smh22@8242 1968 auto-switching. [NB. Default switch-char is `a'.]
smh22@8242 1969 \item [ nmi=xxx ]
smh22@8242 1970 Specify what to do with an NMI parity or I/O error. \\
smh22@8242 1971 `nmi=fatal': Xen prints a diagnostic and then hangs. \\
smh22@8242 1972 `nmi=dom0': Inform DOM0 of the NMI. \\
smh22@8242 1973 `nmi=ignore': Ignore the NMI.
smh22@8242 1974 \item [ mem=xxx ] Set the physical RAM address limit. Any RAM
smh22@8242 1975 appearing beyond this physical address in the memory map will be
smh22@8242 1976 ignored. This parameter may be specified with a B, K, M or G suffix,
smh22@8242 1977 representing bytes, kilobytes, megabytes and gigabytes respectively.
smh22@8242 1978 The default unit, if no suffix is specified, is kilobytes.
smh22@8242 1979 \item [ dom0\_mem=xxx ] Set the amount of memory to be allocated to
smh22@8242 1980 domain0. In Xen 3.x the parameter may be specified with a B, K, M or
smh22@8242 1981 G suffix, representing bytes, kilobytes, megabytes and gigabytes
smh22@8242 1982 respectively; if no suffix is specified, the parameter defaults to
smh22@8242 1983 kilobytes. In previous versions of Xen, suffixes were not supported
smh22@8242 1984 and the value is always interpreted as kilobytes.
smh22@8242 1985 \item [ tbuf\_size=xxx ] Set the size of the per-cpu trace buffers, in
kaf24@9799 1986 pages (default 0).
smh22@8242 1987 \item [ sched=xxx ] Select the CPU scheduler Xen should use. The
smh22@8242 1988 current possibilities are `sedf' (default) and `bvt'.
smh22@8242 1989 \item [ apic\_verbosity=debug,verbose ] Print more detailed
smh22@8242 1990 information about local APIC and IOAPIC configuration.
smh22@8242 1991 \item [ lapic ] Force use of local APIC even when left disabled by
smh22@8242 1992 uniprocessor BIOS.
smh22@8242 1993 \item [ nolapic ] Ignore local APIC in a uniprocessor system, even if
smh22@8242 1994 enabled by the BIOS.
smh22@8242 1995 \item [ apic=bigsmp,default,es7000,summit ] Specify NUMA platform.
smh22@8242 1996 This can usually be probed automatically.
smh22@8242 1997 \end{description}
smh22@8242 1998
smh22@8242 1999 In addition, the following options may be specified on the Xen command
smh22@8242 2000 line. Since domain 0 shares responsibility for booting the platform,
smh22@8242 2001 Xen will automatically propagate these options to its command line.
smh22@8242 2002 These options are taken from Linux's command-line syntax with
smh22@8242 2003 unchanged semantics.
smh22@8242 2004
smh22@8242 2005 \begin{description}
smh22@8242 2006 \item [ acpi=off,force,strict,ht,noirq,\ldots ] Modify how Xen (and
smh22@8242 2007 domain 0) parses the BIOS ACPI tables.
smh22@8242 2008 \item [ acpi\_skip\_timer\_override ] Instruct Xen (and domain~0) to
smh22@8242 2009 ignore timer-interrupt override instructions specified by the BIOS
smh22@8242 2010 ACPI tables.
smh22@8242 2011 \item [ noapic ] Instruct Xen (and domain~0) to ignore any IOAPICs
smh22@8242 2012 that are present in the system, and instead continue to use the
smh22@8242 2013 legacy PIC.
smh22@8242 2014 \end{description}
smh22@8242 2015
smh22@8242 2016
smh22@8242 2017 \section{XenLinux Boot Options}
smh22@8242 2018
smh22@8242 2019 In addition to the standard Linux kernel boot options, we support:
smh22@8242 2020 \begin{description}
smh22@8242 2021 \item[ xencons=xxx ] Specify the device node to which the Xen virtual
smh22@8242 2022 console driver is attached. The following options are supported:
smh22@8242 2023 \begin{center}
smh22@8242 2024 \begin{tabular}{l}
smh22@8242 2025 `xencons=off': disable virtual console \\
smh22@8242 2026 `xencons=tty': attach console to /dev/tty1 (tty0 at boot-time) \\
smh22@8242 2027 `xencons=ttyS': attach console to /dev/ttyS0
smh22@8242 2028 \end{tabular}
smh22@8242 2029 \end{center}
smh22@8242 2030 The default is ttyS for dom0 and tty for all other domains.
smh22@8242 2031 \end{description}
smh22@8242 2032
FMJ@8225 2033
FMJ@8225 2034 %% Chapter Further Support
smh22@8242 2035 \chapter{Further Support}
smh22@8242 2036
smh22@8242 2037 If you have questions that are not answered by this manual, the
smh22@8242 2038 sources of information listed below may be of interest to you. Note
smh22@8242 2039 that bug reports, suggestions and contributions related to the
smh22@8242 2040 software (or the documentation) should be sent to the Xen developers'
smh22@8242 2041 mailing list (address below).
smh22@8242 2042
smh22@8242 2043
smh22@8242 2044 \section{Other Documentation}
smh22@8242 2045
smh22@8242 2046 For developers interested in porting operating systems to Xen, the
smh22@8242 2047 \emph{Xen Interface Manual} is distributed in the \path{docs/}
smh22@8242 2048 directory of the Xen source distribution.
smh22@8242 2049
smh22@8242 2050
smh22@8242 2051 \section{Online References}
smh22@8242 2052
smh22@8242 2053 The official Xen web site can be found at:
smh22@8242 2054 \begin{quote} {\tt http://www.xensource.com}
smh22@8242 2055 \end{quote}
smh22@8242 2056
smh22@8242 2057 This contains links to the latest versions of all online
smh22@8242 2058 documentation, including the latest version of the FAQ.
smh22@8242 2059
smh22@8242 2060 Information regarding Xen is also available at the Xen Wiki at
smh22@8242 2061 \begin{quote} {\tt http://wiki.xensource.com/xenwiki/}\end{quote}
smh22@8242 2062 The Xen project uses Bugzilla as its bug tracking system. You'll find
smh22@8242 2063 the Xen Bugzilla at http://bugzilla.xensource.com/bugzilla/.
smh22@8242 2064
smh22@8242 2065
smh22@8242 2066 \section{Mailing Lists}
smh22@8242 2067
smh22@8242 2068 There are several mailing lists that are used to discuss Xen related
smh22@8242 2069 topics. The most widely relevant are listed below. An official page of
smh22@8242 2070 mailing lists and subscription information can be found at \begin{quote}
smh22@8242 2071 {\tt http://lists.xensource.com/} \end{quote}
smh22@8242 2072
smh22@8242 2073 \begin{description}
smh22@8242 2074 \item[xen-devel@lists.xensource.com] Used for development
smh22@8242 2075 discussions and bug reports. Subscribe at: \\
smh22@8242 2076 {\small {\tt http://lists.xensource.com/xen-devel}}
smh22@8242 2077 \item[xen-users@lists.xensource.com] Used for installation and usage
smh22@8242 2078 discussions and requests for help. Subscribe at: \\
smh22@8242 2079 {\small {\tt http://lists.xensource.com/xen-users}}
smh22@8242 2080 \item[xen-announce@lists.xensource.com] Used for announcements only.
smh22@8242 2081 Subscribe at: \\
smh22@8242 2082 {\small {\tt http://lists.xensource.com/xen-announce}}
smh22@8242 2083 \item[xen-changelog@lists.xensource.com] Changelog feed
smh22@8242 2084 from the unstable and 2.0 trees - developer oriented. Subscribe at: \\
smh22@8242 2085 {\small {\tt http://lists.xensource.com/xen-changelog}}
smh22@8242 2086 \end{description}
smh22@8242 2087
kaf24@2743 2088
kaf24@6977 2089
FMJ@8225 2090 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
kaf24@6977 2091
kaf24@2743 2092 \appendix
kaf24@2743 2093
kaf24@8539 2094 \chapter{Unmodified (VMX) guest domains in Xen with Intel\textregistered Virtualization Technology (VT)}
kaf24@8539 2095
kaf24@8539 2096 Xen supports guest domains running unmodified Guest operating systems using Virtualization Technology (VT) available on recent Intel Processors. More information about the Intel Virtualization Technology implementing Virtual Machine Extensions (VMX) in the processor is available on the Intel website at \\
kaf24@8539 2097 {\small {\tt http://www.intel.com/technology/computing/vptech}}
kaf24@8539 2098
kaf24@8539 2099 \section{Building Xen with VT support}
kaf24@8539 2100
kaf24@8539 2101 The following packages need to be installed in order to build Xen with VT support. Some Linux distributions do not provide these packages by default.
kaf24@8539 2102
kaf24@8539 2103 \begin{tabular}{lp{11.0cm}}
kaf24@8539 2104 {\bfseries Package} & {\bfseries Description} \\
kaf24@8539 2105
kaf24@8539 2106 dev86 & The dev86 package provides an assembler and linker for real mode 80x86 instructions. You need to have this package installed in order to build the BIOS code which runs in (virtual) real mode.
kaf24@8539 2107
kaf24@8539 2108 If the dev86 package is not available on the x86\_64 distribution, you can install the i386 version of it. The dev86 rpm package for various distributions can be found at {\scriptsize {\tt http://www.rpmfind.net/linux/rpm2html/search.php?query=dev86\&submit=Search}} \\
kaf24@8539 2109
kaf24@9603 2110 LibVNCServer & The unmodified guest's VGA display, keyboard, and mouse can be virtualized by the vncserver library. You can get the sources of libvncserver from {\small {\tt http://sourceforge.net/projects/libvncserver}}. Build and install the sources on the build system to get the libvncserver library. There is a significant performance degradation in 0.8 version. The current sources in the CVS tree have fixed this degradation. So it is highly recommended to download the latest CVS sources and install them.\\
kaf24@8539 2111
kaf24@8539 2112 SDL-devel, SDL & Simple DirectMedia Layer (SDL) is another way of virtualizing the unmodified guest console. It provides an X window for the guest console.
kaf24@8539 2113
kaf24@8539 2114 If the SDL and SDL-devel packages are not installed by default on the build system, they can be obtained from {\scriptsize {\tt http://www.rpmfind.net/linux/rpm2html/search.php?query=SDL\&amp;submit=Search}}
kaf24@8539 2115 , {\scriptsize {\tt http://www.rpmfind.net/linux/rpm2html/search.php?query=SDL-devel\&submit=Search}} \\
kaf24@8539 2116
kaf24@8539 2117 \end{tabular}
kaf24@8539 2118
kaf24@8539 2119 \section{Configuration file for unmodified VMX guests}
kaf24@8539 2120
kaf24@8539 2121 The Xen installation includes a sample configuration file, {\small {\tt /etc/xen/xmexample.vmx}}. There are comments describing all the options. In addition to the common options that are the same as those for paravirtualized guest configurations, VMX guest configurations have the following settings:
kaf24@8539 2122
kaf24@8539 2123 \begin{tabular}{lp{11.0cm}}
kaf24@8539 2124
kaf24@8539 2125 {\bfseries Parameter} & {\bfseries Description} \\
kaf24@8539 2126
kaf24@8539 2127 kernel & The VMX firmware loader, {\small {\tt /usr/lib/xen/boot/vmxloader}}\\
kaf24@8539 2128
kaf24@8539 2129 builder & The domain build function. The VMX domain uses the vmx builder.\\
kaf24@8539 2130
kaf24@8539 2131 acpi & Enable VMX guest ACPI, default=0 (disabled)\\
kaf24@8539 2132
kaf24@8539 2133 apic & Enable VMX guest APIC, default=0 (disabled)\\
kaf24@8539 2134
kaf24@9603 2135 pae & Enable VMX guest PAE, default=0 (disabled)\\
kaf24@9603 2136
kaf24@8539 2137 vif & Optionally defines MAC address and/or bridge for the network interfaces. Random MACs are assigned if not given. {\small {\tt type=ioemu}} means ioemu is used to virtualize the VMX NIC. If no type is specified, vbd is used, as with paravirtualized guests.\\
kaf24@8539 2138
kaf24@8539 2139 disk & Defines the disk devices you want the domain to have access to, and what you want them accessible as. If using a physical device as the VMX guest's disk, each disk entry is of the form
kaf24@8539 2140
kaf24@8539 2141 {\small {\tt phy:UNAME,ioemu:DEV,MODE,}}
kaf24@8539 2142
kaf24@8539 2143 where UNAME is the device, DEV is the device name the domain will see, and MODE is r for read-only, w for read-write. ioemu means the disk will use ioemu to virtualize the VMX disk. If not adding ioemu, it uses vbd like paravirtualized guests.
kaf24@8539 2144
kaf24@8539 2145 If using disk image file, its form should be like
kaf24@8539 2146
kaf24@8539 2147 {\small {\tt file:FILEPATH,ioemu:DEV,MODE}}
kaf24@8539 2148
kaf24@8539 2149 If using more than one disk, there should be a comma between each disk entry. For example:
kaf24@8539 2150
kaf24@8539 2151 {\scriptsize {\tt disk = ['file:/var/images/image1.img,ioemu:hda,w', 'file:/var/images/image2.img,ioemu:hdb,w']}}\\
kaf24@8539 2152
kaf24@8539 2153 cdrom & Disk image for CD-ROM. The default is {\small {\tt /dev/cdrom}} for Domain0. Inside the VMX domain, the CD-ROM will available as device {\small {\tt /dev/hdc}}. The entry can also point to an ISO file.\\
kaf24@8539 2154
kaf24@8539 2155 boot & Boot from floppy (a), hard disk (c) or CD-ROM (d). For example, to boot from CD-ROM, the entry should be:
kaf24@8539 2156
kaf24@8539 2157 boot='d'\\
kaf24@8539 2158
kaf24@8539 2159 device\_model & The device emulation tool for VMX guests. This parameter should not be changed.\\
kaf24@8539 2160
kaf24@8539 2161 sdl & Enable SDL library for graphics, default = 0 (disabled)\\
kaf24@8539 2162
kaf24@8539 2163 vnc & Enable VNC library for graphics, default = 1 (enabled)\\
kaf24@8539 2164
kaf24@8539 2165 vncviewer & Enable spawning of the vncviewer (only valid when vnc=1), default = 1 (enabled)
kaf24@8539 2166
kaf24@8539 2167 If vnc=1 and vncviewer=0, user can use vncviewer to manually connect VMX from remote. For example:
kaf24@8539 2168
kaf24@8539 2169 {\small {\tt vncviewer domain0\_IP\_address:VMX\_domain\_id}} \\
kaf24@8539 2170
kaf24@8539 2171 ne2000 & Enable ne2000, default = 0 (disabled; use pcnet)\\
kaf24@8539 2172
kaf24@8539 2173 serial & Enable redirection of VMX serial output to pty device\\
kaf24@8539 2174
kaf24@8539 2175 localtime & Set the real time clock to local time [default=0, that is, set to UTC].\\
kaf24@8539 2176
kaf24@8539 2177 enable-audio & Enable audio support. This is under development.\\
kaf24@8539 2178
kaf24@8539 2179 full-screen & Start in full screen. This is under development.\\
kaf24@8539 2180
kaf24@8539 2181 nographic & Another way to redirect serial output. If enabled, no 'sdl' or 'vnc' can work. Not recommended.\\
kaf24@8539 2182
kaf24@8539 2183 \end{tabular}
kaf24@8539 2184
kaf24@8539 2185
kaf24@8539 2186 \section{Creating virtual disks from scratch}
kaf24@8539 2187 \subsection{Using physical disks}
kaf24@8539 2188 If you are using a physical disk or physical disk partition, you need to install a Linux OS on the disk first. Then the boot loader should be installed in the correct place. For example {\small {\tt dev/sda}} for booting from the whole disk, or {\small {\tt /dev/sda1}} for booting from partition 1.
kaf24@8539 2189
kaf24@8539 2190 \subsection{Using disk image files}
kaf24@8539 2191 You need to create a large empty disk image file first; then, you need to install a Linux OS onto it. There are two methods you can choose. One is directly installing it using a VMX guest while booting from the OS installation CD-ROM. The other is copying an installed OS into it. The boot loader will also need to be installed.
kaf24@8539 2192
kaf24@8539 2193 \subsubsection*{To create the image file:}
kaf24@8539 2194 The image size should be big enough to accommodate the entire OS. This example assumes the size is 1G (which is probably too small for most OSes).
kaf24@8539 2195
kaf24@8539 2196 {\small {\tt \# dd if=/dev/zero of=hd.img bs=1M count=1 seek=1023}}
kaf24@8539 2197
kaf24@8539 2198 \subsubsection*{To directly install Linux OS into an image file using a VMX guest:}
kaf24@8539 2199
kaf24@8539 2200 Install Xen and create VMX with the original image file with booting from CD-ROM. Then it is just like a normal Linux OS installation. The VMX configuration file should have these two entries before creating:
kaf24@8539 2201
kaf24@8539 2202 {\small {\tt cdrom='/dev/cdrom'
kaf24@8539 2203 boot='d'}}
kaf24@8539 2204
kaf24@8539 2205 If this method does not succeed, you can choose the following method of copying an installed Linux OS into an image file.
kaf24@8539 2206
kaf24@8539 2207 \subsubsection*{To copy a installed OS into an image file:}
kaf24@8539 2208 Directly installing is an easier way to make partitions and install an OS in a disk image file. But if you want to create a specific OS in your disk image, then you will most likely want to use this method.
kaf24@8539 2209
kaf24@8539 2210 \begin{enumerate}
kaf24@8539 2211 \item {\bfseries Install a normal Linux OS on the host machine}\\
kaf24@8539 2212 You can choose any way to install Linux, such as using yum to install Red Hat Linux or YAST to install Novell SuSE Linux. The rest of this example assumes the Linux OS is installed in {\small {\tt /var/guestos/}}.
kaf24@8539 2213
kaf24@8539 2214 \item {\bfseries Make the partition table}\\
kaf24@8539 2215 The image file will be treated as hard disk, so you should make the partition table in the image file. For example:
kaf24@8539 2216
kaf24@8539 2217 {\scriptsize {\tt \# losetup /dev/loop0 hd.img\\
kaf24@8539 2218 \# fdisk -b 512 -C 4096 -H 16 -S 32 /dev/loop0\\
kaf24@8539 2219 press 'n' to add new partition\\
kaf24@8539 2220 press 'p' to choose primary partition\\
kaf24@8539 2221 press '1' to set partition number\\
kaf24@8539 2222 press "Enter" keys to choose default value of "First Cylinder" parameter.\\
kaf24@8539 2223 press "Enter" keys to choose default value of "Last Cylinder" parameter.\\
kaf24@8539 2224 press 'w' to write partition table and exit\\
kaf24@8539 2225 \# losetup -d /dev/loop0}}
kaf24@8539 2226
kaf24@8539 2227 \item {\bfseries Make the file system and install grub}\\
kaf24@8539 2228 {\scriptsize {\tt \# ln -s /dev/loop0 /dev/loop\\
kaf24@8539 2229 \# losetup /dev/loop0 hd.img\\
kaf24@8539 2230 \# losetup -o 16384 /dev/loop1 hd.img\\
kaf24@8539 2231 \# mkfs.ext3 /dev/loop1\\
kaf24@8539 2232 \# mount /dev/loop1 /mnt\\
kaf24@8539 2233 \# mkdir -p /mnt/boot/grub\\
kaf24@8539 2234 \# cp /boot/grub/stage* /boot/grub/e2fs\_stage1\_5 /mnt/boot/grub\\
kaf24@8539 2235 \# umount /mnt\\
kaf24@8539 2236 \# grub\\
kaf24@8539 2237 grub> device (hd0) /dev/loop\\
kaf24@8539 2238 grub> root (hd0,0)\\
kaf24@8539 2239 grub> setup (hd0)\\
kaf24@8539 2240 grub> quit\\
kaf24@8539 2241 \# rm /dev/loop\\
kaf24@8539 2242 \# losetup -d /dev/loop0\\
kaf24@8539 2243 \# losetup -d /dev/loop1}}
kaf24@8539 2244
kaf24@8539 2245 The {\small {\tt losetup}} option {\small {\tt -o 16384}} skips the partition table in the image file. It is the number of sectors times 512. We need {\small {\tt /dev/loop}} because grub is expecting a disk device \emph{name}, where \emph{name} represents the entire disk and \emph{name1} represents the first partition.
kaf24@8539 2246
kaf24@8539 2247 \item {\bfseries Copy the OS files to the image}\\
kaf24@8539 2248 If you have Xen installed, you can easily use {\small {\tt lomount}} instead of {\small {\tt losetup}} and {\small {\tt mount}} when coping files to some partitions. {\small {\tt lomount}} just needs the partition information.
kaf24@8539 2249
kaf24@8539 2250 {\scriptsize {\tt \# lomount -t ext3 -diskimage hd.img -partition 1 /mnt/guest\\
kaf24@8539 2251 \# cp -ax /var/guestos/\{root,dev,var,etc,usr,bin,sbin,lib\} /mnt/guest\\
kaf24@8539 2252 \# mkdir /mnt/guest/\{proc,sys,home,tmp\}}}
kaf24@8539 2253
kaf24@8539 2254 \item {\bfseries Edit the {\small {\tt /etc/fstab}} of the guest image}\\
kaf24@8539 2255 The fstab should look like this:
kaf24@8539 2256
kaf24@8539 2257 {\scriptsize {\tt \# vim /mnt/guest/etc/fstab\\
kaf24@8539 2258 /dev/hda1 / ext3 defaults 1 1\\
kaf24@8539 2259 none /dev/pts devpts gid=5,mode=620 0 0\\
kaf24@8539 2260 none /dev/shm tmpfs defaults 0 0\\
kaf24@8539 2261 none /proc proc defaults 0 0\\
kaf24@8539 2262 none /sys sysfs efaults 0 0}}
kaf24@8539 2263
kaf24@8539 2264 \item {\bfseries umount the image file}\\
kaf24@8539 2265 {\small {\tt \# umount /mnt/guest}}
kaf24@8539 2266 \end{enumerate}
kaf24@8539 2267
kaf24@8539 2268 Now, the guest OS image {\small {\tt hd.img}} is ready. You can also reference {\small {\tt http://free.oszoo.org}} for quickstart images. But make sure to install the boot loader.
kaf24@8539 2269
kaf24@8539 2270 \subsection{Install Windows into an Image File using a VMX guest}
kaf24@8539 2271 In order to install a Windows OS, you should keep {\small {\tt acpi=0}} in your VMX configuration file.
kaf24@8539 2272
kaf24@8539 2273 \section{VMX Guests}
kaf24@8539 2274 \subsection{Editing the Xen VMX config file}
kaf24@8539 2275 Make a copy of the example VMX configuration file {\small {\tt /etc/xen/xmeaxmple.vmx}} and edit the line that reads
kaf24@8539 2276
kaf24@8539 2277 {\small {\tt disk = [ 'file:/var/images/\emph{guest.img},ioemu:hda,w' ]}}
kaf24@8539 2278
kaf24@8539 2279 replacing \emph{guest.img} with the name of the guest OS image file you just made.
kaf24@8539 2280
kaf24@8539 2281 \subsection{Creating VMX guests}
kaf24@8539 2282 Simply follow the usual method of creating the guest, using the -f parameter and providing the filename of your VMX configuration file:\\
kaf24@8539 2283
kaf24@8539 2284 {\small {\tt \# xend start\\
kaf24@8539 2285 \# xm create /etc/xen/vmxguest.vmx}}
kaf24@8539 2286
kaf24@8539 2287 In the default configuration, VNC is on and SDL is off. Therefore VNC windows will open when VMX guests are created. If you want to use SDL to create VMX guests, set {\small {\tt sdl=1}} in your VMX configuration file. You can also turn off VNC by setting {\small {\tt vnc=0}}.
kaf24@8539 2288
kaf24@9603 2289 \subsection{Use mouse in VNC window}
kaf24@9603 2290 The default PS/2 mouse will not work properly in VMX by a VNC window. Summagraphics mouse emulation does work in this environment. A Summagraphics mouse can be enabled by reconfiguring 2 services:
kaf24@9603 2291
kaf24@9603 2292 {\small {\tt 1. General Purpose Mouse (GPM). The GPM daemon is configured in different ways in different Linux distributions. On a Redhat distribution, this is accomplished by changing the file `/etc/sysconfig/mouse' to have the following:\\
kaf24@9603 2293 MOUSETYPE="summa"\\
kaf24@9603 2294 XMOUSETYPE="SUMMA"\\
kaf24@9603 2295 DEVICE=/dev/ttyS0\\
kaf24@9603 2296 \\
kaf24@9603 2297 2. X11. For all Linux distributions, change the Mouse0 stanza in `/etc/X11/xorg.conf' to:\\
kaf24@9603 2298 Section "InputDevice"\\
kaf24@9603 2299 Identifier "Mouse0"\\
kaf24@9603 2300 Driver "summa"\\
kaf24@9603 2301 Option "Device" "/dev/ttyS0"\\
kaf24@9603 2302 Option "InputFashion" "Tablet"\\
kaf24@9603 2303 Option "Mode" "Absolute"\\
kaf24@9603 2304 Option "Name" "EasyPen"\\
kaf24@9603 2305 Option "Compatible" "True"\\
kaf24@9603 2306 Option "Protocol" "Auto"\\
kaf24@9603 2307 Option "SendCoreEvents" "on"\\
kaf24@9603 2308 Option "Vendor" "GENIUS"\\
kaf24@9603 2309 EndSection}}
kaf24@9603 2310
kaf24@9603 2311 If the Summagraphics mouse isn't the default mouse, you can manually kill 'gpm' and restart it with the command "gpm -m /dev/ttyS0 -t summa". Note that Summagraphics mouse makes no sense in an SDL window and is therefore not available in this environment.
kaf24@9603 2312
kaf24@8539 2313 \subsection{Destroy VMX guests}
kaf24@8539 2314 VMX guests can be destroyed in the same way as can paravirtualized guests. We recommend that you type the command
kaf24@8539 2315
kaf24@8539 2316 {\small {\tt poweroff}}
kaf24@8539 2317
kaf24@8539 2318 in the VMX guest's console first to prevent data loss. Then execute the command
kaf24@8539 2319
kaf24@8539 2320 {\small {\tt xm destroy \emph{vmx\_guest\_id} }}
kaf24@8539 2321
kaf24@8539 2322 at the Domain0 console.
kaf24@8539 2323
kaf24@8539 2324 \subsection{VMX window (X or VNC) Hot Key}
kaf24@8539 2325 If you are running in the X environment after creating a VMX guest, an X window is created. There are several hot keys for control of the VMX guest that can be used in the window.
kaf24@8539 2326
kaf24@8539 2327 {\bfseries Ctrl+Alt+2} switches from guest VGA window to the control window. Typing {\small {\tt help }} shows the control commands help. For example, 'q' is the command to destroy the VMX guest.\\
kaf24@8539 2328 {\bfseries Ctrl+Alt+1} switches back to VMX guest's VGA.\\
kaf24@8539 2329 {\bfseries Ctrl+Alt+3} switches to serial port output. It captures serial output from the VMX guest. It works only if the VMX guest was configured to use the serial port. \\
kaf24@8539 2330
kaf24@8539 2331 \subsection{Save/Restore and Migration}
kaf24@8539 2332 VMX guests currently cannot be saved and restored, nor migrated. These features are currently under active development.
kaf24@8539 2333
kaf24@8814 2334 \chapter{Vnets - Domain Virtual Networking}
kaf24@8814 2335
kaf24@8814 2336 Xen optionally supports virtual networking for domains using {\em vnets}.
kaf24@8814 2337 These emulate private LANs that domains can use. Domains on the same
kaf24@8814 2338 vnet can be hosted on the same machine or on separate machines, and the
kaf24@8814 2339 vnets remain connected if domains are migrated. Ethernet traffic
kaf24@8814 2340 on a vnet is tunneled inside IP packets on the physical network. A vnet is a virtual
kaf24@8814 2341 network and addressing within it need have no relation to addressing on
kaf24@8814 2342 the underlying physical network. Separate vnets, or vnets and the physical network,
kaf24@8814 2343 can be connected using domains with more than one network interface and
kaf24@8814 2344 enabling IP forwarding or bridging in the usual way.
kaf24@8814 2345
kaf24@8814 2346 Vnet support is included in \texttt{xm} and \xend:
kaf24@8814 2347 \begin{verbatim}
kaf24@8814 2348 # xm vnet-create <config>
kaf24@8814 2349 \end{verbatim}
kaf24@8814 2350 creates a vnet using the configuration in the file \verb|<config>|.
kaf24@8814 2351 When a vnet is created its configuration is stored by \xend and the vnet persists until it is
kaf24@8814 2352 deleted using
kaf24@8814 2353 \begin{verbatim}
kaf24@8814 2354 # xm vnet-delete <vnetid>
kaf24@8814 2355 \end{verbatim}
kaf24@8814 2356 The vnets \xend knows about are listed by
kaf24@8814 2357 \begin{verbatim}
kaf24@8814 2358 # xm vnet-list
kaf24@8814 2359 \end{verbatim}
kaf24@8814 2360 More vnet management commands are available using the
kaf24@8814 2361 \texttt{vn} tool included in the vnet distribution.
kaf24@8814 2362
kaf24@8814 2363 The format of a vnet configuration file is
kaf24@8814 2364 \begin{verbatim}
kaf24@8814 2365 (vnet (id <vnetid>)
kaf24@8814 2366 (bridge <bridge>)
kaf24@8814 2367 (vnetif <vnet interface>)
kaf24@8814 2368 (security <level>))
kaf24@8814 2369 \end{verbatim}
kaf24@8814 2370 White space is not significant. The parameters are:
kaf24@8814 2371 \begin{itemize}
kaf24@8814 2372 \item \verb|<vnetid>|: vnet id, the 128-bit vnet identifier. This can be given
kaf24@8814 2373 as 8 4-digit hex numbers separated by colons, or in short form as a single 4-digit hex number.
kaf24@8814 2374 The short form is the same as the long form with the first 7 fields zero.
kaf24@8814 2375 Vnet ids must be non-zero and id 1 is reserved.
kaf24@8814 2376
kaf24@8814 2377 \item \verb|<bridge>|: the name of a bridge interface to create for the vnet. Domains
kaf24@8814 2378 are connected to the vnet by connecting their virtual interfaces to the bridge.
kaf24@8814 2379 Bridge names are limited to 14 characters by the kernel.
kaf24@8814 2380
kaf24@8814 2381 \item \verb|<vnetif>|: the name of the virtual interface onto the vnet (optional). The
kaf24@8814 2382 interface encapsulates and decapsulates vnet traffic for the network and is attached
kaf24@8814 2383 to the vnet bridge. Interface names are limited to 14 characters by the kernel.
kaf24@8814 2384
kaf24@8814 2385 \item \verb|<level>|: security level for the vnet (optional). The level may be one of
kaf24@8814 2386 \begin{itemize}
kaf24@8814 2387 \item \verb|none|: no security (default). Vnet traffic is in clear on the network.
kaf24@8814 2388 \item \verb|auth|: authentication. Vnet traffic is authenticated using IPSEC
kaf24@8814 2389 ESP with hmac96.
kaf24@8814 2390 \item \verb|conf|: confidentiality. Vnet traffic is authenticated and encrypted
kaf24@8814 2391 using IPSEC ESP with hmac96 and AES-128.
kaf24@8814 2392 \end{itemize}
kaf24@8814 2393 Authentication and confidentiality are experimental and use hard-wired keys at present.
kaf24@8814 2394 \end{itemize}
kaf24@8814 2395 When a vnet is created its configuration is stored by \xend and the vnet persists until it is
kaf24@8814 2396 deleted using \texttt{xm vnet-delete <vnetid>}. The interfaces and bridges used by vnets
kaf24@8814 2397 are visible in the output of \texttt{ifconfig} and \texttt{brctl show}.
kaf24@8814 2398
kaf24@8814 2399 \section{Example}
kaf24@8814 2400 If the file \path{vnet97.sxp} contains
kaf24@8814 2401 \begin{verbatim}
kaf24@8814 2402 (vnet (id 97) (bridge vnet97) (vnetif vnif97)
kaf24@8814 2403 (security none))
kaf24@8814 2404 \end{verbatim}
kaf24@8814 2405 Then \texttt{xm vnet-create vnet97.sxp} will define a vnet with id 97 and no security.
kaf24@8814 2406 The bridge for the vnet is called vnet97 and the virtual interface for it is vnif97.
kaf24@8814 2407 To add an interface on a domain to this vnet set its bridge to vnet97
kaf24@8814 2408 in its configuration. In Python:
kaf24@8814 2409 \begin{verbatim}
kaf24@8814 2410 vif="bridge=vnet97"
kaf24@8814 2411 \end{verbatim}
kaf24@8814 2412 In sxp:
kaf24@8814 2413 \begin{verbatim}
kaf24@8814 2414 (dev (vif (mac aa:00:00:01:02:03) (bridge vnet97)))
kaf24@8814 2415 \end{verbatim}
kaf24@8814 2416 Once the domain is started you should see its interface in the output of \texttt{brctl show}
kaf24@8814 2417 under the ports for \texttt{vnet97}.
kaf24@8814 2418
kaf24@8814 2419 To get best performance it is a good idea to reduce the MTU of a domain's interface
kaf24@8814 2420 onto a vnet to 1400. For example using \texttt{ifconfig eth0 mtu 1400} or putting
kaf24@8814 2421 \texttt{MTU=1400} in \texttt{ifcfg-eth0}.
kaf24@8814 2422 You may also have to change or remove cached config files for eth0 under
kaf24@8814 2423 \texttt{/etc/sysconfig/networking}. Vnets work anyway, but performance can be reduced
kaf24@8814 2424 by IP fragmentation caused by the vnet encapsulation exceeding the hardware MTU.
kaf24@8814 2425
kaf24@8814 2426 \section{Installing vnet support}
kaf24@8814 2427 Vnets are implemented using a kernel module, which needs to be loaded before
kaf24@8814 2428 they can be used. You can either do this manually before starting \xend, using the
kaf24@8814 2429 command \texttt{vn insmod}, or configure \xend to use the \path{network-vnet}
kaf24@8814 2430 script in the xend configuration file \texttt{/etc/xend/xend-config.sxp}:
kaf24@8814 2431 \begin{verbatim}
kaf24@8814 2432 (network-script network-vnet)
kaf24@8814 2433 \end{verbatim}
kaf24@8814 2434 This script insmods the module and calls the \path{network-bridge} script.
kaf24@8814 2435
kaf24@8814 2436 The vnet code is not compiled and installed by default.
kaf24@8814 2437 To compile the code and install on the current system
kaf24@8814 2438 use \texttt{make install} in the root of the vnet source tree,
kaf24@8814 2439 \path{tools/vnet}. It is also possible to install to an installation
kaf24@8814 2440 directory using \texttt{make dist}. See the \path{Makefile} in
kaf24@8814 2441 the source for details.
kaf24@8814 2442
kaf24@8814 2443 The vnet module creates vnet interfaces \texttt{vnif0002},
kaf24@8814 2444 \texttt{vnif0003} and \texttt{vnif0004} by default. You can test that
kaf24@8814 2445 vnets are working by configuring IP addresses on these interfaces
kaf24@8814 2446 and trying to ping them across the network. For example, using machines
kaf24@8814 2447 hostA and hostB:
kaf24@8814 2448 \begin{verbatim}
kaf24@8814 2449 hostA# ifconfig vnif0004 up
kaf24@8814 2450 hostB# ifconfig vnif0004 up
kaf24@8814 2451 hostB# ping
kaf24@8814 2452 \end{verbatim}
kaf24@8814 2453
kaf24@8814 2454 The vnet implementation uses IP multicast to discover vnet interfaces, so
kaf24@8814 2455 all machines hosting vnets must be reachable by multicast. Network switches
kaf24@8814 2456 are often configured not to forward multicast packets, so this often
kaf24@8814 2457 means that all machines using a vnet must be on the same LAN segment,
kaf24@8814 2458 unless you configure vnet forwarding.
kaf24@8814 2459
kaf24@8814 2460 You can test multicast coverage by pinging the vnet multicast address:
kaf24@8814 2461 \begin{verbatim}
kaf24@8814 2462 # ping -b
kaf24@8814 2463 \end{verbatim}
kaf24@8814 2464 You should see replies from all machines with the vnet module running.
kaf24@8814 2465 You can see if vnet packets are being sent or received by dumping traffic
kaf24@8814 2466 on the vnet UDP port:
kaf24@8814 2467 \begin{verbatim}
kaf24@8814 2468 # tcpdump udp port 1798
kaf24@8814 2469 \end{verbatim}
kaf24@8814 2470
kaf24@8814 2471 If multicast is not being forwaded between machines you can configure
kaf24@8814 2472 multicast forwarding using vn. Suppose we have machines hostA on
kaf24@8814 2473 and hostB on and that multicast is not forwarded between them.
kaf24@8814 2474 We use vn to configure each machine to forward to the other:
kaf24@8814 2475 \begin{verbatim}
kaf24@8814 2476 hostA# vn peer-add hostB
kaf24@8814 2477 hostB# vn peer-add hostA
kaf24@8814 2478 \end{verbatim}
kaf24@8814 2479 Multicast forwarding needs to be used carefully - you must avoid creating forwarding
kaf24@8814 2480 loops. Typically only one machine on a subnet needs to be configured to forward,
kaf24@8814 2481 as it will forward multicasts received from other machines on the subnet.
kaf24@8814 2482
kaf24@6977 2483 %% Chapter Glossary of Terms moved to glossary.tex
smh22@8242 2484 \chapter{Glossary of Terms}
smh22@8242 2485
smh22@8242 2486 \begin{description}
smh22@8242 2487
smh22@8242 2488 \item[BVT] The BVT scheduler is used to give proportional fair shares
smh22@8242 2489 of the CPU to domains.
smh22@8242 2490
smh22@8242 2491 \item[Domain] A domain is the execution context that contains a
smh22@8242 2492 running {\bf virtual machine}. The relationship between virtual
smh22@8242 2493 machines and domains on Xen is similar to that between programs and
smh22@8242 2494 processes in an operating system: a virtual machine is a persistent
smh22@8242 2495 entity that resides on disk (somewhat like a program). When it is
smh22@8242 2496 loaded for execution, it runs in a domain. Each domain has a {\bf
smh22@8242 2497 domain ID}.
smh22@8242 2498
smh22@8242 2499 \item[Domain 0] The first domain to be started on a Xen machine.
smh22@8242 2500 Domain 0 is responsible for managing the system.
smh22@8242 2501
smh22@8242 2502 \item[Domain ID] A unique identifier for a {\bf domain}, analogous to
smh22@8242 2503 a process ID in an operating system.
smh22@8242 2504
smh22@8242 2505 \item[Full virtualization] An approach to virtualization which
smh22@8242 2506 requires no modifications to the hosted operating system, providing
smh22@8242 2507 the illusion of a complete system of real hardware devices.
smh22@8242 2508
smh22@8242 2509 \item[Hypervisor] An alternative term for {\bf VMM}, used because it
smh22@8242 2510 means `beyond supervisor', since it is responsible for managing
smh22@8242 2511 multiple `supervisor' kernels.
smh22@8242 2512
smh22@8242 2513 \item[Live migration] A technique for moving a running virtual machine
smh22@8242 2514 to another physical host, without stopping it or the services
smh22@8242 2515 running on it.
smh22@8242 2516
smh22@8242 2517 \item[Paravirtualization] An approach to virtualization which requires
smh22@8242 2518 modifications to the operating system in order to run in a virtual
smh22@8242 2519 machine. Xen uses paravirtualization but preserves binary
smh22@8242 2520 compatibility for user space applications.
smh22@8242 2521
smh22@8242 2522 \item[Shadow pagetables] A technique for hiding the layout of machine
smh22@8242 2523 memory from a virtual machine's operating system. Used in some {\bf
smh22@8242 2524 VMMs} to provide the illusion of contiguous physical memory, in
smh22@8242 2525 Xen this is used during {\bf live migration}.
smh22@8242 2526
smh22@8242 2527 \item[Virtual Block Device] Persistant storage available to a virtual
smh22@8242 2528 machine, providing the abstraction of an actual block storage device.
smh22@8242 2529 {\bf VBD}s may be actual block devices, filesystem images, or
smh22@8242 2530 remote/network storage.
smh22@8242 2531
smh22@8242 2532 \item[Virtual Machine] The environment in which a hosted operating
smh22@8242 2533 system runs, providing the abstraction of a dedicated machine. A
smh22@8242 2534 virtual machine may be identical to the underlying hardware (as in
smh22@8242 2535 {\bf full virtualization}, or it may differ, as in {\bf
smh22@8242 2536 paravirtualization}).
smh22@8242 2537
smh22@8242 2538 \item[VMM] Virtual Machine Monitor - the software that allows multiple
smh22@8242 2539 virtual machines to be multiplexed on a single physical machine.
smh22@8242 2540
smh22@8242 2541 \item[Xen] Xen is a paravirtualizing virtual machine monitor,
smh22@8242 2542 developed primarily by the Systems Research Group at the University
smh22@8242 2543 of Cambridge Computer Laboratory.
smh22@8242 2544
smh22@8242 2545 \item[XenLinux] A name for the port of the Linux kernel that
smh22@8242 2546 runs on Xen.
smh22@8242 2547
smh22@8242 2548 \end{description}
smh22@8242 2549
smh22@2848 2550
kaf24@2743 2551 \end{document}
kaf24@2743 2552
kaf24@2743 2553
kaf24@2743 2554 %% Other stuff without a home
kaf24@2743 2555
kaf24@2743 2556 %% Instructions Re Python API
kaf24@2743 2557
kaf24@2743 2558 %% Other Control Tasks using Python
kaf24@2743 2559 %% ================================
kaf24@2743 2560
kaf24@2743 2561 %% A Python module 'Xc' is installed as part of the tools-install
kaf24@2743 2562 %% process. This can be imported, and an 'xc object' instantiated, to
kaf24@2743 2563 %% provide access to privileged command operations:
kaf24@2743 2564
kaf24@2743 2565 %% # import Xc
kaf24@2743 2566 %% # xc = Xc.new()
kaf24@2743 2567 %% # dir(xc)
kaf24@2743 2568 %% # help(xc.domain_create)
kaf24@2743 2569
kaf24@2743 2570 %% In this way you can see that the class 'xc' contains useful
kaf24@2743 2571 %% documentation for you to consult.
kaf24@2743 2572
kaf24@2743 2573 %% A further package of useful routines (xenctl) is also installed:
kaf24@2743 2574
kaf24@2743 2575 %% # import xenctl.utils
kaf24@2743 2576 %% # help(xenctl.utils)
kaf24@2743 2577
FMJ@7771 2578 %% You can use these modules to write your own custom scripts or you
FMJ@7771 2579 %% can customise the scripts supplied in the Xen distribution.
mwilli2@2746 2580
mwilli2@2762 2581
mwilli2@2762 2582
mwilli2@2746 2583 % Explain about AGP GART
mwilli2@2762 2584
mwilli2@2762 2585
FMJ@7771 2586 %% If you're not intending to configure the new domain with an IP
FMJ@7771 2587 %% address on your LAN, then you'll probably want to use NAT. The
FMJ@7771 2588 %% 'xen_nat_enable' installs a few useful iptables rules into domain0
FMJ@7771 2589 %% to enable NAT. [NB: We plan to support RSIP in future]
mwilli2@2762 2590
mwilli2@2762 2591
mwilli2@2762 2592
mwilli2@2762 2593 %% Installing the file systems from the CD
mwilli2@2762 2594 %% =======================================
mwilli2@2762 2595
FMJ@7771 2596 %% If you haven't got an existing Linux installation onto which you
FMJ@7771 2597 %% can just drop down the Xen and Xenlinux images, then the file
FMJ@7771 2598 %% systems on the CD provide a quick way of doing an install. However,
FMJ@7771 2599 %% you would be better off in the long run doing a proper install of
FMJ@7771 2600 %% your preferred distro and installing Xen onto that, rather than
FMJ@7771 2601 %% just doing the hack described below:
mwilli2@2762 2602
FMJ@7771 2603 %% Choose one or two partitions, depending on whether you want a
FMJ@7771 2604 %% separate /usr or not. Make file systems on it/them e.g.:
FMJ@7771 2605 %% mkfs -t ext3 /dev/hda3
FMJ@7771 2606 %% [or mkfs -t ext2 /dev/hda3 && tune2fs -j /dev/hda3 if using an old
mwilli2@2762 2607 %% version of mkfs]
mwilli2@2762 2608
mwilli2@2762 2609 %% Next, mount the file system(s) e.g.:
mwilli2@2762 2610 %% mkdir /mnt/root && mount /dev/hda3 /mnt/root
mwilli2@2762 2611 %% [mkdir /mnt/usr && mount /dev/hda4 /mnt/usr]
mwilli2@2762 2612
mwilli2@2762 2613 %% To install the root file system, simply untar /usr/XenDemoCD/root.tar.gz:
mwilli2@2762 2614 %% cd /mnt/root && tar -zxpf /usr/XenDemoCD/root.tar.gz
mwilli2@2762 2615
mwilli2@2762 2616 %% You'll need to edit /mnt/root/etc/fstab to reflect your file system
mwilli2@2762 2617 %% configuration. Changing the password file (etc/shadow) is probably a
mwilli2@2762 2618 %% good idea too.
mwilli2@2762 2619
FMJ@7771 2620 %% To install the usr file system, copy the file system from CD on
FMJ@7771 2621 %% /usr, though leaving out the "XenDemoCD" and "boot" directories:
FMJ@7771 2622 %% cd /usr && cp -a X11R6 etc java libexec root src bin dict kerberos
FMJ@7771 2623 %% local sbin tmp doc include lib man share /mnt/usr
mwilli2@2762 2624
mwilli2@2762 2625 %% If you intend to boot off these file systems (i.e. use them for
FMJ@7771 2626 %% domain 0), then you probably want to copy the /usr/boot
FMJ@7771 2627 %% directory on the cd over the top of the current symlink to /boot
FMJ@7771 2628 %% on your root filesystem (after deleting the current symlink)
FMJ@7771 2629 %% i.e.:
mwilli2@2762 2630 %% cd /mnt/root ; rm boot ; cp -a /usr/boot .