direct-io.hg
changeset 2000:1ab8cef48b73
bitkeeper revision 1.1108.35.9 (410a4c2cTzNyYvGTX5H_zUE1Pwg8GQ)
Xend documentation.
Xend documentation.
author | mjw@wray-m-3.hpl.hp.com |
---|---|
date | Fri Jul 30 13:25:00 2004 +0000 (2004-07-30) |
parents | 3eab8d7061ba |
children | 66687fe8d1c7 |
files | .rootkeys docs/xend.tex |
line diff
1.1 --- a/.rootkeys Fri Jul 30 13:24:25 2004 +0000 1.2 +++ b/.rootkeys Fri Jul 30 13:25:00 2004 +0000 1.3 @@ -18,6 +18,7 @@ 3f9e7d564bWFB-Czjv1qdmE6o0GqNg docs/inte 1.4 3f9e7d58t7N6hjjBMxSn-NMxBphchA docs/style.tex 1.5 410144afnSd2Yw68AHGO5gXu2m3y6A docs/user.tex 1.6 40d6ccbfKKBq8jE0ula4eHEzBiQuDA docs/xen_config.html 1.7 +410a4c2bAO_m_l4RsiiPHnZ4ixHWbQ docs/xend.tex 1.8 3f9e7d5bz8BwYkNuwyiPVu7JJG441A docs/xenstyle.cls 1.9 3f815144d1vI2777JI-dO4wk49Iw7g extras/mini-os/Makefile 1.10 3f815144zTnCV5591ulIJQrpe5b-5Q extras/mini-os/README
2.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000 2.2 +++ b/docs/xend.tex Fri Jul 30 13:25:00 2004 +0000 2.3 @@ -0,0 +1,448 @@ 2.4 +% -*- mode: LaTeX -*- 2.5 +\def\seca{\chapter} 2.6 +\def\secb{\section} 2.7 +\def\secc{\subsection} 2.8 +\def\secd{\subsubsection} 2.9 +\def\refa{chapter} 2.10 +\def\refb{section} 2.11 +\def\refc{section} 2.12 +\def\refd{section} 2.13 + 2.14 +%\def\seca{\section} 2.15 +%\def\secb{\subsection} 2.16 +%\def\secc{\subsubsection} 2.17 +%\def\refa{section} 2.18 +%\def\refb{section} 2.19 +%\def\refc{section} 2.20 + 2.21 +\documentclass[11pt,twoside,final,openright]{xenstyle} 2.22 +\usepackage{a4,graphicx,setspace} 2.23 +\setstretch{1.15} 2.24 +\input{style.tex} 2.25 + 2.26 +\begin{document} 2.27 + 2.28 +% TITLE PAGE 2.29 +\pagestyle{empty} 2.30 +\begin{center} 2.31 +\vspace*{\fill} 2.32 +\includegraphics{eps/xenlogo.eps} 2.33 +\vfill 2.34 +\vfill 2.35 +\vfill 2.36 +\begin{tabular}{l} 2.37 +{\Huge \bf Xend} \\[4mm] 2.38 +{\huge Xen v2.0 for x86} \\[80mm] 2.39 + 2.40 +{\Large Xen is Copyright (c) 2004, The Xen Team} \\[3mm] 2.41 +{\Large University of Cambridge, UK} \\[20mm] 2.42 +{\large Last updated 30 August 2004} 2.43 +\end{tabular} 2.44 +\vfill 2.45 +\end{center} 2.46 +\cleardoublepage 2.47 + 2.48 +% TABLE OF CONTENTS 2.49 +\pagestyle{plain} 2.50 +\pagenumbering{roman} 2.51 +{ \parskip 0pt plus 1pt 2.52 + \tableofcontents } 2.53 +\cleardoublepage 2.54 +% PREPARE FOR MAIN TEXT 2.55 +\pagenumbering{arabic} 2.56 +\raggedbottom 2.57 +\widowpenalty=10000 2.58 +\clubpenalty=10000 2.59 +\parindent=0pt 2.60 +\renewcommand{\topfraction}{.8} 2.61 +\renewcommand{\bottomfraction}{.8} 2.62 +\renewcommand{\textfraction}{.2} 2.63 +\renewcommand{\floatpagefraction}{.8} 2.64 + 2.65 +\setstretch{1.15} 2.66 + 2.67 +\seca{Introduction} 2.68 +Xend is the control daemon used to manage a machine running the Xen hypervisor. 2.69 +Xend is responsible for creating and destroying domains and managing their 2.70 +resources, such as virtual block devices and virtual network interfaces. 2.71 + 2.72 +Xend exists because the Xen hypervisor itself only manages the memory image 2.73 +of a domain and its scheduling. Xen provides the event channels that connect 2.74 +a domain to its devices, but is intentionally not involved in setting them up. 2.75 + 2.76 +Xend runs as a daemon in the privileged domain 0 and uses a low-level api 2.77 +to communicate with Xen via the domain 0 kernel. Xend exports its control 2.78 +interface to its clients using HTTP. Most programming languages have 2.79 +HTTP client libraries, so this interface can be used from most popular 2.80 +languages, for example Python, Perl, C, Java. 2.81 +Xend itself is written in Python, as are most of the Xen tools. 2.82 + 2.83 +The xend interface is intended to be a complete interface for the creation 2.84 +and management of domains. It supports domain creation, shutdown, reboot, 2.85 +destruction, save, restore and migration. 2.86 + 2.87 +When xend creates a domain it creates the domain memory image and communicates 2.88 +with the device driver domain(s) to configure the devices for the domain. 2.89 +This sets up connections between the domain and backend device controllers 2.90 +in the driver domain. When a domain shuts down its memory image cannot be fully released 2.91 +unless its backend devices are released and disconnected. This is done by xend. 2.92 +In order to protect against loss of this information when xend is restarted, 2.93 +xend maintains a persistent database of domain configurations. This allows 2.94 +xend to be stopped and restarted without loss of configuration information. 2.95 +For example, in order to upgrade the xend software. 2.96 + 2.97 +\seca{Domain lifecycle} 2.98 +\secb{Domain creation} 2.99 +Xend is instructed to create a domain by posting a domain\_create message to it, 2.100 +containing the domain configuration to be instantiated. The domain configuration 2.101 +is in sxp format and is as far as possible {\em fully-bound}, that is, all 2.102 +parameters are fully-specified. The domain configuration is saved in the filesystem 2.103 +so that it can be reused later if necessary. 2.104 + 2.105 +The domain configuration specifies the domain name, memory size, kernel image 2.106 +and parameters, and all the domain devices. Xend uses the Xen api to create 2.107 +the domain memory image, and then {\em builds} the memory image for the domain 2.108 +using the kernel image. At this point the domain exists, but it cannot be run because 2.109 +it has no devices. Xend then communicates with the device driver domain to create 2.110 +the configured devices. Once the devices are created it sets up event channels 2.111 +for them between the driver domain and the new domain, and notifies the new domain 2.112 +that its devices are connected. At this point the domain can be started. 2.113 + 2.114 +Xend is also responsible for managing domain consoles. When a domain is created, 2.115 +xend sets up a console event channel to the domain, and creates a TCP listening port 2.116 +for the domain console. When a connection is accepted to the port, xend 2.117 +connects input and output for the port to the domain console channel. 2.118 + 2.119 +\secb{Domain destruction} 2.120 +When a domain terminates, because it has been shutdown or it has crashed, the 2.121 +domain resources must be released so that the domain memory image can be 2.122 +finally removed from xen. Xend monitors the domains, and is also signaled by 2.123 +xen (using a VIRQ) when a domain exits. Xend examines the domain states and 2.124 +determines which domains have exited. It then communicates with the driver domain 2.125 +to release the devices for exited domains. Xend also closes any open console 2.126 +connections and removes the TCP listeners for exited domains. 2.127 +Once all devices have been released it instructs xen to destroy the memory image. 2.128 + 2.129 +\seb{Domain restart} 2.130 +Domain restart is the xen equivalent of a machine reboot. When a domain 2.131 +exits because it has been shutdown in reboot mode, its exit code is reboot. 2.132 +When examining domains to find those that have exited and destroy them, 2.133 +xend detects those that have exited for reboot and does not completely destroy 2.134 +them. It disconnects all their devices, and detaches the console listener 2.135 +from its channel to the domain, but does not close it. Instead it schedules 2.136 +a call to rebuild the domain from its configuration. This proceeds almost 2.137 +identically to creating the domain, except that the console listener is 2.138 +reused and connected to the new domain. This allows existing console 2.139 +connections to remain connected across a domain restart. The restarted 2.140 +domain keeps the same name and domain id. 2.141 + 2.142 +The determination of when to restart a domain is in fact slightly more 2.143 +complex than described above. A domain is configured with a 2.144 +{\em restart mode}. If the restart mode is {\em onreboot}, the default, 2.145 +restart happens when the domain is shutdown normally and 2.146 +exits with code reboot. If the restart mode is {\em never} the domain is 2.147 +not restarted. If the restart mode is {\em always} the domain is always 2.148 +restarted, regardless of how it exited. 2.149 + 2.150 +In order to prevent continual domain crash causing restart loops, xend 2.151 +has a {\em minimum restart time}. Xend remembers when a domain was last 2.152 +restarted and will fail a restart that happens inside the minimum 2.153 +restart time. 2.154 + 2.155 +\seca{Devices} 2.156 +\secb{Virtual network interfaces} 2.157 +Each virtual network interface (vif) has 2 parts: the font-end device in its domain, 2.158 +and the back-end device in the driver domain. Usually the driver domain is domain 0, 2.159 +and there is a linux network device corresponding to the vif. The linux device for 2.160 +interface N on domain D is called vifD.N. When a packet is sent on the vif in the 2.161 +domain the packet is received from the linux device. The linux devices are connected 2.162 +to network interfaces using ethernet bridging. 2.163 + 2.164 +The default setup is a bridge xen-br0, with eth0 connected to it, and the routes 2.165 +for eth0 directed at xen-br0. This is controlled by the xend network setup script, 2.166 +default {\tt /etc/xen/network}, which is run when xend starts. 2.167 + 2.168 +When the vifs for a domain are created, a vif control script, default {\tt /etc/xen/vif-bridge}, 2.169 +is run to connect the vif to its bridge. The default script connects the vif 2.170 +to xen-br0 and optionally sets up iptables rules to prevent IP address spoofing. 2.171 +The bridge a vif is connected to can be defined in its configuration, and this is useful 2.172 +for setting up virtual networks using several bridges. 2.173 + 2.174 +\secb{Virtual block devices} 2.175 +Virtual block devices in a domain are interfaces onto back-end device drivers 2.176 +that export physical devices to domains. In the default configuration the back-end 2.177 +driver is in domain 0 and can export any linux block device to a domain. This includes 2.178 +physical disk partitions, LVM volumes and loopback mounts of files. In fact anything 2.179 +that linux can represent as a block device can be exported to a domain as virtual 2.180 +block device. 2.181 + 2.182 +\seca{Xend invocation} 2.183 +Xend is started (by root) using the command 2.184 +\begin{verbatim} 2.185 +xend start 2.186 +\end{verbatim} 2.187 +Xend can be stopped using 2.188 +\begin{verbatim} 2.189 +xend stop 2.190 +\end{verbatim} 2.191 +Xend must be started before any domains (apart from domain 0) can be created. 2.192 +If you try to use the {\tt xm} tool when xend is not running you will get a 2.193 +'connection refused' message. 2.194 + 2.195 +\secb{Xend configuration} 2.196 +Xend reads its own configuration from {\tt /etc/xen/xend-config.sxp}, which is 2.197 +a sequence of s-expressions. The configuration parameters are: 2.198 +\begin{itemize} 2.199 + 2.200 +\item xend-port: Port xend should use for the HTTP interface (default 8000). 2.201 + 2.202 +\item xend-address: Address xend should listen on. 2.203 + Specifying 'localhost' prevents remote connections. 2.204 + Specifying the empty string '' allows all connections, and is the default. 2.205 + 2.206 +\item network-script: The script used to start/stop networking for xend (default network). 2.207 + 2.208 +\item vif-bridge: The default bridge that virtual interfaces should be connected to 2.209 + (default xen-br0). 2.210 + 2.211 +\item vif-script: The default script used to control virtual interfaces 2.212 + (default vif-bridge). 2.213 + 2.214 +\item vif-antispoof: Whether iptables should be set up to prevent IP spoofing for 2.215 + virtual interfaces (default yes). 2.216 +\end{itemize} 2.217 + 2.218 +Configuration scripts ({\it e.g.} for network-script) are looked for in {\tt /etc/xen} 2.219 +unless their name begins with '/'. 2.220 + 2.221 +Xend sends its log output to {\tt /var/log/xend.log}. This is a rotating logfile, 2.222 +and logs are moved onto {\tt xend.log.1} {\it etc.} as they get large. Old logs may 2.223 +be deleted. 2.224 + 2.225 +\secb{Xend database} 2.226 +Xend needs to make some data persistent, and it uses files under {\tt /var/xen/xend-db} 2.227 +for this. The persistent data is stored in files in SXP format. Domain information 2.228 +is saved when domains are created. When xend starts it reads the file {\tt /var/xen/lastboot} 2.229 +(if it exists) to determine the last time the system was rebooted. It compares this time 2.230 +with the last reboot time in {\tt wtmp} to determine if the system has been rebooted 2.231 +since xend last ran. If the system has been rebooted xend removes all its saved data 2.232 +that is not persistent across reboots (for example domain data). 2.233 + 2.234 +\seca{Xend HTTP Interface} 2.235 + The xend interface uses HTTP 1.1 \cite{http} as its transport. 2.236 +Simple PUT and GET calls can encode parameters using the standard url-encoding 2.237 +for parameters: MIME type {\tt application/x-www-form-urlencoded}. 2.238 +When file upload is required, the {\tt multipart/form-data} encoding is used. 2.239 +See the HTML 4.1 specification for details \cite{html}. 2.240 + 2.241 +Xend functions as a webserver and supports two interfaces: one 2.242 +for web-browsers and one for programs. 2.243 +The web-browser interface returns replies in HTML and includes forms 2.244 +for interactive operations such as stopping domains and creating domains 2.245 +from an uploaded configuration. The programmatic interface usually returns replies 2.246 +in s-expression format. Both interfaces are accessed 2.247 +in exactly the same way over HTTP - the only difference is the data returned. 2.248 + 2.249 +The webserver distinguishes browsers from programs using the {\tt User-Agent} 2.250 +and {\tt Accept} headers in the HTTP request. If there is no {\tt User-Agent} or no 2.251 +{\tt Acccept} header, or {\tt Accept} includes the type {\tt application/sxp}, the 2.252 +webserver assumes the client is a program and returns SXP. Otherwise 2.253 +it assumes the client is a webserver and returns HTML. 2.254 +In some cases the return value is essentially a string, so {\tt Content-Type} 2.255 +{\tt text/plain} is returned. 2.256 + 2.257 +The HTTP api supported is listed below. All paths in it are relative to the 2.258 +server root, for example {\tt http://localhost:8000/xend}. 2.259 +As defined in the HTTP specification, we use GET for side-effect free 2.260 +operations that may safely be repeated, and POST for operations with 2.261 +side-effects. For each request we list the HTTP method (GET or POST), 2.262 +the url relative to the server root, the operation name and arguments (if any). 2.263 +The operation name is passed as request parameter 'op', and the arguments 2.264 +are passed by name. Operation name and parameters can be encoded using either 2.265 +encoding described above. We also list the corresponding api function from the 2.266 +Python client interface in {\tt xen.xend.XendClient}. 2.267 + 2.268 +\begin{itemize} 2.269 +\item {\tt GET /},\\ 2.270 + {\tt xend()}:\\ 2.271 + Get list of urls under xend root. 2.272 + 2.273 +\item {\tt GET /node},\\ 2.274 + {\tt xend\_node()}:\\ 2.275 + Get node information. 2.276 + 2.277 +\item {\tt POST /node shutdown()},\\ 2.278 + {\tt xend\_node\_shutdown()}:\\ 2.279 + Shutdown the node 2.280 + 2.281 +\item {\tt POST /node reboot()},\\ 2.282 + {\tt xend\_node\_reboot()}:\\ 2.283 + Reboot the node 2.284 + 2.285 +\item {\tt POST /node notify()}:\\ 2.286 + Set node notification url 2.287 + 2.288 +\item {\tt GET /node/dmesg},\\ 2.289 + {\tt xend\_node\_dmesg()}:\\ 2.290 + Get xen boot message. 2.291 + 2.292 +\item {\tt GET /node/log},\\ 2.293 + {\tt xend\_node\_log()}:\\ 2.294 + Get xend log. 2.295 + 2.296 +\item {\tt GET /domain}\\ 2.297 + {\tt xend\_domains()}:\\ 2.298 + Get list of domains. 2.299 + 2.300 +\item {\tt POST /domain create(config)},\\ 2.301 + {\tt xend\_domain\_create(config)}:\\ 2.302 + Create a domain. 2.303 + 2.304 +\item {\tt POST /domain restore(file)},\\ 2.305 + {\tt xend\_domain\_restore(filename)}:\\ 2.306 + Restore a saved domain. 2.307 + 2.308 +\item {\tt GET /domain/<dom>},\\ 2.309 + {\tt xend\_domain(dom)}:\\ 2.310 + Get domain information. 2.311 + 2.312 +\item {\tt POST /domain/[dom] configure(config)},\\ 2.313 + {\tt xend\_domain\_configure(dom, conf)}:\\ 2.314 + Configure an existing domain (for internal use by restore and migrate). 2.315 + 2.316 +\item {\tt POST /domain/[dom] unpause()},\\ 2.317 + {\tt xend\_domain\_unpause(dom)}:\\ 2.318 + Start domain running 2.319 + 2.320 +\item {\tt POST /domain/[dom] pause()},\\ 2.321 + {\tt xend\_domain\_pause(dom)}:\\ 2.322 + Stop domain running. 2.323 + 2.324 +\item {\tt POST /domain/[dom] shutdown(reason)},\\ 2.325 + {\tt xend\_domain\_shutdown(dom, reason)}:\\ 2.326 + Shutdown domain, reason can be reboot, poweroff, halt. 2.327 + 2.328 +\item {\tt POST /domain/[dom] destroy(reason)},\\ 2.329 + {\tt xend\_domain\_destroy(dom, reason)}:\\ 2.330 + Destroy domain, reason can be reboot, halt. 2.331 + 2.332 +\item {\tt POST /domain/[dom] save(file)},\\ 2.333 + {\tt xend\_domain\_save(dom, filename)}:\\ 2.334 + Save a domain to a file. 2.335 + 2.336 +\item {\tt POST /domain/[dom] migrate(dst)},\\ 2.337 + {\tt xend\_domain\_migrate(dom, dst)}:\\ 2.338 + Migrate a domain. 2.339 + 2.340 +\item {\tt POST /domain/[dom] pincpu(cpu)},\\ 2.341 + {\tt xend\_domain\_pincpu(self, id, cpu)}\\: 2.342 + Pin a domain to a cpu. 2.343 + 2.344 +\item {\tt POST /domain/[dom] bvt\_set(mcuadv, warp, warpl, warpu)},\\ 2.345 + {\tt xend\_domain\_cpu\_bvt\_set(dom, mcuadv, warp, warpl, warpu)}:\\ 2.346 + Set BVT scheduler parameters. 2.347 + 2.348 +\item {\tt POST /domain/[dom] fbvt\_set(mcuadv, warp, warpl, warpu)},\\ 2.349 + {\tt xend\_domain\_cpu\_fbvt\_set(dom, mcuadv, warp, warpl, warpu)}:\\ 2.350 + Set FBVT scheduler parameters. 2.351 + 2.352 +\item {\tt POST /domain/[dom] atropos\_set(period, slice, latency, xtratime)},\\ 2.353 + {\tt xend\_domain\_cpu\_atropos\_set(dom, period, slice, latency, xtratime)}:\\ 2.354 + Set atropos scheduler parameters. 2.355 + 2.356 +\item {\tt POST /domain/[dom] maxmem\_set(memory)},\\ 2.357 + {\tt xend\_domain\_maxmem\_set(dom, memory)}:\\ 2.358 + Set domain maximum memory limit. 2.359 + 2.360 +\item {\tt POST /domain/[dom] device\_create(config)}\\ 2.361 + {\tt xend\_domain\_device\_create(dom, config)}:\\ 2.362 + Add a device to a domain. 2.363 + 2.364 +\item {\tt POST /domain/[dom] device\_destroy(type, index)},\\ 2.365 + {\tt xend\_domain\_device\_destroy(dom, type, index)}:\\ 2.366 + Delete a device from a domain 2.367 + 2.368 +\item {\tt GET /domain/[dom] vifs()},\\ 2.369 + {\tt xend\_domain\_vifs(dom)}:\\ 2.370 + Get virtual network interfaces. 2.371 + 2.372 +\item {\tt GET /domain/[dom] vif(vif)},\\ 2.373 + {\tt xend\_domain\_vif(dom, vif)}:\\ 2.374 + Get virtual network interface. 2.375 + 2.376 +\item {\tt GET /domain/[dom] vbds()},\\ 2.377 + {\tt xend\_domain\_vbds(dom)}:\\ 2.378 + Get virtual block devices. 2.379 + 2.380 +\item {\tt GET /domain/[dom] vbd(vbd)},\\ 2.381 + {\tt xend\_domain\_vbd(dom, vbd)}:\\ 2.382 + Get virtual block device. 2.383 + 2.384 +\item {\tt GET /console},\\ 2.385 + {\tt xend\_consoles()}:\\ 2.386 + Get list of consoles. 2.387 + 2.388 +\item {\tt GET /console/[id]}\\ 2.389 + {\tt xend\_console(id)}:\\ 2.390 + Get information about a console. 2.391 + 2.392 +\item {\tt GET /console/[id] disconnect()}\\ 2.393 + {\tt xend\_console\_disconnect(self, id)}:\\ 2.394 + Disconnect any console TCP connection. 2.395 + 2.396 +\item {\tt GET /vnet}\\ 2.397 + {\tt xend\_vnets()}:\\ 2.398 + Get list of vnets (virtual networks). 2.399 + 2.400 +\item {\tt GET /vnet/[id]}\\ 2.401 + {\tt xend\_vnet(id)}:\\ 2.402 + Get information about a virtual network. 2.403 + 2.404 +\item {\tt POST /vnet create(config)}\\ 2.405 + {\tt xend\_vnet\_create(conf)}:\\ 2.406 + Create a vnet. 2.407 + 2.408 +\item {\tt POST /vnet/[id] delete()}\\ 2.409 + {\tt xend\_vnet\_delete(id)}:\\ 2.410 + Delete a vnet. 2.411 + 2.412 +\item {\tt POST /event inject(event)}\\ 2.413 + {\tt xend\_event\_inject(sxpr)}:\\ 2.414 + Inject an event. 2.415 + 2.416 +\end{itemize} 2.417 + 2.418 +\secb{Xend debugging interface} 2.419 +Xend also listens on port 8001. Connecting to this port (for example via telnet) 2.420 +allows access to some debugging functions: 2.421 +\begin{itemize} 2.422 +\item help: list functions 2.423 +\item traceon: turn xend tracing on 2.424 +\item traceoff: turn xend tracing off 2.425 +\item quit: disconnect. 2.426 +\item info: list console listeners, block and network device controllers. 2.427 +\end{itemize} 2.428 + 2.429 +When tracing is on xend logs all functions calls and exceptions to 2.430 +{\tt /var/log/xend.trace}. 2.431 + 2.432 +\begin{thebibliography}{99} 2.433 + 2.434 +\bibitem{html} 2.435 +HTML 4.01 Specification,\\ 2.436 +http://www.w3.org/TR/html4/,\\ 2.437 +W3C Recommendation, 24 December 1999. 2.438 + 2.439 +\bibitem{http} 2.440 +Hypertext Transfer Protocol -- HTTP/1.1,\\ 2.441 +http://www.ietf.org/rfc/rfc2616.txt,\\ 2.442 +RFC 2616, IETF 1999. 2.443 + 2.444 +\bibitem{ssh} 2.445 +http://www.openssh.org. 2.446 + 2.447 +\bibitem{stunnel} 2.448 +http://www.stunnel.org. 2.449 + 2.450 +\end{thebibliography} 2.451 +\end{document} 2.452 \ No newline at end of file