view tools/xm-test/README @ 9786:a22ce69dd703

Fix the 15_create_smallmem_pos.py test, which was failing because the
set console.limit command in the test was never being run. The select in
Console.py was never timing out because there was always someting to read on
the fd, the OOM messages are constant. So the test would hang.

The fix includes:

1) Changing MEM in 15_create_smallmem_pos.py to 32MBs, which is the default
for the tools that should work.
2) Change the XmConsole init to add an argument to set the console limit
when it's created.
3) Set a default large limit for console so we won't hang in the future.
4) Added a new 16_create_smallmem_neg.py test to handle failure situation.
5) Added comment in README.

Signed-off-by: Daniel Stekloff <dsteklof@us.ibm.com>
author stekloff@dyn9047022152.beaverton.ibm.com
date Wed Apr 19 22:58:03 2006 +0100 (2006-04-19)
parents f1014bb3ad6f
children 4ecfbf08b449
line source
2 xm-test README
4 Copyright (C) International Business Machines Corp., 2005
5 Author(s): Dan Smith <danms@us.ibm.com>
6 Woody Marvel <marvel@us.ibm.com>
8 Overview
9 ========
11 This suite provides a framework for testing the Xen userspace tools.
12 The directory structure is:
14 ./xm-test
15 |
16 +-/lib: Python support libraries
17 |
18 +-/ramdisk: Staging area for building the test ramdisk
19 |
20 +-/tests
21 | |
22 | +-/create: Tests for the 'xm create' command
23 | +-/destroy: Tests for the 'xm destroy' command
24 | . . .
25 |
26 +-/utils: Utility scripts for ramdisk building
28 Reports are posted here:
30 http://xmtest.dague.org
33 Building
34 ========
36 Before the test suite can be used, the ramdisk must be built from
37 source. All source needed for this process is automatically
38 downloaded, extracted, and compiled. Due to the need to create
39 special files, this process must be done as root:
41 # ./autogen
42 # ./configure
43 # make
45 NB: If you have the initrd.img from another installation of xm-test,
46 you can copy it into the ramdisk directory to eliminate the need to
47 rebuild it. If you do this, there is no need to run 'make' again.
48 Simply copy the initrd-X.Y.img file into ramdisk/ and then run:
50 # make existing
52 Or, you can run:
53 # INITRD="http://url.of.initrd.repo/" make existing
55 You do not need to include the name of the image itself in the url,
56 however, an initrd with the right name (initrd.X.Y.img) and version
57 number must exist at that location. The script will determine which
58 version of the initrd it needs and try to download the right file from
59 that location.
61 This will set up the link so that xm-test will use the existing
62 ramdisk. Next, just run "runtest.sh" normally. Note that in general,
63 you should not attempt to use a ramdisk from a previous minor version
64 of xm-test (i.e., don't use a ramdisk from 0.4.0 with 0.5.0. 0.5.0
65 should work for 0.5.3 though)
68 BUILDING with HVM Support
69 =========================
71 If you'd like to build and run this with hardware virtual machine assist
72 (HVM) support to test fully virtualized disk images on VMX/SVM hardware,
73 please add the --enable-hvm-support option to configure:
75 # ./autogen
76 # ./configure --enable-hvm-support
77 # make
79 The ramdisk/bin/create_disk_image script, which builds the full virt
80 disk.img, requires Lilo 22.7+ to be installed on the system. Lilo is
81 used to install the bootloader on the disk.img.
83 If HVM support is enabled, the ramdisk/bin/create_disk_image script
84 will be run to create a full virt disk.img in the ramdisk directory. The
85 script, by default, will look in /boot for the first non-Xen kernel it
86 runs across. If you'd like to set xm-test to use a specific kernel,
87 rather than the first one it finds in /boot, you can configure it in
88 with the "--with-hvm-kernel=KERNEL" option:
90 # ./autogen
91 # ./configure --enable-hvm-support --with-hvm-kernel=KERNEL
92 # make
94 Otherwise, you can always rerun the create script using the -k option
95 to use a specific kernel.
97 The disk.img created for HVM testing must contain a pcnet32 driver for
98 network tests. The ramdisk/bin/create_disk_image script will, by default,
99 look in the /lib/modules directory associated with the kernel being
100 used. If you'd like to specify a different location for the driver or
101 want to tell the script that the driver is built into the kernel, please
102 use the "--with-driver-dir=DRVDIR" configure option. If built into
103 the kernel, please use the key word "builtin" with the option:
105 # ./autogen
106 # ./configure --enable-hvm-support --with-driver-dir=builtin
107 - or -
108 # ./configure --enable-hvm-support --with-driver-dir=/driver/directory
109 # make
111 Xm-test will look for disk.img in the ramdisk directory when run by
112 default.
115 Running
116 =======
118 To run the full test suite, do the following as root:
120 # ./runtest.sh <logfile>
122 This will run all tests, as well as generate and submit a report at
123 the end. All output files will begin with "<logfile>."
124 If you wish to prevent submission of a report, add "-d" to the
125 command line like this:
127 # ./runtest.sh -d <logfile>
129 It may be useful to run tests without submission as above, and then
130 submit the report at a later time. To do so, run runtest.sh with the
131 -s flag and the name of the previously-generated report:
133 # ./runtest.sh -s <logfile>
135 Group test sets are supported in xm-test. This is form of layering of
136 tests groups/cases/tests. In the framework directory "grouptest",
137 files exist for group processing. The user can add groups, casenames
138 and test lists as required. Default group run is "grouptest/default".
140 # ./runtest.sh -g <groupname> <logfile>
142 * NOTE: There is a quick set of tests in group mode, that was added to
143 run certain casenames and tests, and there is a "medium" group, which is a
144 medium-length run (around 20 minutes). Neither is a substitute for the full
145 xm-test test suite.
146 # ./runtest.sh -g quick <logfile>
147 # ./runtest.sh -g medium <logfile>
151 It may be desirable to run a specific test group. This can be
152 accomplished by doing the following:
154 # cd tests/create
155 # TEST_VERBOSE=1 make check
157 When developing or debugging a specific feature, a single test can be
158 run to avoid having to run even a whole test group:
160 # cd tests/create
161 # TEST_VERBOSE=1 make check TESTS=01_create_basic_pos.test
163 The runtest.sh script will create several files, including a .report
164 file, which is the cleaned up, email-friendly report of failures.
165 Additionally, the script will submit your results to the development
166 team for trend analysis. This helps us determine the level of success
167 people "out there" are having with different versions of Xen.
169 Note: you should generally run xm-test with a minimum of memory
170 allocated to Dom0. More memory available for allocation to DomUs
171 means a more rigorous test.
173 BIG FAT WARNING: The test framework assumes it is running on a
174 dedicated machine. As such, the library automatically destroys any
175 running DomUs on the system to provide each test with a "clean slate".
178 Extending
179 =========
181 Additional tests may be added in existing groups to test additional
182 cases for a given xm subcommand. Test programs should be named
183 according to the following scheme:
185 XY_group_name_{pos,neg}.py
187 Where:
188 XY is the next number in line
189 group is the name of the subcommand being tested
190 name is the short name of the test
191 {pos,neg} denotes whether this is a positive or negative test case
193 New subcommand groups should be added as directories named after the
194 subcommand itself. The "Makefile.am.template" should be copied into
195 the new group directory as "Makefile.am".
197 See the Writing_Tests_HOWTO file for more detailed information on
198 adding tests to the suite.
201 Developer Notes
202 ===============
204 Our library provides a DomU console abstraction for automated
205 execution of commands. Please note that this is relatively fragile,
206 and is intended for use only with the ramdisk built by the framework.
207 Because the console experiences some occasional corruption, this
208 method is not completely perfect at the moment, although the authors
209 use it with relatively few problems.
212 Known Issues
213 ============
215 If you create a domain with a small amount of memory, under 32MBs, you
216 may run into out of memory situations for the domain. There's no way
217 to know the amount of memory needed by the kernel and modules used. Xm-test
218 uses 64MBs as default and that should work. If there are out of memory
219 issues, the default can be changed. Edit xm-test/lib/XmTestLib/XenDomain.py
220 and change ParavirtDefaults and HVMDefaults "memory".
222 There are two tests that work with small memory, 15_create_smallmem_pos.py
223 and 16_create_smallmem_neg.py. The first makes sure the default 32 MBs
224 limit works. The second checks a low memory fail situation. These tests
225 are located in the xm-test/tests/create directory and can be easily edited
226 to change the MEM value they should test. If the 32MBs test fails, the
227 failure should be reported to the Xen xen-devel mailing list. The Xen
228 tools use 32MBs as a lower acceptable limit for domain creation. The Xen
229 mailing lists are located here:
231 http://lists.xensource.com/
234 Reporting Bugs
235 ==============
237 If you find a bug in the test framework, report it to:
239 Dan Smith <danms@us.ibm.com>
241 If you find a bug in a specific test case, contact the author of the
242 test case first.