[title]
-[intro]
+[intro, push gate]
osstest is the Xen Project's automated testing system.
-[push gate]
-
We use the production instance to test the mainline development branch
of the Xen hypervisor and tools, as found in xen.git.
osstest, this new facility will give us the ability to evaluate
the alternatives and plan a transition.
-[community]
+[contributor list and commit histogram]
osstest was originally written by Ian Jackson, starting in November
2009. Ian is still the lead maintainer.
But in the past year he has been joined by contributions from Ian
Campbell, Wei Liu, Dario Faggioli, Roger Pau Monne and Anthony Perard.
+The date of the first contribution from each is shown here on the
+slide.
+
+The graph shows the proportion of the osstest code base, by the month
+it was written, for the last three years. (The big spike late in
+2012, which has been chopped off the top of the graph, is an artefact
+due to a code rearrangement which the analysis didn't properly track.)
We have answered a number of enquiries from community members, and
helped advise them about setting up standalone mode and writing tests.
The developer and user documentation has been vastly improved. The
setup process for standalone mode has been simplified.
+We hope to further expand the osstest contributor base, with the move
+to more accessible infrastructure owned by the Xen Project community.
+
+[jobs and branches diagram]
+
+The new contributors have been expanding the range of tests run by
+osstest, and the range of software tested.
+
+Xen's ARM 32-bit support is fully covered by osstest; we have a number
+of ARM systems for both build and test. osstest is ready for testing
+64-bit ARM when we have the appropriate hardware in our test pool.
+
+We have several new series of test runs (called `branches' in osstest
+terminology). These include seabios, qemu upstream mainline and of
+course Xen 4.4.
+
+We are testing linux-next, the Linux kernel's pre-development branch,
+to try to spot upcoming problems earlier.
+
+We are now doing basic build tests of libvirt's libxl integration.
+Recent work on enhancing the capabilities of NetBSD rump kernels on
+Xen has been supported by a rump kernel specific tests in osstest's
+main tests of xen-unstable and also a specific pushgate for the main
+upstream rump kernel git branch; this supplements the rump kernel
+project's own build tests.
-Decisions about sw
+Our guest tests now extend to Debian HVM guests. (We previously only
+tested HVM with RHEL and various versions of Windows, and Debian only
+PV). We now test our OVMF support, and check that we don't break
+FreeBSD guests.
+Tests of the obsolete xend toolstack have been split off into their
+own test jobs.
-what is osstest
+More generally, some obsolete and duplicated tests have been cleaned
+up and the test matrix has been pruned somewhat. This explains the
+downward as well as upward movements in the number of jobs in each
+xen-unstable flight.
- push gates
+We have had to be fairly aggressive about this because our small pool
+of test machines severely limits our test bandwidth. The new test
+system hosting should enable us to continue to expand the range of
+tests.
-hosting
+Work to extend osstest to do performance measurements and ultimately
+detect performance regressions is in the pipeline.
- currently citrix
- Xen Project LF AB moving to own colo
+[references]
-kinds of tests
+osstest has a small but growing contributor community. The move to
+better infrastructure will hopefully grow the code and the community
+even further.
- basic functional
- compatibility
- external projects
+If you want to find out more, here are the URLs for the