]> xenbits.xensource.com Git - people/dariof/osstest.git/log
people/dariof/osstest.git
10 years agocr-for-branches: add linux-3.14
Ian Jackson [Fri, 25 Jul 2014 16:23:52 +0000 (17:23 +0100)]
cr-for-branches: add linux-3.14

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agoREADME.dev: Slightly tart up the new branch instructions
Ian Jackson [Fri, 25 Jul 2014 16:23:40 +0000 (17:23 +0100)]
README.dev: Slightly tart up the new branch instructions

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agomake-flight: Enable qemu tests for 4.5 onwards
Ian Campbell [Wed, 16 Jul 2014 11:45:05 +0000 (12:45 +0100)]
make-flight: Enable qemu tests for 4.5 onwards

Stefano is about to enable qemu support on ARM, we should test it.

standalone-generate-dump-flight-runvars verifies that the affect of this change
is to add armhf jobs/runvars to qemu-mainline and qemu-upstream-unstable as
desired.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Stefano.Stabellini@eu.citrix.com
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
10 years agostandalone-generate-dump-flight-runvars: allow 26 chars for branch name
Ian Campbell [Wed, 16 Jul 2014 11:44:46 +0000 (12:44 +0100)]
standalone-generate-dump-flight-runvars: allow 26 chars for branch name

qemu-upstream-X.Y-testing is 25 characters. Also add a space.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
10 years agorump kernel tests: Run xenstore-ls demo
Ian Jackson [Mon, 14 Jul 2014 12:08:58 +0000 (13:08 +0100)]
rump kernel tests: Run xenstore-ls demo

We can use the newly-generalised ts-rumpuserxen-demo-setup to set up
the config file etc. to run xenstore-ls from the xen.git build.

We have a bespoke script to actually execute it: we extract the actual
program output from the guest console log, and compare the results to
those we expect (based on running xenstore-ls in dom0).

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
v2: Fix copyright date in new script.

10 years agots-guest-destroy-hard: New script
Ian Jackson [Tue, 15 Jul 2014 14:18:00 +0000 (15:18 +0100)]
ts-guest-destroy-hard: New script

This just calls xl destroy; it doesn't check that the guest is
running, or on the network, or anything.  (It also doesn't check that
the guest is down afterwards.)

We are going to want this to be able to destroy the rumpuserxen
xenstorels domain, which is short-lived (and will normally have
terminated by the time we want to destroy it).

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v2: Fix copyright date in new script.

10 years agorump kernel tests: Specify cmdline via runvar
Ian Jackson [Mon, 14 Jul 2014 11:58:57 +0000 (12:58 +0100)]
rump kernel tests: Specify cmdline via runvar

The runvar <gn>_cmdline is used for the rumpuserxen guest command
line.  Previously "3" (a binary flags word meaningful to the test/demo
program included in rumpuser-xen.git) was hardcoded.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
10 years agorump kernel tests: Specify kernel via runvar
Ian Jackson [Tue, 15 Jul 2014 14:40:08 +0000 (15:40 +0100)]
rump kernel tests: Specify kernel via runvar

The runvar <gn>_builtimage is set to <distpart>:<distpath>.
Previously we hardcoded `/usr/local/lib/xen/rump-kernel' in
`rumpuserxen'.

The ffs filesystem images, and the demo config template, still come
from a single particular build, which we find from the
`guests_rumpuserxenbuildjob' runvar.  This runvar is then doing double
duty as the build job for the ffs filesystem images and example
configs, and the `default' build job for guest kernels.

The subpath /usr/local/lib/xen/rump-kernel is duplicated.  It
now appears:
 1. To find the demo guest's kernel: in make-flight, setting a runvar,
    with the actual rump-kernel kernel image leafname appended.
 2. For the filesystem images and config file: hardcoded in
    ts-rumpuserxen-demo-setup.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v2: Fix demo application kernel image filename.
    Disentangle wording of last para of commit message.

10 years agorump kernel tests: Use guest-specific fs images location
Ian Jackson [Tue, 15 Jul 2014 14:38:11 +0000 (15:38 +0100)]
rump kernel tests: Use guest-specific fs images location

We are going to want to have multiple different guests in the same
job.  Don't use the same locations for all of their fs images.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
10 years agorump kernel tests: Do a xen.git tools build
Ian Jackson [Mon, 14 Jul 2014 12:04:35 +0000 (13:04 +0100)]
rump kernel tests: Do a xen.git tools build

Having laid the groundwork, we can now:

* In make-flight, for the rumpuserxen builds, set the runvars that
  specify which xen.git bits etc. to build.  (We pass qemuu now,
  because we intend that eventually we will want to build that too.)

* In ts-rumpuserxen-build, record the locations of the app-tools
  configure and make wrappers in the runvars which will be honoured by
  ts-xen-build.  Thus ts-xen-build in rumpuserxen build jobs will use
  these app-tools and do a rump cross build.

* Add a ts-xen-build step to the build-rumpuserxen recipe, passing
  `tools' as the target to build.

ts-rumpuserxen-build does not generate a formal set of build outputs.
Instead, the ts-xen-build step references the files generated by
ts-rumpuserxen-build directly (and is told where to find them via the
new command prefix runvars).

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v2: Improve commit message.

10 years agots-xen-build: Honour cmdprefix_configure, cmdprefix_make
Ian Jackson [Mon, 14 Jul 2014 12:03:16 +0000 (13:03 +0100)]
ts-xen-build: Honour cmdprefix_configure, cmdprefix_make

These runvars can be set to specify a string (containing shell code)
to be prefixed to the executions of ./configure and make.

If not set, the behaviour is as as before.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
10 years agots-xen-build: Allow specification of make targets
Ian Jackson [Mon, 14 Jul 2014 12:01:37 +0000 (13:01 +0100)]
ts-xen-build: Allow specification of make targets

This allows sg-run-job (or a manual caller of ts-xen-build) to specify
which target(s) to pass to `make'.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
10 years agomake-flight: rumpuserxen tests: use same buildjob for all guests
Ian Jackson [Mon, 14 Jul 2014 11:52:28 +0000 (12:52 +0100)]
make-flight: rumpuserxen tests: use same buildjob for all guests

Change the name of the runvar for the build job to be
guests_rumpuserxenbuildjob, which is the default (fallback) for
guest_var.

(We are going to introduce another step which uses another guest but
wants the same build job.)

No resulting change in the system's overall behaviour.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v2: Fix stray leftover in commit message.

10 years agobuilding: Delete when cloning, not when preparing $builddir
Ian Jackson [Mon, 14 Jul 2014 11:37:43 +0000 (12:37 +0100)]
building: Delete when cloning, not when preparing $builddir

Make prepbuilddirs no longer delete $builddir; instead:
 - have prepbuilddirs delete *-stamp files and dist/;
 - have build_clone delete the specific source directories;
 - any individual ts-*-build scripts that use other things
   would have to delete them (but searching for $builddir
   reveals none).

This makes it possible for later steps within a build job to look at
the build trees from previous steps.  This is helpful if for any
reason it isn't easy or desirable to package up a formal set of build
outputs for reuse.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
v2: Provide motivation in the commit message.

10 years agosg-run-job: Support toggling `adding' by repeating `+'
Ian Jackson [Mon, 14 Jul 2014 12:07:30 +0000 (13:07 +0100)]
sg-run-job: Support toggling `adding' by repeating `+'

Specifying `+' as an argument to spawn-ts suppresses the automatic
addition of subsequent arguments to the testid.  Make it possible to
toggle this by repeating `+'.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
10 years agoBuildSupport: make selectbuildhost modify @ARGV
Ian Jackson [Fri, 11 Jul 2014 14:49:10 +0000 (15:49 +0100)]
BuildSupport: make selectbuildhost modify @ARGV

We pass it \@ARGV everywhere now, and it eats the argument (if it
finds one).  This allows build scripts to take extra arguments.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
10 years agoselectguest: log a message
Ian Jackson [Tue, 15 Jul 2014 14:15:02 +0000 (15:15 +0100)]
selectguest: log a message

Print the guest's name and the host it's on; this can help with
interpreting the logs.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
10 years agots-rumpuserxen-build: Cope with new layout, disabling bodges
Ian Jackson [Fri, 11 Jul 2014 14:27:11 +0000 (15:27 +0100)]
ts-rumpuserxen-build: Cope with new layout, disabling bodges

The rumpsrc psuedo-submodule has been integrated with the nblibs
submodule and made into a proper submodule.

Also, in the intervening time, rumpuserxen/Config.mk now uses ?= when
setting XEN_HEADERS.

So if we detect that we have a new rumpuser-xen, we can disable:
 - ad-hoc arrangements for rumpsrc
 - grievous-bodge
 - patching Config.mk

We consider a rumpuserxen old iff it has nblibs as a submodule and
lacks rumpsrc as a submodule.

We have to have a different name for the rumpsrc submodule (rather
than using rumpsrc), because in the new arrangements it needs to be
fetched from the location specified in rumpuserxen's .gitmodules
rather than our own configured location for the old rumpsrc.  (And the
local name is in the runvars etc.)

The old runvars etc. for rumpsrc need to stay around to cope with old
rumpuserxen, for now, but are going to be obsolete.  Leave some
comments about this in relevant places.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
10 years agoBuildSupport: provide submodule_have
Ian Jackson [Fri, 11 Jul 2014 14:13:03 +0000 (15:13 +0100)]
BuildSupport: provide submodule_have

Convenience function.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
10 years agoadhoc-revtuple-generator: Explicitly specify refs to fetch
Ian Jackson [Tue, 8 Jul 2014 10:27:59 +0000 (11:27 +0100)]
adhoc-revtuple-generator: Explicitly specify refs to fetch

Some of the git trees in $c{Repos} may have been created (eg by
earlier versions of git or earlier versions of osstest) without the
usual
  remote.origin.fetch = +refs/heads/*:refs/remotes/origin/*
git config setting.

Pass this refspec explicitly, instead of relying on it being set as
the default.

Amongst other things, this will prevent a possible error during
bisection
  git fetch -p origin
  [stuff]
  fatal: bad object 3e3d5f6146c3f684f4b7bd71feffc31e1c668754
which is currently appearing in our cron mail.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agoexecutive-postgresql-schema: Schema dump
Ian Jackson [Mon, 30 Jun 2014 10:13:37 +0000 (11:13 +0100)]
executive-postgresql-schema: Schema dump

We really ought to have a proper schema script.  But for now, include
the output of pg_dump --schema-only on the osstestdb, for the benefit
of anyone trying to set up a hosted (rather than standalone) instance
(or just trying to see how the code works).

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
v2: New patch in v2 of the series.

10 years agomake-flight: Test rump kernels only when needed
Ian Jackson [Fri, 27 Jun 2014 10:20:19 +0000 (11:20 +0100)]
make-flight: Test rump kernels only when needed

Introduce a new subroutine branch_wants_rumpkernel_tests and call it
before making the rumpuserxen build and test jobs.

The default is not to include a rumpuserxen test.  We include it for
rumpuserxen itself, xen, linux (in case there is a frontend/backend
incompatibility), and osstest.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agomake-flight: Break out do_rumpkernel_tests
Ian Jackson [Fri, 27 Jun 2014 10:19:47 +0000 (11:19 +0100)]
make-flight: Break out do_rumpkernel_tests

No functional change in this patch.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agostandalone-generate-dump-flight-runvars: New utility
Ian Jackson [Fri, 27 Jun 2014 14:40:57 +0000 (15:40 +0100)]
standalone-generate-dump-flight-runvars: New utility

This is helpful for showing the effect of changes to make-flight,
cr-daily-branch, et al.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
v2: Do not set DAILY_BRANCH_POSTMAKE_HOOK, since standalone now does
    that.

10 years agomg-list-all-branches: New script.
Ian Jackson [Fri, 27 Jun 2014 11:45:11 +0000 (12:45 +0100)]
mg-list-all-branches: New script.

The approach here is a little unpleasant but it has the virtue of
getting the ones out of crontab too, and of not depending on the
production database (which will in any case contain obsolete flights).

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agostandalone-config-example: Provide a setting of Repos
Ian Jackson [Fri, 27 Jun 2014 15:51:46 +0000 (16:51 +0100)]
standalone-config-example: Provide a setting of Repos

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
10 years agostandalone: Use DAILY_BRANCH_POSTMAKE_HOOK
Ian Jackson [Mon, 30 Jun 2014 10:12:08 +0000 (11:12 +0100)]
standalone: Use DAILY_BRANCH_POSTMAKE_HOOK

DAILY_BRANCH_PREEXEC_HOOK is after we have made half the prospective
email, but this is not useful for standalone users.

Reported-by: Ian Campbell <Ian.Campbell@citrix.com>
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
v2: New patch in v2 of the series.

10 years agostandalone: Honour OSSTEST_CONFIG
Ian Jackson [Fri, 27 Jun 2014 15:51:27 +0000 (16:51 +0100)]
standalone: Honour OSSTEST_CONFIG

If OSSTEST_CONFIG is set in the environment, do not default it
to ~/.xen-osstest/config.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agosubmodulefixup: Tell git cache [fetch=try]
Ian Jackson [Fri, 27 Jun 2014 10:42:31 +0000 (11:42 +0100)]
submodulefixup: Tell git cache [fetch=try]

Make git_massage_url take an optional %xopts list, currently with
only GitFetchBestEffort as a possible key.

Use that in submodulefixup, unconditionally.  Often the submodule
update will work fine with an older version of the submodule, and if
the submodule we get is out of date then the submodule update will
fail anyway.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agoadhoc-revtuple-generator: Do not always reclone
Ian Jackson [Fri, 27 Jun 2014 14:24:18 +0000 (15:24 +0100)]
adhoc-revtuple-generator: Do not always reclone

Instead, try setting the `origin' remote's URL and fetching.  If that
fails it may be that the local repo is corrupted, so then we try
recloning.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agoadhoc-revtuple-generator: Avoid an undefined variable warning
Ian Jackson [Fri, 27 Jun 2014 13:44:58 +0000 (14:44 +0100)]
adhoc-revtuple-generator: Avoid an undefined variable warning

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agoadhoc-revtuple-generator: Include $treename in some error messages
Ian Jackson [Fri, 27 Jun 2014 11:30:35 +0000 (12:30 +0100)]
adhoc-revtuple-generator: Include $treename in some error messages

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agoadhoc-revtuple-generator: Cope with parentless git commits
Ian Jackson [Fri, 27 Jun 2014 11:30:13 +0000 (12:30 +0100)]
adhoc-revtuple-generator: Cope with parentless git commits

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agots-xen-install: Correct mlog to logm
Ian Campbell [Mon, 16 Jun 2014 08:31:03 +0000 (09:31 +0100)]
ts-xen-install: Correct mlog to logm

Only logm exists.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
10 years agomg-debian-installer-update: Print messages to stderr.
Anthony PERARD [Mon, 9 Jun 2014 15:16:50 +0000 (16:16 +0100)]
mg-debian-installer-update: Print messages to stderr.

This fix standalone-reset which rely on the output of this script.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Dario Faggioli <dario.faggioli@citrix.com>
10 years agomg-hosts: Usage documentation.
Ian Jackson [Wed, 25 Jun 2014 14:05:34 +0000 (15:05 +0100)]
mg-hosts: Usage documentation.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agomg-hosts: Allow spaces in setflagexpr exprs
Ian Jackson [Wed, 25 Jun 2014 14:05:21 +0000 (15:05 +0100)]
mg-hosts: Allow spaces in setflagexpr exprs

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agocr-for-branches: clean up tmp/
Ian Jackson [Wed, 25 Jun 2014 09:33:01 +0000 (10:33 +0100)]
cr-for-branches: clean up tmp/

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
10 years agoImplement direct IPMI power control.
Ian Campbell [Mon, 16 Jun 2014 08:51:21 +0000 (09:51 +0100)]
Implement direct IPMI power control.

The xenuse module currently used to control some machines (specifically the ARM
marilith machines) is unreliable and subject to breakages on the xenrt server
side.

Since xenuse/xenrt is just wrapping IPMI for these machines implement direct
control via a new Osstest::PDI::impi module.

This is distinct from the existing impiextra (arbitrary ipmi commands on power
status change) because IPMI power control is not idempotent, therefore it is
necessary to query the current status before turning the machine on or off.

After this change the correct power method for a marilith node is:
    ipmiextra on marilith-n0-mgmt.uk.xensource.com <user> <pass> chassis bootdev pxe;ipmi marilith-n0-mgmt.uk.xensource.com <user> <pass>

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
Ian, when we discussed this on IRC you expressed a preference for using
freeipmitools, but since the existing ipmiextra module uses ipmitool I figured
it would be better to be consistent.

10 years agocrontab: reduce frequency of linux-next
Ian Jackson [Mon, 9 Jun 2014 14:29:52 +0000 (15:29 +0100)]
crontab: reduce frequency of linux-next

10 years agomake-flight: Do not filter the qemu-upstream-* jobs
Ian Jackson [Mon, 9 Jun 2014 14:20:05 +0000 (15:20 +0100)]
make-flight: Do not filter the qemu-upstream-* jobs

When "qemu-xen-traditional" was the default, this filtering was
correct.  But now, "qemu-xen" is the default, and that version may
also be used in other tests for ancillary functions (eg, in dom0 for
the bootloader).

So simply do all of the jobs when testing qemu-upstream-*.

Retain the old behaviour for qemu-xen-4.2-testing, which has
qemu-xen-traditional as the default.  (There is no
qemu-xen-4.1-testing, and 4.3 has the change to the default.)

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agobuild_clone: When guessing vcs from url suffix, strip cache info
Ian Jackson [Thu, 5 Jun 2014 10:15:46 +0000 (11:15 +0100)]
build_clone: When guessing vcs from url suffix, strip cache info

We assume that anything not containing `/'s after `%20' is
uninteresting.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agoap-common: Use besteffort_repo for rumpsrc
Ian Jackson [Wed, 4 Jun 2014 13:47:54 +0000 (14:47 +0100)]
ap-common: Use besteffort_repo for rumpsrc

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agocri-common, ap-common: abstract out besteffort_repo
Ian Jackson [Wed, 4 Jun 2014 13:46:49 +0000 (14:46 +0100)]
cri-common, ap-common: abstract out besteffort_repo

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agocr-daily-branch: Fix URLs used for trees which should be using push gate output
Ian Campbell [Wed, 4 Jun 2014 12:49:16 +0000 (13:49 +0100)]
cr-daily-branch: Fix URLs used for trees which should be using push gate output

For several trees we were using the upstream URL for tests of all branches when
we should have been using the push gate output on xenbits (to help isolate us
from external failures). i.e. tree_libvirt was git://libvirt.org/libvirt.git
for every branch but for branches other than libvirt it should be
git://xenbits.xen.org/libvirt.git.

This is down to a missing third argument to determine_version. Fix this for all
but osstest (which shouldn't be touched).

Tested with:
 $ BRANCHES="xen-unstable libvirt rumpuserxen seabios qemu-mainline qemu-xen-unstable linux-next linuxfirmware osstest"
 $ for b in $BRANCHES ; do f=T${b//-}; ./standalone make-flight -f $f $b; done
 $ check() { \
     f=T${1//-} ; \
     echo $1 ; \
     ./mg-show-flight-runvars $f | \
         awk '$2 ~ /^tree_/ && $3 !~ /^$|xenbits.xen.org/ { printf "    %-28s %s\n", $2, $3 }' | \
         sort -u; \
   }
 $ for b in xen-unstable libvirt rumpuserxen seabios qemu-mainline qemu-xen-unstable linux-next osstest ; do check $b; done

Which produces:

xen-unstable
    tree_gnulib_libvirt          git://drall.uk.xensource.com:9419/git://git.sv.gnu.org/gnulib.git%20[fetch=try]
    tree_rumpuserxen_rumpsrc     https://github.com/rumpkernel/rumpkernel-netbsd-src
libvirt
    tree_gnulib_libvirt          git://drall.uk.xensource.com:9419/git://git.sv.gnu.org/gnulib.git%20[fetch=try]
    tree_libvirt                 git://libvirt.org/libvirt.git
    tree_rumpuserxen_rumpsrc     https://github.com/rumpkernel/rumpkernel-netbsd-src
rumpuserxen
    tree_rumpuserxen             https://github.com/rumpkernel/rumpuser-xen
    tree_rumpuserxen_rumpsrc     https://github.com/rumpkernel/rumpkernel-netbsd-src
seabios
    tree_gnulib_libvirt          git://drall.uk.xensource.com:9419/git://git.sv.gnu.org/gnulib.git%20[fetch=try]
    tree_rumpuserxen_rumpsrc     https://github.com/rumpkernel/rumpkernel-netbsd-src
    tree_seabios                 git://git.seabios.org/seabios.git
qemu-mainline
    tree_gnulib_libvirt          git://drall.uk.xensource.com:9419/git://git.sv.gnu.org/gnulib.git%20[fetch=try]
    tree_qemuu                   git://git.qemu.org/qemu.git
    tree_rumpuserxen_rumpsrc     https://github.com/rumpkernel/rumpkernel-netbsd-src
qemu-xen-unstable
linux-next
    tree_gnulib_libvirt          git://drall.uk.xensource.com:9419/git://git.sv.gnu.org/gnulib.git%20[fetch=try]
    tree_linux                   git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
    tree_rumpuserxen_rumpsrc     https://github.com/rumpkernel/rumpkernel-netbsd-src
linuxfirmware
    tree_gnulib_libvirt          git://drall.uk.xensource.com:9419/git://git.sv.gnu.org/gnulib.git%20[fetch=try]
    tree_rumpuserxen_rumpsrc     https://github.com/rumpkernel/rumpkernel-netbsd-src
osstest
    tree_gnulib_libvirt          git://drall.uk.xensource.com:9419/git://git.sv.gnu.org/gnulib.git%20[fetch=try]
    tree_rumpuserxen_rumpsrc     https://github.com/rumpkernel/rumpkernel-netbsd-src

The presence of tree_gnulib_libvirt is expected while the
tree_rumpuserxen_rumpsrc is erroneous (I think) but not intended to be fixed
here.

Note that linuxfirmware is broken since tree_linuxfirmware should be
$GIT_KERNEL_ORG/pub/scm/linux/kernel/git/firmware/linux-firmware.git. This is
also not addressed here.

For comparison the xen-unstable check output prior to this fix was:

xen-unstable
    tree_gnulib_libvirt          git://drall.uk.xensource.com:9419/git://git.sv.gnu.org/gnulib.git%20[fetch=try]
    tree_libvirt                 git://libvirt.org/libvirt.git
    tree_rumpuserxen             https://github.com/rumpkernel/rumpuser-xen
    tree_rumpuserxen_rumpsrc     https://github.com/rumpkernel/rumpkernel-netbsd-src

Note the incorrect tree_libvirt and tree_rumpuserxen.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson <ian.jackson@eu.citrix.com

10 years agomake-flight: include cri-common before ap-common
Ian Campbell [Wed, 4 Jun 2014 11:05:56 +0000 (12:05 +0100)]
make-flight: include cri-common before ap-common

The latter wants cached_repo which the former provides.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson <ian.jackson@eu.citrix.com

10 years agostandalone: ensure log dir exists for all users of with_logging
Ian Campbell [Wed, 4 Jun 2014 11:00:02 +0000 (12:00 +0100)]
standalone: ensure log dir exists for all users of with_logging

By moving the check into that function. Previously at least make-flight was
failing to all ensure_logs

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson <ian.jackson@eu.citrix.com

10 years agots-logs-capture: collect /proc/partitions
Ian Jackson [Wed, 4 Jun 2014 11:29:36 +0000 (12:29 +0100)]
ts-logs-capture: collect /proc/partitions

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agots-logs-capture: collect resize2fs.log
Ian Jackson [Wed, 4 Jun 2014 11:28:48 +0000 (12:28 +0100)]
ts-logs-capture: collect resize2fs.log

This is generated by ts-xen-build-prep but previously we didn't
collect it.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agots-xen-build-prep: Run the backgrounded resize2fs with ionice
Ian Jackson [Wed, 4 Jun 2014 11:24:06 +0000 (12:24 +0100)]
ts-xen-build-prep: Run the backgrounded resize2fs with ionice

We wrap this in
  ionice -cBest-Effort -n7

This is an attempt to stop resize2fs from starving builds of IO
bandwidth.  We have been seeing that some IO-intensive build steps
(notably, configure) can take unreasonable amounts of time and we
suspect resize2fs as the culprit.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agoREADME.dev: more info about mg-blockage
Ian Jackson [Tue, 3 Jun 2014 15:08:49 +0000 (16:08 +0100)]
README.dev: more info about mg-blockage

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agoTolerate outages of git://git.sv.gnu.org/gnulib.git
Ian Campbell [Tue, 20 May 2014 10:54:36 +0000 (11:54 +0100)]
Tolerate outages of git://git.sv.gnu.org/gnulib.git

gnulib is a bit special in that it isn't part of a push gate and
therefore doesn't have an output repo on xenbits. This means that if
git.sv.gnu.org is down for some reason then any flight which uses it
will potentially fail, even ones which are not specifically trying to
test gnulib.git. Normally unrelated flights are isolated from 3rd
party failures because they use the relevant push gates' outputs.

The git-proxy which we use supports passing [fetch=try] as an option
which will cause it to try and fetch from upstream and fall back to
the cached version if upstream does not respond. This is exactly what
we want for gnulib.git. Proxy options in the URL are separate with a
space which we encode here as %20 to avoid having to go around adding
quotes to all the uses of shell variables which contain a URL.

Provide a helper shell function cached_repo which conditionally
arranges for the given repo to be cached with the given options only
if the osstest config sets GitCacheProxy, so as not to break uses of
osstest outside of the production environment where a cache may not be
available (of course such uses will then be subject to outages of 3rd
party repos)

It was also necessary to make repo_tree_rev_fetch_git idempotent WRT
the proxy prefix to avoid trying to clone
git://proxy/git://proxy/git://some.real.domain/foo.git.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
10 years agocri-args-hostlists, etc.: Send rumpuserxen reports to rumpkernel-builds
Ian Jackson [Tue, 3 Jun 2014 14:59:51 +0000 (15:59 +0100)]
cri-args-hostlists, etc.: Send rumpuserxen reports to rumpkernel-builds

There is now a list, rumpkernel-builds@lists.sourceforge, for this
kind of information relating to rumpuser-xen.

Provide a facility in cri-args-hostlists for start_email to look for
$OSSTEST_EMAIL_HEADER--$branch.

Introduce two new files
  daily-cron-email-real--rumpuserxen
  daily-cron-email-real-bisectcomplete--rumpuserxen
which override the general settings.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agocs-bisection-step: Remove a bit of duplication
Ian Jackson [Tue, 3 Jun 2014 14:52:18 +0000 (15:52 +0100)]
cs-bisection-step: Remove a bit of duplication

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agocs-bisection-step: Cope with url-less trees
Ian Jackson [Mon, 2 Jun 2014 10:09:17 +0000 (11:09 +0100)]
cs-bisection-step: Cope with url-less trees

Since osstest 64a9c064, some flights have a tree with an empty
tree_<foo> variable.  This breaks the bisector.  Make it more robust
as follows:

When searching for the basis flight, ignore not only trees with no url
specified but also ones with an empty url.

When preparing the set of trees we care about, eliminate any where the
latest flight has no url (or an empty url).

(It would still be better for the flight constructor to leave
tree_<foo> unset, and for the ts-* build script to set it in as a part
of the post-build data collection.  That would allow the bisector to
work on that tree where relevant.  But doing this is a task for
another day - if for no other reason than that we have to decide
whether to put it in a new variable built_tree_<foo>.)

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agoap-qemu-revision: Make it actually work
Ian Jackson [Fri, 23 May 2014 09:55:51 +0000 (10:55 +0100)]
ap-qemu-revision: Make it actually work

This is used (only) for obtaining lists of the corresponding qemu
revisions (for the report email) when doing a Xen push.  It has never
worked because we didn't provide a type argument to git-cat-file.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agocr-daily-branch: Only set TREE_SEABIOS for seabios flights
Ian Campbell [Fri, 23 May 2014 09:26:26 +0000 (10:26 +0100)]
cr-daily-branch: Only set TREE_SEABIOS for seabios flights

For all other flights we want to use the Config.mk settings.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
10 years agoDefault vcs to git
Ian Jackson [Thu, 22 May 2014 11:24:19 +0000 (12:24 +0100)]
Default vcs to git

adhoc-revtuple-generator and cs-bisection-step try to guess the VCS
from the URL, by looking at the end of the URL for a suitable file
extension.  If none was found, things would break.

This relies on a convention about tree URLs which is sadly no longer
being widely observed.  In particular many projects have git URLs
which do not mention git.

Instead, have these scripts assume that urls which don't end in .git
or .hg are for git.

This renders hg into a second-class VCS in osstest, which is (I think)
tolerable.  If it isn't, then someone ought to come up with some
better way of ensuring that the vcs is explicitly specified to
adhoc-revtuple-generator and cs-bisection-step.  That would probably
be doable but is too much work right now.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agosg-run-job: Do blocked check globally, in run-job
Ian Jackson [Wed, 21 May 2014 11:36:02 +0000 (12:36 +0100)]
sg-run-job: Do blocked check globally, in run-job

Only run-job can fully abandon the whole job.  With the previous
arrangements (in bc63a901), a blocked build job would go on to try to
run ts-logs-capture and become broken (rather than blocked).

The downside of this is that it is not possible in the future to
introduce a kind of job which can selectively carry on if some of its
dependencies fail.  This notion is IMO an acceptable casualty.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agoTestSupport::broken: Update status of `preparing' jobs
Ian Jackson [Wed, 21 May 2014 11:34:01 +0000 (12:34 +0100)]
TestSupport::broken: Update status of `preparing' jobs

If the current job is preparing, and we call broken, then we should
update the status to that supplied by broken's caller.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agomake-flight: rumpuserxen amd64 too
Ian Jackson [Wed, 21 May 2014 10:54:10 +0000 (11:54 +0100)]
make-flight: rumpuserxen amd64 too

Build and test amd64 rump kernel.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agocrontab: Rearrange actually-daily branches
Ian Jackson [Wed, 21 May 2014 10:37:48 +0000 (11:37 +0100)]
crontab: Rearrange actually-daily branches

Make them start at roughly the same time, by using cr-for-branches for
them all in one go.  That increases the extent to which build hosts
will be reused (saving on reinstallations), and hence also the extent
to which the build hosts' ccache will be effective (saving on cpu).

Set them off at 04:18 UK time (currently equivalent to 21:18 Pacific
time), in the hope that at least some of them might have produced
answers during following the UK working day.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agorumpuserxen: Increase build timeout for grievous bodge
Ian Jackson [Wed, 21 May 2014 10:42:37 +0000 (11:42 +0100)]
rumpuserxen: Increase build timeout for grievous bodge

This involves copying 90Mby of data.  If the test machine is busy (in
the worst case, running the kernel side of resize2fs which due to
Linux's poor IO scheduling hogs the entire IO system) it can take a
long time.  In flight 26359 it was still running after 280s, when the
machine was released.

Increase the timeout to 2000s.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agocr-publish-flight-logs: Provide ref path for osstest branch push
Ian Jackson [Mon, 19 May 2014 10:58:47 +0000 (11:58 +0100)]
cr-publish-flight-logs: Provide ref path for osstest branch push

Without this, the push fails if the osstest tree is in detached head
state.  (Which is a situation which can occur during by-hand testing.)

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agoAdd my developer/deployment notes as README.dev
Ian Campbell [Mon, 19 May 2014 11:08:09 +0000 (12:08 +0100)]
Add my developer/deployment notes as README.dev

These are mostly targetted at people setting up things on the production
infra.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
10 years agoproduction-config: Update to pickup latest Debian Installer
Ian Campbell [Mon, 19 May 2014 11:32:16 +0000 (12:32 +0100)]
production-config: Update to pickup latest Debian Installer

Pickup the changes made in
     mg-debian-installer-update: Updates for newer armhf kernels from backports.
     mg-debian-installer-update: grab Xen PV domU capable images too

I have run

    for i in i386 amd64 armhf ; do
        OSSTEST_CONFIG=production-config ./mg-debian-installer-update wheezy $i 'firmware-bnx2'
    done

on the osstest vm. Point the config to these new versions.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
10 years agosg-run-job: check for prerequisite builds when running build jobs
Ian Campbell [Thu, 15 May 2014 10:30:39 +0000 (11:30 +0100)]
sg-run-job: check for prerequisite builds when running build jobs

For example build-$ARCH-libvirt depends on build-$ARCH for the Xen bits.
Performing this check early saves allocating, installing, preparating etc a
host just to find the prerequisite is missing.

Rename ts-xen-build-check to ts-build-check while we are there. The script
checks all prerequisite builds, not just Xen.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
10 years agostandalone: allow making a baseline flight.
Ian Campbell [Thu, 15 May 2014 10:30:38 +0000 (11:30 +0100)]
standalone: allow making a baseline flight.

While there make the existing --lvextend argument actually work (by passing it
to getopt) and document both.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
10 years agots-xen-build: Move Xen build tree from xen-unstable to just xen
Ian Campbell [Thu, 15 May 2014 10:30:37 +0000 (11:30 +0100)]
ts-xen-build: Move Xen build tree from xen-unstable to just xen

It's not always xen-unstable we are testing, and I had to check that it was
the right generic name, so reduce this confusion.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
10 years agots-xen-build: stash the Xen and SeaBIOS .config's
Ian Campbell [Thu, 15 May 2014 10:30:36 +0000 (11:30 +0100)]
ts-xen-build: stash the Xen and SeaBIOS .config's

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
10 years agoAdd a flight to test seabios.org's master branch
Ian Campbell [Thu, 15 May 2014 10:30:35 +0000 (11:30 +0100)]
Add a flight to test seabios.org's master branch

Most of this is the usual stuff for adding a new branch.

cr-daily-branch needed to set REVISION_SEABIOS for the seabios branch and leave
it blank for others (so that Config.mk's version will be used). ts-xen-build
already consumes the resulting version runvar but needed to be taught about the
tree URL runvar.

make-flight and mfi-common are updated to only build+test seabios on x86.

The new branch is added to cr-daily-branches.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
10 years agoAdd a flight to test qemu.org's ("mainline") master branch.
Ian Campbell [Thu, 15 May 2014 10:30:34 +0000 (11:30 +0100)]
Add a flight to test qemu.org's ("mainline") master branch.

The naming here is slightly confused because we call our branches of qemuu on
xenbits "qemu-upstream-foo", so I went with qemu-mainline to be distinct.

I've introduced $qemuubranch along the lines of $xenbranch, it is normally set
to qemu-upstream-{X.Y,unstable} but for the qemu-mainline flight it instead
points to qemu-mainline.

I've run up to the make-flight bit of cr-daily-branch and it differs only in
the expected ways from a standard xen-unstable branch (i.e. uses xen.git#master
and qemu.git#master).

I did consider causing make-flight:job_create_test_filter_callback to omit any
test which didn't use qemuu but I decided not to because it is used for PV
qdisk backends too.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
10 years agomg-debian-installer-update: grab Xen PV domU capable images too
Ian Campbell [Mon, 28 Apr 2014 10:41:24 +0000 (11:41 +0100)]
mg-debian-installer-update: grab Xen PV domU capable images too

Some test cases are going to want this, so they should be grabbed and
versioned here.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
10 years agoBuild against liblzma to support .xz compressed kernels
Ian Campbell [Mon, 28 Apr 2014 10:41:23 +0000 (11:41 +0100)]
Build against liblzma to support .xz compressed kernels

Install the library at runtime.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
10 years agocr-external-linux: remove
Ian Campbell [Mon, 28 Apr 2014 10:41:22 +0000 (11:41 +0100)]
cr-external-linux: remove

Seems to be unused.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
10 years agocr-daily-branch: Only build oldkern for xen-unstable flights
Ian Campbell [Thu, 15 May 2014 10:35:44 +0000 (11:35 +0100)]
cr-daily-branch: Only build oldkern for xen-unstable flights

This is just supposed to be a build test, it doesn't actually get used.
There's no need to be doing this for every flight, doing it for xen-unstable
is enough

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
10 years agocrontab: Bisect the libvirt branch
Ian Campbell [Wed, 14 May 2014 08:14:16 +0000 (09:14 +0100)]
crontab: Bisect the libvirt branch

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
10 years agomg-debian-installer-update: Updates for newer armhf kernels from backports.
Ian Campbell [Mon, 28 Apr 2014 09:18:38 +0000 (10:18 +0100)]
mg-debian-installer-update: Updates for newer armhf kernels from backports.

In newer kernels sd_mod.ko requires crypto symbols, so include
kernel/crypto/*.ko in the installer overlay cpio.

Also prefer the newest versions of packages pulled in from backports, by
reversing the sort. Anchor the linux-image package name to we only pick up the
-armmp flavour and not -armmp-lpae (which is unnecessary in an installer
environment).

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agorump kernels: Create a "branch" for rump kernel tests
Ian Jackson [Fri, 16 May 2014 16:22:10 +0000 (17:22 +0100)]
rump kernels: Create a "branch" for rump kernel tests

This is a traditional push gate.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agorump kernels: Provide two test jobs
Ian Jackson [Fri, 16 May 2014 15:57:16 +0000 (16:57 +0100)]
rump kernels: Provide two test jobs

10 years agorump kernels: Provide a build job
Ian Jackson [Thu, 15 May 2014 15:48:56 +0000 (16:48 +0100)]
rump kernels: Provide a build job

Provide:
  - test script ts-rumpuserxen-build
  - recipe build-rumpuserxen
  - in flights, job build-i386-rumpuserxen
  - in ap-common, TREE and TREEVCS for RUMPUSERXEN and RUMPUSERXEN_RUMPSRC

mfi-common also honours REVISION_RUMPUSERXEN, but provides no way to
specify the revisions of the subtrees (the two submodules and
rumpsrc).  The upstream rumpuser-xen git repository contains the
commit hashes of those.

We had to do some pretty unpleasant things to get the rumpuser-xen
build system to work with rumpsrc we cloned ourselves.

During bisection the bisector must choose a unification of the various
trees' revision graphs, into a single graph.  But we do not here teach
adhoc-revtuple-generator about these interrelationships.  As a result
the bisector may choose a unification inconsistent with the subtree
references in rumpuser-xen.git.  This might impair the performance of
the bisector in the presence of cross-tree version incompatibilities,
but the results should be tolerable for now.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agomfi-common: Honour REVISION_LIBVIRT=disable
Ian Jackson [Fri, 16 May 2014 16:56:54 +0000 (17:56 +0100)]
mfi-common: Honour REVISION_LIBVIRT=disable

This makes it possible for (for example) cr-daily-branch to disable
the libvirt build, when it is not needed.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agoprepareguest: Tolerate $mb=undef
Ian Jackson [Fri, 16 May 2014 15:51:59 +0000 (16:51 +0100)]
prepareguest: Tolerate $mb=undef

Allow callers to pass undef for $mb, to indicate that there is no need
for prepareguest to do anything about storage.

No functional change for any of the existing callers.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agoguest_umount_lv: Tolerate lack of volume group
Ian Jackson [Fri, 16 May 2014 15:27:25 +0000 (16:27 +0100)]
guest_umount_lv: Tolerate lack of volume group

Check if the guest has a volume group, and if not do nothing.

We are going to introduce a kind of guest with no significant storage.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agotarget_jobdir: Create the directory
Ian Jackson [Fri, 16 May 2014 15:53:43 +0000 (16:53 +0100)]
target_jobdir: Create the directory

Callers of target_jobdir can now rely on the directory existing after
it returns.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agoTestSupport: Break out target_extract_jobdistpath_subdir
Ian Jackson [Fri, 16 May 2014 15:26:26 +0000 (16:26 +0100)]
TestSupport: Break out target_extract_jobdistpath_subdir

This is the meat of what was Osstest::BuildSupport::xendist, which now
becomes a simple convenience wrapper.

No functional change.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agoTestSupport: Break out target_jobdir
Ian Jackson [Fri, 16 May 2014 15:20:12 +0000 (16:20 +0100)]
TestSupport: Break out target_jobdir

No functional change.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agoBuildSupport: Provide some support for git submodules
Ian Jackson [Thu, 15 May 2014 15:01:13 +0000 (16:01 +0100)]
BuildSupport: Provide some support for git submodules

* Provide a new "submodulefixup" function which, given a cloned
  tree which has submodules:
    - massages the .gitmodules file according to the tree_*
      runvars and massage_url
    - arranges for the submodules to be fetched and checked out
    - stores the revisions of the submodules, as necessary
    - honours any revision_ tags for the submodules
    - allows the caller to give the submodules names which are
      suitable for use in runvars etc. (and might differ from
      the names or paths used in the supermodule)

* In a submodule, .git is actually a file containing a reference
  to a subdirectory of the main git tree.  Cope with that.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agoBuildSupport, ts-*-build: Remove some clone-and-hack
Ian Jackson [Wed, 14 May 2014 13:30:54 +0000 (14:30 +0100)]
BuildSupport, ts-*-build: Remove some clone-and-hack

Introduce Osstest::BuildSupport containing selectbuildhost,
builddirsprops and prepbuilddirs.  The corresponding replicated code
from the various ts-*-build scripts is abolished.

Also break out xendist, which we are going to want to reuse.

No functional change.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agogit_massage_url: Make idempotent
Ian Jackson [Thu, 15 May 2014 15:47:46 +0000 (16:47 +0100)]
git_massage_url: Make idempotent

If called more than once, it should not produce a url starting with
multiple copies of the git cache proxy prefix.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
10 years agoTestSupport: Provide target_editfile
Ian Jackson [Thu, 15 May 2014 15:00:57 +0000 (16:00 +0100)]
TestSupport: Provide target_editfile

... as well as target_editfile_root.  No callers yet.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
11 years agomake-flight etc.: More Debian HVM testcases
Wei Liu [Wed, 23 Apr 2014 17:03:28 +0000 (18:03 +0100)]
make-flight etc.: More Debian HVM testcases

ts-debian-hvm-install was written to work as general HVM testcase. It
can be tuned to easily use another BIOS other than OVMF.

Modify make-flight and friends to create more Debian HVM 64bit testcases
to run with SeaBIOS and ROMBIOS.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoadhoc-people-touched-filter: Include patch authors
Ian Jackson [Mon, 12 May 2014 13:36:10 +0000 (14:36 +0100)]
adhoc-people-touched-filter: Include patch authors

The git "Author:" tag of each commit should be included.

(We used to include only special pseudo-headers from the patch
comment, because the old hg practice used to be to put the committer
in the hg commit author.  But git separates committer and author.)

This involves a slight refactoring.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
11 years agolibvirt: Fixes for older versions of Xen
Ian Campbell [Fri, 2 May 2014 13:08:04 +0000 (14:08 +0100)]
libvirt: Fixes for older versions of Xen

libvirt supports Xen 4.2 onwards. However:
  - We were not correctly setting the libvirtbuildjob runvar on 4.3 and
    earlier.
  - Xen 4.2 installs to /usr not /usr/local so the CFLAGS/LDFLAGS used to build
    libvirt were wrong.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoDrop unnecessary TREEVCS settings for libvirt and gnulib-libvirt
Ian Campbell [Tue, 6 May 2014 08:05:23 +0000 (09:05 +0100)]
Drop unnecessary TREEVCS settings for libvirt and gnulib-libvirt

This will be correctly inferred from the URL.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agostandalone: Export BUILD_LVEXTEND_MAX when running make-flight.
Ian Campbell [Fri, 2 May 2014 10:23:04 +0000 (11:23 +0100)]
standalone: Export BUILD_LVEXTEND_MAX when running make-flight.

This is stops the build taking over the entire VG and allows tests to be run
without reinstalling the host.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agostandalone: Use WebspaceLog for apache log check
Ian Campbell [Thu, 1 May 2014 16:10:55 +0000 (17:10 +0100)]
standalone: Use WebspaceLog for apache log check

Also print the result in the error message for cut-and-paste convenience.

While there I noticed that run-test was setting OSSTEST_CONFIG twice.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoTestSupport: stash installer files etc.
Ian Jackson [Fri, 9 May 2014 11:04:19 +0000 (12:04 +0100)]
TestSupport: stash installer files etc.

Stash the pxeboot config file, the preseed file, et al.  This is done
centrally in file_link_contents.

We don't stash things whose $contents is actually a coderef because
(a) doing so would be inconvenient and (b) many of them are large and
not very interesting.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
11 years agots-xen-build: collect built seabios and ovmf versions
Ian Campbell [Fri, 25 Apr 2014 16:11:08 +0000 (17:11 +0100)]
ts-xen-build: collect built seabios and ovmf versions

The seabios path was incorrect and ovmf was not present.

Take the opportunity to rename collectversion_qemu to collectversions.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
11 years agocri-common: Only use git proxy for git, http and https URLs
Ian Campbell [Fri, 25 Apr 2014 15:15:20 +0000 (16:15 +0100)]
cri-common: Only use git proxy for git, http and https URLs

For testing I had ap-fetch-version-old returning a file:// URL to a local repo.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>