Ian Jackson [Thu, 15 May 2014 15:48:56 +0000 (16:48 +0100)]
rump kernels: Provide a build job
Provide:
- test script ts-rumpuserxen-build
- recipe build-rumpuserxen
- in flights, job build-i386-rumpuserxen
- in ap-common, TREE and TREEVCS for RUMPUSERXEN and RUMPUSERXEN_RUMPSRC
mfi-common also honours REVISION_RUMPUSERXEN, but provides no way to
specify the revisions of the subtrees (the two submodules and
rumpsrc). The upstream rumpuser-xen git repository contains the
commit hashes of those.
We had to do some pretty unpleasant things to get the rumpuser-xen
build system to work with rumpsrc we cloned ourselves.
During bisection the bisector must choose a unification of the various
trees' revision graphs, into a single graph. But we do not here teach
adhoc-revtuple-generator about these interrelationships. As a result
the bisector may choose a unification inconsistent with the subtree
references in rumpuser-xen.git. This might impair the performance of
the bisector in the presence of cross-tree version incompatibilities,
but the results should be tolerable for now.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 15 May 2014 15:01:13 +0000 (16:01 +0100)]
BuildSupport: Provide some support for git submodules
* Provide a new "submodulefixup" function which, given a cloned
tree which has submodules:
- massages the .gitmodules file according to the tree_*
runvars and massage_url
- arranges for the submodules to be fetched and checked out
- stores the revisions of the submodules, as necessary
- honours any revision_ tags for the submodules
- allows the caller to give the submodules names which are
suitable for use in runvars etc. (and might differ from
the names or paths used in the supermodule)
* In a submodule, .git is actually a file containing a reference
to a subdirectory of the main git tree. Cope with that.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Wed, 14 May 2014 13:30:54 +0000 (14:30 +0100)]
BuildSupport, ts-*-build: Remove some clone-and-hack
Introduce Osstest::BuildSupport containing selectbuildhost,
builddirsprops and prepbuilddirs. The corresponding replicated code
from the various ts-*-build scripts is abolished.
Also break out xendist, which we are going to want to reuse.
No functional change.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Mon, 12 May 2014 13:36:10 +0000 (14:36 +0100)]
adhoc-people-touched-filter: Include patch authors
The git "Author:" tag of each commit should be included.
(We used to include only special pseudo-headers from the patch
comment, because the old hg practice used to be to put the committer
in the hg commit author. But git separates committer and author.)
This involves a slight refactoring.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Campbell [Fri, 2 May 2014 13:08:04 +0000 (14:08 +0100)]
libvirt: Fixes for older versions of Xen
libvirt supports Xen 4.2 onwards. However:
- We were not correctly setting the libvirtbuildjob runvar on 4.3 and
earlier.
- Xen 4.2 installs to /usr not /usr/local so the CFLAGS/LDFLAGS used to build
libvirt were wrong.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Fri, 9 May 2014 11:04:19 +0000 (12:04 +0100)]
TestSupport: stash installer files etc.
Stash the pxeboot config file, the preseed file, et al. This is done
centrally in file_link_contents.
We don't stash things whose $contents is actually a coderef because
(a) doing so would be inconvenient and (b) many of them are large and
not very interesting.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Campbell [Tue, 29 Apr 2014 12:16:34 +0000 (13:16 +0100)]
ts-debian-hvm-install: increase boot_timeout to 200s
Tianocore can be slow to post, especially on large guests. We've been hitting
the 100s timeout on lake-frog and fire-frog. Even when succeeding the duration
is 90s+.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Ian Campbell [Mon, 28 Apr 2014 07:55:06 +0000 (08:55 +0100)]
production-config: Update di version again.
This contains the current version for amd64 and i386 (actually the same as
yesterday) which picks up the latest point release but for armhf this is a link
to the old (2013-12-12) image since armhf uses a backports kernel and is
unaffected by the point release but does seem to have a deendency on some newer
infrastructure (depmod et al, I think) which needs sorting out.
Ian Campbell [Wed, 23 Apr 2014 10:07:01 +0000 (11:07 +0100)]
Add libvirt build and test jobs.
I have run this through a "cr-daily-branch --real" and the push gate machinery
seem functional. The test cases run through to starting a guest which fails
because virsh support has not been implemented yet.
The libvirt.git initscript is horribly redhat specific, and the debian package
one is large and complex, so ts-libvirt-build inserts its own simple
initscript.
I have observed the autogen.sh phase taking 34 minutes on army (arm build
cluster) hence the surprisingly large timeout of an hour. The build took around
20 minutes on the same system, but I used a timeout of an hour anyway to be
safe.
The standalone wrapper is updated to handle path_libvirt in set-paths.
libvirt.git's autogen.sh involves initialising/cloning a git submodule, which
doesn't use osstest's git cache and introduces unpredicability into what is
built. Therefore ap-* are taught about a new libvirt-gnulib tree (prefixed
libvirt incase we end up with other users of gnulib in the future). However
this doesn't use the usual push gate mechanism, since this was deemed to be
overkill and just always uses the tip of gnulib. The results are recorded
though so the bisector can still operate over this tree in the event of
breakage.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com> Cc: Jim Fehlig <jfehlig@suse.com> Cc: Dario Faggioli <dario.faggioli@citrix.com>
Ian Jackson [Thu, 10 Apr 2014 17:20:18 +0000 (18:20 +0100)]
cr-*, ap-*: Facilities for testing linux-next
* Define the new branch's git source repo in ap-common.
* Provide a branch-settings file which disables running additional
baseline tests, listing changesets in flight reports, pushing,
and which arranges for the baseline revision for the flight
report to be determined "late", ie after the to-be-tested
versions of everything have been decided.
* When the baseline revision is to be determined "late", run
the new script ap-fetch-version-baseline-late. This is another
collection of per-branch rules. We introduce currently one
per-branch rule, for linux-next:
* The baseline version for a test of linux-next is
- the version tested by the most recent flight on the
linux-linus branch;
- provided that said version is an ancestor of the linux-next
version under test;
- otherwise we try an earlier linux-linus flight.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 10 Apr 2014 17:18:54 +0000 (18:18 +0100)]
sg-check-tested: support --git-ancestor-of
This will be used for the linux-next branch, whose baseline is the
most recently tested linux-linus which is an ancestor of the
being-tested linux-next.
We do not provide this for any VCS other than git. Hopefully we will
never need to.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 17 Apr 2014 15:06:05 +0000 (16:06 +0100)]
ts-xen-install: nodhcp: Make idempotent
If run more than once, this function would erroneously determine that
xenbr0 was the primary physical interface name and try to enroll it
into the bridge (also named xenbr0).
Instead, spot the "bridge_ports" line and use it to extract the
physical interface name. Eliminate from consideration any interfaces
matching xenbr\d+.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Tested-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson [Thu, 17 Apr 2014 15:32:33 +0000 (16:32 +0100)]
cr-daily-branch: Print better message re lack of push
wantpush might be false because
- OSSTEST_PUSH is false (only set to true in daily-cron-settings-real)
- baseline untested (what we had here)
- "$tree" is unknown (although I think this would fail elsewhere)
- input and output revisions are the same
Print the message "Published tested tree is already up to date" only
if that's actually the case.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Tested-by: Ian Campbell <Ian.Campbell@citrix.com>
Ian Jackson [Thu, 17 Apr 2014 11:00:36 +0000 (12:00 +0100)]
preseed_base: drop obsolete references to NetGateway and NetNetmask
NetGateway and NetNetmask are not honoured (their lines in the preseed
file are commented out) and I'm not aware of anyone who sets them.
They have been there from the dawn of time.
Removing them gets rid of this warning (which appears twice)
Use of uninitialized value in concatenation (.) or string at Osstest/Debian.pm line 432.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Wei Liu [Mon, 14 Apr 2014 11:04:34 +0000 (12:04 +0100)]
ts-debian-hvm-install: fix guest memory size
If the host has more than 5G free memory, change guest memory size to 5G
to catch any error that triggers by crossing 4G boundary. Otherwise
guest memory size is set to 768 MB (taken from ts-redhat-install) which
should be a safe value to run this testcase on every host.
Signed-off-by: Wei Liu <wei.liu2@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Mon, 14 Apr 2014 10:31:15 +0000 (11:31 +0100)]
standalone: force an explicit default config
Some scripts (e.g. cri-args-hostlists) default to production-config instead of
the more expected ~/.xen-osstest/config. I keep forgetting this, so set the
more appropriate default in the helper script.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Mon, 14 Apr 2014 10:31:00 +0000 (11:31 +0100)]
ts-xen-build: remove frobbing of xen.git/buildconfigs/enable-xen-config
This shouldn't be touched since osstest.git commit c1fba9810435 "import
enable-xen-config from xen.git, use for ts-kernel-build" and is just wrong
since xen.git 40305629dc48 "build: remove Linux kernel build integration."
since the file no longer exists.
Of the options which were being set here only CONFIG_HIGHPTE isn't already
handled by ts-kernel-build. I don't think CONFIG_HIGHPTE needs to be forced to
off these days (pvops kernels do it automagically). If I'm wrong I'm sure
osstest will tell us...
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com> Cc: Jan Beulich <JBeulich@suse.com>
Wei Liu [Wed, 4 Dec 2013 17:19:32 +0000 (17:19 +0000)]
Introduce ts-debian-hvm-install
This is debian hvm guest test case. It resembles ts-redhat-install:
1. prepare a auto install CD
2. install debian hvm guest, currently using OVMF
3. test if the guest is up
The installtion CD is also bootable with seabios.
Signed-off-by: Wei Liu <wei.liu2@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Fri, 4 Apr 2014 08:55:56 +0000 (09:55 +0100)]
make-flight: really drop the stray xend test from 4.5 onwards
The precedence rules meant the test was:
( xend enabled AND arch = i386 ) OR ( arch = amd64 )
rather than the intended:
( xend enabled ) AND ( arch = i386 OR arch = amd64 )
I've confirmed that a xen-instable flight no longer has test jobs with
toolstack == xend and that a Xen 4.4 flight is unchanged.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Fri, 21 Mar 2014 09:58:00 +0000 (09:58 +0000)]
standalone mode convenience frontend
I can never remember all the various env vars which I can/should set so this
tool provides a command line veneer over the basics.
It also does some sanity checks for things which keep tripping me up
(inability to read apache logs, lack of ssh-agent) which fail in more or less
obscure ways.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
[ijc -- remove stray debug comment]
Ian Campbell [Thu, 20 Mar 2014 15:56:27 +0000 (15:56 +0000)]
Use ts-kernel-build for build-*-oldkern
instead of using the kernel build integration in xen.git, which is going away.
There is no difference to the .config produced. No test jobs seem to rely on
these kernels so I have not worried about making the contents of dist be
identical (specifically the filenames under /boot have lost their -xen suffix)
Remove the now unused xen and qemu(u) tree/revision stuff from the jobs'
runvars. Add the appropriate kconfighow and kimagefile runvars and implement
an appropriate kconfighow handler to use the create_config.sh present in this
tree (the xen.git intergration called the same script)
If the 2.6.18 build starts to fail, xen-unstable pushes will be broken
until they are fixed, as before. Previously such a build failure might be due
to changes in xen.git so that was slightly justifiable. But in practice this
is quite rare. This situation, while anomalous, is tolerable.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Thu, 20 Mar 2014 10:49:07 +0000 (10:49 +0000)]
Capture "lspci -vvv" and "lspci -tv" output.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
[ijc -- added "lspci -tv" and adjusted changelog to match]
Ian Campbell [Wed, 12 Mar 2014 16:47:44 +0000 (16:47 +0000)]
make-flight: only test xl in unstable
xend has been deprecated for enough releases now, lets focus on xl and/or make
space to add libvirt tests.
I have verified that a xen-4.4-testing flight does not change, while a
xen-unstable flight changes in the following expected ways:
- build-.*-xend jobs are dropped
- test-.*-xend-.* jobs become test-.*-xl-.*, the corresponding toolstack
runvar changes to xl and the buildjob runvar loses its -xen suffix. This
affects:
- test-amd64-i386-xend-qemut-winxpsp3
- test-amd64-i386-xend-winxpsp3
- a new job test-amd64-i386-xl-qemuu-winxpsp3 is added, previously the
non-sensical test-amd64-i386-xend-qemuu-winxpsp3 was suppressed by
job_create_test_filter_callback.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Tue, 18 Mar 2014 14:56:51 +0000 (14:56 +0000)]
Allow per-host TFTP setup
I run osstest against machines which are in both the XenServer and XenClient
administrative domains, and hence which have different TFTP servers, accessible
locally via different NFS mounted paths.
Make it possible to specify various bits of TFTP path via ~/.xen-osstest/config
by introducing the idea of Tftp "scope" and allowing that to be configurable on
a per-host basis. For example I have:
TftpDefaultScope xenserver
and I am now able to install on both cam-st16 (a xenserver world test box) and
marilith-n4 (an osstest machine hosted in the xenclient network) without
messing around with my configuration every time. I ran build-$ARCH,
build-$ARCH-pvops and test-$ARCH-$ARCH-xl in both cases.
Per-host Tftp settings are now in the $ho->{Tftp} hash instead of in $c.
$c{TftpHost} is unused -- remove the setting of its default.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Wed, 22 Jan 2014 16:39:28 +0000 (16:39 +0000)]
standalone: add blessing to flights table.
sg-check-tested wants this to exist.
I haven't implemented Osstest/JobDB/Standalone::job_ensure_started, which
should transition from 'contructing' to 'running' because I'm not hitting that
path.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Mon, 17 Mar 2014 14:58:59 +0000 (14:58 +0000)]
TestSupport: Don't use git proxy for non-git:// or http urls.
After this I was stymied by ssh host keys and other roadblocks and just pushed
the branch to my xenbits tree but I think this is still correct as far as it
goes.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Wed, 12 Feb 2014 11:10:51 +0000 (11:10 +0000)]
ts-hosts-allocate-Standalone: abort if the host to use has changed
When a job has been run once then the selected host is stored in a runvar and
used from then on. This means that if you try to run on a different host (by
changing the config or by changing OSSTEST_HOST_HOST) then you may be
surprised when things happen to the original host and not the new one.
Abort when this is detected.
Changing host requires you to run:
./cs-adjust-flight -v $flight runvar-del $job host
$flight = standalone by default
$job = test-x-y-z or build-x etc
host = the literal string host (not either of the host names)
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Mon, 10 Feb 2014 13:36:10 +0000 (13:36 +0000)]
cr-daily-branch: Make it possible to suppress the forcing of a baseline test
This is undesirable (most of the time) in a standalone environment, where you
are most likely to be interested in the current version and not historical
comparissons.
Not sure there isn't a better way.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Send the voluminous host allocation debug output to a compressed
logfile "hosts-allocate.debug.gz". Also send a copy of the logm
output to the debug log, by manipulating $logm_handle.
Remove the bodge which unshifted -D onto the arguments, and then
parsed it. Now the script takes no options.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
projects / osstest.git / log