Ian Campbell [Fri, 21 Mar 2014 09:58:00 +0000 (09:58 +0000)]
standalone mode convenience frontend
I can never remember all the various env vars which I can/should set so this
tool provides a command line veneer over the basics.
It also does some sanity checks for things which keep tripping me up
(inability to read apache logs, lack of ssh-agent) which fail in more or less
obscure ways.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
[ijc -- remove stray debug comment]
Ian Campbell [Thu, 20 Mar 2014 15:56:27 +0000 (15:56 +0000)]
Use ts-kernel-build for build-*-oldkern
instead of using the kernel build integration in xen.git, which is going away.
There is no difference to the .config produced. No test jobs seem to rely on
these kernels so I have not worried about making the contents of dist be
identical (specifically the filenames under /boot have lost their -xen suffix)
Remove the now unused xen and qemu(u) tree/revision stuff from the jobs'
runvars. Add the appropriate kconfighow and kimagefile runvars and implement
an appropriate kconfighow handler to use the create_config.sh present in this
tree (the xen.git intergration called the same script)
If the 2.6.18 build starts to fail, xen-unstable pushes will be broken
until they are fixed, as before. Previously such a build failure might be due
to changes in xen.git so that was slightly justifiable. But in practice this
is quite rare. This situation, while anomalous, is tolerable.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Thu, 20 Mar 2014 10:49:07 +0000 (10:49 +0000)]
Capture "lspci -vvv" and "lspci -tv" output.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
[ijc -- added "lspci -tv" and adjusted changelog to match]
Ian Campbell [Wed, 12 Mar 2014 16:47:44 +0000 (16:47 +0000)]
make-flight: only test xl in unstable
xend has been deprecated for enough releases now, lets focus on xl and/or make
space to add libvirt tests.
I have verified that a xen-4.4-testing flight does not change, while a
xen-unstable flight changes in the following expected ways:
- build-.*-xend jobs are dropped
- test-.*-xend-.* jobs become test-.*-xl-.*, the corresponding toolstack
runvar changes to xl and the buildjob runvar loses its -xen suffix. This
affects:
- test-amd64-i386-xend-qemut-winxpsp3
- test-amd64-i386-xend-winxpsp3
- a new job test-amd64-i386-xl-qemuu-winxpsp3 is added, previously the
non-sensical test-amd64-i386-xend-qemuu-winxpsp3 was suppressed by
job_create_test_filter_callback.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Tue, 18 Mar 2014 14:56:51 +0000 (14:56 +0000)]
Allow per-host TFTP setup
I run osstest against machines which are in both the XenServer and XenClient
administrative domains, and hence which have different TFTP servers, accessible
locally via different NFS mounted paths.
Make it possible to specify various bits of TFTP path via ~/.xen-osstest/config
by introducing the idea of Tftp "scope" and allowing that to be configurable on
a per-host basis. For example I have:
TftpDefaultScope xenserver
and I am now able to install on both cam-st16 (a xenserver world test box) and
marilith-n4 (an osstest machine hosted in the xenclient network) without
messing around with my configuration every time. I ran build-$ARCH,
build-$ARCH-pvops and test-$ARCH-$ARCH-xl in both cases.
Per-host Tftp settings are now in the $ho->{Tftp} hash instead of in $c.
$c{TftpHost} is unused -- remove the setting of its default.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Wed, 22 Jan 2014 16:39:28 +0000 (16:39 +0000)]
standalone: add blessing to flights table.
sg-check-tested wants this to exist.
I haven't implemented Osstest/JobDB/Standalone::job_ensure_started, which
should transition from 'contructing' to 'running' because I'm not hitting that
path.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Mon, 17 Mar 2014 14:58:59 +0000 (14:58 +0000)]
TestSupport: Don't use git proxy for non-git:// or http urls.
After this I was stymied by ssh host keys and other roadblocks and just pushed
the branch to my xenbits tree but I think this is still correct as far as it
goes.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Wed, 12 Feb 2014 11:10:51 +0000 (11:10 +0000)]
ts-hosts-allocate-Standalone: abort if the host to use has changed
When a job has been run once then the selected host is stored in a runvar and
used from then on. This means that if you try to run on a different host (by
changing the config or by changing OSSTEST_HOST_HOST) then you may be
surprised when things happen to the original host and not the new one.
Abort when this is detected.
Changing host requires you to run:
./cs-adjust-flight -v $flight runvar-del $job host
$flight = standalone by default
$job = test-x-y-z or build-x etc
host = the literal string host (not either of the host names)
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Mon, 10 Feb 2014 13:36:10 +0000 (13:36 +0000)]
cr-daily-branch: Make it possible to suppress the forcing of a baseline test
This is undesirable (most of the time) in a standalone environment, where you
are most likely to be interested in the current version and not historical
comparissons.
Not sure there isn't a better way.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Send the voluminous host allocation debug output to a compressed
logfile "hosts-allocate.debug.gz". Also send a copy of the logm
output to the debug log, by manipulating $logm_handle.
Remove the bodge which unshifted -D onto the arguments, and then
parsed it. Now the script takes no options.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
This driver is broken on ARM:
ERROR: "__bad_udelay" [drivers/scsi/bfa/bfa.ko] undefined!
I've taken the lazy way out and disabled it on all platforms. I think it isn't
especially likely that any of the current osstest hosts are using Fibre
Channel right now. The code to enable it came from a big batch addition of
drivers in 451f39c6149e.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Fri, 14 Feb 2014 18:16:29 +0000 (18:16 +0000)]
production-config: do not set WebspaceUrl, WebspaceFile
The defaults,
$c{WebspaceFile} ||= "$ENV{'HOME'}/public_html/";
$c{WebspaceUrl} ||= "http://$myfqdn/~$whoami/";
are adequate. This means the configuration can work on both
woking and the new osstest VM.
Signed-off-by: Ian Jackson <iwj@osstest.cam.xci-test.com>
Ian Campbell [Wed, 12 Feb 2014 14:27:56 +0000 (14:27 +0000)]
Do not attempt migration tests if the platform doesn't support it
Doing so blocks the rest of the tests in the job, which may be able to
indepentently complete. So arrange for a ts-migrate-support-check test to run
and gate the remaining migration tests on that.
This relies on the xen patch "xl: suppress suspend/resume functions on
platforms which do not support it" to actually suppress migration support on
arm.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
This needs to go in after the Xen patch. Otherwise this new step will appear to
pass and then start to fail when the Xen patch is applied.
Ian Campbell [Wed, 12 Feb 2014 10:30:49 +0000 (10:30 +0000)]
Configure the Calxeda fabric on host boot
The fabric on the Calxeda midway boxes (marilith-* in osstest) does not learn
mac address (at least not with the firmware we have, and with Calxeda folding
this seems unlikely to get fixed). This means that guests do not get network
connectivity unless their mac addresses explicitly registered with the fabric.
Registrations can be done with the bridge(8) tool from the iproute2 package
which unfortunately is only present in Jessie+ and not in Wheezy. So I have
done my own backport and placed it in $images/wheezy-iproute2 and arranged for
it to be installed along with the transitional iproute package (from the same
source) which is needed to satisfy various dependencies.
The registrations are ephemeral and need to be renewed on each reboot, so add
the necessary commands to rc.local during ts-xen-install.
This required leaking a certain amount of the implementation of select_ether.
Unless we want to do the bodge on every ts-guest-start, reboot, migrate etc
then this seems to be the best way.
test-armhf-armhf-xl now gets past ts-guest-start and as far as the migration
tests.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v2: hardcode 8 mac addresses and gate on equiv-marilith host flag
simplify code to install iproute debs
Ian Jackson [Mon, 10 Feb 2014 16:40:37 +0000 (16:40 +0000)]
ts-guests-nbd-mirror: set "oldstyle=true"
Newer NBDs (wheezy's and later) need a config option to say we're
using the old-style port-based addressing. At some point we will
probably switch to using the new-style addressing, but not yet.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Mon, 10 Feb 2014 16:40:22 +0000 (16:40 +0000)]
ts-guests-nbd-mirror: add checkaccessible test
If the NBD devices are not properly accessible on the client, bomb out
here rather than futilely starting a guest, and then timing out when
the guest fails to boot because it can't find its root fs.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Mon, 10 Feb 2014 16:39:37 +0000 (16:39 +0000)]
ts-guests-nbd-mirror: purge old packages first
Purge any old nbd-client and nbd-server _before_ we make their config.
This only has any effect if the packages are installed before this
script starts, which isn't the case in any of the automatically-run
recipes. But it can occur when the script is being tested by hand.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Thu, 23 Jan 2014 17:50:02 +0000 (17:50 +0000)]
ts-xen-install: nodhcp: restructure
Move target_editfile_root to contain all the meat of the function.
This is useful because we're going to possibly want to read the input
interfaces file twice.
No functional change.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Tue, 11 Feb 2014 12:30:21 +0000 (12:30 +0000)]
cr-daily-branch: make sure we test the correct tree for Linux branches
These branches should test the specific Linux tree which they and so should
not apply the per-arch overrides which are only intended to be used to pick up
an already verified tested Linux branch for use when testing some other
non-linux branch.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Fri, 7 Feb 2014 09:57:22 +0000 (09:57 +0000)]
ts-kernel-build: make sure CONFIG_PACKET is enabled
It is required by the dhcp client and is not present in the arm
multi_v7_defconfig.
Also stash the config file in the build results for easy reference, it is
already in kerndist.tar.gz but that's a 30+M download compared with a few tens
of K.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Campbell [Wed, 5 Feb 2014 16:46:19 +0000 (16:46 +0000)]
Enable armhf tests for linux-linus branch
Rather than whitelisting interesting branches, instead blacklist the hisotrical
branches which are not interesting to armhf, ensuring that any new branches
will pick up armhf automatically.
This creates the expected build-armhf, build-armhf-pvops and
test-armhf-armhf-xl jobs for the linux-linus branch.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Mon, 20 Jan 2014 17:52:10 +0000 (17:52 +0000)]
make-flight: reduce indentation in test_matrix_do_one
Now that the body of the multiple nested loops is in a function it doesn't need
to be so deeply indented. This wasn't done in the previous patch for clarity.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Ian Campbell [Mon, 20 Jan 2014 17:20:57 +0000 (17:20 +0000)]
make-flight: refactor job_create_test into mfi-common
Now that it uses a callback it can trivially be moved to mfi-common
Arguably the setting of *buildjob could also be a callback, but my intended use
case doesn't need that and it seems reasonable enough for now that users of the
common job_create_test also use the common create_build_jobs (or produce
something sufficiently similar).
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Ian Campbell [Mon, 20 Jan 2014 16:48:11 +0000 (16:48 +0000)]
make-flight: Remove md5sum based job filtering
JOB_MD5_PATTERN was intended to allow making randomly smaller flights, but is
not used in practice.
The filtering of the qemmu*-win jobs was intended to reduce the number of
combinations but ended up suppressing only:
test-amd64-i386-xl-qemuu-winxpsp3-vcpus1
test-amd64-i386-xl-qemuu-win7-amd64
Both of which seem useful so allow them to be enabled.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Ian Campbell [Mon, 20 Jan 2014 13:48:35 +0000 (13:48 +0000)]
make-flight: expand hard tabs
Run expand on the file.
Add an emacs magic var block disabling indent-tabs-mode and setting the sh
mode basic offset to 2 (which appears to be the common, but not universal,
case).
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Ian Campbell [Fri, 10 Jan 2014 09:55:10 +0000 (09:55 +0000)]
make-flight: Adjust buildjob runvar for xend vs non-xend tests
Currently we only do so for xenbuildjob which only contains the hypervisor and
not the tools, for those we need buildjob. In principal the xenbuildjob output
should be identical for both xend and non-xend builds, but we use a matching
set of consistency.
This adds a xend suffix to buildjob for exactly test-amd64-i386-pv,
test-amd64-amd64-pv, test-amd64-i386-xend-winxpsp3,
test-amd64-i386-xend-qemut-winxpsp3 which is the expected set of xend jobs.
Ian Campbell [Fri, 20 Dec 2013 12:35:27 +0000 (12:35 +0000)]
do not install xend for xl tests
We need to check that xl works correctly when xend is not even installed (in
case we are subtly relying on some file which xend installs).
Therefore for xen 4.4 onwards never build xend in the default build job and
instead create two new build jobs (for i386 and amd64) with xend enabled.
Update the tests to use the correct xenbuildjob.
Tested only to the extent of running make-flight for xen-4.{2,3,4}-testing and
xen-unstable and observing that the jobs do not differ for 4.2 and 4.3 and the
4.4 and unstable have gained the new build-{i386,and64}-xend jobs and that the
relevant tests have switched their xenbuildjob runvar to have the suffix.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson [Mon, 6 Jan 2014 13:52:58 +0000 (13:52 +0000)]
cs-bisection-step: Use tree URLs from latest flight, not basis pass
We (both in adhoc-revtuple-generator and in the eventual build job)
(when using git) clone a particular url and then expect that the
resulting tree has all of the relevant revisions in it.
If a push gate has a different input tree to output tree, it is
necessary to use the output tree. And, if the tree has moved, it is
probably more useful to use the new url than the old.
So it is more proper to use the tree url from the last flight found,
rather than that from the basis pass. Do that.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Campbell [Wed, 18 Dec 2013 09:32:04 +0000 (09:32 +0000)]
ts-debian-install: work around xen-create-image reliance on xend
The xen-tools script contains some sanity checks of the xend configuration and
fails if the configuration file is not present. This issue has been reported
upstream http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=732456.
Workaround this issue by creating a dummy xend-config.sxp if it does not
already exist. Include network- and vif- script options to keep the sanity
checks happy.
The workaround is enabled for all releases up to and including Wheezy in the
hopes that this will be fixed by the time Jessie is released, if not then the
osstest pushgate will hopefully catch this when Jessie is pushed to it.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Roger Pau Monné [Mon, 2 Dec 2013 15:22:53 +0000 (16:22 +0100)]
target_guest_lv_name: move into TestSupport; fix FreeBSD test
Move from lvm_lv_name in Debian.pm, and use it where appropriate in
ts-freebsd-install. This will fix the bug where ts-freebsd-install
fails on hsots with "-" in their hostname (and hence their vg name).
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Thu, 28 Nov 2013 18:20:04 +0000 (18:20 +0000)]
target_file_exists: Use non-root access for this check
The new call of this in built_stash_file makes ts-*-build require root
access, which is not desirable.
The only other caller of target_file_exists is in ts-xen-install where
it is called on
/etc/default/xencommons
/etc/sysconfig/xencommons
/etc/default/xend
/etc/sysconfig/xend
all of which should be accessible as a normal user in a default config.
So change the access to use target_cmd_output rather than
target_cmd_output_root.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
If this property is set then the environment variable
(a) DISTCC_HOSTS is set to the value of the host property
(a) CCACHE_PREFIX is set to "distcc"
which will result in distcc being used as configured.
distcc had better be installed for such hosts.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 21 Nov 2013 12:26:43 +0000 (12:26 +0000)]
cr-for-branches: when doing pre-pull from incoming, take out lock correctly
I.e. with -f if caller specified -q, rather than always using -w. -q
would make the script carry on if it didn't manage to get the lock and
do the pull, which would be wrong. -w is wrong because it can block
for another long-running script.
projects / osstest.git / log