]> xenbits.xensource.com Git - people/dariof/osstest.git/log
people/dariof/osstest.git
11 years agoIntroducing Osstest/Fedora.pm fedora-guest
Dario Faggioli [Tue, 10 Dec 2013 13:37:01 +0000 (14:37 +0100)]
Introducing Osstest/Fedora.pm

which, for now, only handles the kickstart files (the
equivalent of d-i preseeds).

Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
11 years agots-xen-build: bump "configure" timeout to 600s
Ian Jackson [Wed, 9 Apr 2014 10:36:44 +0000 (11:36 +0100)]
ts-xen-build: bump "configure" timeout to 600s

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
11 years agoiso_create_genisoimage: Honour $isotimeout
Ian Jackson [Wed, 9 Apr 2014 10:20:22 +0000 (11:20 +0100)]
iso_create_genisoimage: Honour $isotimeout

Fixes regression introduced in 20387acf, where the newly-moved code
used a hardwired value of 60 rather than the parameter provided for
the purpose.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
11 years agoSwitch back to genisoimage
Wei Liu [Mon, 7 Apr 2014 11:34:41 +0000 (12:34 +0100)]
Switch back to genisoimage

Xorriso is too slow. This is a patch to modify all related sites to use
genisoimage.

This effectively reverts the introduction of iso_create_xorriso and
introduces iso_create_genisoimage.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
11 years agots-redhat-install: use double quote in paths
Wei Liu [Sat, 5 Apr 2014 19:01:26 +0000 (20:01 +0100)]
ts-redhat-install: use double quote in paths

Otherwise variables are not substituted.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
11 years agots-redhat-install: copy_iso_content_to_dir -> iso_copy_content_from_image
Wei Liu [Fri, 4 Apr 2014 10:31:13 +0000 (11:31 +0100)]
ts-redhat-install: copy_iso_content_to_dir -> iso_copy_content_from_image

I renamed the function and missed this callsite.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
11 years agomake-flight: OVMF test flight
Wei Liu [Fri, 21 Mar 2014 12:12:27 +0000 (12:12 +0000)]
make-flight: OVMF test flight

Output of following runes attached:
  ./standalone-reset
  ./mg-show-flight-runvars standalone | grep ovmf | sed 's/[ \t]*$//'

test-amd64-amd64-xl-qemuu-ovmf-amd64     all_hostflags             arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test,hvm
test-amd64-i386-xl-qemuu-ovmf-amd64      all_hostflags             arch-i386,arch-xen-amd64,suite-wheezy,purpose-test,hvm
test-amd64-amd64-xl-qemuu-ovmf-amd64     arch                      amd64
test-amd64-i386-xl-qemuu-ovmf-amd64      arch                      i386
test-amd64-amd64-xl-qemuu-ovmf-amd64     buildjob                  build-amd64
test-amd64-i386-xl-qemuu-ovmf-amd64      buildjob                  build-i386
test-amd64-amd64-xl-qemuu-ovmf-amd64     device_model_version      qemu-xen
test-amd64-i386-xl-qemuu-ovmf-amd64      device_model_version      qemu-xen
build-amd64                              enable_ovmf               true
build-armhf                              enable_ovmf               true
build-i386                               enable_ovmf               true
test-amd64-amd64-xl-qemuu-ovmf-amd64     kernbuildjob              build-amd64-pvops
test-amd64-i386-xl-qemuu-ovmf-amd64      kernbuildjob              build-i386-pvops
test-amd64-amd64-xl-qemuu-ovmf-amd64     kernkind                  pvops
test-amd64-i386-xl-qemuu-ovmf-amd64      kernkind                  pvops
test-amd64-amd64-xl-qemuu-ovmf-amd64     ovmf_image                debian-7.2.0-amd64-CD-1.iso
test-amd64-i386-xl-qemuu-ovmf-amd64      ovmf_image                debian-7.2.0-amd64-CD-1.iso
test-amd64-amd64-xl-qemuu-ovmf-amd64     toolstack                 xl
test-amd64-i386-xl-qemuu-ovmf-amd64      toolstack                 xl
test-amd64-amd64-xl-qemuu-ovmf-amd64     xenbuildjob               build-amd64
test-amd64-i386-xl-qemuu-ovmf-amd64      xenbuildjob               build-amd64

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agosg-run-job: OVMF job
Wei Liu [Tue, 10 Dec 2013 11:56:41 +0000 (11:56 +0000)]
sg-run-job: OVMF job

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoIntroduce ts-debian-hvm-install
Wei Liu [Wed, 4 Dec 2013 17:19:32 +0000 (17:19 +0000)]
Introduce ts-debian-hvm-install

This is debian hvm guest test case. It resembles ts-redhat-install:
1. prepare a auto install CD
2. install debian hvm guest, currently using OVMF
3. test if the guest is up

The installtion CD is also bootable with seabios.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoDebian.pm: factor out preseed_base
Wei Liu [Wed, 11 Dec 2013 13:26:22 +0000 (13:26 +0000)]
Debian.pm: factor out preseed_base

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoTestSupport.pm: introduce iso_copy_content_from_image
Wei Liu [Wed, 2 Apr 2014 18:04:59 +0000 (19:04 +0100)]
TestSupport.pm: introduce iso_copy_content_from_image

... which returns a string of commands to pass into target_cmd_root.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoTestSupport.pm: introduce iso_gen_flags_basic
Wei Liu [Wed, 2 Apr 2014 18:01:39 +0000 (19:01 +0100)]
TestSupport.pm: introduce iso_gen_flags_basic

... which returns the flags that can be passed to xorriso to create a
bootable image (not EFI capable though).

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoTestSupport.pm: introduce iso_create_empty
Wei Liu [Wed, 2 Apr 2014 17:58:01 +0000 (18:58 +0100)]
TestSupport.pm: introduce iso_create_empty

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agots-redhat-install: use $flight, $job and $gn when creating iso
Wei Liu [Fri, 21 Mar 2014 11:06:44 +0000 (11:06 +0000)]
ts-redhat-install: use $flight, $job and $gn when creating iso

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoTestSupport.pm: introduce guest_create
Wei Liu [Wed, 2 Apr 2014 17:50:40 +0000 (18:50 +0100)]
TestSupport.pm: introduce guest_create

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoTestSupport.pm: move rewrite_config_nocd to support library
Wei Liu [Wed, 2 Apr 2014 17:49:27 +0000 (18:49 +0100)]
TestSupport.pm: move rewrite_config_nocd to support library

... and rename it to guest_editconfig_nocd.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoTestSupport.pm: introduce iso_create_xorriso
Wei Liu [Wed, 2 Apr 2014 17:21:22 +0000 (18:21 +0100)]
TestSupport.pm: introduce iso_create_xorriso

... and convert ts-redhat-install to make use of it.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoTestSupport.pm: add bios option to guest config file
Wei Liu [Thu, 5 Dec 2013 11:16:21 +0000 (11:16 +0000)]
TestSupport.pm: add bios option to guest config file

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agomake-flight: really drop the stray xend test from 4.5 onwards
Ian Campbell [Fri, 4 Apr 2014 08:55:56 +0000 (09:55 +0100)]
make-flight: really drop the stray xend test from 4.5 onwards

The precedence rules meant the test was:
  ( xend enabled AND arch = i386 ) OR ( arch = amd64 )
rather than the intended:
  ( xend enabled ) AND ( arch = i386 OR arch = amd64 )

I've confirmed that a xen-instable flight no longer has test jobs with
toolstack == xend and that a Xen 4.4 flight is unchanged.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agostandalone-reset: fix typo
Wei Liu [Wed, 2 Apr 2014 19:12:30 +0000 (20:12 +0100)]
standalone-reset: fix typo

The script name is sa-init-db not sa-db-init.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agomake-flight: drop another stray xend test from 4.5 onwards
Ian Campbell [Mon, 24 Mar 2014 11:50:11 +0000 (11:50 +0000)]
make-flight: drop another stray xend test from 4.5 onwards

I believe this configuration is covered for xl by the existing
test-amd64-i386-xl so test-amd64-i386-pv is dropped rather than replaced.

Changes for xen-4.4-testing flights:
$ diff -ub <(./mg-show-flight-runvars before44) <(./mg-show-flight-runvars after44)
$

Changes to unstable flights:
$ diff -ub <(./mg-show-flight-runvars beforeunstable) <(./mg-show-flight-runvars afterunstable) | grep -E '^[-+]'

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoCorrect crontab syntax to run most jobs at 30 minute intervals.
Ian Campbell [Mon, 31 Mar 2014 11:01:22 +0000 (12:01 +0100)]
Correct crontab syntax to run most jobs at 30 minute intervals.

4/30 means 4-4/30, the correct (if counter intuitive) syntax is 4-59/30.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
11 years agostandalone mode convenience frontend
Ian Campbell [Fri, 21 Mar 2014 09:58:00 +0000 (09:58 +0000)]
standalone mode convenience frontend

I can never remember all the various env vars which I can/should set so this
tool provides a command line veneer over the basics.

It also does some sanity checks for things which keep tripping me up
(inability to read apache logs, lack of ssh-agent) which fail in more or less
obscure ways.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
[ijc -- remove stray debug comment]

11 years agostandalone: refactor out some useful bits of standalone-reset
Ian Campbell [Fri, 21 Mar 2014 09:57:59 +0000 (09:57 +0000)]
standalone: refactor out some useful bits of standalone-reset

I sometimes want just these bits.

I went with a new "sa" prefix, but I left standalone-reset alone since it is
already reasonably well known.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agots-xen-build: Drop ability to build kernels
Ian Campbell [Thu, 20 Mar 2014 15:56:28 +0000 (15:56 +0000)]
ts-xen-build: Drop ability to build kernels

These are now all handled by ts-kernel-build.

We continue to specify KERNELS='' so that when building ancient versions of
Xen they won't try and build anything.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoUse ts-kernel-build for build-*-oldkern
Ian Campbell [Thu, 20 Mar 2014 15:56:27 +0000 (15:56 +0000)]
Use ts-kernel-build for build-*-oldkern

instead of using the kernel build integration in xen.git, which is going away.

There is no difference to the .config produced. No test jobs seem to rely on
these kernels so I have not worried about making the contents of dist be
identical (specifically the filenames under /boot have lost their -xen suffix)

Remove the now unused xen and qemu(u) tree/revision stuff from the jobs'
runvars. Add the appropriate kconfighow and kimagefile runvars and implement
an appropriate kconfighow handler to use the create_config.sh present in this
tree (the xen.git intergration called the same script)

If the 2.6.18 build starts to fail, xen-unstable pushes will be broken
until they are fixed, as before.  Previously such a build failure might be due
to changes in xen.git so that was slightly justifiable.  But in practice this
is quite rare.  This situation, while anomalous, is tolerable.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoimport enable-xen-config from xen.git, use for ts-kernel-build
Ian Campbell [Thu, 20 Mar 2014 15:56:26 +0000 (15:56 +0000)]
import enable-xen-config from xen.git, use for ts-kernel-build

I plan to remove it from xen.git. Take the version from a03984e7bcdf which
hasn't changed since 8dedf6431393 in 2010.

Update ts-kernel-build to not require xen.git.

config_intree_buildconfigs was unused so remove it.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
[ijc -- reduce timeout to 100, wrap log line]

11 years agomfi-common: remove xen_kernels runvar from build-*-pvops
Ian Campbell [Thu, 20 Mar 2014 15:56:25 +0000 (15:56 +0000)]
mfi-common: remove xen_kernels runvar from build-*-pvops

ts-kernel-build does not use this.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoCapture "lspci -vvv" and "lspci -tv" output.
Ian Campbell [Thu, 20 Mar 2014 10:49:07 +0000 (10:49 +0000)]
Capture "lspci -vvv" and "lspci -tv" output.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
[ijc -- added "lspci -tv" and adjusted changelog to match]

11 years agomake-flight: only test xl in unstable
Ian Campbell [Wed, 12 Mar 2014 16:47:44 +0000 (16:47 +0000)]
make-flight: only test xl in unstable

xend has been deprecated for enough releases now, lets focus on xl and/or make
space to add libvirt tests.

I have verified that a xen-4.4-testing flight does not change, while a
xen-unstable flight changes in the following expected ways:
  - build-.*-xend jobs are dropped
  - test-.*-xend-.* jobs become test-.*-xl-.*, the corresponding toolstack
    runvar changes to xl and the buildjob runvar loses its -xen suffix. This
    affects:
     - test-amd64-i386-xend-qemut-winxpsp3
     - test-amd64-i386-xend-winxpsp3
  - a new job test-amd64-i386-xl-qemuu-winxpsp3 is added, previously the
    non-sensical test-amd64-i386-xend-qemuu-winxpsp3 was suppressed by
    job_create_test_filter_callback.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoRevert "allow.all: Tolerate all build-*-oldkern failures"
Ian Jackson [Wed, 19 Mar 2014 15:20:38 +0000 (15:20 +0000)]
Revert "allow.all: Tolerate all build-*-oldkern failures"

The network braindamage has been worked around in the firewall.

This reverts commit b4fdc54eac9ed82e0e056573a11f3ad106f97c67.

11 years agoAllow per-host TFTP setup
Ian Campbell [Tue, 18 Mar 2014 14:56:51 +0000 (14:56 +0000)]
Allow per-host TFTP setup

I run osstest against machines which are in both the XenServer and XenClient
administrative domains, and hence which have different TFTP servers, accessible
locally via different NFS mounted paths.

Make it possible to specify various bits of TFTP path via ~/.xen-osstest/config
by introducing the idea of Tftp "scope" and allowing that to be configurable on
a per-host basis. For example I have:
  TftpDefaultScope xenserver

  TftpPath_xenserver /usr/groups/netboot/
  TftpPxeDir_xenserver pxelinux.cfg/

  TftpPath_xenclient /home/xc_tftpboot/pxe/
  TftpPxeDir_xenclient /
  TftpPxeTemplates_xenclient %ipaddrhex%/pxelinux.cfg

  HostProp_marilith-n4_TftpScope xenclient

and I am now able to install on both cam-st16 (a xenserver world test box) and
marilith-n4 (an osstest machine hosted in the xenclient network) without
messing around with my configuration every time. I ran build-$ARCH,
build-$ARCH-pvops and test-$ARCH-$ARCH-xl in both cases.

Per-host Tftp settings are now in the $ho->{Tftp} hash instead of in $c.

$c{TftpHost} is unused -- remove the setting of its default.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoDebian: die if we are unable to find the override kernel deb
Ian Campbell [Tue, 18 Mar 2014 10:22:56 +0000 (10:22 +0000)]
Debian: die if we are unable to find the override kernel deb

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agostandalone: add blessing to flights table.
Ian Campbell [Wed, 22 Jan 2014 16:39:28 +0000 (16:39 +0000)]
standalone: add blessing to flights table.

sg-check-tested wants this to exist.

I haven't implemented Osstest/JobDB/Standalone::job_ensure_started, which
should transition from 'contructing' to 'running' because I'm not hitting that
path.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agocri-args-hostlists: Allow environment to control OSSTEST_CONFIG
Ian Campbell [Wed, 22 Jan 2014 16:39:23 +0000 (16:39 +0000)]
cri-args-hostlists: Allow environment to control OSSTEST_CONFIG

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoTestSupport: Don't use git proxy for non-git:// or http urls.
Ian Campbell [Mon, 17 Mar 2014 14:58:59 +0000 (14:58 +0000)]
TestSupport: Don't use git proxy for non-git:// or http urls.

After this I was stymied by ssh host keys and other roadblocks and just pushed
the branch to my xenbits tree but I think this is still correct as far as it
goes.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoAllow forcing the use of current osstest HEAD for branch=osstest
Ian Campbell [Tue, 11 Feb 2014 13:31:43 +0000 (13:31 +0000)]
Allow forcing the use of current osstest HEAD for branch=osstest

Otherwise cr-daily-branch expects $HOME/testing.git to exist and will
git-reset it etc, which is rather annoying when in standalone mode...

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agots-hosts-allocate-Standalone: abort if the host to use has changed
Ian Campbell [Wed, 12 Feb 2014 11:10:51 +0000 (11:10 +0000)]
ts-hosts-allocate-Standalone: abort if the host to use has changed

When a job has been run once then the selected host is stored in a runvar and
used from then on. This means that if you try to run on a different host (by
changing the config or by changing OSSTEST_HOST_HOST) then you may be
surprised when things happen to the original host and not the new one.

Abort when this is detected.

Changing host requires you to run:
    ./cs-adjust-flight -v $flight runvar-del $job host
$flight = standalone by default
$job = test-x-y-z or build-x etc
host = the literal string host (not either of the host names)

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agocs-adjust-flight: fix runvar-del
Ian Campbell [Tue, 11 Feb 2014 16:49:29 +0000 (16:49 +0000)]
cs-adjust-flight: fix runvar-del

Missing $.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agocr-daily-branch: Make it possible to suppress the forcing of a baseline test
Ian Campbell [Mon, 10 Feb 2014 13:36:10 +0000 (13:36 +0000)]
cr-daily-branch: Make it possible to suppress the forcing of a baseline test

This is undesirable (most of the time) in a standalone environment, where you
are most likely to be interested in the current version and not historical
comparissons.

Not sure there isn't a better way.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoCXFabric: Use a vif script hook to (de)register guest mac address
Ian Campbell [Thu, 6 Mar 2014 14:12:45 +0000 (14:12 +0000)]
CXFabric: Use a vif script hook to (de)register guest mac address

Since mac table is apparently small this avoids filling it with mac addresses
which may never get used.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
11 years agoRevert "CXFabric: Register more MAC addresses with the fabric"
Ian Jackson [Thu, 6 Mar 2014 12:36:23 +0000 (12:36 +0000)]
Revert "CXFabric: Register more MAC addresses with the fabric"

This reverts commit 9be39d3917d33abeba14f7f282e0b896ebc25ab7.

It turns out that the Calxeda fabric has a limit on the number of
addresses, which this exceeds.  For now, go back to 8.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agomg-allocate: allow alternatives
Ian Jackson [Fri, 21 Feb 2014 15:34:24 +0000 (15:34 +0000)]
mg-allocate: allow alternatives

11 years agomg-allocate: fix typo in message
Ian Jackson [Fri, 21 Feb 2014 13:44:39 +0000 (13:44 +0000)]
mg-allocate: fix typo in message

11 years agots-hosts-allocate-Executive: Compress debug output
Ian Jackson [Fri, 21 Feb 2014 13:31:39 +0000 (13:31 +0000)]
ts-hosts-allocate-Executive: Compress debug output

Send the voluminous host allocation debug output to a compressed
logfile "hosts-allocate.debug.gz".  Also send a copy of the logm
output to the debug log, by manipulating $logm_handle.

Remove the bodge which unshifted -D onto the arguments, and then
parsed it.  Now the script takes no options.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
11 years agoTestSupport: logm: allow $logm_handle to be an aref
Ian Jackson [Fri, 21 Feb 2014 13:30:29 +0000 (13:30 +0000)]
TestSupport: logm: allow $logm_handle to be an aref

If $logm_handle is an array reference, iterate over it.  This allows
calling code to duplicate the messages.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
11 years agoExecutive: support DebugFh xparam to alloc_resources
Ian Jackson [Fri, 21 Feb 2014 13:27:49 +0000 (13:27 +0000)]
Executive: support DebugFh xparam to alloc_resources

This redirects some of the more verbose output (the json dumps)
elsewhere.  If DebugFh is not set, this output is suppressed.

ts-hosts-allocate-Executive provides DebugFh pointing to its DEBUG fh.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
11 years agostandalone-reset: actually honour '-f' option
Dario Faggioli [Wed, 29 Jan 2014 14:32:35 +0000 (14:32 +0000)]
standalone-reset: actually honour '-f' option

standalone-reset's usage says:

  usage: ./standalone-reset [<options>] [<branch> [<xenbranch> [<buildflight>]]]
   branch and xenbranch default, separately, to xen-unstable
  options:
   -f<flight>     generate flight "flight", default is "standalone"

but then there is no place where '-f' is processed, and hence
no real way to pass a specific flight name to make-flight.

Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoREADME: Add some core concepts and terminology
Ian Campbell [Mon, 20 Jan 2014 13:20:10 +0000 (13:20 +0000)]
README: Add some core concepts and terminology

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoCXFabric: Register more MAC addresses with the fabric
Ian Campbell [Sun, 2 Mar 2014 10:22:44 +0000 (10:22 +0000)]
CXFabric: Register more MAC addresses with the fabric

I've seen 0x20 used in a single flight. Register up to 0x40.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agocr-for-branches: Branch for Xen 4.4
Ian Jackson [Fri, 21 Feb 2014 16:44:15 +0000 (16:44 +0000)]
cr-for-branches: Branch for Xen 4.4

Process the new branches xen-4.4-testing and qemu-upstream-4.4-testing.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
11 years agots-xen-build: Set LIBLEAFDIR_x86_64
Ian Jackson [Fri, 21 Feb 2014 18:27:12 +0000 (18:27 +0000)]
ts-xen-build: Set LIBLEAFDIR_x86_64

Xen 4.1 puts things in /usr/lib64 otherwise.  This is wrong for
Debian, and does not work at all on wheezy.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
11 years agots-kernel-build: Disable "Brocade BFA Fibre Channel Support"
Ian Campbell [Mon, 24 Feb 2014 09:50:57 +0000 (09:50 +0000)]
ts-kernel-build: Disable "Brocade BFA Fibre Channel Support"

This driver is broken on ARM:
    ERROR: "__bad_udelay" [drivers/scsi/bfa/bfa.ko] undefined!

I've taken the lazy way out and disabled it on all platforms. I think it isn't
especially likely that any of the current osstest hosts are using Fibre
Channel right now. The code to enable it came from a big batch addition of
drivers in 451f39c6149e.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoallow.all: Tolerate all build-*-oldkern failures
Ian Jackson [Thu, 27 Feb 2014 15:00:11 +0000 (15:00 +0000)]
allow.all: Tolerate all build-*-oldkern failures

We are getting very many of these due to network braindamage.
For now, tolerate them.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
11 years agoExecutive.pm: Change default ControlDaemonHost
Ian Jackson [Tue, 18 Feb 2014 18:58:12 +0000 (18:58 +0000)]
Executive.pm: Change default ControlDaemonHost

We are going to move these daemons from woking to the new osstest VM.
So use a DNS alias for now.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
11 years agoproduction-config: do not set WebspaceUrl, WebspaceFile
Ian Jackson [Fri, 14 Feb 2014 18:16:29 +0000 (18:16 +0000)]
production-config: do not set WebspaceUrl, WebspaceFile

The defaults,
    $c{WebspaceFile} ||= "$ENV{'HOME'}/public_html/";
    $c{WebspaceUrl} ||= "http://$myfqdn/~$whoami/";
are adequate.  This means the configuration can work on both
woking and the new osstest VM.

Signed-off-by: Ian Jackson <iwj@osstest.cam.xci-test.com>
11 years agoproduction-config: authorise iwj@osstest key
Ian Jackson [Tue, 18 Feb 2014 18:05:53 +0000 (18:05 +0000)]
production-config: authorise iwj@osstest key

Just like the iwj@woking key.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoreadglobalconfig: change default DhcpWatchMethod
Ian Jackson [Tue, 18 Feb 2014 18:05:32 +0000 (18:05 +0000)]
readglobalconfig: change default DhcpWatchMethod

Use the TCP connection to woking, so that it works after we switch
to the new osstest VM.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoproduction-config: use /home/xc_tftpboot, not /tftpboot
Ian Jackson [Fri, 14 Feb 2014 19:09:32 +0000 (19:09 +0000)]
production-config: use /home/xc_tftpboot, not /tftpboot

This is available on both woking and on the new osstest VM.

Signed-off-by: Ian Jackson <iwj@osstest.cam.xci-test.com>
11 years agogit: Use "git foo" rather than "git-foo"
Ian Jackson [Fri, 14 Feb 2014 16:08:00 +0000 (16:08 +0000)]
git: Use "git foo" rather than "git-foo"

All of our machines now have "git foo" and have done for some time,
and some are going to not have "git-foo".

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
11 years agots-xen-build: Apply python workaround in wheezy too
Ian Jackson [Wed, 19 Feb 2014 15:58:20 +0000 (15:58 +0000)]
ts-xen-build: Apply python workaround in wheezy too

Debian #693721 is not yet fixed.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
11 years agoDo not attempt migration tests if the platform doesn't support it
Ian Campbell [Wed, 12 Feb 2014 14:27:56 +0000 (14:27 +0000)]
Do not attempt migration tests if the platform doesn't support it

Doing so blocks the rest of the tests in the job, which may be able to
indepentently complete. So arrange for a ts-migrate-support-check test to run
and gate the remaining migration tests on that.

This relies on the xen patch "xl: suppress suspend/resume functions on
platforms which do not support it" to actually suppress migration support on
arm.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
This needs to go in after the Xen patch. Otherwise this new step will appear to
pass and then start to fail when the Xen patch is applied.

11 years agoConfigure the Calxeda fabric on host boot
Ian Campbell [Wed, 12 Feb 2014 10:30:49 +0000 (10:30 +0000)]
Configure the Calxeda fabric on host boot

The fabric on the Calxeda midway boxes (marilith-* in osstest) does not learn
mac address (at least not with the firmware we have, and with Calxeda folding
this seems unlikely to get fixed). This means that guests do not get network
connectivity unless their mac addresses explicitly registered with the fabric.

Registrations can be done with the bridge(8) tool from the iproute2 package
which unfortunately is only present in Jessie+ and not in Wheezy. So I have
done my own backport and placed it in $images/wheezy-iproute2 and arranged for
it to be installed along with the transitional iproute package (from the same
source) which is needed to satisfy various dependencies.

The registrations are ephemeral and need to be renewed on each reboot, so add
the necessary commands to rc.local during ts-xen-install.

This required leaking a certain amount of the implementation of select_ether.
Unless we want to do the bodge on every ts-guest-start, reboot, migrate etc
then this seems to be the best way.

test-armhf-armhf-xl now gets past ts-guest-start and as far as the migration
tests.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v2: hardcode 8 mac addresses and gate on equiv-marilith host flag
    simplify code to install iproute debs

11 years agofreebsd: switch to 10.0-RELEASE (20140116-r260789)
Ian Jackson [Mon, 10 Feb 2014 12:09:51 +0000 (12:09 +0000)]
freebsd: switch to 10.0-RELEASE (20140116-r260789)

11 years agomake-flight: abolish special-casing of suite for armhf
Ian Jackson [Thu, 12 Dec 2013 12:08:31 +0000 (12:08 +0000)]
make-flight: abolish special-casing of suite for armhf

Since the suite is now wheezy by default this is no longer needed.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
11 years agoDebian: Switch to wheezy
Ian Jackson [Mon, 18 Nov 2013 16:04:45 +0000 (16:04 +0000)]
Debian: Switch to wheezy

This involves updating the d-i version too.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agots-guests-nbd-mirror: set "oldstyle=true"
Ian Jackson [Mon, 10 Feb 2014 16:40:37 +0000 (16:40 +0000)]
ts-guests-nbd-mirror: set "oldstyle=true"

Newer NBDs (wheezy's and later) need a config option to say we're
using the old-style port-based addressing.  At some point we will
probably switch to using the new-style addressing, but not yet.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agots-guests-nbd-mirror: add checkaccessible test
Ian Jackson [Mon, 10 Feb 2014 16:40:22 +0000 (16:40 +0000)]
ts-guests-nbd-mirror: add checkaccessible test

If the NBD devices are not properly accessible on the client, bomb out
here rather than futilely starting a guest, and then timing out when
the guest fails to boot because it can't find its root fs.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agots-guests-nbd-mirror: purge old packages first
Ian Jackson [Mon, 10 Feb 2014 16:39:37 +0000 (16:39 +0000)]
ts-guests-nbd-mirror: purge old packages first

Purge any old nbd-client and nbd-server _before_ we make their config.

This only has any effect if the packages are installed before this
script starts, which isn't the case in any of the automatically-run
recipes.  But it can occur when the script is being tested by hand.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoTestSupport: Suppress prompting by apt
Ian Jackson [Mon, 10 Feb 2014 16:38:57 +0000 (16:38 +0000)]
TestSupport: Suppress prompting by apt

Always set DEBIAN_PRIORITY=critical UCF_FORCE_CONFFOLD=y in
environment of apt, to suppress some prompts.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoTestSupport: break out target_run_apt
Ian Jackson [Mon, 10 Feb 2014 16:34:52 +0000 (16:34 +0000)]
TestSupport: break out target_run_apt

We are going to add some environment variables to the apt invocation,
so centralise where this happens.  No functional change.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agots-xen-install: default the interface to the one in /etc/network/interfaces
Ian Jackson [Thu, 23 Jan 2014 18:30:47 +0000 (18:30 +0000)]
ts-xen-install: default the interface to the one in /etc/network/interfaces

The default was simply eth0.  This is the other piece of automatically
coping with the boot interface not being eth0.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agots-xen-install: nodhcp: restructure
Ian Jackson [Thu, 23 Jan 2014 17:50:02 +0000 (17:50 +0000)]
ts-xen-install: nodhcp: restructure

Move target_editfile_root to contain all the meat of the function.
This is useful because we're going to possibly want to read the input
interfaces file twice.

No functional change.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agots-host-install: set `IPAPPEND 2' (if interface isn't forced)
Ian Jackson [Thu, 23 Jan 2014 16:33:44 +0000 (16:33 +0000)]
ts-host-install: set `IPAPPEND 2' (if interface isn't forced)

This causes BOOTIF=<mac-address> to appear on command line.  This
makes d-i use that interface.  (See also Debian #615600.)

This is a better approach to interface name instability than setting
the force interface host property.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agots-kernel-build: force CONFIG_BLK_DEV_NBD=y
Ian Jackson [Thu, 23 Jan 2014 11:29:52 +0000 (11:29 +0000)]
ts-kernel-build: force CONFIG_BLK_DEV_NBD=y

Otherwise, with wheezy and Linux 3.4.77:

Jan 23 04:36:42 lake-frog nbd_client[4274]: Cannot open NBD: No such file or directory#012Please ensure the nbd module is loaded.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agots-xen-build-prep: avoid lvextend segfault (Debian #736173) with wheezy
Ian Jackson [Tue, 21 Jan 2014 18:53:13 +0000 (18:53 +0000)]
ts-xen-build-prep: avoid lvextend segfault (Debian #736173) with wheezy

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agocr-daily-branch: make sure we test the correct tree for Linux branches
Ian Campbell [Tue, 11 Feb 2014 12:30:21 +0000 (12:30 +0000)]
cr-daily-branch: make sure we test the correct tree for Linux branches

These branches should test the specific Linux tree which they and so should
not apply the per-arch overrides which are only intended to be used to pick up
an already verified tested Linux branch for use when testing some other
non-linux branch.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agots-kernel-build: make sure CONFIG_PACKET is enabled
Ian Campbell [Fri, 7 Feb 2014 09:57:22 +0000 (09:57 +0000)]
ts-kernel-build: make sure CONFIG_PACKET is enabled

It is required by the dhcp client and is not present in the arm
multi_v7_defconfig.

Also stash the config file in the build results for easy reference, it is
already in kerndist.tar.gz but that's a 30+M download compared with a few tens
of K.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
11 years agocr-daily-branch: Limit "people who touched" listing
Ian Jackson [Wed, 5 Feb 2014 16:55:51 +0000 (16:55 +0000)]
cr-daily-branch: Limit "people who touched" listing

Sometimes this can be very large, which can result in the results
postings getting discarded.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
11 years agoEnable armhf tests for linux-linus branch
Ian Campbell [Wed, 5 Feb 2014 16:46:19 +0000 (16:46 +0000)]
Enable armhf tests for linux-linus branch

Rather than whitelisting interesting branches, instead blacklist the hisotrical
branches which are not interesting to armhf, ensuring that any new branches
will pick up armhf automatically.

This creates the expected build-armhf, build-armhf-pvops and
test-armhf-armhf-xl jobs for the linux-linus branch.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agomake-flight: refactor test_matrix_do_one
Ian Campbell [Tue, 21 Jan 2014 11:52:54 +0000 (11:52 +0000)]
make-flight: refactor test_matrix_do_one

Pull some test creation out into their own subdirectories. This allows the
total level of indentation to be reduced.

It also allows us to invert the pre-condition test and simply return at the
top of the subroutine, which further reduces indentation.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
11 years agomfi-common: onetoolstack does not vary for a given $xenbranch
Ian Campbell [Mon, 20 Jan 2014 17:58:33 +0000 (17:58 +0000)]
mfi-common: onetoolstack does not vary for a given $xenbranch

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
11 years agomake-flight: Refactor test matrix iteration into mfi-common
Ian Campbell [Mon, 20 Jan 2014 17:56:28 +0000 (17:56 +0000)]
make-flight: Refactor test matrix iteration into mfi-common

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
11 years agomake-flight: reduce indentation in test_matrix_do_one
Ian Campbell [Mon, 20 Jan 2014 17:52:10 +0000 (17:52 +0000)]
make-flight: reduce indentation in test_matrix_do_one

Now that the body of the multiple nested loops is in a function it doesn't need
to be so deeply indented. This wasn't done in the previous patch for clarity.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
11 years agomake-flight: Separate matrix iteration from test job creation
Ian Campbell [Mon, 20 Jan 2014 17:48:35 +0000 (17:48 +0000)]
make-flight: Separate matrix iteration from test job creation

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
11 years agomake-flight: refactor test case filter over $branch
Ian Campbell [Mon, 20 Jan 2014 17:47:15 +0000 (17:47 +0000)]
make-flight: refactor test case filter over $branch

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
11 years agomake-flight: refactor job_create_test into mfi-common
Ian Campbell [Mon, 20 Jan 2014 17:20:57 +0000 (17:20 +0000)]
make-flight: refactor job_create_test into mfi-common

Now that it uses a callback it can trivially be moved to mfi-common

Arguably the setting of *buildjob could also be a callback, but my intended use
case doesn't need that and it seems reasonable enough for now that users of the
common job_create_test also use the common create_build_jobs (or produce
something sufficiently similar).

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
11 years agomake-flight: refactor job_create_test filters
Ian Campbell [Mon, 20 Jan 2014 17:18:22 +0000 (17:18 +0000)]
make-flight: refactor job_create_test filters

This will allow job_create_test to be moved mfi-common.

No (intentional) change to the set of jobs which are created.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
11 years agomake-flight: Remove md5sum based job filtering
Ian Campbell [Mon, 20 Jan 2014 16:48:11 +0000 (16:48 +0000)]
make-flight: Remove md5sum based job filtering

JOB_MD5_PATTERN was intended to allow making randomly smaller flights, but is
not used in practice.

The filtering of the qemmu*-win jobs was intended to reduce the number of
combinations but ended up suppressing only:
  test-amd64-i386-xl-qemuu-winxpsp3-vcpus1
  test-amd64-i386-xl-qemuu-win7-amd64

Both of which seem useful so allow them to be enabled.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
11 years agomfi-common: restrict scope of local vars in create_build_jobs
Ian Campbell [Mon, 20 Jan 2014 15:47:01 +0000 (15:47 +0000)]
mfi-common: restrict scope of local vars in create_build_jobs

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
11 years agomfi-common: Allow caller of create_build_jobs to include/exclude xend builds
Ian Campbell [Mon, 20 Jan 2014 15:43:12 +0000 (15:43 +0000)]
mfi-common: Allow caller of create_build_jobs to include/exclude xend builds

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v2: Corect WANT_XEND test.

11 years agoRemove support for building the XCP kernel
Ian Campbell [Mon, 20 Jan 2014 15:38:26 +0000 (15:38 +0000)]
Remove support for building the XCP kernel

This kernel is well and truly dead.

I've left some of the general support for using other kernel types in place
even though it is now unused.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
11 years agomake-flight: refactor build job creation into mfi-common
Ian Campbell [Mon, 20 Jan 2014 15:32:27 +0000 (15:32 +0000)]
make-flight: refactor build job creation into mfi-common

This is pure code motion *except* I have aligned the backslash continuation
lines at the same time -- I was unable to resist doing so.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
11 years agomake-flight: Drop obsolete/unused xenrt_images variable.
Ian Campbell [Mon, 20 Jan 2014 15:12:24 +0000 (15:12 +0000)]
make-flight: Drop obsolete/unused xenrt_images variable.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
11 years agomake-flight: refactor common function "stripy" into helper library
Ian Campbell [Mon, 20 Jan 2014 15:07:18 +0000 (15:07 +0000)]
make-flight: refactor common function "stripy" into helper library

Will be useful for other make-flight variants.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
11 years agomake-flight: expand hard tabs
Ian Campbell [Mon, 20 Jan 2014 13:48:35 +0000 (13:48 +0000)]
make-flight: expand hard tabs

Run expand on the file.

Add an emacs magic var block disabling indent-tabs-mode and setting the sh
mode basic offset to 2 (which appears to be the common, but not universal,
case).

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
11 years agostandalone: Correct arguments to JobDB flight_create
Ian Campbell [Tue, 21 Jan 2014 16:44:29 +0000 (16:44 +0000)]
standalone: Correct arguments to JobDB flight_create

The real jobdb takes $intended and $branch in the other order, meaning that
the standalone db ends up with them backwards.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agoRemove obsolete scripts.
Ian Campbell [Mon, 20 Jan 2014 12:43:58 +0000 (12:43 +0000)]
Remove obsolete scripts.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
11 years agots-kernel-build: override kernel config from runvars
Ian Campbell [Fri, 10 Jan 2014 14:58:32 +0000 (14:58 +0000)]
ts-kernel-build: override kernel config from runvars

Use this to build EXT4 into the kernel statically for armhf build to work
around the lack of guest initrd support on ARM at the moment.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
11 years agomake-flight: override the Debian guest suite on armhf
Ian Campbell [Fri, 10 Jan 2014 11:41:47 +0000 (11:41 +0000)]
make-flight: override the Debian guest suite on armhf

We already override the host Debian suite but we need to override the guest too
(since armhf doesn't exist in squeeze).

Consolidate the Debian guest vars in to one place and add  debian_suite runvar
when appropriate.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>