Wei Liu [Fri, 27 Oct 2017 10:42:18 +0000 (11:42 +0100)]
Set mac address in interfaces(5) if force-mac-address is set
Nic with that flag set doesn't have a valid mac address and should use
the one assigned by osstest. Put the mac address in interfaces(5) in
the installed OS.
Wei Liu [Fri, 20 Oct 2017 14:35:06 +0000 (15:35 +0100)]
ts-guests-nbd-mirror: make it work with stretch
On the server side, only add oldstyle= and port= on Wheezy and Jessie.
Stretch doesn't support or need those anymore.
On the client side, generate new style configuration file.
Reorder nbd-client setup a bit. Install it first, then write our own
configuration file, then start it. This make dpkg to not ask what to
do regarding configuration files.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
v3: invert some tests, rearrange client setup code.
Wei Liu [Tue, 17 Oct 2017 16:10:10 +0000 (17:10 +0100)]
ts-debian-fixup: merge origin extra= to our own if necessary
The original extra= was not removed, so there were two extra= in the
resulting config file.
It wasn't a problem for xl because the second extra= took precedence.
However libvirt tests would only pick up the first extra= -- they
worked by chance.
Fix this issue by merging the original. If there isn't already extra=
in $cfg, use our own.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
v3: handle situation when no extra= is in $cfg
Ian Jackson [Fri, 27 Apr 2018 13:43:56 +0000 (14:43 +0100)]
production-config: Use something outside ~osstest/.ssh for TestHostKeypairPath
ansible nowadays chmods ~/.ssh to 700 for every user whose
authorized_keys it touches. This includes osstest@osstest.
The result is that other users on osstest.test-lab cannot access this
file. I have cp -a'd the keys, which are a piece of static
configuration we don't expect to change often, to a different
directory which will not be attacked by ansible. Refer to them there.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Mon, 3 Jul 2017 16:54:19 +0000 (17:54 +0100)]
db schema: Make the testid column NOT NLLL
We have to fix some old data. We insist that the old data is indeed
old (more than 5 years old) and not part of proper flights (ie,
blessed "play" or "crashed" or "unknown").
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 5 Apr 2018 14:40:00 +0000 (15:40 +0100)]
email output: Add MIME headers
We universally use UTF-8 in git commit messages and other kinds of
messages. The RFC-*822 default is us-ascii. Fix this by providing a
set of MIME headers.
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 5 Apr 2018 14:38:39 +0000 (15:38 +0100)]
cr-ensure-disk-space: Actually quit before taking lock if all is well
5d2466dc0f26 "cr-ensure-disk-space: Correct stdout output" was
supposed to change an `exit 0' into a `quit_ok' but erroneously
changed it into `check_space'. Fix this.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
These are the same as the corresponding ones without -shadow, except
that they set xen_boot_append to `hap=false', so that that will be
passed to the hypervisor to force shadow paging.
CC: Jan Beulich <jbeulich@suse.com> CC: George Dunlap <george.dunlap@citrix.com> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Partially-Reviewed-by: Wei Liu <wei.liu2@citrix.com> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Mon, 5 Mar 2018 15:48:57 +0000 (15:48 +0000)]
rumpkernel: clone from github rather than the official URL
Because, on wheezy:
$ GIT_SSL_NO_VERIFY=1 HOME=/ git clone http://repo.rumpkernel.org/rumprun
Cloning into 'rumprun'...
error: RPC failed; result=22, HTTP code = 501
fatal: The remote end hung up unexpectedly
$
strace shows a baroque sequence of redirects etc. involving servers
from rumpkernel.org, github.com, and some kind of gandi.net CDN.
On jessie it seems to just work. This commit should therefore be
reverted at some point, when all the instances are running jessie or
later. (Right now the VMs in the Mass colo are mostly wheezy.)
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Wed, 31 Jan 2018 11:18:55 +0000 (11:18 +0000)]
sg-report-host-history: Multiply size of reported history by 10
Right now,
http://logs.test-lab.xenproject.org/osstest/results/host/laxton1.html
contains ~200 jobs as expected, but that covers only 4 days. We
obviously would like more like a month.
The effect ought to be some more db work, but not worse concurrency.
CC: Julien Grall <julien.grall@linaro.org> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Mon, 29 Jan 2018 17:27:18 +0000 (17:27 +0000)]
daily-cron-email-*: Replace lists.xensource.com with lists.xenproject.org
The .xensource.com domain is quite deprecated now.
Reported-by: Doug Goldstein <cardoe@cardoe.com> Reported-by: Andrew Cooper <andrew.cooper3@citrix.com> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 14 Dec 2017 16:28:21 +0000 (16:28 +0000)]
smoke tests: Fix job reuse globs in cr-daily-branch
A symptom that the old patterns lack !build-arm64-xsm, so the xsm job
might be reused. The overall cause is that it contained a (partial)
list of architectures.
Instead, we observe that:
* The things we want to avoid reusing are Xen and libvirt (which
builds against Xen.
* Non-Xen builds are invariably build-<arch>-<thing> (or even longer)
so we can match them all with build-*-*.
* We need to exclude the Xen builds and the libvirt builds. The
libvirt builds are obvious. The Xen builds are either build-<arch>
which does not match build-*-*, or build-<arch>-xsm.
CC: Stefano Stabellini <sstabellini@kernel.org> Acked-by: Julien Grall <julien.grall@linaro.org> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 14 Dec 2017 16:03:44 +0000 (16:03 +0000)]
smoke tests: Build arm64 with XSM, instead of without
The smoke flight contains test-arm64-arm64-xl-xsm so it should contain
build-arm64-xsm (and not contain build-arm64).
I have checked the results with
OSSTEST_CONFIG=standalone-config-example eatmydata ./standalone-generate-dump-flight-runvars
and looking at the diff shows precisely the expected change.
CC: Stefano Stabellini <sstabellini@kernel.org> Acked-by: Julien Grall <julien.grall@linaro.org> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
This variable can be set to the absolute pathname of a kernel .deb to
use. It will be used only for hosts for which the corresponding
hostflag "need-kernel-deb-<suite>-special" is set.
There is not currently any facility for more than one special kernel
for each architecture.
As with backports kernels, the normal (Debian-supplied) kernel is
available too, and is used for hosts without any special hostflag.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
* In the first half of the backports kernel processing, set
the new variable specialkernel to the string "backports".
(This token occurs in the output .deb and kernel name, and
is also referenced by hostflags of the form
need-kernel-deb-<suite>-backports.)
* Break out the second half of the backports procesing into its
own section, conditional on $specialkernel.
* Replace occurrences of the "backports" token, after that point,
with $specialkernel.
Overall, this will allow other kinds of special kernel to be
introduced. But there is, for now, no functional change.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 14 Dec 2017 13:09:52 +0000 (13:09 +0000)]
di_special_kernel: Use di_installer_path
This avoids the caller having to supply $d_i. This is good because
there is a site we want to call this from which uses that name for a
different value!
No functional change.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Thu, 14 Dec 2017 12:23:24 +0000 (12:23 +0000)]
di_special_kernel: Replace open-coding in ts-host-install
The effect is simply to reuse the loop in di_special_kernel. The
extra tests etc. to compute $k and $c in di_special_kernel are of no
import here, and are harmless. We have already called
di_special_kernel so if it was going to fail due to this extra
computation, it would do so earlier.
No overall functional change.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Thu, 30 Nov 2017 17:40:13 +0000 (17:40 +0000)]
mg-hosts: mknetbootdir: Correctly use Path again
In fcd93edd00a8c9b08da3994d8bee9fa3809c3d93
"netboot: Rework interface to host_netboot_file (was host_pxefile)"
we unintentionally dropped $ho->{Tftp}{Path} from the
effective path. Reinstate it.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Tue, 5 Dec 2017 11:47:53 +0000 (11:47 +0000)]
production-config: Increase LogsMinSpaceMby etc.
osstest does not currently try to estimate the prospective disk use in
logs/ etc. of a flight before it starts; nor does it aggregate the
disk requirements of multiple concurrent flights.
Instead, it has a hardcoded minimum, which is enforced (by expiry) at
the start of each flight. It must be large enough for all in-progress
flights. This turns out not to be true any more, because of expansion
of both the number of different osstest branches, and because of
expansion of the number of jobs per flight, and because of expansion
of the activities within each job.
So, well, let's double it.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Fri, 1 Dec 2017 11:11:38 +0000 (11:11 +0000)]
cs-adjust-flight: Correct pseudo-BNF synopsys for runvar-build-set
In 497b2c6c933d13a05b01c6a654ce470be16dd78a
cs-adjust-flight: Rework runvar-build-set new value handling
the interpretation of this parameter was changed completely, but the
synopsis was not updated and thus became wrong.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Tue, 21 Nov 2017 15:45:23 +0000 (15:45 +0000)]
ms-planner: Including Unshare event for previous sharing state
Arrange for the initial plan to contain an Unshare event for resources
which are unallocated, but currently recorded as shared (ie, they
might be ready for reuse).
Specifically, we need to:
* Include the Wear in the information for a host with no
unallocatable shares (since the consumers might want to reuse the
current state).
* When processing the allocations in cmd_reset, look for sharing
entries (ie, things we found allocated by magic/shared, and made a
note in %currentshare) that didn't have any corresponding
allocations of shares. Generate an Unshare event for them.
* When we find an Unshare event in the plan, we used to remove it.
Now we keep it if it's the first thing in the plan. (For
subsequent entries callers don't generally need it because they can
find the sharing info from previous events.) When it is the first
thing in the plan, we do some checks on it.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Wed, 15 Nov 2017 11:00:07 +0000 (11:00 +0000)]
ts-xen-build-prep: Install libelf-dev for benefit of linux.git
Linux upstream has started needing libelf-dev. Without it, recent tip
fails (in our configuration) like this:
Makefile:938: *** "Cannot generate ORC metadata for CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel". Stop.
This is because kernel now is using objtool to create unwind
information. This needs libelf to work. Advantage is that this
approach no longer depends on assembler sources being heavily
annotated with unwind hints.
CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> CC: Stefano Stabellini <sstabellini@kernel.org> CC: Boris Ostrovsky <boris.ostrovsky@oracle.com> CC: Juergen Gross <jgross@suse.com> CC: Paul Durrant <paul.durrant@citrix.com> CC: Wei Liu <wei.liu2@citrix.com> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Juergen Gross <jgross@suse.com>
Roger Pau Monne [Wed, 25 Oct 2017 10:30:24 +0000 (11:30 +0100)]
osstest: Add freebsd runvars to examine job in generic flights
Previous patches only added the FreeBSD runvars to the jobs in the
examine flight, but failed to also add them to the examine job in the
generic flight.
This is necessary because the examine jobs now contain a step which
uses freebsd - the memdisk append determination.
On installations without an anointed freebsd job, and with no
freebsd-related config settings, that step will simply and harmlessly
fail.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Hook the memdisk parameter detection and the saving of the host
properties into the examine jobs.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v2:
- Do not pass a host ident to ts-examine-hostprops-save.
- Use .- for ts-memdisk-try-append so that the rest of the job will
run even if this step fails.
Changes since v1:
- Run the memdisk test first (so that we don't leave the host in a
weird state).
- Pass a host to the examine-hostprops-save.
ts-examine-hostprops-save: introduce a script to save properties
This script turns the properties stored in the runvars using the
format hostprop/$ident/$prop=$val into host properties stored in the
database.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v12:
- Introduce a dry_run variable and print a message only once if not
saving the properties in the database.
Changes since v2:
- Call selecthost based on the idents passed in the putative
hostprops runvar.
- Fix commit message.
- Use '/' instead of '_' in the runvars.
- Do a dry run if flight blessing != real.
- Fix parentheses indentation.
Changes since v1:
- Select a host for setting the properties.
- Print a message before exiting if blessing != real.
- Skip properties that don't contain the selected host.
ts-memdisk-try-append: introduce a script to test memdisk options
The intended usage is to run this script against every host in order
to record the possible needed memdisk flags.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v12:
- Add hyphens to the boot test options.
Changes since v1:
- Get the arch of the job and exit with 0 if it's not supported.
- Pass the --recordappend argument to ts-memdisk-try-append.
ts-freebsd-host-install: add arguments to test memdisk append options
This is needed in order to figure out which memdisk options should be
used to boot the images on each specific box.
Note that when passed the --record-append argument upon success the
script stores the tentative host property in the runvars.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v12:
- Add hyphens between words in command line options.
- Die if --record-append is specified without --test-boot.
Changes since v2:
- Fix commit message.
Changes since v1:
- Provide a --recordappend argument to force the recording the
memdisk parameters.
- Exit gracefully if a bootonly test is attempted against a
non-supported architecture.
- Use NONE instead of an empty string when calling
setup_netboot_memdisk if nothing should be appended.
- Do not perform any arch test in ts-freebsd-host-install.
This is used to store tentative host properties in the runvars of a
job, with the expectation that at some point (ie: at the end of the
job) they will be turned into real properties stored in the database.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v2:
- Use the following runvar format to store the putative host props:
hostprop/$ident/$prop=$val.
Roger Pau Monne [Mon, 23 Oct 2017 11:41:16 +0000 (12:41 +0100)]
netboot_memdisk: allow each host to have different append values
Some hosts require "append raw" [0] when booting with memdisk, while
others don't. This is based on the hardware/BIOS, and needs to be set
on a per-host basis.
In order to do this, add a new "MemdiskAppend" host property and make
use of it in the setup_netboot_memdisk helper.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v1:
- Explicitly use NONE for no options set (instead of an empty string,
which is the default).
- Allow to manually pass append parameters.
Roger Pau Monne [Mon, 23 Oct 2017 09:57:00 +0000 (10:57 +0100)]
HostDB: introduce set_property
And provide a helper in TestSupport to use it. This allows osstest to
set host properties from test script themselves (instead of using
the mg-hosts clu).
Note that the setting of host properties is limited to flights with
intended blessing real, and it will fail for any other blessing.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v12:
- Use intended blessing in the error message.
Changes since v2:
- Die if attempting to modify a host prop with intended blessing !=
real.
projects / people / liuw / osstest.git / log