Livepatch compiles and works on x86/ARM{32|64} so we can enable it
in the Xen config when requested by the enable_livepatch runvar.
When we are trying to build with enable_livepatch, run `make
dist-tests' as well, to ship the test cases. This depends on a
corresponding change to xen.git.
Finally, split the livepatch tests into their own stashed deliverable
from this job.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Reviewed-by: Konrad Rzeszutek Wilk <konrad@kernel.org>
mfi-common: Add an enable_livepatch runvar to the Xen build jobs
Set it to true on branches that support livepatching (Xen versions 4.9
and higher). Currently nothing reads this variable, so no overall
functional change.
Changes to the flights are as follows. On these branches:
osstest
xen-4.8-testing
xen-4.9-testing
xen-unstable
xen-unstable-smoke
in these jobs:
build-amd64 build-amd64-xsm
build-arm64 build-arm64-xsm
build-armhf build-armhf-xsm
build-i386 build-i386-xsm
add the runvar setting `enable_livepatch=true'.
Signed-off-by: Konrad Rzeszutek Wilk <konrad@kernel.org> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com> Reviewed-by: Konrad Rzeszutek Wilk <konrad@kernel.org>
TestSupport: target_cmd_root_status: New sub which returns return code.
All the different target_cmd_* end up calling tcmdex
which has the unfortunate side-effect of calling 'die' if
the SSH sessions results in any return code not zero.
That is fine, except for tests where we want to get a non-zero
return value.
This patch adds the $badstatusok to tcmdex - and makes all
the existing callers pass in the value of zero to it. This
way the commands behave the normal old way.
to all the other functions which use tcmdex.
The only exposed function that does it differently
is target_cmd_root_status.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Fri, 26 May 2017 10:41:52 +0000 (11:41 +0100)]
arm64, armhf: Use dtuart for console by default
We don't want to have to set a host property for each ARM host.
Julien writes:
This command line should always work for arm64. If the device-tree
does not provide the property `stdout-path' then it is a bug. We
want the command line to be as agnostic as possible from the
platform.
Reported-by: Julien Grall <julien.grall@arm.com> Acked-by: Julien Grall <julien.grall@arm.com> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 25 May 2017 11:47:55 +0000 (12:47 +0100)]
ts-hosts-allocate-Executive: Make build failures not be host-sticky
Builds (jobs whose recipe contains `build') are not supposed to
contain host-specific tests. We already don't care about varying
which host they run on, to hunt for host-specific bugs.
We should also not make build failures sticky to the host they last
failed on.
Fix this by defaulting $prevfail_bonus to 0 in that case, before the
week-long later default is applied. This means that a build job
runvar would override the built-in default whether that's 7 days or 0.
Reported-by: Roger Pau Monné <roger.pau@citrix.com> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Tue, 23 May 2017 18:06:42 +0000 (19:06 +0100)]
ts-examine-logs-save: Do not do readdir $! check
In perl 5.14 in Massachusetts, this produces this message
/home/logs/logs/109694/test-amd64-amd64-examine Bad file descriptor at ./ts-examine-logs-save line 100.
Sadly this means there is no way to tell failure apart from end of
directory. This patch should be reverted when we have a fixed version
of perl.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
(Verified with
OSSTEST_CONFIG=standalone-config-example eatmydata ./standalone-generate-dump-flight-runvars
Of course the libvirt branches lack the xl tests already.)
CC: Stefano Stabellini <sstabellini@kernel.org> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Julien Grall <julien.grall@arm.com>
Ian Jackson [Wed, 17 May 2017 20:58:22 +0000 (21:58 +0100)]
host examination: bootloader: better handling of multiple runs
If ts-examine-serial-pre was run more than once (which happens by hand
during development, for example), it would reuse the same ids (which
is not idea for manual testing, since we normally want to know if the
last run was good). Instead, generate new cookies each time (and
overwrite the old ones in the runvars).
Also, it owuld keep adding new copies to the grub menu entries
(obscuring the actual interesting menu text completely). Instead,
prefix the mentries with a fixed string so they can be stripped out
again. They now look a bit sillier after one run, but less silly
after several.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Wed, 17 May 2017 22:22:16 +0000 (23:22 +0100)]
host examination: bootloader: edit all grub2 menu entries
edit all the menuentry and submenu entries.
This is a good idea because in princple the first menu and submenu
entries in the autogenerated grub.cfg might be in a function which is
not actually called.
The resulting grub menu now looks even sillier.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Wed, 17 May 2017 20:45:37 +0000 (21:45 +0100)]
host examination: bootloader: Do not use ed
These ed runes are what I used to develop this approach, but they are
rather opaque to those very few programmers who are not proficient
users of ed, the standard text editor.
Also, using target_editfile_root means we keep a record of the
bootloader config in the logs, which is desirable.
The regexps used to edit the grub2 confiug are slightly more
sophisticated than the ed ones were.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Wed, 17 May 2017 15:30:03 +0000 (16:30 +0100)]
host examination: serial, grub2: put cookie in menu lines
Empirically, the message printed by `echo' is not always shown. It
seems to depend on the host BIOS. I suspect that there's a
VGA-to-serial conversion step which only triggers when grub asks for
keyboard input, or something. In any case, empirically, increasing
the sleep (and sleeping before) did not work on the godellos.
Instead, stuff the cookie into the start of grub menu entries.
This is gross but effective.
We put it in the first menu, and the first submenu, which means it
will appear on the screen somewhere.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Tue, 16 May 2017 14:36:21 +0000 (15:36 +0100)]
host examination: Run a regular exam flight
Examine every host, once a month.
This flight contains these jobs:
build-amd64 build-amd64-pvops
build-arm64 build-arm64-pvops
build-armhf build-armhf-pvops
build-i386 build-i386-pvops
examine-*
The build jobs are very similar to normal build jobs and I won't
recount their contents here. In any case, normally they will be
replaced with build jobs from a recent xen-unstable or osstest flight.
The examine-SOMEHOST job looks like this:
all_host_di_version current
all_host_suite jessie
arch {amd64,i386,arm64,armhf}
buildjob build-{amd64,i386,arm64,armhf}
enable_xsm false
host SOMEHOST
hostalloc_maxwait_max 20000
kernbuildjob build-{amd64,i386,arm64,armhf}-pvops
kernkind pvops
toolstack xl
xenbuildjob build-{amd64,i386,arm64,armhf}
The above is from standalone-generate-dump-flight-runvars so contains
dummy data. The suite, architectures, host and recipe will vary
depending on the output from cs-list-hosts.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Tue, 16 May 2017 14:35:28 +0000 (15:35 +0100)]
host examination: Add a check to many flights
Check that nothing in the principal other branches is likely to
regress the host examination machinery. This is particularly relevant
for osstest itself, of course.
Runvars look like this (e.g., from test-amd64-amd64-examine):
Ian Jackson [Tue, 16 May 2017 13:35:06 +0000 (14:35 +0100)]
make-hosts-flight: Impose a timeout
If we cannot acquire a particular host for more than 20ks (~5.5h),
give up on it. This allows flights to complete if hosts are blocked
for a long time.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Fri, 12 May 2017 15:45:56 +0000 (16:45 +0100)]
host examination: Permanently save the logs
After this, we retain some of the logs from the last examination of
each host, in a directory in results/ named after the host.
The logic for computing the destination directory has similar env var
logic to that used in the invocation of sg-report-host-history but if
those are not set (as they usually won't be) has config-based
fallback.
In Standalone mode enough of these variables must be set to avoid
calling blessing_suffix, since we don't want the filename to depend on
a not-really-very-useful (and not implemented by intended_blessing)
Standalone flight's intended blessing.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Fri, 12 May 2017 15:44:28 +0000 (16:44 +0100)]
TestSupport, Executive: Provide intended_blessing
This will be used by a ts script which permanently saves some logs in
a host-specific rather than flight-specific directory. Non-real
flights ought not to mess up real data.
The function is not provided for Standalone mode, so a test step that
tries to use it will fail.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Wed, 10 May 2017 14:25:09 +0000 (15:25 +0100)]
host examination: Initial skeleton
Introduce an initial cut of job recipies host-examine-linux and
host-examine-xen, and a flight generation script which can make a
flight to examine all hosts with a particular blessing.
So far all this does is provide a way to make a flight which:
* Allocates each relevant host
* Installs Debian and, if applicable, Xen
* Reboots (if applicable, into Xen)
* Collects the logs to the flight log directory
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Tue, 16 May 2017 17:40:39 +0000 (18:40 +0100)]
cs-hosts-list, standalone-generate-dump-flight-runvars: make compatible
Honour OSSTEST_HOSTSLIST_DUMMY, to cause cs-hosts-list to produce
dummy output. This means that standalone-generate-dump-flight-runvars
can now try generating flights whose contents depend on the host
database, and which are therefore normally only useable in Executive
mode.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Wed, 17 May 2017 20:58:09 +0000 (21:58 +0100)]
tcmd: Work around ssh bug with `ssh host ""'
This runs an interactive shell session on the host, rathern than
running `sh -c ""' on it.
Evidently ssh checks for the presence of a command line specification
after (foolishly, but now historically unavoidably) concatenating all
the command line arguments with spaces in between.
Turn ssh host "" into ssh host " " which is the expected no-op.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Fri, 12 May 2017 11:05:51 +0000 (12:05 +0100)]
ts-hosts-allocate-Executive: tolerate lack of hostflags
Jobs generated by make-hosts-flight do not need to (and therefore do
not) set any hostflags: they set a `host' runvar instead, for the
specific host. Do not pointlessly bomb out in this situation.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Fri, 12 May 2017 13:46:35 +0000 (14:46 +0100)]
sg-report-flight: Do not report <none executed> unless it's true
This fallback $storefail->() call is normally run because the job
failed without running any steps. But it can also happen because the
job execution itself failed (eg, sg-run-job crashed).
If there were any steps, replace the <none executed> text with
<job status>. This is accurate, if rather uninformative.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Tue, 16 May 2017 13:42:58 +0000 (14:42 +0100)]
README: Advise using `play' for playground flights
Any flight eventually blessed `adhoc' is supposed to contain, in the
db, accurate information corresponding to a real clean run. This is
not appropriate for playing about.
Using `play' usefully disables a number of safety catches, including
one which prevents post-startup flight modification.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 4 May 2017 11:57:32 +0000 (12:57 +0100)]
make-flight: Drop Windows XP tests
XP is really quite obsolete now.
From diffing standalone-generate-dump-flight-runvars output,
the tests that are dropped are:
test-amd64-amd64-xl-qemut-winxpsp3
test-amd64-amd64-xl-qemuu-winxpsp3
test-amd64-i386-xl-qemut-winxpsp3
test-amd64-i386-xl-qemut-winxpsp3-vcpus1
test-amd64-i386-xl-qemuu-winxpsp3
test-amd64-i386-xl-qemuu-winxpsp3-vcpus1
This means we are introducing 8 tests and dropping 7.
We drop the tests only on the branches:
linux-3.0
linux-3.10
linux-3.14
linux-3.16
linux-3.18
linux-3.4
linux-4.1
linux-4.9
linux-linus
linux-next
osstest
qemu-mainline
qemu-upstream-4.7-testing
qemu-upstream-4.8-testing
qemu-upstream-unstable
seabios
xen-4.6-testing
xen-4.7-testing
xen-4.8-testing
xen-unstable
The other branches are mostly out-of-support Xen branches. These are
either old ones we are still doing security support for (and would
like to know about regressions on, even for old guests), or very old
ones which we don't expect to change ever.
CC: Paul Durrant <Paul.Durrant@citrix.com> CC: Jan Beulich <JBeulich@suse.com> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
v2: Drop the tests on 4.6 as well.
Ian Jackson [Thu, 4 May 2017 11:33:10 +0000 (12:33 +0100)]
make-flight: Add tests for some more recent Windows versions
We call this function `do_hvm_win_2017_tests' because 2017 is the year
in which we are adding these tests and there isn't otherwise a good
common name. This is in the expectation we might want to retire them
at a similar point.
Do these new tests for all branches. If they fail on some old
branches, they fail; but even very old branches should not regress
even for these newer guests.
I have run standalone-generate-dump-flight-runvars and eyeballed the
diff and the diff appears appropriate. The new jobs are
test-amd64-amd64-xl-qemut-win10-i386
test-amd64-amd64-xl-qemut-ws16-amd64
test-amd64-amd64-xl-qemuu-win10-i386
test-amd64-amd64-xl-qemuu-ws16-amd64
test-amd64-i386-xl-qemut-win10-i386
test-amd64-i386-xl-qemut-ws16-amd64
test-amd64-i386-xl-qemuu-win10-i386
test-amd64-i386-xl-qemuu-ws16-amd64
The images are from Citrix XenRT's autoinstall image directory,
retrieved today (2017-05-04). As with the existing images, we are
using them by virtue of the MSDN subscriptions of all the developers
who will work with them. (Pending possible future changes to the
licensing arrangements for our Windows tests.)
CC: Paul Durrant <Paul.Durrant@citrix.com> CC: Jan Beulich <JBeulich@suse.com> CC: Lars Kurth <lars.kurth@citrix.com> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 4 May 2017 11:17:22 +0000 (12:17 +0100)]
make-flight: Factor out do_hvm_win_test_one
No functional change. (Verified with
standalone-generate-dump-flight-runvars.)
Don't bother messing with do_hvm_winxp_tests as 1. that uses testids
without the guest arch, so isn't compatible with this unless we make
it more general 2. we intend to abolish that for most branches
shortly.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Tue, 25 Apr 2017 13:15:40 +0000 (13:15 +0000)]
proxy config: Actually set https_proxy too
05406e5aaffb "proxy config: Set https_proxy too" was ineffective
because in d22b80bb "proxy config: Factor out http_proxy_envsettings"
the variable name $var was not substituted into the new
supposedly-more-general shell rune, which consequently lacked the
appropriate generality.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Tue, 25 Apr 2017 13:17:45 +0000 (13:17 +0000)]
mg-repro-setup: Cope with flights referencing other builds
We attempted to do this by trying to adjust build runvars only if they
didn't have a `.'. But cs-adjust-flight runvar-build-set has special
handling for <old-value>: it is always qualified with the flight.
So change the match to look for the current flight number.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 20 Apr 2017 13:04:36 +0000 (13:04 +0000)]
sg-report-flight: Do not report many non-regressions as allowable
In bd648aea2ebe
sg-report-flight: report allowable regressions separately in summary
whether the regression was allowable was put in $allow, but this
was erroneously not used.
In 4a210cda9cc8
sg-report-flight: fix allowable failure logic not to reuse $allow for two purposes (!)
that use of the variable, which was never used, was removed.
However, what ought to happen, is that $failv->{Allow} would be set to
1 if the entry matched an allow pattern, and only if it was a blocker.
That makes the logic in "report ... separately" correct.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Tue, 18 Apr 2017 13:09:38 +0000 (13:09 +0000)]
sg-run-job: Honour skip_testids runvar
This works like truncate_testids, except that the steps which match
are never run (truncate_testids skips steps *after* matching steps,
and also doesn't skip some things like log capture).
If the programmed testid for a step ends in `(*)' to request
substitutions of the stepno, the skip matching takes place against the
unsubstituted value (since it happens before a stepno is allocated).
There is no way to skip only some of a set of steps whose testids are
distinguished only by stepno.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Tue, 18 Apr 2017 12:39:29 +0000 (12:39 +0000)]
truncation: Support globs, and multiple patterns
Rename the variable to truncate_testids. It contains glob patterns as
for Tcl `string match', space-separated.
Adjust the two places which set it: cs-bisection-step (which needs to
quote any special characters) and mg-repro-setup (which does not
really process the value, but ought to be able to cope with
space-separated lists.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Mon, 10 Apr 2017 15:33:42 +0000 (15:33 +0000)]
mg-repro-setup: New script for setting up repros automatically
Given a previously failed job, this will:
* Create a flight for the repro attempt
* (Optionally) allocate a host to the user's personal task
* (Optionally) wipe the host
* Install the version of Xen and Linux used by that flight
* Run the job until the specified step finishes
* Email the user
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Mon, 10 Apr 2017 15:31:26 +0000 (15:31 +0000)]
mg-allocate: Provide --stdout-output
This prints RESTYPE/RESNAME/SHAREIX to stdout, after allocation is
successful (outside the db retry loop). This means that a
programmatic caller can now tell what was allocated, even if the spec
was complex.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Thu, 6 Apr 2017 17:06:10 +0000 (17:06 +0000)]
proxy config: Add ability to install MITM TLS cert
We want things like build jobs to be able to download things via
https. But we want them to be cached. To this end, we are having our
squid treat CONNECT as a request to MITM the TLS connection.
But this means that clients will see squid's cert, not the real one.
So placate them by installing the cert on each test box.
(The squid becomes part of the TCB for our coverity upload password,
but that is fine.)
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Mon, 20 Mar 2017 15:17:31 +0000 (15:17 +0000)]
cs-adjust-flight: Work properly on db transaction retry
The variable @changes is eaten during processing. So on db retry,
there would be changes missing. (When creating a new flight, this
would often mean a flight with no jobs, which cannot be executed.)
CC: Roger Pau Monné <roger.pau@citrix.com> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Wed, 22 Feb 2017 11:45:58 +0000 (11:45 +0000)]
html output: ReportHtmlUnpubBaseUrl: New config variable
This allows us to generate private urls for unpublished flights. This
will be useful for links from the ongoing flights summary - since,
naturally, none of those logs are published yet.
The URLs will be of more limited use; the exact usefulness will depend
on the deployment.
If no separate Unpub url is specified, simply use the Pub one.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 26 Jan 2017 16:26:22 +0000 (16:26 +0000)]
bisection: Really avoid flail
The flail detector needs to search for flights which will look like
the one it is creating, when this one is done. The intended blessing
always lacks `-bisect' so is no use.
We can use the last blessing for this: both cr-try-bisect and
cr-try-bisect-adhoc pass the same value for the last blessing and for
the 2nd parameter to perhaps_bisect_step_core (which is passed to
sg-execute flight to properly bless the flight when it's done).
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Wed, 18 Jan 2017 16:13:21 +0000 (16:13 +0000)]
d-i: backports: When using backports kernel for a host, install linux-base
Newer kernels may need a newer linux-base. This call has the happy
side effect of adding backports to its sources.list, which is also
generally needed.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
projects / people / aperard / osstest.git / log