Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Reviewed-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v1:
- Request hosts with iommu flag.
Roger Pau Monne [Wed, 11 Mar 2020 16:51:14 +0000 (17:51 +0100)]
ts-examine-hostprops-save: record hostflags also
Commit putative hotflags into the database if present on the runvars.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v4:
- New in this version.
---
Requested by Ian on IRC:
17:08:58 Diziet royger: I think your ts-examine-hostprops-save hunk in 2/ belongs in 1/ ? (Or in
a separate 1.5/ along with hostflag_putative_record.)
Roger Pau Monne [Wed, 11 Mar 2020 13:59:49 +0000 (14:59 +0100)]
host: introduce a helper to modify hostflags
Add a generic function to perform database changes related to a host
flag and add a wrapper to TestSupport.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v4:
- Move addition of hostflag_putative_record to a different patch.
- Introduce a single helper in TestSupport: modify_host_flag.
Changes since v3:
- Introduce modify_flag instead of {set/remove}_flag.
- Introduce a generic modify_host helper.
- Split from patch 1.
---
Requested on IRC:
17:08:58 Diziet royger: I think your ts-examine-hostprops-save hunk in 2/ belongs in 1/ ? (Or in
a separate 1.5/ along with hostflag_putative_record.)
Roger Pau Monne [Wed, 11 Mar 2020 16:33:32 +0000 (17:33 +0100)]
host: introduce modify_host
Abstract the set_property checks and DB call into a helper.
No functional change.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Requested on IRC:
17:09:30 Diziet Also if it were me I would put the modify_host refactoring in its own nfc patch,
but I won't insist on that...
---
changes since v4:
- New in this version.
Ian Jackson [Thu, 2 Jan 2020 17:59:24 +0000 (17:59 +0000)]
Switch to linux-4.19 by default (from 4.14)
This affects only x86 and only the branches that aren't linux-*, since
obviously the latter use whatever version they are using.
I compared the most recent linux-4.19 results with the most recent
linux-4.14 ones, and there was only one new failure (in 143841):
test-amd64-i386-qemuu-rhel6hvm-intel 12 guest-start/redhat.repeat fail REGR. vs. 143911
This step has failed twice in the web-visible history of this job on
4.19; and once recently in 4.14. Because of the low update rate of
these trees nowadays, these tests are a while ago and we don't have
the logs any more.
I think given that it's already not perfect this is not a blocker and
we should update osstest to 4.14.
CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> CC: Boris Ostrovsky <boris.ostrovsky@oracle.com> CC: Juergen Gross <jgross@suse.com> CC: Wei Liu <wei.liu2@citrix.com> CC: Paul Durrant <paul.durrant@citrix.com> Acked-by: Roger Pau Monné <royger@FreeBSD.org> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Thu, 19 Dec 2019 16:35:06 +0000 (16:35 +0000)]
Revert "Arrange to upgrade microcode on x86 test hosts."
We are going to do this differently, by installing the packages from
Debian. We couldn't do that in 2015 because not every version of Xen
supported scanning the dom0 initrd for microcode, but now all
supported versions do. So we will do that, in a moment.
Ian Jackson [Thu, 19 Dec 2019 15:05:45 +0000 (15:05 +0000)]
ts-xen-install: Drop gdb= parameter
This has been there forever and I doubt anyone has ever used it.
It needs an L or H suffix to instruct Xen how to mux the console with
the gdb remote protocol based on the high bit on 8-bit clean line. So
probably doesn't work (and maybe never has). Recent changes to more
conspicuously report command line parsing failures highlighted this
issue.
Suggested-by: Andrew Cooper <Andrew.Cooper3@citrix.com> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
These bugs are checked for by xtf/test-hvm64-lbr-tsx-vmentry. So
those tests fail in xen-4.8-testing but only on applicable hardware.
Because we don't pin these tests to individual hosts (because that
would involve running the XTF tests on each host pair) this can show
up as a "regression". Force pushing it makes it go away for a bit,
until for some reason the test runs on a different host.
Instead, treat these "regressions" as allowable but only in 4.8.
I have tested this with
./sg-report-flight --that-flight=144558 144726
and diff'd before and after. The difference is as expected, that
test-xtf-amd64-amd64-5 50 xtf/test-hvm64-lbr-tsx-vmentry fail REGR. vs. 144558
is now
Regressions which are regarded as allowable (not blocking):
Reviewed-by: Andrew Cooper <Andrew.Cooper3@citrix.com> Release-acked-by: Juergen Gross <jgross@suse.com> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Thu, 5 Dec 2019 17:14:40 +0000 (17:14 +0000)]
ts-xen-build-prep: Install python3-docutils
This is the package (or, one of the packages) containing rst2html.
This is now needed for builds of libvirt upstream.
Really this packages install call should be ts-libvirt-build, but:
Historically we have done it all in ts-xen-build-prep. In the
meantime we have put a lock around the call to the package manager,
but this has only been lightly tested. At this stage of the Xen
release we would rather be cautious.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com> Release-acked-by: Juergen Gross <jgross@suse.com>
Now we have the cacheing we can put this back and have useful host
histories again.
Some performance figures (individual measurements):
limit=200 limit=2000
before this series 3m32 some very long times
with this series, --regenerate 3m06 13m56 29m05
with this series, reusing cache 2m22 1m49 3m10 3m36
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Fri, 8 Nov 2019 17:17:26 +0000 (17:17 +0000)]
sg-report-host-history: Move job runvars query later
This query is just used for the power methods. Put it near there.
Also, indent it in a `do' block. These changes will make the next
change easier to read.
No functional change.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Fri, 8 Nov 2019 17:22:08 +0000 (17:22 +0000)]
sg-report-host-history: Write cache entries for tail, too
mainquery fetches a number of rows supposed to be larger than needed
for the output limit $limit. And then for each host we sort them by
time of the last step - which means we must have the last step, which
is a separate query for each job. We want to cache this information
even for jobs we do not actually report in the html output.
(There is still nothing which actually reads this cache data.)
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Mon, 11 Nov 2019 15:49:37 +0000 (15:49 +0000)]
ts-libvirt-build: Do an out-of-tree build
Recent versions of libvirt do not support in-tree builds (!)
Cope with this by always building in a subdirectory `build' (a
subdirectory of the source tree); this is the arrangement which the
libvirt upstream messages and documentation now seem to recommend (at
least where things have been updated).
I compared the differences in build output between the results of this
branch and a previous passing xen-unstable flight. The libvirt
library version increased and a file
usr/local/share/libvirt/cpu_map/arm_features.xml
appeared. I think this is just due to changes in the libvirt version, 2cff65e4c60e..70218e10bcde, in particular 0de541bfc575
cpu_map: Ship arm_features.xml
I also tested that a test job, built with current libvirt and these
osstest changes, passes as expected.
CC: Jim Fehlig <jfehlig@suse.com> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com> Tested-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Mon, 11 Nov 2019 17:34:48 +0000 (17:34 +0000)]
ts-libvirt-build: Provide PKG_CONFIG_PATH
In osstest we do not install the xen tree in /usr/local because the
build environment is shared with many different build jobs which might
be using different versions of Xen. We put it in a job-specific
directory in ~osstest on the build host, and set environment variables
to ensure that it all gets picked up.
Recent versions of libvirt insist on finding xenlight.pc; otherwise
they disable libxl support. So we must add a PKG_CONFIG_PATH setting.
(In all cases, contrary to the usual protocol for path-like variables,
we do not append but instead simply set the variable. This is OK
because this is an osstest build script run via ssh to the build host,
so the variables won't have been set already.)
CC: Jim Fehlig <jfehlig@suse.com> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Contrary to the assertions in the commit message, this unmount etc. is
actually used by some tests. So removing it breaks things.
Now, we have a different workaround: a 10s sleep before we attempt the
umount. The combination of ea6626f7 guest_prepare_disk: Only do the umount if we set an env var 1d3a97b0 xl guest creation: Pause 10s to work around libxl/blkback races 3a208c18 all guest creation: Pause 10s to work around libxl/blkback races
and this revert is simply this:
@@ -1938,6 +1938,8 @@ sub guest_create_paused ($) {
sub guest_prepare_disk ($) {
my ($gho) = @_;
Ian Jackson [Mon, 11 Nov 2019 11:47:10 +0000 (11:47 +0000)]
all guest creation: Pause 10s to work around libxl/blkback races
In 1d3a97b06d2c
xl guest creation: Pause 10s to work around libxl/blkback races
we added a 10s delay to work around a race bug in Linux blkback.
This was intended to be used in combination with ea6626f7edd9
guest_prepare_disk: Only do the umount if we set an env var
after which it is only xl which is vulnerable to this race.
But that commit was wrong, so we must revert it. After we do
that the sleep in the xl driver will come too late.
So, move the 10s sleep from the osstest xl and libvirt drivers to the
general guest preparation step, right next to where the affected lv in
use check is.
This is still a bodge, unfortunately.
CC: Jürgen Groß <jgross@suse.com> CC: Wei Liu <wl@xen.org> CC: Anthony PERARD <anthony.perard@citrix.com> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Thu, 7 Nov 2019 17:46:48 +0000 (17:46 +0000)]
make-flight: Drop all win10 tests in all flights
These are failing and have been for some time and it does not appear
that anyone has the capability to fix them. Running them in these
circumstances seems wasteful.
Effect is to drop test-*-win10-* jobs (checked with
standalone-generate-dump-flight-runvars).
CC: Sander Eikelenboom <linux@eikelenboom.it> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com> Release-acked-by: Juergen Gross <jgross@suse.com>
Ian Jackson [Wed, 6 Nov 2019 16:49:36 +0000 (16:49 +0000)]
sg-report-host-history: Reduce limit from 2000 to 200
Currently the "sg-report-host-history" part of most flights is taking
an inordinate amount of time. Hours. These are serialised and this
is a big problem, seriously impeding throughput.
Reducing this limit by a factor of 10 will reduce the available
history when we are looking at host-specific problems. It is an
emergency fix.
I am working on an arrangement which will avoid having to rescan all
of history each time and which will instead reuse previous output.
CC: Jürgen Groß <jgross@suse.com> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Mon, 4 Nov 2019 16:50:38 +0000 (16:50 +0000)]
adhoc-revtuple-generator: Bisect over 5000 commits (really)
In e9b0653875b3 we changed one of the `1000' values to `5000'. But
this magic number had been duplicated. Urgh!
The result is that adhoc-revtuple-generator might generate a weirdly
truncated output which causes cs-bisection-stop to fail with messages
like this:
*** not RelvUp at 3d40147282670d597b336be5599b5cc4c2ff7ddd at ./cs-bisection-step line 554.
*** not RelvDown at 2fa3479cfadb0bb3fe694dbfd29f2350eb2570df at ./cs-bisection-step line 554.
*** not RelvUp at 2fa3479cfadb0bb3fe694dbfd29f2350eb2570df at ./cs-bisection-step line 554.
...
Use of uninitialized value in concatenation (.) or string at ./cs-bisection-step line 747.
Should test .
BROKEN see earlier errors. at ./cs-bisection-step line 1454, <SVGI> line 10089.
Fix this by (i) plumbing the magic value we already edited properly
back to the (command-line controlled) global variable (ii) changing
the global variable from 1000 to 5000.
git-grep '\b1000\b' still produces a fair amount of output but most
of it is timeouts, which is fair enough. There is also a flight
count limit in sg-report-flight, which limits how far back it is
willing to look. We don't want to change that here.
With this change, cs-bisection-step on the currently-failing freebsd
build job does this:
Searching for interesting versions
Result found: flight 141420 (pass), for basis pass
Result found: flight 143397 (fail), for basis failure
Need to reproduce basis pass (pass); had 1 already.
Should test 2fa3479cfadb0bb3fe694dbfd29f2350eb2570df.
This looks plausible: it is picking up where it left off before the
basis pass fell over its horizon.
CC: Roger Pau Monné <royger@FreeBSD.org> CC: Jürgen Groß <jgross@suse.com> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Thu, 31 Oct 2019 14:57:45 +0000 (14:57 +0000)]
grub2 setup: Set GRUB_TERMINAL=console, if no other setting
The default for d-i, if it doesn't know better, is to let update-grub
set grub's terminal to `gfxterm'. But in osstest we do not really
ever want to use a graphical console.
Let us discuss some of the cases in a bit more detail:
On UEFI systems with a serial console, the UEFI console ought (and in
our installation is in all cases so far) typically linked to the
serial console. So GRUB_TERMINAL=console would be right for UEFI.
This appears to be correct on the albanas, our one pair of in-service
x86 boxes with a UEFI firmware configuration.
But on x86 systems, we generally pass console=ttyS... arguments on the
d-i command line, and d-i arranges for GRUB_TERMINAL=serial and
appropriate other settings. We already have a workaround that changes
that to "serial console", which is fine whether "console" means a VGA
console we don't look at, or some kind of BIOS console redirection.
This currently works on all our x86 machines including o UEFI.
On our ThunderX (arm64) boxes, `gfxterm' does not work at all.
`console', does, because it goes to the UEFI console which UEFI sends
to the serial port.
The best approach to unpicking this mess seems to be to apply a
default setting of GRUB_TERMINAL=console. The effect of this is to
change `gfxterm' in grub.cfg. In practice all our x86
boxes (including our x86 UEFI boxes, where `console' would work) have
it set to `serial' (modified by us to `serial console') so remain
unchanged.
The net result is that on ARM, we now set `GRUB_TERMINAL=console', and
we now get all of the bootloader serial output on the rochesters.
I have tested this on:
rochester0 - arm64 uefi ThunderX, used not to work
laxton1 - arm64 uefi SoftIron
albana0 - x86 uefi
huxelrebe0 - x86 bios
arndale-westfield - armhf u-boot
cubietruck-gleizes - armhf u-boot
Thanks to Brian Woods for poking at rochester0 and making the key
suggestions.
CC: Brian Woods <brian.woods@xilinx.com> CC: Stefano Stabellini <sstabellini@kernel.org> CC: Julien Grall <julien.grall@arm.com> CC: Jürgen Groß <jgross@suse.com> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Wed, 30 Oct 2019 16:04:49 +0000 (16:04 +0000)]
xl guest creation: Pause 10s to work around libxl/blkback races
In ea6626f7edd9eb40a3510eaf6816a77cac4f63d0
guest_prepare_disk: Only do the umount if we set an env var
we removed (in the usual case) a check for the guest disk
already being mounted in dom0 etc. This check is there for
ad-hoc testing.
We removed it because it exposes what we think is an annoying race in
blkback.
Unfortunately this change seems to have made guest-rapid-restart races
worse rather than better. Steps test-* guest-start/debian.repeat seem
to fail a lot more now.
We are in the throes of preparing the Xen 4.13 release. These guest
restart races have existed for a long time.
Bodge this for now by adding a sleep :-/.
We do this in the xl toolstack, during domain creation. And also in
the libvirt toolstack because that uses xl but doesn't inherit the
sleep from the Osstest module.
Release-acked-by: Jürgen Groß <jgross@suse.com> CC: Wei Liu <wl@xen.org> CC: Anthony PERARD <anthony.perard@citrix.com> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
We aren't actually interested in bisecting the FreeBSD
version (usually, the anointed version) which was used as the platform
for the failed builds. We are thereby making the assumption that any
build failure (or indeed test failure) is the result in changes to the
recent FreeBSD being actually built or used, not the version being
used as a build host.
Achieve ignoring this by having other_revision_job_suffix return a new
magic new value DISCARD, which all callers must know means `skip
this one'. There are three call sites:
In cs-bisection-step:flight_rmap, we skip those rows in the Perl
loop. (We can't skip them conveniently in the SQL because we can't
refer to the column `othrev'; we'd have to duplicate the expression,
or have a subquery. This doesn't seem likely to matter much.)
In cs-bisection-step:preparejob, we always compare the returned suffix
with a fixed value (which eventually came from the previous call). So
DISCARD will never match. No change is needed here.
In Osstest.pm:main_revision_job_cond, we compare the returned suffix
with ''. Again, it will never match and no change is needed.
I have checked that now a cs-bisection-step run chooses a single
FreeBSD master commit to try to build.
CC: Roger Pau Monné <royger@FreeBSD.org> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Thu, 24 Oct 2019 10:46:13 +0000 (11:46 +0100)]
power_cycle_sleep: Change default sleep to 15s
5s is so short that when a host fails to respond we aren't sure if it
was just very idle and ran off its PSU's internal energy storage for
that period.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Wed, 23 Oct 2019 14:55:06 +0000 (15:55 +0100)]
make-flight: Drop arm64 with Linux before 4.10
The driver for the laxtons' network cards is not in 4.4 (and that's
quite old). Our ThunderX's may even require something more recent but
we will cross that bridge when we see it.
Effect is to drop the following jobs:
linux-4.1 *arm64*
linux-4.4 *arm64*
linux-4.9 *arm64*
(Checked by eyeballing standalone-generate-dump-flight-runvars diff.)
CC: Julien Grall <julien.grall@arm.com> CC: Stefano Stabellini <sstabellini@kernel.org> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com> Acked-by: Julien Grall <julien.grall@arm.com>
Now we have two lists of things not supported on ARM: one of branches
where that's inherent in the branch somehow, and one for those where
the kernel is simply too old. The latter are going to differ between
armhf and arm64.
No functional change.
(Verified with standalone-generate-dump-flight-runvars.)
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Mon, 21 Oct 2019 13:23:55 +0000 (14:23 +0100)]
guest_prepare_disk: Only do the umount if we set an env var
This call to guest_umount_lv is here for the benefit of ad-hoc reruns
of (eg) ts-guest-start tidy up any ad-hoc messing about (eg from
earlier runs of ts-debian-fixup or something). It is not needed in
production runs.
Serendipitously, this osstest code discovered a bug in the Linux
blkback: when tearing down, it sets the backend state to 6 before it
has closed the underlying block devices. This ultimately means that
after "xl destroy" or "xl shutdown -w" there is a period when the
guest's open handle onto its storage is still open. This is wrong.
This detection depends on us winning a tricky race. So it shows up in
osstest as a very low probability heisenbug. The bug is currently in
all versions of Linux and causing a bit of a nuisance.
It would be best to add a proper check for this bug. However, this is
quite fiddly: really, it ought to be done as close to the xl command
completion as possible, in the same ssh invocation. That would
involve a fair bit of plumbing and ad-hocery. I don't think that
would be proportionate for such a low-impact bug.
So instead in this patch I just disable this cleanup code in the
troublesome case, unless it is explicitly requested by the user
setting OSSTEST_GUEST_DISK_MOUNT_CLEANUP to a trueish value. (This
would be reasonably convenient for the ad-hoc testing that this call
serves.)
Thanks to Roger for diagnosing the Linux kernel bug.
CC: Jürgen Groß <jgross@suse.com> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com> Reviewed-by: Roger Pau Monne <roger.pau@citrix.com>
Ian Jackson [Thu, 3 Oct 2019 09:48:38 +0000 (10:48 +0100)]
Osstest.pm: Fix main_revision_job_cond after 0964bab7a9ea
In
other_revision_job_suffix: Take and pass referring runvar name
we updated main_revision_job_cond to pass a dummy 'x' for the new
parameter. But the parameter is a sql expression, not a value,
and so an extra pair of quotes are needed.
This error broke sg-check-tested and this fix fixes it.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Mon, 30 Sep 2019 14:16:04 +0000 (15:16 +0100)]
freebsd build job bisection: add special case
other_revision_job_suffix contains ad-hoc code which returns an
identifier distinguishing certain jobs which are expected to refer to
different revisions within their flight.
Add the special case for freebsdbuildjob's recursion.
After this change we are now willing to tolerate the fact that a
freebsd build job has as input multiple different revisions of
freebsd.
cs-bisection-step has code to avoid creating recursive build jobs: the
created top-level job will therefore reuse the same freebsdbuildjob as
the template. Hopefully that will be the previously anointed one and
still be available.
The bisector wants to repro on the same host as before. This means it
won't necessarily use the most recent pass as the basis build. So
long as the previous build has not been expired, this is fine. It
does involve building an earlier freebsd on a later one but this
should be OK.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Thu, 29 Aug 2019 09:12:44 +0000 (10:12 +0100)]
mg-hosts mknetbootdir: Introduce and require -F<firmware>
If one runs
./mg-hosts mknetbootdir HOST
before having sorted out all the host configuration, it uses the
default configuration value for the host's firmware kind, which is
"bios". If the configuration is then changed, things don't work.
This is confusing.
So ask the user to specify one or more -F<firmware>, or -Fany.
CC: Dominic Brekau <dominic.brekau@credativ.de> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Tue, 2 Jul 2019 16:26:02 +0000 (17:26 +0100)]
ts-hosts-allocate-Executive: Treat "no suitable host" as starved
In particular, this means that
* platform-* jobs will not cause problems in old Xen branches when
there a platform supports only newer Xen
* commissioning flights will complain less about the architectures
that aren't included in the particular set of hosts
The motivation for this patch, now, is that the first of these applies
to `platform-thunderx', recently introduced in the Xen Project colo.
CC: Julien Grall <julien.grall@arm.com> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Tue, 2 Jul 2019 16:16:47 +0000 (17:16 +0100)]
cr-daily-branch, mfi-common: Use tested seabios and ovmf for -prev
Introduce {TREE,REVISION}_{OVMF,SEABIOS}_PREV, so that -prev builds
use the tested ovmf too. This should be true in all branches,
including xen-unstable. (In the seabios and ovmf branches, there
are no -prev builds.)
Checked with standalone-generate-dump-flight-runvars
and the result is to these runvars
revision_ovmf
revision_ovmf
revision_seabios
revision_seabios
to jobs
build-i386-prev
build-amd64-prev
in the branches
xen-*-testing values are baseline
xen-unstable values are the empty string
The empty string is equivalent to unset: see config in ts-xen-build.
CC: Wei Liu <wl@xen.org> CC: Anthony PERARD <anthony.perard@citrix.com> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
The glob syntax here was wrong, and the code cs-adjust-flight did not
handle it properly either. So --rebuild -r has not worked since it
first appeared in: a1e0e5846f7bb7d82a5db1d7cd643b9f5ca1b9a9
mg-repro-flight: Provide --rebuild to make variant build jobs
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v2: New patch
Ian Jackson [Fri, 17 May 2019 17:24:12 +0000 (17:24 +0000)]
mg-repro-setup: Introduce `statictask' variable
We are going to make a mode where we don't set OSSTEST_TASK. The
result is that our subprocesses will do whatever they usually do.
Those are mg-allocate (which would allocate for our static task) and
mg-execute-flight which will make a dynamic task. We must therefore
prevent mg-allocate from running since the allocations would not be
useable for the flight execution.
No functional change yet, since nothing sets statictask=false and
therefore OSSTEST_TASK would always be set.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Thu, 30 May 2019 16:47:42 +0000 (17:47 +0100)]
ts-kernel-build: Disable CONFIG_ARCH_QCOM in Xen Project CI
drivers/firmware/qcom_scm.c:469:47: error: passing argument 3 of `dma_alloc_coherent' from incompatible pointer type [-Werror=incompatible-pointer-types]
This is fixed by
firmware: qcom_scm: Use proper types for dma mappings
but this is not present in all relevant stable branches.
We currently have no Qualcomm hardware in the Xen Project test lab so
we do not need this enabled.
CC: Stefano Stabellini <sstabellini@kernel.org> CC: linux-arm-msm@vger.kernel.org CC: linux-kernel@vger.kernel.org CC: Stephen Boyd <swboyd@chromium.org> CC: Andy Gross <agross@kernel.org> CC: Bjorn Andersson <bjorn.andersson@linaro.org> CC: Avaneesh Kumar Dwivedi <akdwived@codeaurora.org> Acked-by: Julien Grall <julien.grall@arm.com> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Fri, 17 May 2019 13:39:47 +0000 (13:39 +0000)]
installs: Disable cron
The presence of cron causes leak check failures, since cron may run
processes that the leak checker detects. Disable it, since none of
our installs live long enough for this to matter.
Do this in host_install_postboot_complete since it seems to me like we
don't want this in guests any more than we want it in hosts.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Fri, 3 May 2019 13:38:53 +0000 (13:38 +0000)]
cr-daily-branch: seabios: "usually" use xen-tested-master
This branch is supposed to be suitable for all versions of Xen.
Conversely, older versions of seabios do not build on newer
compilers (as provided, eg, in stretch).
So, for "branches" other than xen-unstable and xen-unstable-smoke, use
the usual "determine_version" machinery, which will select
xen-tested-master for branches other than the ovmf branch itself.
No change for the seabios "branch", nor for xen-unstable*. The effect
is to switch xen-*-testing, qemu-*, linux-*, etc., to all use ovmf
xen-tested-master.
Ian Jackson [Thu, 2 May 2019 15:59:16 +0000 (16:59 +0100)]
mg-repro-flight: Provide --rebuild to make variant build jobs too
This allows a single command to repro a particular job with a variety
of different source code.
The implementation technique is:
- run the build job in a separate flight, so that it can run
with a separate task which gives its host up after the build
- do much of the heavy lifting of runvar fiddling etc. in
a new helper routine in cs-adjust-flight
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v2: Add a missing `continue' (without which everything goes quite wrong)
Ian Jackson [Wed, 1 May 2019 10:43:08 +0000 (11:43 +0100)]
Drop Xen 4.5 and earlier
These releases are out of security support. They are known not to
build on Debian stretch, which is what we are using, and we do not
intend to ever update them to fix that.
Xen 4.6 is also out of security support but we want osstest to be able
to continue to build it so that we can test 4.6->4.7 migration, for
the purposes of testing Xen 4.7, which is still supported right now.
So we have recently applied some build fixes to the 4.6 tree, and for
now we retain 4.6 in osstest so that build fixes applied to
staging-4.6 can propagate to stable-4.6.
CC: committers@xenproject.org Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Fri, 26 Apr 2019 14:58:47 +0000 (15:58 +0100)]
cross builds: Build armhf kernels on amd64 hosts
Our armhf hosts are devboards and very slow, as well as scarce. It
takes 17ks or so for a kernel build. This will go *much* faster on
an amd64 box and we have lots of those too.
standalone-generate-dump-flight-runvars shows that the only change is
to change host_arch from armhf to amd64 in build-armhf-pvops jobs.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com> CC: Stefano Stabellini <sstabellini@kernel.org> Acked-by: Julien Grall <julien.grall@arm.com>
---
v2: Fix typo in commit message.
Ian Jackson [Fri, 26 Apr 2019 14:22:55 +0000 (15:22 +0100)]
cross builds: mfi-common: Prepare for kernel cross building
Introduce job_create_build_crossable, which takes a target->host
architecture map in its arguments, and use it for build-kern,
passing an empty architecture map.
Overall functional change is only to add
host_arch=$arch
to the kernel build jobs, which has no ultimate effect because it's
the same as the arch=$arch. (Difference in flight construction
verified with standalone-generate-dump-flight-runvars.)
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Fri, 26 Apr 2019 13:44:04 +0000 (14:44 +0100)]
arch replumbing: ts-debian-di-install: Use $gho->{Arch}
This is just tidying up. The only effect is that now these would
honour $r{all_guest_arch} as a fallback. But right now,
$r{GUEST_arch} will always be set, and that is what ends up in
$gho->{Arch}.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
This comment was lamenting the very problem we are fixing now. It
would now be possible to test i386->amd64 tools migration, by writing
an appropriate test job with different src_host_arch and
dst_host_arch etc.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Fri, 26 Apr 2019 13:27:12 +0000 (14:27 +0100)]
arch replumbing: Replace many $r{arch} with $[g]ho->{Arch}
No functional change with existing flights. But the effect is that
now, generally, ts-* scripts and the support code will honour
host_arch, if it is set, in preference to arch.
This patch contains only replacements of $r{arch} with $ho->{Arch} or
$gho->{Arch}. In fact, perhaps surprisingly, there were no locations
where $gho was wanted rather than $ho (I have double checked this).
Exceptions, where we left $r{arch} alone, are:
* make-flight: a comment, which we are about to deal with;
* ts-kernel-build: we are going to support cross building and
$r{arch} is going to be the architecture of the kernel we want
rather than of the build host.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
projects / people / royger / osstest.git / log