osstest: save/retrieve the last successfully tested FreeBSD build
And use it in order to install the hosts for the next FreeBSD flight.
Note that the anoints are done after the push, but there's a recovery
mechanism in case osstest manages to do a successful push, but the
artifacts are not anointed for some reason. Osstest will continue to
create flights until the revision of the anointed artifacts matches
the one pushed to the repository (even if that means running flights
with OLD_REVISION == NEW_REVISION).
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v7:
- Add space between $freebsd_branch and * in list-prepared.
Changes since v6:
- Use just the branch name for anoint (instead of freebsd-$branch).
- Force the creation of flights if the revision of the current
anoints don't match OLD_VERSION.
- Iterate over the list of current anoints instead of open-coding the
arches.
- Split the retrieve/anoint code blocks into it's own sections.
Add the missing pieces so that cr-daily-branch knows how to create a
FreeBSD flight. This has been tested by calling `./cr-daily-branch
freebsd-master`.
Note that there are two missing pieces. The first one is correctly
setting the FREEBSD_<arch>_BUILDJOB environment variable when calling
make-freebsd-flight, without it the installer media will always be
picked from the configuration file. This requires keeping track of
previous FreeBSD successful build jobs, marking them as permanent and
fetching them from the database in order to pass them on the env
variable.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v8.1: fix ap-fetch-version-old for the freebsd-* branch
(The remote branch has been changed to tested/master or
tested/stable/<version> instead of plain master or stable/11.)
Changes since v6:
- Set the path to the FreeBSD git trees in this patch.
- Push to tested/master or tested/stable/<version> instead of using
the same branches as the origin tree.
Changes since v5:
- Add support for testing multiple FreeBSD branches, default to
branch master.
Changes since v3:
- Fix the daily-cron-email-* settings for FreeBSD branch flights.
osstest: introduce a script to create a FreeBSD flight
The logic to create a FreeBSD build job is added to
make-freebsd-flight. This includes creating a FreeBSD build job, and
also testing the output of that build job (by creating another build
job that depends on the output of the first).
Note that the FreeBSD build job needs some input in order to setup a
FreeBSD host, and that can be fetched from different places:
1. Env variable FREEBSD_<arch>_BUILDJOB: use the output from a
previous build-<arch>-freebsd job.
2. Env variables FREEBSD_DIST and FREEBSD_VERSION: set before calling
into make-flight, provide the path to the installer image and sets
folder and the version being installed.
3. Config file FreeBSDDist and FreeBSDVersion: same as 2. except that
they are set on the config file.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v6:
- Re-wrap lines.
- Remove the freebsd hostflag.
- Do not set the FreeBSD git trees in this patch.
- Set all_host_os runvar to freebsd for the FreeBSD build jobs.
Changes since v5:
- Append the "freebsd" hostflag, the specific version will be set
dynamically by the ts-freebsd-set-hostflags script.
- Use FREEBSD_<arch>_BUILDJOB to set the freebsdbuildjob runvar.
- Loop around arches, although only amd64 is supported ATM. This
should make it easier to add more arches later on.
- Expand README comment to describe the expected layout of the
FreeBSD install media.
Changes since v4:
- Convert get_freebsdjob_runvars into set_freebsd_runvars, mimic the
behavior of set_hostos_runvars.
Changes since v3:
- Split the sg-run-job code into pre-patches.
Changes since v1:
- Replace freebsd_buildjob with freebsdbuildjob.
- Replace FREEBSD_SETS/IMAGE with a single FREEBSD_DIST that points
to a folder that should contain both things.
- Document the FreeBSDDist and FreeBSDVersion config file options in
the README file.
osstest: add support for FreeBSD buildjobs to sg-run-job
Add support and introduce a FreeBSD build job to sg-run-job.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v5:
- Add a '+' to the arguments passed to ts-freebsd-set-hostflags, so
they are hidden from testid.
Changes since v4:
- Use a switch in allocate-build-host.
Changes since v3:
- New in this version (split from existing patch).
Make need_build_host store a string instead of a boolean. This is
later going to be expanded to handle the FreeBSD build jobs.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v5:
- Use a regsub to set need_build_host.
- NB: I would prefer to keep the tolower instead of changing the
procs to -FREEBSD/-LINUX since I don't really like the SHOUTING.
Changes since v4:
- Fix the usage of the catching-otherwise blocks.
Changes since v3:
- New in this version (split from patch).
osstest: introduce a script to set the runtime hostflags runvar for FreeBSD jobs
Due to the nature of the FreeBSD install media, which is
self-generated from the ts-freebsd-build script, the hostflags runvar
set to FreeBSD jobs are related to the current version under test.
The following hostflags might need to be fetched from the runvars of a
previous build-$arch-freebsd job:
- share-build-freebsd-$arch-$hash: the $hash used here is calculated
from the checksum of the installer image used by this specific job.
This allows osstest to share FreeBSD build hosts, and be sure the
exact desired FreeBSD version is used.
- PropMinVer:Freebsd:$version: $version contains the major FreeBSD
version under test. This version is obtained from the FreeBSD
source code used to create the install media.
Note that the share-build-* hostflag is optional, and will only be set
for build jobs.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v5:
- Fetch the host idents from the arguments, instead of assuming
"host" always.
- Set PropMinVer:Freebsd:<version> instead of freebsd-<version> to
better cope with FreeBSD version bumps. This should only happen
every ~2 years.
Changes since v4:
- Make the share-build- flag optional.
Changes since v3:
- Make use of the set_runtime_hostflag helper.
- Make use of the sha256file helper.
This is required for FreeBSD, that will need to set some of the
hostflags at runtime. The current IDENT_hostflags will be keep as-is,
and they should only be set at job creation time.
Also introduce a helper to set the runtime hostflags.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v4:
- Allow runtime flags to be appended instead of overwritten.
In order to generate the FreeBSD installer image and the install
media.
The install sets are the vanilla ones generated by the 'ftp' release
target. The installer image is handcrafted based on the filesystem
created by the 'bootonly' target, which is then populated with the ssh
host keys, and setup in order to use the serial console. The other
difference from upstream FreeBSD installer images is that the one
built by osstest uses a ramdisk instead of relying on the installer
media to be somehow attached, either on a CD or USB drive. This is
required in order to boot the image from pxelinux (where no CD or USB
is actually attached to the host, and everything is fetched from
tftp).
Due to the nature of the FreeBSD build, the outputs are different from
what osstest expects from a buildjob, more specifically
path_freebsddist points to a folder that contains the several outputs
form this buildjob.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v4:
- Remove some useless comments.
- Use qw(git) instead of 'git'.
- Replace unicode nonbreaking space.
- Remove trail ','.
- Improve the way to fetch the version number.
Changes since v3:
- Introduce two helpers to perform the build process.
- Install packages using target_install_packages.
Changes since v2:
- Increase build target timeouts.
- Use sysrc instead echo to set rc.conf options.
Changes since v1:
- Remove the ts-build-check FreeBSD hack.
- Use pkg-static instead of pkg.
- Introduce buildcmd_stamped_logged_root and target_cmd_build_root.
- Use target_cmd_build_root and buildcmd_stamped_logged_root in the
ts-freebsd-build script.
- Fix the script snippets to use <<END.<<'END' in order to avoid
escaping the shell variables.
- Set path_freebsddist runvar to point to the folder where the build
files are stashed.
- Add a comment at the top of the file describing what runvars are
consumed/produced by the build script.
osstest: add support for the FreeBSD package manager
FreeBSD support is added to target_install_packages and
target_install_packages_norec, although there's no equivalent to the
--no-install-recommends in the FreeBSD package manager.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v6:
- Use $ho->{Os} in order to detect the target OS.
Changes since v5:
- Add prototype for package_install_cmd.
- Use $ho->{Ident} instead of hardcoding "host" in get_hostflags.
Changes since v4:
- Unify more code between FreeBSD and Debian: introduce an
OS-agnostic package_install_cmd to generate the install cmd.
Some steps of the FreeBSD build process need to be done as root, so
introduce some helpers to perform those steps:
buildcmd_stamped_logged_root and target_cmd_build_root. They behave
exactly the same as the non-root counterparts.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v3:
- New in this version.
The installation is performed using the bsdinstall tool, which is part
of the FreeBSD base system. The installer image is setup with the
osstest ssh keys and sshd enabled by default, which allows the test
harness to just ssh into the box, create the install config file and
launch the scripted install.
Currently the installation is done with ZFS only, in stripe mode, and
a single disk.
In order to support the FreeBSD installer the setup_netboot_memdisk
helper is used, that allows setting the pxe boot of a host using a
memdisk.
The install script either picks the binary images from the output of a
previous FreeBSD buildjob (yet to be introduced), or from the folder
pointed by freebsd_distpath. This folder should contain the installer
image (install.img) and the compressed install sets (kernel.txz,
base.txz) together with the MANIFEST file that contains the checksums.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v4:
- Remove get_sets_path routine and xopts variable.
- Call resource_shared_mark_ready when the host is ready to be used.
Changes since v3:
- Use sha256file instead of the sha256sum clu.
- Shift parameters passed to the lock and copy script, so that
parameter 0 is ignored.
- Copy to .tmp and then move the FreeBSD image.
- Use a single-line find rune to cleanup old images.
- Add some missing "set -e".
- Don't check the output of target_cmd_output_root against undefined,
that's never going to be true.
- Split the setup_netboot_memdisk helper code into a pre-patch.
Changes since v2:
- Use sysrc instead of echo to modify rc.conf.
Changes since v1:
- Use pkg bootstrap instead of pkg update.
- Use freebsdbuildjob instead of freebsd_buildjob.
- add "append raw" to the memdisk command line.
- Use with-lock-ex instead of flock.
- Use system_checked and an array to hold the locked command to
execute.
- Use an array to store the list of possible disk devices.
- Use target_putfilecontents_root_stash in order to upload the
install script to the target.
- Remove the usage of TftpFreeBSDBase.
- Use target_tftp_prefix in order to get the prefix for the tftp
target image path.
- Add a missing linewrap.
- Replace the split used to get the runvar paths with a regexp.
- Place the install sets at /root/osstest_sets instead of
/tmp/osstest_sets.
- Pick the sets/installer image from the freebsdbuildjob
path_freebsdist or from the runvar variable freebsd_distpath.
- Add a comment about which FreeBSD specific inputs the script
consumes.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v4:
- Add a deployment note to the commit message.
Changes since v3:
- New in this version (split from install patch).
osstest: move known_hosts generation to TestSupport
This is equivalent to the already existing authorized_keys function,
and generates the contents of the known_hosts file that should be
installed on targets.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Mon, 24 Jul 2017 16:48:36 +0000 (17:48 +0100)]
anointments: Provide ./mg-anoint list-prepared
Deployment note: Requires Text::Glob, which on Debian is in
libtext-glob-perl (which I have installed in the Xen Project and
Citrix Cambridge primary VMs).
CC: Roger Pau Monne <roger.pau@citrix.com> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Mon, 24 Jul 2017 14:30:02 +0000 (15:30 +0100)]
README.planner: Improve internals documentation a bit
* share-flight resources may end up owned by a different task to their
shareix, perhaps as a result of test database operations or perhaps
as a result of donation with mg-allocate. This should not be a
problem.
* Document the xdbref task type.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Mon, 22 May 2017 15:09:23 +0000 (16:09 +0100)]
ap-common: Switch to Linux 4.9 by default
I ran a special report[1] to see what to expect and:
Tests which did not succeed and are blocking,
including tests which could not be run:
test-amd64-i386-xl-qemuu-win7-amd64 15 guest-localmigrate/x10 fail REGR.
test-amd64-i386-xl-qemut-win7-amd64 15 guest-localmigrate/x10 fail REGR.
These Windows 7 migration tests have been failing on many branches and
don't look like they are something to do with the version of Linux
used in dom0.
Accordingly I intend to push this change to switch osstest to using
Linux 4.9 by default. ARM tests are not affected at this time.
CC: Boris Ostrovsky <boris.ostrovsky@oracle.com> CC: Juergen Gross <jgross@suse.com> CC: Stefano Stabellini <sstabellini@kernel.org> CC: Wei Liu <wei.liu2@citrix.com> CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> CC: Roger Pau Monné <roger.pau@citrix.com> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 29 Jun 2017 12:10:38 +0000 (13:10 +0100)]
step logfiles: Make substep_* take $logfile, not $script
The $script parameter was used by Standalone to print a message, but
this is no longer done.
It is used by Executive as the value for the "step" column in the
steps table, which shows up as "script" in the output from
sg-report-flight. But, when running under sg-run-job,
JobDB::Executive already has a better source of the script name: the
parent step (as reported in OSSTEST_TESTID). When not running under
sg-run-job the use of $0 is good enough.
Finally, it is used by Executive as a logfile name. But when running
under sg-run-job, JobDB::Executive gets the right logfile name anyway
(as above); and when not running under sg-run-job there is no default
logfile.
Conversely, substeps might organise their own logfile.
So abolish $script from the TestSupport substep API. Replace it in
substep_start and substep_eval with a $logfile parameter.
Update all call sites to no longer pass $script. None make their own
logfiles, so none pass the new parameter $logfile.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 29 Jun 2017 12:05:18 +0000 (13:05 +0100)]
step logfiles: Honour new logfile column
There are the following places in the tree where we construct the
logfile name from stepno and step:
* tcl/JobDB-Executive main decision about logfile name } OK
* JobDB::Executive final fallback for new steps } keep
* steps-logfile-constraint.sql default for old data }
* sg-report-flight (twice) } both read stepno and step from the db;
* determine-failure-reasons } here we teach them to try logfile first
Now substep logs will have correct links in the job output, as soon as
the logfile column has the right data.
This is part of a "Populate-then-rely" schema change, as described in
schema/README.updates. Specifically, this is the step:
6. Optionally commit: code which reads new column, but which tolerates
it containing NULL/DEFAULT. (`add' is already Needed.)
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 29 Jun 2017 12:02:24 +0000 (13:02 +0100)]
substep logfiles: Set logfile and step columns correctly for substeps
If we are running under sg-run-job, OSSTEST_TESTID will be set to the
testid of the parent step (ie, the step created by sg-run-job which
relates to the execution of the whole script).
Unless the caller has specified otherwise, the log and script are
the same as for the parent step.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 29 Jun 2017 11:55:54 +0000 (12:55 +0100)]
substep logfiles: Populate new logfile column, at all, for substeps
For now we populate it with the sometimes-wrong value based on
$script. In detail:
* Add a parameter $logfile to $mjobdb->step_start, which currently
always gets passed undef.
* In JobDB::Executive::step_start, try to use it as the value for the
new logfile column. But, if it is not set (which is currently
always the case), default it to "<stepno>.<script>.log".
This is the same log filename currently assumed (and therefore used)
by sg-report-flight. So with current versions of sg-report-flight,
this has no visible functional change.
This is part of a "Populate-then-rely" schema change, as described in
schema/README.updates. Specifically, this is part of the step:
5. Commit: code to populate new column; changing `add' to status
Needed and `constraint' to status Ready.
We don't set `constraint' to Ready yet because we haven't completed
the code to populate the new column.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 29 Jun 2017 11:43:00 +0000 (12:43 +0100)]
step logfiles: Provide new column
We will write things into it, and use it, soon.
This is the start of a "Populate-then-rely" schema change, as
described in schema/README.updates. Specifically, the steps:
1. Commit: new schema update `add', status Preparatory
2. Commit: new schema update `constraint', status Unfinished
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 29 Jun 2017 11:08:48 +0000 (12:08 +0100)]
step logfiles: Standalone mode: Do not print uninformative $script value
Right now, this is sometimes wrong; even if it were right, it's not
going to be interesting in the transcript since the actual script name
is obvious from context.
Now this function does not care if $script is undef.
While we're here, fix the prototype (which is mostly commentary here,
as it is not used for method calls).
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 29 Jun 2017 12:40:00 +0000 (13:40 +0100)]
db write transactions: Make all INSERTs name columns explicitly
This makes the code more robust against schema updates. Specifically,
if any columns were to be deleted, these runes would break. (Also if
any column was inserted other than after the existing columns,
although this is not something Postgres can currently do AIUI.)
Affected tables are:
flights_harness_touched
jobs
resources
runvars
steps
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 29 Jun 2017 11:41:50 +0000 (12:41 +0100)]
schema updates: mg-schema-test-database: Cope with applied updates
The schema_updates table contents are written by the call to
mg-schema-update. Trying to copy the contents from the source db
correctly fails with duplicate key errors. So don't do that.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Wed, 21 Jun 2017 11:46:26 +0000 (12:46 +0100)]
sg-run-job: Abolish `anyfailed'
This is very similar to !$ok.
anyfailed is set to 1 only by catching-otherwise, which also then sets
ok to 0. The differences are solely some sites which set ok to 0
without setting anyfailed to 1:
(i) In run-job, if there are escaped steps
(ii) In run-job, if any steps were not run due to the skip_testids
or truncate_testids runvars
(iii) If a per-host-ts step fails
The only places where anyfailed was used were:
* To decide whether to do log capture in build jobs. This happens
before (i) and (ii); and (iii) is not applicable because it only
applies if $need_xen_hosts is nonempty which is never true if
$need_build_host is false. We can simply use $ok instead,
without any functional change.
* In printing a message to stdout at the end of the job.
We update this use site to use $ok and change the message
accordingly.
So overall, no functional change other than to a message in the
transcript.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Wed, 21 Jun 2017 11:40:39 +0000 (12:40 +0100)]
sg-run-job: Make catching-otherwise honour $ok
This is more regular and abolishes a number of explicit $ok tests in
run-job.
Of the call sites:
* "catching-otherwise blocked check-not-blocked" near the top of
run-job is soon after $ok is set, so NFC.
* The other call sites in run-job all test $ok explicitly, so NFC.
* The one call site outside run-job (in examine-host-examine)
contains only calls to run-ts, and to examine-host-install-* which
itself only contains similar calls. These all therefore honour $ok
already. So NFC.
While we are here:
* Add a doc comment.
* Do some slight whitespace fixups.
Overall, no functional change.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 15 Jun 2017 18:06:13 +0000 (19:06 +0100)]
sg-run-job: Spawn ts-syslog-server
Every job now gets a syslog server, which starts up after host
allocation.
The server's address and port are recorded by ts-syslog-server in a
runvar and can be used by subsequent test steps.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v2: Change mistaken TESTID `.' to `='. `.' means literally `.'.
Do not start ts-syslog-server if !$ok.
Do not start ts-syslog-server if job is being truncated now.
Use new | (stdin pipe) spawn-ts feature; avoids total failure.
Ian Jackson [Thu, 15 Jun 2017 18:05:50 +0000 (19:05 +0100)]
ts-syslog-server: New test script
This is a "complete implementation of the UDP syslog protocol".
In fact, the UDP syslog protocol is trivial: the UDP packets contain
newline-terminated unix text strings. We can simply chomp them and
log them (along with the sending address and port).
This script is designed to be run with the new "|" spawn-ts mode: it
will keep running until it is reaped.
Currently it is not part of any jobs.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Fri, 16 Jun 2017 16:40:04 +0000 (17:40 +0100)]
make-flight: Provide Windows 10 and Windows Server 16 guests with 3.5G of RAM
Paul advises that 4G may be needed, but we have some 8G hosts in
various places. If this is not sufficient in practice, we should
consider restricting the test to bigger hosts.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> CC: Wei Liu <wei.liu2@citrix.com> CC: Dario Faggioli <dario.faggioli@citrix.com> Reviewed-by: Paul Durrant <paul.durrant@citrix.com>
Ian Jackson [Fri, 16 Jun 2017 15:55:26 +0000 (16:55 +0100)]
ts-windows-install: Provide guest with 32G of disk rather than 10G
Paul Durrant examined a VM after a migration failure and found it
short of free space with some essential services not running, and
conjectures that this is cauxed by disk shortage.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Reviewed-by: Paul Durrant <paul.durrant@citrix.com>
Ian Jackson [Wed, 14 Jun 2017 14:52:16 +0000 (14:52 +0000)]
mg-repro-setup: Use new syntax for cs-adjust-flight
In 497b2c6c933d "Rework runvar-build-set new value handling" we fixed
bugs in cs-adjust-flight which would make mg-repro-setup's attempt to
adjust build jobs sometimes not match at all, but we also changed the
syntax for specifying just a new flight. I forgot to update the
one in-tree user of that feature.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Tue, 6 Jun 2017 15:50:37 +0000 (16:50 +0100)]
cs-adjust-flight: Rework runvar-build-set new value handling
Previously, if it didn't contain a `.', it would be taken as a flight
name and completed with the old job name. (This was not documented.)
This meant that there was no way to adjust to refer to a differnet job
in the flight being manipulated without specifying the flight number
(which is not desirable, nor even possible with new:)
Instead, we adopt the convention that a trailing . completes the value
with the old job name. Values without a . are taken literally as a
job name, resulting in intra-flight references to that job.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
make-flight: Add livepatch build/test target in the matrix.
So we can have "test-amd64-amd64-livepatch" or such.
Changes to the flights are as follows. In these branches:
osstest
xen-4.8-testing
xen-4.9-testing
xen-unstable
add the new jobs:
test-amd64-amd64-livepatch
test-amd64-i386-livepatch
which look a bit like this (test-amd64-amd64-livepatch example):
all_host_di_version current
all_host_suite jessie
all_hostflags arch-amd64,arch-xen-amd64,suite-jessie,purpose-test
arch amd64
buildjob build-amd64
kernbuildjob build-amd64-pvops
kernkind pvops
toolstack xl
xen_boot_append loglvl=all
xenbuildjob build-amd64
Note that we do not add these jobs to xen-unstable-smoke of course,
although we did (earlier) turn livepatching for the Xen build
configuration in those flights, and try building the livepatch test
cases (since those things are cheap).
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Livepatch compiles and works on x86/ARM{32|64} so we can enable it
in the Xen config when requested by the enable_livepatch runvar.
When we are trying to build with enable_livepatch, run `make
dist-tests' as well, to ship the test cases. This depends on a
corresponding change to xen.git.
Finally, split the livepatch tests into their own stashed deliverable
from this job.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Reviewed-by: Konrad Rzeszutek Wilk <konrad@kernel.org>
mfi-common: Add an enable_livepatch runvar to the Xen build jobs
Set it to true on branches that support livepatching (Xen versions 4.9
and higher). Currently nothing reads this variable, so no overall
functional change.
Changes to the flights are as follows. On these branches:
osstest
xen-4.8-testing
xen-4.9-testing
xen-unstable
xen-unstable-smoke
in these jobs:
build-amd64 build-amd64-xsm
build-arm64 build-arm64-xsm
build-armhf build-armhf-xsm
build-i386 build-i386-xsm
add the runvar setting `enable_livepatch=true'.
Signed-off-by: Konrad Rzeszutek Wilk <konrad@kernel.org> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com> Reviewed-by: Konrad Rzeszutek Wilk <konrad@kernel.org>
TestSupport: target_cmd_root_status: New sub which returns return code.
All the different target_cmd_* end up calling tcmdex
which has the unfortunate side-effect of calling 'die' if
the SSH sessions results in any return code not zero.
That is fine, except for tests where we want to get a non-zero
return value.
This patch adds the $badstatusok to tcmdex - and makes all
the existing callers pass in the value of zero to it. This
way the commands behave the normal old way.
to all the other functions which use tcmdex.
The only exposed function that does it differently
is target_cmd_root_status.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Fri, 26 May 2017 10:41:52 +0000 (11:41 +0100)]
arm64, armhf: Use dtuart for console by default
We don't want to have to set a host property for each ARM host.
Julien writes:
This command line should always work for arm64. If the device-tree
does not provide the property `stdout-path' then it is a bug. We
want the command line to be as agnostic as possible from the
platform.
Reported-by: Julien Grall <julien.grall@arm.com> Acked-by: Julien Grall <julien.grall@arm.com> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 25 May 2017 11:47:55 +0000 (12:47 +0100)]
ts-hosts-allocate-Executive: Make build failures not be host-sticky
Builds (jobs whose recipe contains `build') are not supposed to
contain host-specific tests. We already don't care about varying
which host they run on, to hunt for host-specific bugs.
We should also not make build failures sticky to the host they last
failed on.
Fix this by defaulting $prevfail_bonus to 0 in that case, before the
week-long later default is applied. This means that a build job
runvar would override the built-in default whether that's 7 days or 0.
Reported-by: Roger Pau Monné <roger.pau@citrix.com> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Tue, 23 May 2017 18:06:42 +0000 (19:06 +0100)]
ts-examine-logs-save: Do not do readdir $! check
In perl 5.14 in Massachusetts, this produces this message
/home/logs/logs/109694/test-amd64-amd64-examine Bad file descriptor at ./ts-examine-logs-save line 100.
Sadly this means there is no way to tell failure apart from end of
directory. This patch should be reverted when we have a fixed version
of perl.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
(Verified with
OSSTEST_CONFIG=standalone-config-example eatmydata ./standalone-generate-dump-flight-runvars
Of course the libvirt branches lack the xl tests already.)
CC: Stefano Stabellini <sstabellini@kernel.org> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Julien Grall <julien.grall@arm.com>
Ian Jackson [Wed, 17 May 2017 20:58:22 +0000 (21:58 +0100)]
host examination: bootloader: better handling of multiple runs
If ts-examine-serial-pre was run more than once (which happens by hand
during development, for example), it would reuse the same ids (which
is not idea for manual testing, since we normally want to know if the
last run was good). Instead, generate new cookies each time (and
overwrite the old ones in the runvars).
Also, it owuld keep adding new copies to the grub menu entries
(obscuring the actual interesting menu text completely). Instead,
prefix the mentries with a fixed string so they can be stripped out
again. They now look a bit sillier after one run, but less silly
after several.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Wed, 17 May 2017 22:22:16 +0000 (23:22 +0100)]
host examination: bootloader: edit all grub2 menu entries
edit all the menuentry and submenu entries.
This is a good idea because in princple the first menu and submenu
entries in the autogenerated grub.cfg might be in a function which is
not actually called.
The resulting grub menu now looks even sillier.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Wed, 17 May 2017 20:45:37 +0000 (21:45 +0100)]
host examination: bootloader: Do not use ed
These ed runes are what I used to develop this approach, but they are
rather opaque to those very few programmers who are not proficient
users of ed, the standard text editor.
Also, using target_editfile_root means we keep a record of the
bootloader config in the logs, which is desirable.
The regexps used to edit the grub2 confiug are slightly more
sophisticated than the ed ones were.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Wed, 17 May 2017 15:30:03 +0000 (16:30 +0100)]
host examination: serial, grub2: put cookie in menu lines
Empirically, the message printed by `echo' is not always shown. It
seems to depend on the host BIOS. I suspect that there's a
VGA-to-serial conversion step which only triggers when grub asks for
keyboard input, or something. In any case, empirically, increasing
the sleep (and sleeping before) did not work on the godellos.
Instead, stuff the cookie into the start of grub menu entries.
This is gross but effective.
We put it in the first menu, and the first submenu, which means it
will appear on the screen somewhere.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Tue, 16 May 2017 14:36:21 +0000 (15:36 +0100)]
host examination: Run a regular exam flight
Examine every host, once a month.
This flight contains these jobs:
build-amd64 build-amd64-pvops
build-arm64 build-arm64-pvops
build-armhf build-armhf-pvops
build-i386 build-i386-pvops
examine-*
The build jobs are very similar to normal build jobs and I won't
recount their contents here. In any case, normally they will be
replaced with build jobs from a recent xen-unstable or osstest flight.
The examine-SOMEHOST job looks like this:
all_host_di_version current
all_host_suite jessie
arch {amd64,i386,arm64,armhf}
buildjob build-{amd64,i386,arm64,armhf}
enable_xsm false
host SOMEHOST
hostalloc_maxwait_max 20000
kernbuildjob build-{amd64,i386,arm64,armhf}-pvops
kernkind pvops
toolstack xl
xenbuildjob build-{amd64,i386,arm64,armhf}
The above is from standalone-generate-dump-flight-runvars so contains
dummy data. The suite, architectures, host and recipe will vary
depending on the output from cs-list-hosts.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Tue, 16 May 2017 14:35:28 +0000 (15:35 +0100)]
host examination: Add a check to many flights
Check that nothing in the principal other branches is likely to
regress the host examination machinery. This is particularly relevant
for osstest itself, of course.
Runvars look like this (e.g., from test-amd64-amd64-examine):
Ian Jackson [Tue, 16 May 2017 13:35:06 +0000 (14:35 +0100)]
make-hosts-flight: Impose a timeout
If we cannot acquire a particular host for more than 20ks (~5.5h),
give up on it. This allows flights to complete if hosts are blocked
for a long time.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Fri, 12 May 2017 15:45:56 +0000 (16:45 +0100)]
host examination: Permanently save the logs
After this, we retain some of the logs from the last examination of
each host, in a directory in results/ named after the host.
The logic for computing the destination directory has similar env var
logic to that used in the invocation of sg-report-host-history but if
those are not set (as they usually won't be) has config-based
fallback.
In Standalone mode enough of these variables must be set to avoid
calling blessing_suffix, since we don't want the filename to depend on
a not-really-very-useful (and not implemented by intended_blessing)
Standalone flight's intended blessing.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Fri, 12 May 2017 15:44:28 +0000 (16:44 +0100)]
TestSupport, Executive: Provide intended_blessing
This will be used by a ts script which permanently saves some logs in
a host-specific rather than flight-specific directory. Non-real
flights ought not to mess up real data.
The function is not provided for Standalone mode, so a test step that
tries to use it will fail.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Wed, 10 May 2017 14:25:09 +0000 (15:25 +0100)]
host examination: Initial skeleton
Introduce an initial cut of job recipies host-examine-linux and
host-examine-xen, and a flight generation script which can make a
flight to examine all hosts with a particular blessing.
So far all this does is provide a way to make a flight which:
* Allocates each relevant host
* Installs Debian and, if applicable, Xen
* Reboots (if applicable, into Xen)
* Collects the logs to the flight log directory
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Tue, 16 May 2017 17:40:39 +0000 (18:40 +0100)]
cs-hosts-list, standalone-generate-dump-flight-runvars: make compatible
Honour OSSTEST_HOSTSLIST_DUMMY, to cause cs-hosts-list to produce
dummy output. This means that standalone-generate-dump-flight-runvars
can now try generating flights whose contents depend on the host
database, and which are therefore normally only useable in Executive
mode.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
projects / people / iwj / osstest.git / log