Anthony PERARD [Wed, 7 Feb 2018 18:42:48 +0000 (18:42 +0000)]
hopfully fixing centos 6 install
the https issue with yum seams to be that the proxy certificate (MITM)
was not taken into account. It should be now.
Also, using baseurl instead of mirrorlist does not work with fedora.
The squid is caching a new url every time, probably because of the use
of https.
Anthony PERARD [Tue, 21 Nov 2017 18:05:13 +0000 (18:05 +0000)]
make-centos-flight: Create a flight with CentOS as dom0
This is based on make-flight, with the added all_host_os=centos runvar,
and without test that can not be run.
Anything based on the recipe "test-debian" or "test-pair" is remove, as
they require xen-tools. There is no XSM tests as the CentOS packages is
built without. There is no build jobs as the Xen packages been tested
are built in the CentOS CBS.
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Anthony PERARD [Wed, 15 Nov 2017 12:31:04 +0000 (12:31 +0000)]
WORKAROUND: Osstest/TestSupport: Make target_reboot works with systemd
On host running with systemd as init, doing `ssh host reboot` will
result in ssh returning an error.
This patch works around by not waiting for the reboot command to return.
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Anthony PERARD [Wed, 15 Nov 2017 15:18:46 +0000 (15:18 +0000)]
CentOS: Setup osstest-confirm-booted.service
This emulate the osstest-confirm-booted service that is done on Debian.
In order to have the service been started last:
- it is made dependent on multi-user.target which is the default target on systemd.
- it is part of osstest.target which depend on multi-user.target.
- the osstest.target is set as the new default.
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Anthony PERARD [Wed, 15 Nov 2017 18:56:01 +0000 (18:56 +0000)]
JobDB-Standalone.tcl: Fix read-runvar
This fix the error bellow, when the runvar exist:
can't read "runvarinfo()": no such element in array
while executing
"set val $runvarinfo($val)"
invoked from within
"osstestdb eval {
SELECT val FROM runvars
WHERE flight = $flight
AND job = $job
AND name = $name
} runvarinfo {
se..."
(procedure "jobdb::read-runvar" line 3)
invoked from within
"jobdb::read-runvar $flight $job all_host_os"
(procedure "run-job" line 13)
invoked from within
"run-job $job"
(file "./sg-run-job" line 767)
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Roger Pau Monne [Thu, 30 Nov 2017 09:14:17 +0000 (09:14 +0000)]
osstest: use -DWITHOUT_AUTO_OBJ with FreeBSD release targets
Due to a recent FreeBSD change the default output directory of the release
targets is changed to the object directory instead of the source
directory. Use WITHOUT_AUTO_OBJ to restore previous behavior. This is
harmless if used with previous versions, it will be ignored.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Roger Pau Monne [Wed, 8 Nov 2017 14:53:34 +0000 (14:53 +0000)]
osstest: remove the loader timeout from the install image
When the FreeBSD installer is booted on the godello{0/1} boxes it
receives spurious key strokes. This doesn't happen so far when booted
from disk, or with any other boxes.
In order to cope with this remove the loader timeout on the install
image. Note that failure to boot will still drop the loader into a
manual prompt.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Roger Pau Monne [Wed, 8 Nov 2017 14:50:04 +0000 (14:50 +0000)]
osstest: hook FreeBSD flight into cr-daily-branch
Add the missing pieces so that cr-daily-branch knows how to create a
FreeBSD flight. This has been tested by calling `./cr-daily-branch
freebsd-master`.
Note that there are two missing pieces. The first one is correctly
setting the FREEBSD_<arch>_BUILDJOB environment variable when calling
make-freebsd-flight, without it the installer media will always be
picked from the configuration file. This requires keeping track of
previous FreeBSD successful build jobs, marking them as permanent and
fetching them from the database in order to pass them on the env
variable.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v8.1: fix ap-fetch-version-old for the freebsd-* branch
(The remote branch has been changed to tested/master or
tested/stable/<version> instead of plain master or stable/11.)
Changes since v6:
- Set the path to the FreeBSD git trees in this patch.
- Push to tested/master or tested/stable/<version> instead of using
the same branches as the origin tree.
Changes since v5:
- Add support for testing multiple FreeBSD branches, default to
branch master.
Changes since v3:
- Fix the daily-cron-email-* settings for FreeBSD branch flights.
Roger Pau Monne [Wed, 8 Nov 2017 14:50:03 +0000 (14:50 +0000)]
osstest: introduce a script to create a FreeBSD flight
The logic to create a FreeBSD build job is added to
make-freebsd-flight. This includes creating a FreeBSD build job, and
also testing the output of that build job (by creating another build
job that depends on the output of the first).
Note that the FreeBSD build job needs some input in order to setup a
FreeBSD host, and that can be fetched from different places:
1. Env variable FREEBSD_<arch>_BUILDJOB: use the output from a
previous build-<arch>-freebsd job.
2. Env variables FREEBSD_DIST and FREEBSD_VERSION: set before calling
into make-flight, provide the path to the installer image and sets
folder and the version being installed.
3. Config file FreeBSDDist and FreeBSDVersion: same as 2. except that
they are set on the config file.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v6:
- Re-wrap lines.
- Remove the freebsd hostflag.
- Do not set the FreeBSD git trees in this patch.
- Set all_host_os runvar to freebsd for the FreeBSD build jobs.
Changes since v5:
- Append the "freebsd" hostflag, the specific version will be set
dynamically by the ts-freebsd-set-hostflags script.
- Use FREEBSD_<arch>_BUILDJOB to set the freebsdbuildjob runvar.
- Loop around arches, although only amd64 is supported ATM. This
should make it easier to add more arches later on.
- Expand README comment to describe the expected layout of the
FreeBSD install media.
Changes since v4:
- Convert get_freebsdjob_runvars into set_freebsd_runvars, mimic the
behavior of set_hostos_runvars.
Changes since v3:
- Split the sg-run-job code into pre-patches.
Changes since v1:
- Replace freebsd_buildjob with freebsdbuildjob.
- Replace FREEBSD_SETS/IMAGE with a single FREEBSD_DIST that points
to a folder that should contain both things.
- Document the FreeBSDDist and FreeBSDVersion config file options in
the README file.
Roger Pau Monne [Wed, 8 Nov 2017 14:50:02 +0000 (14:50 +0000)]
osstest: add support for FreeBSD buildjobs to sg-run-job
Add support and introduce a FreeBSD build job to sg-run-job.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v5:
- Add a '+' to the arguments passed to ts-freebsd-set-hostflags, so
they are hidden from testid.
Changes since v4:
- Use a switch in allocate-build-host.
Changes since v3:
- New in this version (split from existing patch).
Roger Pau Monne [Wed, 8 Nov 2017 14:50:02 +0000 (14:50 +0000)]
osstest: change the meaning of need_build_host
Make need_build_host store a string instead of a boolean. This is
later going to be expanded to handle the FreeBSD build jobs.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v5:
- Use a regsub to set need_build_host.
- NB: I would prefer to keep the tolower instead of changing the
procs to -FREEBSD/-LINUX since I don't really like the SHOUTING.
Changes since v4:
- Fix the usage of the catching-otherwise blocks.
Changes since v3:
- New in this version (split from patch).
Roger Pau Monne [Wed, 8 Nov 2017 14:50:00 +0000 (14:50 +0000)]
osstest: introduce a script to set the runtime hostflags runvar for FreeBSD jobs
Due to the nature of the FreeBSD install media, which is
self-generated from the ts-freebsd-build script, the hostflags runvar
set to FreeBSD jobs are related to the current version under test.
The following hostflags might need to be fetched from the runvars of a
previous build-$arch-freebsd job:
- share-build-freebsd-$arch-$hash: the $hash used here is calculated
from the checksum of the installer image used by this specific job.
This allows osstest to share FreeBSD build hosts, and be sure the
exact desired FreeBSD version is used.
- PropMinVer:Freebsd:$version: $version contains the major FreeBSD
version under test. This version is obtained from the FreeBSD
source code used to create the install media.
Note that the share-build-* hostflag is optional, and will only be set
for build jobs.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v5:
- Fetch the host idents from the arguments, instead of assuming
"host" always.
- Set PropMinVer:Freebsd:<version> instead of freebsd-<version> to
better cope with FreeBSD version bumps. This should only happen
every ~2 years.
Changes since v4:
- Make the share-build- flag optional.
Changes since v3:
- Make use of the set_runtime_hostflag helper.
- Make use of the sha256file helper.
Roger Pau Monne [Wed, 8 Nov 2017 14:49:59 +0000 (14:49 +0000)]
osstest: add support for runtime_IDENT_hostflags
This is required for FreeBSD, that will need to set some of the
hostflags at runtime. The current IDENT_hostflags will be keep as-is,
and they should only be set at job creation time.
Also introduce a helper to set the runtime hostflags.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v4:
- Allow runtime flags to be appended instead of overwritten.
Roger Pau Monne [Wed, 8 Nov 2017 14:49:59 +0000 (14:49 +0000)]
osstest: introduce a FreeBSD build script
In order to generate the FreeBSD installer image and the install
media.
The install sets are the vanilla ones generated by the 'ftp' release
target. The installer image is handcrafted based on the filesystem
created by the 'bootonly' target, which is then populated with the ssh
host keys, and setup in order to use the serial console. The other
difference from upstream FreeBSD installer images is that the one
built by osstest uses a ramdisk instead of relying on the installer
media to be somehow attached, either on a CD or USB drive. This is
required in order to boot the image from pxelinux (where no CD or USB
is actually attached to the host, and everything is fetched from
tftp).
Due to the nature of the FreeBSD build, the outputs are different from
what osstest expects from a buildjob, more specifically
path_freebsddist points to a folder that contains the several outputs
form this buildjob.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v4:
- Remove some useless comments.
- Use qw(git) instead of 'git'.
- Replace unicode nonbreaking space.
- Remove trail ','.
- Improve the way to fetch the version number.
Changes since v3:
- Introduce two helpers to perform the build process.
- Install packages using target_install_packages.
Changes since v2:
- Increase build target timeouts.
- Use sysrc instead echo to set rc.conf options.
Changes since v1:
- Remove the ts-build-check FreeBSD hack.
- Use pkg-static instead of pkg.
- Introduce buildcmd_stamped_logged_root and target_cmd_build_root.
- Use target_cmd_build_root and buildcmd_stamped_logged_root in the
ts-freebsd-build script.
- Fix the script snippets to use <<END.<<'END' in order to avoid
escaping the shell variables.
- Set path_freebsddist runvar to point to the folder where the build
files are stashed.
- Add a comment at the top of the file describing what runvars are
consumed/produced by the build script.
Roger Pau Monne [Wed, 8 Nov 2017 14:49:58 +0000 (14:49 +0000)]
osstest: add support for the FreeBSD package manager
FreeBSD support is added to target_install_packages and
target_install_packages_norec, although there's no equivalent to the
--no-install-recommends in the FreeBSD package manager.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v6:
- Use $ho->{Os} in order to detect the target OS.
Changes since v5:
- Add prototype for package_install_cmd.
- Use $ho->{Ident} instead of hardcoding "host" in get_hostflags.
Changes since v4:
- Unify more code between FreeBSD and Debian: introduce an
OS-agnostic package_install_cmd to generate the install cmd.
Roger Pau Monne [Wed, 8 Nov 2017 14:49:55 +0000 (14:49 +0000)]
osstest: introduce build helpers for FreeBSD
Some steps of the FreeBSD build process need to be done as root, so
introduce some helpers to perform those steps:
buildcmd_stamped_logged_root and target_cmd_build_root. They behave
exactly the same as the non-root counterparts.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes since v3:
- New in this version.
Roger Pau Monne [Wed, 8 Nov 2017 14:49:53 +0000 (14:49 +0000)]
osstest: move known_hosts generation to TestSupport
This is equivalent to the already existing authorized_keys function,
and generates the contents of the known_hosts file that should be
installed on targets.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Thu, 14 Dec 2017 16:28:21 +0000 (16:28 +0000)]
smoke tests: Fix job reuse globs in cr-daily-branch
A symptom that the old patterns lack !build-arm64-xsm, so the xsm job
might be reused. The overall cause is that it contained a (partial)
list of architectures.
Instead, we observe that:
* The things we want to avoid reusing are Xen and libvirt (which
builds against Xen.
* Non-Xen builds are invariably build-<arch>-<thing> (or even longer)
so we can match them all with build-*-*.
* We need to exclude the Xen builds and the libvirt builds. The
libvirt builds are obvious. The Xen builds are either build-<arch>
which does not match build-*-*, or build-<arch>-xsm.
CC: Stefano Stabellini <sstabellini@kernel.org> Acked-by: Julien Grall <julien.grall@linaro.org> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 14 Dec 2017 16:03:44 +0000 (16:03 +0000)]
smoke tests: Build arm64 with XSM, instead of without
The smoke flight contains test-arm64-arm64-xl-xsm so it should contain
build-arm64-xsm (and not contain build-arm64).
I have checked the results with
OSSTEST_CONFIG=standalone-config-example eatmydata ./standalone-generate-dump-flight-runvars
and looking at the diff shows precisely the expected change.
CC: Stefano Stabellini <sstabellini@kernel.org> Acked-by: Julien Grall <julien.grall@linaro.org> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
This variable can be set to the absolute pathname of a kernel .deb to
use. It will be used only for hosts for which the corresponding
hostflag "need-kernel-deb-<suite>-special" is set.
There is not currently any facility for more than one special kernel
for each architecture.
As with backports kernels, the normal (Debian-supplied) kernel is
available too, and is used for hosts without any special hostflag.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
* In the first half of the backports kernel processing, set
the new variable specialkernel to the string "backports".
(This token occurs in the output .deb and kernel name, and
is also referenced by hostflags of the form
need-kernel-deb-<suite>-backports.)
* Break out the second half of the backports procesing into its
own section, conditional on $specialkernel.
* Replace occurrences of the "backports" token, after that point,
with $specialkernel.
Overall, this will allow other kinds of special kernel to be
introduced. But there is, for now, no functional change.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 14 Dec 2017 13:09:52 +0000 (13:09 +0000)]
di_special_kernel: Use di_installer_path
This avoids the caller having to supply $d_i. This is good because
there is a site we want to call this from which uses that name for a
different value!
No functional change.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Thu, 14 Dec 2017 12:23:24 +0000 (12:23 +0000)]
di_special_kernel: Replace open-coding in ts-host-install
The effect is simply to reuse the loop in di_special_kernel. The
extra tests etc. to compute $k and $c in di_special_kernel are of no
import here, and are harmless. We have already called
di_special_kernel so if it was going to fail due to this extra
computation, it would do so earlier.
No overall functional change.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Thu, 30 Nov 2017 17:40:13 +0000 (17:40 +0000)]
mg-hosts: mknetbootdir: Correctly use Path again
In fcd93edd00a8c9b08da3994d8bee9fa3809c3d93
"netboot: Rework interface to host_netboot_file (was host_pxefile)"
we unintentionally dropped $ho->{Tftp}{Path} from the
effective path. Reinstate it.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Tue, 5 Dec 2017 11:47:53 +0000 (11:47 +0000)]
production-config: Increase LogsMinSpaceMby etc.
osstest does not currently try to estimate the prospective disk use in
logs/ etc. of a flight before it starts; nor does it aggregate the
disk requirements of multiple concurrent flights.
Instead, it has a hardcoded minimum, which is enforced (by expiry) at
the start of each flight. It must be large enough for all in-progress
flights. This turns out not to be true any more, because of expansion
of both the number of different osstest branches, and because of
expansion of the number of jobs per flight, and because of expansion
of the activities within each job.
So, well, let's double it.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Fri, 1 Dec 2017 11:11:38 +0000 (11:11 +0000)]
cs-adjust-flight: Correct pseudo-BNF synopsys for runvar-build-set
In 497b2c6c933d13a05b01c6a654ce470be16dd78a
cs-adjust-flight: Rework runvar-build-set new value handling
the interpretation of this parameter was changed completely, but the
synopsis was not updated and thus became wrong.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Wed, 15 Nov 2017 11:00:07 +0000 (11:00 +0000)]
ts-xen-build-prep: Install libelf-dev for benefit of linux.git
Linux upstream has started needing libelf-dev. Without it, recent tip
fails (in our configuration) like this:
Makefile:938: *** "Cannot generate ORC metadata for CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel". Stop.
This is because kernel now is using objtool to create unwind
information. This needs libelf to work. Advantage is that this
approach no longer depends on assembler sources being heavily
annotated with unwind hints.
CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> CC: Stefano Stabellini <sstabellini@kernel.org> CC: Boris Ostrovsky <boris.ostrovsky@oracle.com> CC: Juergen Gross <jgross@suse.com> CC: Paul Durrant <paul.durrant@citrix.com> CC: Wei Liu <wei.liu2@citrix.com> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Juergen Gross <jgross@suse.com>
Roger Pau Monne [Wed, 25 Oct 2017 10:30:24 +0000 (11:30 +0100)]
osstest: Add freebsd runvars to examine job in generic flights
Previous patches only added the FreeBSD runvars to the jobs in the
examine flight, but failed to also add them to the examine job in the
generic flight.
This is necessary because the examine jobs now contain a step which
uses freebsd - the memdisk append determination.
On installations without an anointed freebsd job, and with no
freebsd-related config settings, that step will simply and harmlessly
fail.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
projects / people / aperard / osstest.git / log