Ian Jackson [Fri, 3 Jul 2015 10:06:40 +0000 (11:06 +0100)]
sg-report-flight: Fix --include (broken by 3d56a191)
In 3d56a191 "sg-report-flight: Make bodyprint[f] defer the output" it
became wrong for anything in the main printout part of
sg-report-flight to write directly to STDOUT. However, the two (very
similar) bits of code for handling the two kinds of --include still
used cat.
The result is emails looking like this:
From: osstest service owner <osstest-admin@xenproject.org>
To: <xen-devel@lists.xensource.com>, <rumpkernel-builds@freelists.org> CC: <ian.jackson@eu.citrix.com>
Subject: [rumpuserxen test] People who touched revisions under test:
Alice Bob <alice@example.com>
Ian Jackson <Ian.Jackson@eu.citrix.com>
Carol David <carol@example.com>
Date: Fri, 3 Jul 2015 06:50:53 +0000
flight 59037 rumpuserxen real [real]
Fix this by replacing those two near-identical bits of code with calls
to a single function which slurps the required include file into the
body text accumulation variable.
(There is no need to provide a facility for header includes, since
there is no need for a caller to pass headers through
sg-report-flight: they can simply be written into the output file
before running sg-report-flight, as indeed the string `Subject: ' is.
Despite the fact that --include-begin is used only by cr-daily-branch
which passes it a file `heading', the text is intended for the start
of the body, not the email headers.)
(b) The output after this patch differs from the output produced by 1f0c168d (the previous osstest production commit) only in addition of
the `Last test of basis' and `Testing same since' information, in the
appropriate place.
Deployment note: Because this patch changes only sg-report-flight, it
is safe to deploy under the feet of a running flight. I intend to do
this for all the production branches in the colo.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Dario Faggioli [Tue, 30 Jun 2015 17:36:59 +0000 (19:36 +0200)]
make-flight etc.: stop testing SEDF at all
SEDF has been broken and unmaintained at least until Xen 4.2, and most
likely even before! Tests are failing without anyonce caring, and yet
we're keeping using test resources for them.
Let's stop doing this!
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Tue, 30 Jun 2015 13:31:28 +0000 (14:31 +0100)]
Emails: Provide X-Osstest-Failures
Specifically:
* $cat now contains SORTLETTER BLOCKSTATUS DESCRIPTION...
* Rewrap the lines setting $cat
* Add $notsucceeds{$cat}[][2] containing one line for the new header
* Generate the new header
* Document the new header
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson [Tue, 30 Jun 2015 13:29:21 +0000 (14:29 +0100)]
Emails: Provide X-Osstest-Versions-This and That headers
Specifically:
* Provide headerprint() in sg-report-flight
* Use it to add X-Osstest-Versions headers based on %specver
* Cause cr-daily-branch to always enable the feature
* Document this in README.email
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v2: Document This and That
Ian Jackson [Tue, 30 Jun 2015 13:17:05 +0000 (14:17 +0100)]
Emails: Provide README.email and link to it
Also link to README, from the email footer.
I have tested that the link to README works right now. The link to
README.email does not yet, of course, but it will start to work when
osstest pushes this change to production.
CC: Andrew Cooper <andrew.cooper3@citrix.com> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson [Tue, 30 Jun 2015 10:08:37 +0000 (11:08 +0100)]
sg-report-flight: Use `bodyprint' and `bodyprintf'
This will allows us to divert the output so that we can (optionally)
produce email headers too. We move the error checking into the
function, obviously.
For now, this is a simple replacement with no functional change,
except that some call sites now have error checking which previously
did not.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson [Mon, 29 Jun 2015 17:06:11 +0000 (18:06 +0100)]
Reporting: sg-report-flight: Put overall pushgate stats in email
This adds something like this (would have appeared in 37638):
Last test of basis 37629 2015-06-24 21:36:10 Z 4 days
Failing since 37630 2015-06-26 10:35:44 Z 3 days 9 attempts
Testing same since 37635 2015-06-28 03:05:56 Z 1 days 4 attempts
or this (would have appeared in 37629):
Last test of basis 37617 2015-06-20 04:06:01 Z 9 days
Testing same since 37628 2015-06-24 13:35:23 Z 5 days 2 attempts
(Both examples from the Cambridge instance.)
This is added just after the versions are reported.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v2: Swap order of `Failing since' and `Testing same since' output lines
v3: Actually return when no pushgate summary wanted.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v3: Fix bug resulting in ERROR: missing FROM-clause entry for table "r"
Ian Jackson [Mon, 29 Jun 2015 15:46:36 +0000 (16:46 +0100)]
Reporting: In history tables, show multiple failure steps
In report_run_getinfo, used by sg-report-job-history and
sg-report-host-history, make provision for reporting multiple failed
steps.
The failed steps are put side by side (in a single table cell). The
cell background is set by the job status, but the individual steps may
vary their colour, as seen in this example:
http://xenbits.xen.org/people/iwj/2015/linux-next.html
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v2: Remove unnecessary space in ColourAttr
Ian Jackson [Mon, 29 Jun 2015 15:23:08 +0000 (16:23 +0100)]
Reporting: Break out report_blessingscond
The returned SQL is a self-contained expression, and does not require
additional bind parameters. To spot SQL quoting problems, we die if
a blessing is not reasonable.
The use sites of the $blessingcond in sg-report-flight are adjusted to
no longer pass @blessings to execute.
No overall functional change with reasonable blessings.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson [Mon, 29 Jun 2015 14:55:13 +0000 (15:55 +0100)]
Reporting: report_run_getinfo produces Content
Previously we produced Summary, and the call sites were confused about
whether it had already been html-escaped. Now we produce something
which is explicitly already html.
No functional change except that some sg-report-host-history output is
escaped as it ought to have been (but this is only relevant if the
database contains strange things which none of ours do).
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson [Mon, 29 Jun 2015 14:51:22 +0000 (15:51 +0100)]
Reporting: report_run_getinfo produces ColourAttr
This includes the bgcolour attribute name and the quotes. This will
make it possible for this function to sometimes not set a background
for the whole table cell in the future (which will be part of putting
multiple step results in each cell).
No functional change.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v2: Remove an unneeded space in ColourAttr
Ian Jackson [Mon, 29 Jun 2015 15:37:14 +0000 (16:37 +0100)]
Reporting: sg-report-host-history: Support --max-flight
We can't use report_blessingscond because we handle blessings
differently in sg-report-host-history (since we want the history of
even adhoc etc. flights).
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson [Fri, 26 Jun 2015 18:21:34 +0000 (19:21 +0100)]
Reporting: sg-report-flight: Fix undefined value in $worstrow[]
The fallback entry in the worstrow search, used when a job has not
been run at all, was wrong. Also, fix the doc comment which induced
the mistake, and add a comment about the contents of @worst.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v2: Added the comment about @worst.
Ian Jackson [Tue, 30 Jun 2015 09:22:25 +0000 (10:22 +0100)]
ap-common: Abolish linux-3.0 special case
This is identical to the effect of the linux-[3-9] etc. glob.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v2: New patch in this version of the series.
Ian Jackson [Mon, 29 Jun 2015 10:00:48 +0000 (11:00 +0100)]
Test Linux 4.1
The kernel.org stable git branch layout is the same for 4.1 as for
3.x, so simply extend the pattern, and add the branch to BRANCHES.
We move the linux-stable.git pattern to the end, and extend it to
linux-[3-9].[0-9]*, to provide a bit more future-proofing. (There is
no particular reason other than shell glob convenience to stop at
linux-9.x.)
DEPLOYMENT NOTE: After this patch is acked, but just before it is
pushed to pretest, run (this version of)
./mg-branch-setup linux-4.1 bisect b953c0d234bc72e8489d3bf51a276c5c4ec85345
to create the working trees on the osstest VM and push the base of
v4.1 to the tested output branch.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v2: Extend glob pattern in ap-common and move to the end.
Ian Campbell [Mon, 15 Jun 2015 11:37:44 +0000 (12:37 +0100)]
Debian: Arrange to be able to chainload a xen.efi from grub2
Xen cannot (currently) be booted directly via the usual multiboot
path on EFI systems of any arch. Instead it is necessary to either
launch xen.efi direct from the UEFI shell or to chainload it from
grub. In both cases the Xen command line as well as what would
normally be the multiboot modules (kernel+command line, XSM policy,
initrd) must be configured in a Xen configuration file.
Here we add a new overlay/etc/grub.d/15_osstest_uefi grub script which
arranges that if a xen.efi is found in the EFI System Partition (as
arrange by a previous patch) a suitable entry is created in grub.cfg
as well.
When parsing the grub.cfg look for such an entry into addition to the
regular/multiboot one and when necessary write the configuration file
based on the regular entry and return the chainload one such that it
gets booted.
This is currently enabled only for Jessie ARM64 systems.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v4: Use a separate grub script snippet, there's nothing we need in
20_linux_xen
v3: Rewrap and reindent to avoid long lines
Ian Campbell [Mon, 15 Jun 2015 10:42:18 +0000 (11:42 +0100)]
Collect xen.efi into xendist and install in appropriate place
Previously these binaries would have been included in the regular
("tools") dist file, whereas they really belong in the xen one.
Install into /boot/efi/EFI/osstest ready for use when chainloading.
Note that /boot/efi is (or should be) a VFAT filesystem. So a bit of
care is needed WRT symlinks etc. This is also what prevents us from
just including /boot/efi/EFI/osstest/xen.efi in the dist tarball since
untarring over a VFAT needs a little care WRT the case of directories
etc.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Thu, 28 May 2015 11:44:34 +0000 (12:44 +0100)]
Debian: grub2: Use GRUB_CMDLINE_LINUX_XEN_REPLACE(_DEFAULT)
This overrides GRUB_CMDLINE_LINUX(_DEFAULT) which we were previously
editing but only for the Xen entries, meaning that we don't switch to
console=hvc0 for the native cases (i.e. don't break them).
We do however want to edit GRUB_CMDLINE_LINUX(_DEFAULT) to remove
"quiet" if present, since it is useful to people, especially those
using standalone mode as a provisioning tool, wanting to e.g. compare
a boot under Xen with the native case.
This has been supported since Wheezy but in any case we supply our own
20_linux_xen grub generator based on Wheezy which supports this.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Wed, 24 Jun 2015 08:23:16 +0000 (09:23 +0100)]
Debian: Do not add Debian to UEFI boot order on install
By default Debian inserts itself before any existing entries,
including the PXE one, meaning we otherwise cannot remotely regroove
the box.
Add a hook to /usr/lib/base-installer.d/ which installs a diversion of
/usr/sbin/grub-install and installs a wrapper which adds the
--no-nvram argument which prevents grub-install from adding the boot
entries.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v4: Die on !jessie
v3: New patch replacing "Debian: Fixup UEFI boot order during install"
Ian Campbell [Thu, 21 May 2015 09:57:41 +0000 (10:57 +0100)]
ts-host-install: Set dtbs in the non-special kernel case too.
When DTB support was added for armhf it required a backports kernel
for use with Wheezy and therefore DTBs were only collected for that
special case. However as we move forward to newer kernels it should
be possible to run using the proper kernel (e.g. Jessie should work
for armhf). Therefore we will need the DTBs for those kernels too.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v3: Improved commit message
Ian Campbell [Wed, 20 May 2015 15:58:16 +0000 (16:58 +0100)]
ts-kernel-build: Additional kernel options for Mustang
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Note: ARM testing currently uses a 3.16 based linux-arm-xen kernel
(compared to 3.14 currently used as the default). In order to support
Mustang something newer is needed, I've been testing 4.1 (next LTS
kernel), 3.18 (the intervening LTS kernel) might be sufficient but I
haven't tried it. We should plan to either update linux-arm-xen or the
default kernel version at some point before we get Mustang h/w.
v3: Set CONFIG_NET_XGENE (was commented out before)
Remove setting of CONFIG_SPARSEMEM_VMEMMAP
Ian Campbell [Thu, 21 May 2015 12:58:27 +0000 (13:58 +0100)]
Enable chain loading to local disk for UEFI PXE systems.
First arrange for bootloader to be installed to removable media path,
by using a new in Jessie preseed option. Then use that to chainload a
bootloader from the disk.
The removable media path is well known (part of the UEFI spec) which
saves us having to worry about which OS is on the host (so long as the
OS can be configured to populate the removable media path)
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Tue, 27 Jan 2015 12:06:41 +0000 (12:06 +0000)]
ts-host-install: Support UEFI PXE boot using grub.efi
Handling of chainloading from the local hdd in
setup_pxeboot_local_uefi will be added in the next patch.
(Wrap DiBase entry in README too for consistency)
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v4: Turn grub "echo todo" into a die.
v3: Reindent README (fix DiBase too for consistency)
Adjust for reordering of setup_{firmware}_pxeboot to put
{firmware} at end
Rewrap setup_grub_efi_bootcfg
Adjust for setup_pxeboot -> setup_pxeboot_di
Ian Campbell [Thu, 21 May 2015 12:46:24 +0000 (13:46 +0100)]
Refactor pxelinux configuration
The mechanism used to PXE boot can differ depending on the firmware
type. Therefore refactor into Osstest::TestSupport and key off a new
host property "firmware".
Currently supported is "bios" (the default) and "uboot", both of which
use pxelinux.cfg style files.
The default for the firmware property is "bios", hence no change for
any existing system (including those which use uboot, regardless of
whether they are now configured with the firmware proprty or not)
As part of this "setup_pxeboot" becomes "setup_pxeboot_di" since it is
somewhat d-i specific.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v3: reorder setup_${firmware}_pxeboot to setup_pxeboot_${firmware}
join $dicmd in setup_pxeboot instead of caller
Ian Campbell [Thu, 21 May 2015 11:49:55 +0000 (12:49 +0100)]
ts-host-install: split the "di" from the "host" command line
(i.e. the bit before/after the -- marker). When abstracting over
different bootloaders in a future patch this will be convenient since
it allows the code to add to either.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Thu, 21 May 2015 11:46:16 +0000 (12:46 +0100)]
ts-host-install: Split initrd out of @installcmdline
Other bootloaders handle this with an explicit separate option rather
than parsing it out of the command line as pxelinux does. Prepare for
supporting these.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Tue, 27 Jan 2015 10:07:21 +0000 (10:07 +0000)]
Introduce mg-pxe-loader-update
The story for PXE booting via UEFI (at least on arm64) is not so
straightforward as with pxelinux on x86. There seems to no good
bootloader to launch via UEFI+pxe, in fact all I could find was grub
(syslinux, and by extension pxelinux.efi, is x86 only).
Add mg-pxe-loader-update modelled on mg-debian-installer-update which
will download the necessary grub binaries and produce a grub image
which can be used to pxe boot.
grub lacks the convenient ability to search for config file based on
(substrings of) the MAC or IP address. So we arrange for the grub.cfg
in TftpGrubBase to chain load another config file from
TftpTmpDir/'$net_default_mac' where $net_default_mac is a grub
variable which is substituted at boot time.
Actually using this requires that bootp/dhcp provide a next-file so
UEFI knows what to boot (usually this would be pxelinux.0 which we
can't use here). Locally we have configured this as $name/pxe.img, so
we can use different loaders.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v4: Just mgi-common now that mgi-debian merged therein.
v3: Use mgi-common and mgi-debian
Ian Campbell [Fri, 19 Jun 2015 10:56:15 +0000 (11:56 +0100)]
mg-*: Make Debian package fetching common in mgi-common
This pattern appears several times in mg-debian-installer-update and
I'm about to add another script which uses it.
The centralised version always does "sort -n -r | head -n" to pick out
only the newest version, which only some of the open coded sites did.
Likewise the centralised version checks that $pkgfile is non-empty,
unlike some of the open coded sites.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v4: Fold into mgi-common instead
v3: New patch
Which has caught out one or two people using standalone mode.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Reviewed-and-Tested-by: Dario Faggioli <dario.faggioli@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Dario Faggioli [Fri, 26 Jun 2015 10:58:51 +0000 (12:58 +0200)]
Stop testing SEDF, start testing RTDS
the SEDF scheduler is about to be deprecated and go
away (see [1]). OTOH, the RTDS scheduler is here to
stay.
It therefore makes sense to stop smoke testing the
former in favour of the latter. Note that the
-sedf-pin jobs where only added in order to try to
debug a long standing issue with SEDF; it is not
necessary to have anything like that for RTDS.
For now, as RTDS is still marked as experimental,
test failures are allowed, as it is for SEDF.
Also, RTDS was introduced in Xen 4.5, so add the
test only for the relevant brnaches. Similarly,
SEDF is removed in Xen 4.6, so keep testing it in
older branches.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Fri, 26 Jun 2015 10:31:31 +0000 (11:31 +0100)]
mg-debian-installer-update: Update "current" symlink, if appropriate
Where "appropriate" means if TftpDiVersion is set to current, which is
the default in standalone mode. The assumption is that someone wuth
that configration runs mg-debian-installer-update then they would
expected the update to be immediately effective.
There was some existing, but commented, code to do this update,
reinstate it with the correct condition and adjusting for the addition
of -$suite to the patch many moons ago.
There is no impact on any production configuration, since they always
set TftpDiVersion.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Dario Faggioli [Wed, 24 Jun 2015 16:11:22 +0000 (18:11 +0200)]
ts-logs-capture: some more info when capturing logs
more specifically:
* '-n' switch to xl info, to show the topology
of the host,
* 'xl vcpu-list', to show what the various vcpus
are up to.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Wei Liu [Fri, 12 Jun 2015 22:50:55 +0000 (23:50 +0100)]
ts-kernel-build: enable CONFIG_FHANDLE
According to Debian bug #764539, systemd needs this kernel feature to be
able to mount disks. Complete list of required kernel features can be
found in /usr/share/doc/systemd/README.gz.
Signed-off-by: Wei Liu <wei.liu2@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Wei Liu [Wed, 20 May 2015 17:56:28 +0000 (18:56 +0100)]
ts-xen-build-prep: reverse the test for installing libc6-dev-i386
Starting from wheezy, Debian introduced multiarch support, so we need to
install libc6-dev-i386 to build tools.
Since multiarch will be permanent, we reverse the test to not install
libc6-dev-i386 on releases older than wheezy (i.e. wheezy and jessie
will have that package).
Signed-off-by: Wei Liu <wei.liu2@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Wei Liu [Mon, 15 Jun 2015 11:07:38 +0000 (12:07 +0100)]
Cope with Jessie's d-i vg name
In Jessie the default vg name is changed to "$hostname-vg". Make that
default case and check for wheezy, squeeze and lenny for backward
compatibility.
Signed-off-by: Wei Liu <wei.liu2@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes in v4:
1. Add missing "use Osstest::Debian;".
Changes in v3:
1. Introduce a function to return vg name and use it.
Ian Campbell [Fri, 12 Jun 2015 10:24:11 +0000 (11:24 +0100)]
Force sysvinit with Jessie
Systemd doesn't honor osstest-confirm-booted service, which breaks
ts-leak-check. Fall back to SysV init for now.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
[wei: rewrite commit message and code comment] Signed-off-by: Wei Liu <wei.liu2@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Wei Liu [Tue, 16 Jun 2015 16:10:00 +0000 (17:10 +0100)]
mg-debian-installer-update: Also collect DTBs for standard kernels
For armhf and arm64 for Jessie we will need these in the normal case
as well as in the backports case. Arrange to download.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
[wei: fix rebase conflict] Signed-off-by: Wei Liu <wei.liu2@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Mon, 26 Jan 2015 16:31:35 +0000 (16:31 +0000)]
mg-debian-installer-update: updates to better handle Jessie onwards.
In mg-debian-installer-update:
- Expand the list of (suite,arch) combinations which don't exist and
move it to the top.
- Tweak the backports.org handling to allow it to be specified on a
per (suite,arch) basis, and specify it only for (wheezy,armhf)
since it is not currently needed for jessie. Make the
initramfs-tools fetching conditional too.
- Only extract backports-dtbs if they are present in the package.
- Handle arm64.
- Collect the correct armhf kernel for Jessie onwards (vexpress
removed from path).
Have mg-debian-installer-update-all fetch arm64 too.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
[wei: wrap lines] Signed-off-by: Wei Liu <wei.liu2@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Mon, 26 Jan 2015 16:08:43 +0000 (16:08 +0000)]
More flexible handling of need-kernel-deb-$flavour host flag
The current behaviour does not account for the need to apply this
workaround only for particular Debian releases, i.e. Midway will be
supported just fine in Jessie.
It also does not allow for kernels from alternative sources other than
backports (e.g. experimental, although that is hypothetical right
now).
The flavour name is actually irrelevant in practice (none of our
supported arch's need multiple kernel flavours at install time, which
is all we care about). So omit it from the new scheme.
The new scheme is "need-kernel-deb-$suite-$altsource", i.e.
need-kernel-deb-wheezy-backports. Possibly in the future e.g.
need-kernel-deb-jessie-experimental indicating that the experimental
kernel is needed on jessie.
mg-debian-installer-update is modified to download to the new name.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Also update TftpDiVersion, after having run
mg-debian-installer-update-all on the colo controller VM.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Thu, 18 Jun 2015 16:25:03 +0000 (17:25 +0100)]
ts-kernel-build: Support --reuse to keep same build tree
This is very useful when iterating over kernel configurations, since
it avoids blowing away the build tree and all the existing built
objects. The Linux build system does the right thing when .config
changes and only rebuilds the affected bits.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Tue, 16 Jun 2015 13:54:44 +0000 (14:54 +0100)]
sg-report-host-history: Move query preparation into jobquery
To get the scope of database locking, and of the SET LOCAL workaround,
right, it is convenient to move these query preparations into the
jobquery function.
No functional change.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Fri, 29 May 2015 15:26:16 +0000 (15:26 +0000)]
sg-report-host-history: Introduce new script
This script is still a work in progress. It does currently mostly
work and can generate a useful report. However, it needs:
* Some database locking (to avoid deadlock errors)
* A --flight= option to make it automatically report host usage
for all hosts touched in a flight, for the use of cr-*
* Hooking into cr-*
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v2: Improved commit message.
Fix a debug bug.
Actually install the html; do not leave it as .new
Ian Jackson [Wed, 17 Jun 2015 13:42:56 +0000 (13:42 +0000)]
reporting: sg-report-flight should ignore missing jobs
The function findaflight should not, when the caller specifies a job,
find a flight which does not contain that job at all.
One effect of allowing it to find such flights is that it might find a
bisection flight and try to use it as a basis for claiming a
regression, or as a justification for something not being a
regression, and then complain that all the missing steps in the
bisection flight are `blocked'.
This can be seen in the report for 58627:
test-amd64-i386-xl-qemuu-winxpsp3 6 xen-boot fail blocked in 56366-bisect
After this patch, a report generated for 58627 no longer mentions
56366 at all.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson [Fri, 29 May 2015 14:14:41 +0000 (14:14 +0000)]
sg-report-job-history: Cope if history too short
If there have been less than 99 relevant flights, the inner SELECT (to
determine the minimum flight number) would return NULL. And anything
> NULL is NULL and NULL is treated as false. So the host runvar
identification would break.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Campbell [Mon, 15 Jun 2015 08:36:02 +0000 (09:36 +0100)]
Arrange to upgrade microcode on x86 test hosts.
Both Xen and Linux support extracting a microcode update from an
initramfs early during boot. This requires prepending a suitable
uncompressed cpio archive containing the necessary files to the
initrd.
Xen also supports loading the microcode cpio from any multiboot
module, but for in order to allow the possibility of loading on native
boots (e.g. for build jobs) we prefer the prepend method.
This patch provides mg-cpu-microcode-update which creates a suitable
microcode cpio in the images directory and arranges for it to be added
to the host during installation (so it is done before the kernel is
installed and initramfs generated etc, saving faff). It also adds
"ucode=scan" to the Xen command line when necessary.
The version of initramfs-tools in Wheezy does not yet support
prepending things to the initrm, so we use a custom compression
command which sneakily does it for us. This could be done better from
Jessie onwards.
Note that Linux only supports this from v3.8 onwards, so this doesn't
work for Wheezy (which uses v3.2). From Jessie onwards we should
benefit from microcode updates even for native (build) jobs.
Tested on both Intel and AMD where it appeared to have the desired
effect under Xen. Under native I only tried Wheezy which doesn't
support early microcode.
Note that I've not bothered to implement late microcode updates, which
would work on v3.2 too, since that would need a different set of files
etc and it doesn't seem especially pressing.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Cc: Jan Beulich <JBeulich@suse.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Roger Pau Monne [Fri, 29 May 2015 10:38:18 +0000 (11:38 +0100)]
osstest: reduce FreeBSD install timeouts
Only the first block is expected to take longer (because it decompresses
the image and writes it to a LVM volume), the remaining commands should
execute much faster, so reduce the timeout.
Ian Campbell [Tue, 26 May 2015 11:36:00 +0000 (12:36 +0100)]
ts-libvirt-build: initscript should depend on xenstored service
In order that xencommons is guarenteed to have been started before
libvirtd. Otherwise sometimes libvirt can be started first resulting
in:
error: invalid argument: unsupported config type xen-xl
Because xen wasn't available when libvirt started.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Tue, 9 Jun 2015 09:54:00 +0000 (10:54 +0100)]
Ignore guest-stop failures for windows 7 tests.
As described in <1432284841.10746.136.camel@citrix.com> /
http://lists.xen.org/archives/html/xen-devel/2015-05/msg03016.html
Windows 7 does not appear to reliably actually shutdown when asked to
via the ACPI power button.
This test step has been failing forever until the "Toolstack::xl:
Support for ACPI fallback for shutdown" patch to osstest, and now it
passes only very infrequently, so we may as well ignore it.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Cc: Andrew Cooper <Andrew.Cooper3@citrix.com> Cc: Paul Durrant <paul.durrant@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>, Cc: Jan Beulich <JBeulich@suse.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
projects / osstest.git / log