Ian Campbell [Wed, 24 Jun 2015 08:23:16 +0000 (09:23 +0100)]
Debian: Do not add Debian to UEFI boot order on install
By default Debian inserts itself before any existing entries,
including the PXE one, meaning we otherwise cannot remotely regroove
the box.
Add a hook to /usr/lib/base-installer.d/ which installs a diversion of
/usr/sbin/grub-install and installs a wrapper which adds the
--no-nvram argument which prevents grub-install from adding the boot
entries.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v4: Die on !jessie
v3: New patch replacing "Debian: Fixup UEFI boot order during install"
Ian Campbell [Thu, 21 May 2015 09:57:41 +0000 (10:57 +0100)]
ts-host-install: Set dtbs in the non-special kernel case too.
When DTB support was added for armhf it required a backports kernel
for use with Wheezy and therefore DTBs were only collected for that
special case. However as we move forward to newer kernels it should
be possible to run using the proper kernel (e.g. Jessie should work
for armhf). Therefore we will need the DTBs for those kernels too.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v3: Improved commit message
Ian Campbell [Wed, 20 May 2015 15:58:16 +0000 (16:58 +0100)]
ts-kernel-build: Additional kernel options for Mustang
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Note: ARM testing currently uses a 3.16 based linux-arm-xen kernel
(compared to 3.14 currently used as the default). In order to support
Mustang something newer is needed, I've been testing 4.1 (next LTS
kernel), 3.18 (the intervening LTS kernel) might be sufficient but I
haven't tried it. We should plan to either update linux-arm-xen or the
default kernel version at some point before we get Mustang h/w.
v3: Set CONFIG_NET_XGENE (was commented out before)
Remove setting of CONFIG_SPARSEMEM_VMEMMAP
Ian Campbell [Thu, 21 May 2015 12:58:27 +0000 (13:58 +0100)]
Enable chain loading to local disk for UEFI PXE systems.
First arrange for bootloader to be installed to removable media path,
by using a new in Jessie preseed option. Then use that to chainload a
bootloader from the disk.
The removable media path is well known (part of the UEFI spec) which
saves us having to worry about which OS is on the host (so long as the
OS can be configured to populate the removable media path)
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Tue, 27 Jan 2015 12:06:41 +0000 (12:06 +0000)]
ts-host-install: Support UEFI PXE boot using grub.efi
Handling of chainloading from the local hdd in
setup_pxeboot_local_uefi will be added in the next patch.
(Wrap DiBase entry in README too for consistency)
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v4: Turn grub "echo todo" into a die.
v3: Reindent README (fix DiBase too for consistency)
Adjust for reordering of setup_{firmware}_pxeboot to put
{firmware} at end
Rewrap setup_grub_efi_bootcfg
Adjust for setup_pxeboot -> setup_pxeboot_di
Ian Campbell [Thu, 21 May 2015 12:46:24 +0000 (13:46 +0100)]
Refactor pxelinux configuration
The mechanism used to PXE boot can differ depending on the firmware
type. Therefore refactor into Osstest::TestSupport and key off a new
host property "firmware".
Currently supported is "bios" (the default) and "uboot", both of which
use pxelinux.cfg style files.
The default for the firmware property is "bios", hence no change for
any existing system (including those which use uboot, regardless of
whether they are now configured with the firmware proprty or not)
As part of this "setup_pxeboot" becomes "setup_pxeboot_di" since it is
somewhat d-i specific.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v3: reorder setup_${firmware}_pxeboot to setup_pxeboot_${firmware}
join $dicmd in setup_pxeboot instead of caller
Ian Campbell [Thu, 21 May 2015 11:49:55 +0000 (12:49 +0100)]
ts-host-install: split the "di" from the "host" command line
(i.e. the bit before/after the -- marker). When abstracting over
different bootloaders in a future patch this will be convenient since
it allows the code to add to either.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Thu, 21 May 2015 11:46:16 +0000 (12:46 +0100)]
ts-host-install: Split initrd out of @installcmdline
Other bootloaders handle this with an explicit separate option rather
than parsing it out of the command line as pxelinux does. Prepare for
supporting these.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Tue, 27 Jan 2015 10:07:21 +0000 (10:07 +0000)]
Introduce mg-pxe-loader-update
The story for PXE booting via UEFI (at least on arm64) is not so
straightforward as with pxelinux on x86. There seems to no good
bootloader to launch via UEFI+pxe, in fact all I could find was grub
(syslinux, and by extension pxelinux.efi, is x86 only).
Add mg-pxe-loader-update modelled on mg-debian-installer-update which
will download the necessary grub binaries and produce a grub image
which can be used to pxe boot.
grub lacks the convenient ability to search for config file based on
(substrings of) the MAC or IP address. So we arrange for the grub.cfg
in TftpGrubBase to chain load another config file from
TftpTmpDir/'$net_default_mac' where $net_default_mac is a grub
variable which is substituted at boot time.
Actually using this requires that bootp/dhcp provide a next-file so
UEFI knows what to boot (usually this would be pxelinux.0 which we
can't use here). Locally we have configured this as $name/pxe.img, so
we can use different loaders.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v4: Just mgi-common now that mgi-debian merged therein.
v3: Use mgi-common and mgi-debian
Ian Campbell [Fri, 19 Jun 2015 10:56:15 +0000 (11:56 +0100)]
mg-*: Make Debian package fetching common in mgi-common
This pattern appears several times in mg-debian-installer-update and
I'm about to add another script which uses it.
The centralised version always does "sort -n -r | head -n" to pick out
only the newest version, which only some of the open coded sites did.
Likewise the centralised version checks that $pkgfile is non-empty,
unlike some of the open coded sites.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v4: Fold into mgi-common instead
v3: New patch
Which has caught out one or two people using standalone mode.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Reviewed-and-Tested-by: Dario Faggioli <dario.faggioli@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Dario Faggioli [Fri, 26 Jun 2015 10:58:51 +0000 (12:58 +0200)]
Stop testing SEDF, start testing RTDS
the SEDF scheduler is about to be deprecated and go
away (see [1]). OTOH, the RTDS scheduler is here to
stay.
It therefore makes sense to stop smoke testing the
former in favour of the latter. Note that the
-sedf-pin jobs where only added in order to try to
debug a long standing issue with SEDF; it is not
necessary to have anything like that for RTDS.
For now, as RTDS is still marked as experimental,
test failures are allowed, as it is for SEDF.
Also, RTDS was introduced in Xen 4.5, so add the
test only for the relevant brnaches. Similarly,
SEDF is removed in Xen 4.6, so keep testing it in
older branches.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Fri, 26 Jun 2015 10:31:31 +0000 (11:31 +0100)]
mg-debian-installer-update: Update "current" symlink, if appropriate
Where "appropriate" means if TftpDiVersion is set to current, which is
the default in standalone mode. The assumption is that someone wuth
that configration runs mg-debian-installer-update then they would
expected the update to be immediately effective.
There was some existing, but commented, code to do this update,
reinstate it with the correct condition and adjusting for the addition
of -$suite to the patch many moons ago.
There is no impact on any production configuration, since they always
set TftpDiVersion.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Dario Faggioli [Wed, 24 Jun 2015 16:11:22 +0000 (18:11 +0200)]
ts-logs-capture: some more info when capturing logs
more specifically:
* '-n' switch to xl info, to show the topology
of the host,
* 'xl vcpu-list', to show what the various vcpus
are up to.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Wei Liu [Fri, 12 Jun 2015 22:50:55 +0000 (23:50 +0100)]
ts-kernel-build: enable CONFIG_FHANDLE
According to Debian bug #764539, systemd needs this kernel feature to be
able to mount disks. Complete list of required kernel features can be
found in /usr/share/doc/systemd/README.gz.
Signed-off-by: Wei Liu <wei.liu2@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Wei Liu [Wed, 20 May 2015 17:56:28 +0000 (18:56 +0100)]
ts-xen-build-prep: reverse the test for installing libc6-dev-i386
Starting from wheezy, Debian introduced multiarch support, so we need to
install libc6-dev-i386 to build tools.
Since multiarch will be permanent, we reverse the test to not install
libc6-dev-i386 on releases older than wheezy (i.e. wheezy and jessie
will have that package).
Signed-off-by: Wei Liu <wei.liu2@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Wei Liu [Mon, 15 Jun 2015 11:07:38 +0000 (12:07 +0100)]
Cope with Jessie's d-i vg name
In Jessie the default vg name is changed to "$hostname-vg". Make that
default case and check for wheezy, squeeze and lenny for backward
compatibility.
Signed-off-by: Wei Liu <wei.liu2@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Changes in v4:
1. Add missing "use Osstest::Debian;".
Changes in v3:
1. Introduce a function to return vg name and use it.
Ian Campbell [Fri, 12 Jun 2015 10:24:11 +0000 (11:24 +0100)]
Force sysvinit with Jessie
Systemd doesn't honor osstest-confirm-booted service, which breaks
ts-leak-check. Fall back to SysV init for now.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
[wei: rewrite commit message and code comment] Signed-off-by: Wei Liu <wei.liu2@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Wei Liu [Tue, 16 Jun 2015 16:10:00 +0000 (17:10 +0100)]
mg-debian-installer-update: Also collect DTBs for standard kernels
For armhf and arm64 for Jessie we will need these in the normal case
as well as in the backports case. Arrange to download.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
[wei: fix rebase conflict] Signed-off-by: Wei Liu <wei.liu2@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Mon, 26 Jan 2015 16:31:35 +0000 (16:31 +0000)]
mg-debian-installer-update: updates to better handle Jessie onwards.
In mg-debian-installer-update:
- Expand the list of (suite,arch) combinations which don't exist and
move it to the top.
- Tweak the backports.org handling to allow it to be specified on a
per (suite,arch) basis, and specify it only for (wheezy,armhf)
since it is not currently needed for jessie. Make the
initramfs-tools fetching conditional too.
- Only extract backports-dtbs if they are present in the package.
- Handle arm64.
- Collect the correct armhf kernel for Jessie onwards (vexpress
removed from path).
Have mg-debian-installer-update-all fetch arm64 too.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
[wei: wrap lines] Signed-off-by: Wei Liu <wei.liu2@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Mon, 26 Jan 2015 16:08:43 +0000 (16:08 +0000)]
More flexible handling of need-kernel-deb-$flavour host flag
The current behaviour does not account for the need to apply this
workaround only for particular Debian releases, i.e. Midway will be
supported just fine in Jessie.
It also does not allow for kernels from alternative sources other than
backports (e.g. experimental, although that is hypothetical right
now).
The flavour name is actually irrelevant in practice (none of our
supported arch's need multiple kernel flavours at install time, which
is all we care about). So omit it from the new scheme.
The new scheme is "need-kernel-deb-$suite-$altsource", i.e.
need-kernel-deb-wheezy-backports. Possibly in the future e.g.
need-kernel-deb-jessie-experimental indicating that the experimental
kernel is needed on jessie.
mg-debian-installer-update is modified to download to the new name.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Also update TftpDiVersion, after having run
mg-debian-installer-update-all on the colo controller VM.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Thu, 18 Jun 2015 16:25:03 +0000 (17:25 +0100)]
ts-kernel-build: Support --reuse to keep same build tree
This is very useful when iterating over kernel configurations, since
it avoids blowing away the build tree and all the existing built
objects. The Linux build system does the right thing when .config
changes and only rebuilds the affected bits.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Tue, 16 Jun 2015 13:54:44 +0000 (14:54 +0100)]
sg-report-host-history: Move query preparation into jobquery
To get the scope of database locking, and of the SET LOCAL workaround,
right, it is convenient to move these query preparations into the
jobquery function.
No functional change.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Fri, 29 May 2015 15:26:16 +0000 (15:26 +0000)]
sg-report-host-history: Introduce new script
This script is still a work in progress. It does currently mostly
work and can generate a useful report. However, it needs:
* Some database locking (to avoid deadlock errors)
* A --flight= option to make it automatically report host usage
for all hosts touched in a flight, for the use of cr-*
* Hooking into cr-*
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v2: Improved commit message.
Fix a debug bug.
Actually install the html; do not leave it as .new
Ian Jackson [Wed, 17 Jun 2015 13:42:56 +0000 (13:42 +0000)]
reporting: sg-report-flight should ignore missing jobs
The function findaflight should not, when the caller specifies a job,
find a flight which does not contain that job at all.
One effect of allowing it to find such flights is that it might find a
bisection flight and try to use it as a basis for claiming a
regression, or as a justification for something not being a
regression, and then complain that all the missing steps in the
bisection flight are `blocked'.
This can be seen in the report for 58627:
test-amd64-i386-xl-qemuu-winxpsp3 6 xen-boot fail blocked in 56366-bisect
After this patch, a report generated for 58627 no longer mentions
56366 at all.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson [Fri, 29 May 2015 14:14:41 +0000 (14:14 +0000)]
sg-report-job-history: Cope if history too short
If there have been less than 99 relevant flights, the inner SELECT (to
determine the minimum flight number) would return NULL. And anything
> NULL is NULL and NULL is treated as false. So the host runvar
identification would break.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Campbell [Mon, 15 Jun 2015 08:36:02 +0000 (09:36 +0100)]
Arrange to upgrade microcode on x86 test hosts.
Both Xen and Linux support extracting a microcode update from an
initramfs early during boot. This requires prepending a suitable
uncompressed cpio archive containing the necessary files to the
initrd.
Xen also supports loading the microcode cpio from any multiboot
module, but for in order to allow the possibility of loading on native
boots (e.g. for build jobs) we prefer the prepend method.
This patch provides mg-cpu-microcode-update which creates a suitable
microcode cpio in the images directory and arranges for it to be added
to the host during installation (so it is done before the kernel is
installed and initramfs generated etc, saving faff). It also adds
"ucode=scan" to the Xen command line when necessary.
The version of initramfs-tools in Wheezy does not yet support
prepending things to the initrm, so we use a custom compression
command which sneakily does it for us. This could be done better from
Jessie onwards.
Note that Linux only supports this from v3.8 onwards, so this doesn't
work for Wheezy (which uses v3.2). From Jessie onwards we should
benefit from microcode updates even for native (build) jobs.
Tested on both Intel and AMD where it appeared to have the desired
effect under Xen. Under native I only tried Wheezy which doesn't
support early microcode.
Note that I've not bothered to implement late microcode updates, which
would work on v3.2 too, since that would need a different set of files
etc and it doesn't seem especially pressing.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Cc: Jan Beulich <JBeulich@suse.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Roger Pau Monne [Fri, 29 May 2015 10:38:18 +0000 (11:38 +0100)]
osstest: reduce FreeBSD install timeouts
Only the first block is expected to take longer (because it decompresses
the image and writes it to a LVM volume), the remaining commands should
execute much faster, so reduce the timeout.
Ian Campbell [Tue, 26 May 2015 11:36:00 +0000 (12:36 +0100)]
ts-libvirt-build: initscript should depend on xenstored service
In order that xencommons is guarenteed to have been started before
libvirtd. Otherwise sometimes libvirt can be started first resulting
in:
error: invalid argument: unsupported config type xen-xl
Because xen wasn't available when libvirt started.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Tue, 9 Jun 2015 09:54:00 +0000 (10:54 +0100)]
Ignore guest-stop failures for windows 7 tests.
As described in <1432284841.10746.136.camel@citrix.com> /
http://lists.xen.org/archives/html/xen-devel/2015-05/msg03016.html
Windows 7 does not appear to reliably actually shutdown when asked to
via the ACPI power button.
This test step has been failing forever until the "Toolstack::xl:
Support for ACPI fallback for shutdown" patch to osstest, and now it
passes only very infrequently, so we may as well ignore it.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Cc: Andrew Cooper <Andrew.Cooper3@citrix.com> Cc: Paul Durrant <paul.durrant@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>, Cc: Jan Beulich <JBeulich@suse.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Roger Pau Monne [Thu, 28 May 2015 15:59:08 +0000 (16:59 +0100)]
FreeBSD: use custom image containing atkbd fix
Recent failures are due to a bug in x86emu when trying to call into
the BIOS in order to get the repeat rate and keyboard frequency. This
image contains the following fix, which should prevent the issue by
not calling into the BIOS anymore:
Ian Campbell [Wed, 6 May 2015 10:41:11 +0000 (11:41 +0100)]
grub: remove patch to disable submenu from 20_linux_xen overlay
setupboot_grub2 now supports submenus, so we can reduce our delta vs
upstream a bit.
I started by extracting 20_linux_xen from
http://snapshot.debian.org/archive/debian/20130703T094657Z/pool/main/g/grub2/grub-common_1.99-27%2Bdeb7u2_amd64.deb
and then applying the patch at
http://savannah.gnu.org/file/grub.patch?file_id=32276 (the patch from
grub bug #42420 at http://savannah.gnu.org/bugs/?43420) and
reinstating the comment at the top of the file (modified to drop the
reference to the Debian bug.
This left me with some spurious changes:
@@ -93,7 +93,7 @@ linux_entry ()
if test ! -e "${xen_dirname}/${xenpolicy}" ; then
return
fi
- xen_args=`echo $xen_args flask=enforcing`
+ xen_args=`echo $xen_args flask_enabled=1 flask_enforcing=1`
if ${recovery} ; then
title="$(gettext_quoted "%s, with Xen %s (XSM enabled) and Linux %s (recovery mode)")"
else
@@ -137,7 +137,6 @@ EOF
echo '$message'
module ${rel_dirname}/${xenpolicy}
EOF
- fi
cat << EOF
}
EOF
I think these are bugs in the patch in the grub BTS, which were fixed
while iterating over the XSM series in osstest but didn't make it into
the upstream version, the fixes to those bugs are reverted byu the
above. So I have manually reverted them.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com> Cc: Wei.Lui@citrix.com Cc: longtaox.pang@intel.com
longtao.pang [Tue, 26 May 2015 09:08:19 +0000 (17:08 +0800)]
Parsing grub which has 'submenu' primitive
Now auto-gen kernel grub2 config file's boot menu entries can have
2-level hierarchy, containing 'submenu' primitive, which is comprised by
several sub-menuentries. Xen boot entries are grouped into such kind of
'submenu' block. This patch adds setupboot_grub2() ability to handle
such new grub.cfg format
Signed-off-by: longtao.pang <longtaox.pang@intel.com> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
[ ijc -- minor coding style fixup and dropped comment made obsolete by
this change ]
longtao.pang [Wed, 13 May 2015 03:36:40 +0000 (11:36 +0800)]
Move the code for setting memory size into prep()
Move the code for setting memory size from outside into prep(),
where these $ram_mb operations are only used; And in next patch,
for L2 guest installation, $ram_mb will be overridden by a runvar value,
which needs $gho in scope.
This is pure code motion. No functional change.
Signed-off-by: longtao.pang <longtaox.pang@intel.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
longtao.pang [Wed, 13 May 2015 03:36:39 +0000 (11:36 +0800)]
Edit some APIs in TestSupport.pm for nested test
1. If vif model is defined for guest, use it in guest's configuration
2. In L2 installation context, its host (L1) shall not have a static IP
but stored in runvar in L1 installation before; use this stored runvar
IP.
Signed-off-by: longtao.pang <longtaox.pang@intel.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
[ ijc -- resolved conflict with addition of ${viftype}, made
${vifmodel} a prefix rather than postfix thing for
consistency with ${viftype}. ]
longtao.pang [Wed, 13 May 2015 03:36:38 +0000 (11:36 +0800)]
Refactor installation of overlays
Apply overlay to guests as well as hosts by moving to preseed_base.
We do this because we want to be able to:
* Use '/etc/init.d/osstest-confirm-booted' for guests as
well as hosts (in particular when testing nestedhvm
we sometimes want to treat a guest as a host).
* Use grub's 20_linux_xen which is harmless unless Xen is
installed in the guest which is the case only for
nestedhvm testing when we want these changes.
The other things in the overlay are some initscripts
(xenbridge and xenlightdaemons) which are not enabled for guests and
hence are therefore harmless.
Signed-off-by: longtao.pang <longtaox.pang@intel.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson [Wed, 20 May 2015 12:43:39 +0000 (13:43 +0100)]
Revert "cs-bisection-step: allow -bisect blessed flights for basis pass"
If the bisector is allowed to consider its own output flights as
candidates for the basis pass, then it will (generally) restart with a
new basis each time it finds a pass.
This is very slow and wasteful.
There is not much explanation in 01edca47 of the change I am now
reverting, but I think I probably created by some semi-manual process
a flight to serve as the basis. I now think that my mistake was to
bless that flight `adhoc-bisect' or some such, rather than `adhoc'.
Ian Jackson [Wed, 20 May 2015 12:36:15 +0000 (13:36 +0100)]
cs-bisection-step: Do not treat repro attempts as flail
The need_repro machinery deliberarely makes attempts to reproduce
various results.
This can cause the flail detector to trigger when not intended. In
particular, the bisector may have (for some reason[1]) restarted with
a new baseline, and the temporarally-stripy pass/fail requirement
would then require the basis fail to be repro'd, again.
[1] Currently this happens much more often than is desirable. This
will be fixed in a moment.
Fix this by only considering, for the purposes of flail, flights which
are no older than the first repro (the basis pass).
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson [Wed, 20 May 2015 11:30:19 +0000 (12:30 +0100)]
Revert "cs-bisection-step: Abandon repro attempts after a bit"
This safety catch is unnecessary and unhelpful.
It is unnecessary because 489773b4 "Detect flailing" will detect
attempts by the bisector to repeatedly run the same flight and hope
for different results.
It is unhelpful because it can happen for good reasons that a
particular revision has been tested many times. In particular:
- The osstest push gate input tree may have not been advanced for a
long time and been failing its push gate.
- The bisector may have (for some reason[1]) restarted with a new
baseline, and the temporarally-stripy pass/fail requirement would
then require the basis fail to be repro'd, again.
[1] Currently this happens much more often than is desirable. This
will be fixed in a moment.
Ian Jackson [Wed, 20 May 2015 12:11:41 +0000 (13:11 +0100)]
cs-bisection-step: Flail detection: look only at our blessing
When looking for identical previously flights, consider only ones
which have the same blessing as our prospective flight will have.
There are good reasons why apparently identical flights might appear
with other in-scope blessings: notably, a single-test-job main branch
might produce many failures from its push gate, which would all have
the main branch blessing (rather than the bisector's blessing).
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson [Wed, 20 May 2015 11:49:05 +0000 (12:49 +0100)]
cs-bisection-step: Refer to jobs we create just by job name
When we make a fresh build job, rather than referring to an existing
job in another flight, pass to the rest of the machinery only the job
name, not <flight>.<job>.
This means that the generated flight refers to its own jobs without
specifying the flight number. This allows the flail detector to
operate properly: without this, we might have repeated attempts to
test and build the same thing, but they would look identical because
their self-referential runvars would be different due to their
different flight numbers.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson [Fri, 15 May 2015 16:54:03 +0000 (16:54 +0000)]
ts-host-install: New --poweron-test-only option
We are having a difficulty with one of the test boxes, which can be
most easily reproduced by running ts-host-install to power cycle the
box and then see if it wakes up enough to fetch a preseed file.
Keep this mode of operation in tree in case it's useful in future.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson [Fri, 15 May 2015 17:03:01 +0000 (18:03 +0100)]
FreeBSD: use custom image containing BSD netfront bugfix
Roger has supplied these images which contain a fix for a netfront bug
in FreeBSD: it would send the gratuitous ARP before netback was ready,
so it could get lost, leading to stochastic migration failures:
https://svnweb.freebsd.org/base?view=revision&revision=282908
Here is the runvar diff:
- test-amd64-i386-freebsd10-amd64 freebsd_image FreeBSD-10.1-RELEASE-amd64.raw.xz
- test-amd64-i386-freebsd10-i386 freebsd_image FreeBSD-10.1-RELEASE-i386.raw.xz
+ test-amd64-i386-freebsd10-amd64 freebsd_image FreeBSD-10.1-CUSTOM-amd64-20150518.raw.xz
+ test-amd64-i386-freebsd10-i386 freebsd_image FreeBSD-10.1-CUSTOM-i386-20150518.raw.xz
I have confirmed that these are the right filenames.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Campbell [Fri, 15 May 2015 18:54:36 +0000 (19:54 +0100)]
Debian: correct u-boot command to load flask policy
The use of the $flaskpolocy variable is escaped only once, meaning it
is interpreted by the shell on the test host, a context in which it
is not set (it is set in Perl, and it would be set in u-boot env by
the line above). The symptom was that the ext2load command got given
no file to load and so tries to load the default "pxelinux.0":
May 14 20:47:47.108243 ## Executing script at 43100000
May 14 20:47:47.111115 Loading dtbs/sun7i-a20-cubietruck.dtb
May 14 20:47:47.114489 22007 bytes read in 114 ms (188.5 KiB/s)
May 14 20:47:47.252237 820116 bytes read in 114 ms (6.9 MiB/s)
May 14 20:47:47.396931 Loaded xen-4.6-unstable to 0x41000000 (c8394)
May 14 20:47:47.400989 command line: [...]
May 14 20:47:47.413380 4963576 bytes read in 214 ms (22.1 MiB/s)
May 14 20:47:47.653005 Loaded vmlinuz-3.16.7-ckt4+ to 0x42000000 (4bbcf8)
May 14 20:47:47.657999 command line: [...]
May 14 20:47:47.671376 12501593 bytes read in 510 ms (23.4 MiB/s)
May 14 20:47:48.203299 Loaded initrd.img-3.16.7-ckt4+ to 0x43300000 (bec259)
May 14 20:47:48.208406 ** File not found pxelinux.0 **
May 14 20:47:48.294291 Loaded xenpolicy-4.6-unstable to 0x41200000 (bec259)
The filesize in the log message remains the same as the initrd because
nothing is actually loaded so the variable isn't updated.
Rather than adding additional backticks to quote it to be interpreted
by u-boot, remove the quoting so it is interpreted by Perl, making
the setting of the u-boot var (which in any case used the wrong
syntax) unnecessary. This matches what we do for the kernel etc too.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Cc: Wei Liu <Wei.Liu2@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Wed, 11 Jun 2014 20:38:27 +0000 (21:38 +0100)]
ts-xen-build-prep: mkfs a new /home/osstest, don't resize2fs
Online resize is 40x slower than mkfs. It appears that the
backgrounded resize2fs can starve build tasks of IO bandwidth.
So instead, use mkfs to make a new filesystem for /home/osstest.
We use rsync to copy in the old contents.
For convenience of (a) review (b) possible reversion, we keep (for
now) the lvextend machinery. So we create a new 1-extent LV for the
lvextend machinery to work on.
But we don't call resize2fs when we extend it, because now it doesn't
have a fs on it yet. We make the filesystem later.
We move the ccache_setup until after this is done because it's a bit
pointless to put things in the to-be-removed /home/osstest when they
could be put in the new one after it had been set up.
We take slight care to make the rune slightly idempotent: if it
completed successfully we detect this and do not run it again. But if
it didn't, things may be messed up and running it again is unlikely to
help and may make things worse.
I have tested this on rice-weevil and the whole new target command
(including rsync, mkfs, mount etc.) takes 126s.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson [Fri, 15 May 2015 10:31:35 +0000 (11:31 +0100)]
i18n/l10n: Make Timezone configurable and change the default
* Introduce a new config option Timezone
* Replace hardcoded Europe/London everywhere with $c{Timezone}
* The default is UTC
* But in production-config-cambridge set it to Europe/London
The overall effect is:
* No change in Cambridge
* Default timezone changes to UTC but can now be overridden
* Production instance timezone changes to UTC
(It appears that there is no reasonable way to find out the Olson TZ
name of the controller host's default timezone. If there were, or we
discover one, we should arrange that the default is set
appropriately.)
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson [Thu, 14 May 2015 17:12:30 +0000 (18:12 +0100)]
production-config: Use /home/logs, not /home/osstest/pub
The logs and images (including .../logs, .../results, etc.) are now on
their own filesystem on the production osstest VM, which I have called
/home/logs.
Changing this in production config will allow us to tidy up by
removing the symlink I left behind.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
projects / people / liuw / osstest.git / log