Dario Faggioli [Tue, 9 Dec 2014 14:06:41 +0000 (15:06 +0100)]
ts-bench-hostcmp-post: add plotting facilities
in order to have an additional graph, comparing host and
guests performance when running kernbench.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
Dario Faggioli [Mon, 17 Nov 2014 18:01:36 +0000 (18:01 +0000)]
Kernbench perf comparison between host and guest
Recipes are defined for running kernbench on baremetal,
and on PV and HVM guests. Jobs making use of those recipes
are instantiated too.
Aim is making investigating performances loss due to
virtualization overhead easy and automatable.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
Dario Faggioli [Fri, 21 Nov 2014 12:09:07 +0000 (12:09 +0000)]
ts-bench-hostcmp-post: add plotting facilities
in order to have an additional graph, comparing host and
guests performance when running unixbench.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
Dario Faggioli [Thu, 6 Nov 2014 18:04:37 +0000 (19:04 +0100)]
Recipes and jobs for running unixbench both on host and guest
Recipes are defined for running unixbench on baremetal,
PV and HVM guests, with similar HW resources. Jobs making
use of those recipes are instantiated too.
Aim is making investigating performances loss due to
virtualization overhead easy and automatable.
In this case, rebooting the host is necessary (to
boot it baremetal, rather than on Dom0), and that makes
leak checks impossible. A mechanism is therefore introduced
for specifying, in sg-run-job, that for a particular job,
we don't want the leak checks.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
Dario Faggioli [Thu, 13 Nov 2014 17:45:49 +0000 (18:45 +0100)]
ts-bench-hostcmp-host-reset: new script
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
Dario Faggioli [Mon, 10 Nov 2014 17:20:33 +0000 (18:20 +0100)]
ts-bench-hostcmp-host-prep: new script
the goal is to run a benchmark both in a guest and
on baremetal, to investigate the performances loss
due to the virtualization overhead.
In order to help accomplishing this, the new script
introduced by this commit modifies the host's boot
configuration as follows:
- it makes it boot baremetal Linux, rather than Xen
and a Dom0 kernel;
- it limits the host's pcpus and amount of memory
to the values contained in the specific runvars
(if defined).
This is done under the assumption that the benchmark
will (or has been already) run on one (or more)
guest(s) too. If the runvars for limiting host's
resources are defined, it is assumed that they will
be (were) defined, and that they will have (had) the
same values, also when prepping the run of the
benchmark in the guest.
The test script only alter the host's boot config;
it is left to the caller to actually reboot the host,
and also to restore the old config, if wanted, after
the benchmark has been run.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
Dario Faggioli [Mon, 10 Nov 2014 12:57:43 +0000 (13:57 +0100)]
ts-bench-hostcmp-guest-prep: new script
the goal is to run a benchmark both in a guest and
on baremetal, to investigate the performances loss
due to the virtualization overhead.
In order to help accomplishing this, the new script
introduced by this commit modifies a guest's config
file in order for it to have the same number of
vcpus and (almost) the same amount of memory of the
underlying host. This is done under the assumption
that the benchmark will (or has been already) run
on the host too.
It is possible to make the guest have less vcpus
than the host has pcpus, and less memory than the
host, by defining two specific runvars. In fact, in
case of really "beefy" hosts, one may not want to
run the benchmark in an unrealistically large guest.
Of course, specific measures to also limit the host
resources, when running the benchmark on it, should
be taken, if comparing apples to apples is important.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
if defined, in the form of host properties. In standalone
mode, that should happen via the config file.
Methods are introduced to read those host properties or,
if they are not defined, to fetch the information by
querying the host directly.
The host properties always take precedence. This means
that, if they're defined, no command is run on the host,
and the values stored in the properties are used.
This commit also introduces a simple bash script that,
if run on the host, retrieves and prints such host
hardware properties, for convenience and/or testing.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
Recipes are defined for prepping and running kernbench
on the host and on Debian PV and HVM guests.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
Dario Faggioli [Tue, 9 Dec 2014 14:05:25 +0000 (15:05 +0100)]
ts-kernbench-reslts: process and plot bench results
Extract the data from the output of kernbench and produce
the tables, the gnuplot script and the plots.
All is saved in $stash, for the running flight and job.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
ts-unixbench-reslts: retrieve and stash kernbench results
in a file named according to the following convention:
$hostname--$benchname-$benchparams
i.e., something like this:
debian--kernbench-n2
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
ts-kernbench-run: kick off the benchmark on the target
There is a runvar called 'kernbench_params', for specifying
the benchmark's runtime arguments.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
ts-kernbench-build: prep the environment for running kernbench
by shipping the benchmark (it's just a script) and the linux
kernel sources to the target. The dependences installed are
the ones required to build a linux kernel, plus a few more
packages required by the kernbench script.
As for the unixbench equivalent, this accepts two parametrs,
in the form 'host=somehost someguest'. If only the first one is
provided, it must be 'host=somehost', and the script will prep
the host.
This also installs some packages so, if host sharing is to
be allowed (which, BTW, is highly *not* recommended), the
following patch from Ian Campbell is required:
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
mg-kernbench-download: new script for downloading kernbench
It downloads the benchmark (it's just a script) and a linux
kernel archive, necessary for running the benchmark itself,
and store them in c{Images}/benchs.
Default values for the repo URL and actual filename are embedded
in the script itself, and can be overridden as usual (e.g., via
standalone.config).
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
Dario Faggioli [Tue, 24 Jun 2014 14:53:12 +0000 (16:53 +0200)]
standalone-reset: introduce a new -t option
for making it possible to call the new make-bench-flight
script, and generating the benchmarking jobs. It can be
combined with the existing '-f' option, to create a
benchmarking flight containing all the benchmarking jobs.
This is generic, so, when passing '-t sometype', a script
called make-sometype-flight is what will be invoked.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
Changes from RFCv1:
* this into "standalone make-flight" too, as requested
during review.
Dario Faggioli [Tue, 24 Jun 2014 14:55:49 +0000 (16:55 +0200)]
make-bench-flight: to create a benchmarking flight
This is all done in a new script, to keep these jobs
separated from regular testing jobs defined by make-flight.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
Dario Faggioli [Tue, 24 Jun 2014 14:58:13 +0000 (16:58 +0200)]
sg-run-job: recipes for the unixbench jobs
Recipes are defined for prepping and running
the unixbench benchmark on the host and on
Debian PV and HVM guests.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
Dario Faggioli [Fri, 21 Nov 2014 00:20:58 +0000 (00:20 +0000)]
ts-unixbench-reslts: process and plot bench results
Mangle the results of a run of unixbench a bit, so that
they can be plotted. This also produces a (gnu)plot script
and the plot itself. All is saved in $stash, for the
running flight and job.
This is done in a new Osstest/Benchmarking.pm module, as
the functions introduced may turn out useful somewhere else
too.
The results are read from the original unixbench results
file and a new file, with basically a table in it is
produced. Gnuplot uses such file as data for the plotting.
A gnuplot script is produced and invoked (and saved in $stash)
rather than using the gnuplot perl binding because, this way,
one can modify the plotting commands a bit, and regen the
plot(s).
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
Dario Faggioli [Wed, 25 Jun 2014 01:40:06 +0000 (03:40 +0200)]
ts-unixbench-reslts: for retrieving the results
and store them in $c{Stash}, in a file named according
to the following convention:
$hostname--$benchname-$benchparams
i.e., something like this:
debian--unixbench-i3-c2
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
Changes from RFCv1:
* use target_getfile_root_stash, as suggested during review;
Dario Faggioli [Tue, 24 Jun 2014 14:55:01 +0000 (16:55 +0200)]
ts-unixbench-run: kick off the benchmark on the target
There is a runvar called 'unixbench_params', for specifying
the benchmark's runtime arguments.
The commit also adds a couple of generic functions in
TestSupport.pm, for `cat'-ing the content of a file on
the target into a corresponding file in the stash area.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
Changes from RFCv1:
* add default value for runtime arguments.
Dario Faggioli [Mon, 23 Jun 2014 15:17:18 +0000 (17:17 +0200)]
ts-unixbench-build: prep the environment for running unixbench
by installing some dependencies, shipping the archive, untaring
and building the sources.
This accepts two parametrs, in the form 'host=somehost someguest',
as most of the ts-guest-xxx scripts. If only the first one is
provided, it must be 'host=somehost', and the script will prep
the host.
As this installs some packages, troubles may arise if running
on a shered test host. This is not a too big deal, as hosts
used for benchmarking should not be shared anyway. In any case,
this patch from Ian Campbell will make the issue go away:
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
Changes from RFCv1:
* make selecthost() interact directly with @ARGV, as
requested during review;
* renamed to -build (from -prep), as suggested during review;
* use 'xaf' for automatic archive type detection, as suggested
during review.
Dario Faggioli [Tue, 24 Jun 2014 14:00:47 +0000 (16:00 +0200)]
mg-unixbench-download: new script for downloading the unixbench archive
The script fetches it, and saves it in c{Images}/benchs.
Default values for the repo URL and actual filename are embedded
in the script itself, and can be overridden as usual (e.g., via
standalone.config).
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
Changes from RFCv1:
* stop using ap-common for default values (for URL and remote
filename), as requested during review.
As an analogue to target_putfilecontents_[root_]stash().
(While at it, fix one whitespace damaged line.)
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
Changes from RFCv1:
* adding this was requested during review.
Dario Faggioli [Tue, 4 Nov 2014 17:18:07 +0000 (18:18 +0100)]
Guest setup: allow the amount of RAM to be a runvar
the value of which can be retrieved via guest_var('memory');.
This works for both PV and HVM Debian guests.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
Dario Faggioli [Thu, 13 Nov 2014 10:48:44 +0000 (11:48 +0100)]
Osstest/Debian.pm: fix identifying a Linux baremetal grub2 entry
In fact, in setupboot_grub2(), if we are interested
in a Linux baremetal entry, there is no point in
asking for the entry to contain an hypervisor line
("Hv").
Also, in such entry, Linux kernel and initrd are to be
found in "linux" and "initrd" lines, rather than in
"multiboot" and "module" ones.
I haven't actually checked whether also grub1 and/or
uboot needs fixing.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
ts-devbian-hvm-install: prune "cdrom:" from install sources
in sources.list, so installing packages in the guest with
apt-get does not stall waiting for the install CD to be
inserted.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Ian Campbell <Ian.Campbell@citrix.com> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Campbell [Fri, 21 Nov 2014 13:16:52 +0000 (13:16 +0000)]
Add simple helper to update DI for all architectures.
Uses DebianNonfreeFirmware, even (especially) for production, so move
the README stanza out of standalone only section. The current default
matches what is in the current production versions of DI.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Fri, 21 Nov 2014 13:16:51 +0000 (13:16 +0000)]
ts-kernel-build: enable CONFIG_IKCONFIG{_PROC}
This makes the kernel's .config available in /proc/config.gz and
embeds a copy which can be extracted with
linux/scripts/extract-ikconfig (which I've not tried, but have no
reason to doubt).
Having this around can be handy with an older osstest installed test
box and to confirm you've booted the kernel you think you have when
you are messing with .config options.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Thu, 20 Nov 2014 13:48:56 +0000 (13:48 +0000)]
linux-next tests: Use correct branch for baseline
Make cr-daily-branch honour an environment or setting variable
EXTRA_SGR_ARGS. In branch-settings.linux-next set it appropriately to
arrange that the linux-next test reports consider linux-linus tests as
interesting as well as just linux-next ones.
(We already use a flight from linux-linus for selecting the baseline
linux version.)
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Use a logarithmic scale. Cap the bonus at 12h rather than 5d/30 = 4h.
When we have previously failed, make sure we apply a reverse bonus,
rather than a penalty.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
v2: Fix cap to really be 12h not 12d.
Ian Jackson [Tue, 11 Nov 2014 19:30:38 +0000 (19:30 +0000)]
ts-hosts-allocate-Executive: Score for equivalent previous failures
Look to see whether the last run on any hosts which are equivalent to
the ones we're looking at, failed. This means that when host X is
failing and we are considering host Y which is equivalent to X, we
give Y a selection bonus.
This means that osstest will be less obsessive about sticking to the
very same failing host.
Also, no longer multiply by the number of previously-failing hosts;
instead, just consider whether _all_ the hosts are previously-failing
in their corresponding slots (*_host runvars). It doesn't make sense
to make the bonus for using failing hosts depend on the number of
hosts used by the test.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
v2: Improve commit message.
Ian Jackson [Thu, 13 Nov 2014 15:53:20 +0000 (15:53 +0000)]
ts-rumpuserxen-demo-xenstorels: set `on_poweroff="preserve"'
We rely on the domain existing after xenstore-ls's main has called
exit, so that we can do our own xenstore-ls in dom0 and check the
results.
Previously, this happened by accident because the rump kernel would,
after _exit, call a minios function which crashes the domain. New
rump kernels don't do this, and instead shut down cleanly.
Setting `on_poweroff="preserve"' has the desired effect.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
We have `install-append <suite>' but that goes before d-i's -- so only
applies during installation. Provide an option that applies
post-installation too.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Fri, 31 Oct 2014 11:20:04 +0000 (11:20 +0000)]
DhcpWatch::leases: Check errors
Check error returns from connect() et al (which present as an undef
return from IO::Socket::INET), and from read() (which can be detected
via IO::Handle::error).
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Campbell [Wed, 29 Oct 2014 10:41:30 +0000 (10:41 +0000)]
ts-host-install: Add force-mac-address host flag
This causes us to emit udev runes which force the device named in the host's
Interface_Force property to have the MAC address given in the host's Ether
property. This is the reverse of the behaviour of the Interface_Force property
without this flag, which is to rename the device with MAC address Ether to have
the requested name.
This is necessary because the arndale's USB Ethernet device has no burned in
MAC address and there is no mechanism to configure this e.g. via the bootloader
in a way which Linux will then notice.
Without this change Linux will pick a random mac address when loading the
driver, which breaks osstest's attempts to observe when the test host has
picked up its preseed file by watching the webserver logs, since DHCP will not
map a random mac address to the IP we are expecting.
This is only useful on systems which have exactly one Ethernet device,
otherwise there is no reliable way to determine which device is which.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Wed, 29 Oct 2014 10:41:28 +0000 (10:41 +0000)]
ts-host-install: include console before *and* after the -- marker.
As of v3.15 Linux now stop parsing its command line at the -- mark and passes
the remainder to init. This has broken Debian Installer's feature where
anything after the -- is propagated to the installed system.
See also https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=762007
To work around this we need to pass console= twice, once before the -- for the
current kernel's use and then again after the -- which is propagated to the
installed system.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Wed, 29 Oct 2014 10:41:27 +0000 (10:41 +0000)]
ts-host-install: Refactor code to construct a Linux console= for a host
And reuse it in when writing the u-boot script for initial boot. This was
previously hardcoded to ttyAMA which matches the production host DB for the
current set of hosts which use this script.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Wed, 29 Oct 2014 10:41:25 +0000 (10:41 +0000)]
mg-debian-installer-update: Add more modules when creating initrd overlay
The arndale platform has a usb network device. In addition both the SATA and
NIC/USB hardware on this platform requires additional regulator (power), clk
and phy modules, which in turn require i2c support.
Add everything which is needed on this platform.
Sort the list of drivers while here.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Campbell [Wed, 29 Oct 2014 10:41:24 +0000 (10:41 +0000)]
mg-debian-installer-update: Include any device tree blobs in the tftp dir
We will need to supply these for some arm boards.
We provide both an unpacked version, which can be referenced via u-boot's
support for the "fdtdir" property in pxelinux.cfg and a tarball which is more
convenient for installing into the target filesystem on host install
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Mon, 6 Oct 2014 16:05:32 +0000 (17:05 +0100)]
cr-for-branches: Keep many more logs
Keep 300 cycles of cr-for-branches.log.
Also, remove the spurious `500' from the command line, which looks
like it was there because I thought that `-n' took a number argument
(but in fact that's -c), whereas it was actually interpreted as a
filename (which never exists so savelog silently ignores it).
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Mon, 6 Oct 2014 15:54:55 +0000 (16:54 +0100)]
bisection: Do not get bisection start points from broken flights
cs-bisection-step ignores any flight with broken steps, on the grounds
that the results in such a flight maybe unreliable.
Therefore, if a main flight has both broken steps and regressions, it
is wrong to take its mro as the new basis mro for bisections. In the
worst case a newly failing job will be determined to be unbisectable
(since the only relevant failure is in a broken flight), have the
stamp file created, and then never be properly bisected.
So: in sg-report-flight, report `broken-step' for each broken step (in
the top-level flight we are reporting - broken jobs in flights we are
referring to to justify heisenbugs are OK); and, in cr-daily-branch,
do not install the new mro if such indications are present.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Fri, 3 Oct 2014 15:02:32 +0000 (16:02 +0100)]
cat-dhcp-leases: new script
ISC dhcpd treates /var/lib/dhcp3/leases as a log-structured database,
appending records to it and periodically rewriting it. When it
decides to rewrite it it writes the new contents to a temporary file -
fine.
But rather than renaming the temporary file into place, over the top
of the old file, using rename(2), it first renames the old leases file
to leases~. So briefly there is no leases file.
This means that the service used by Osstest/DhcpWatch/leases.pm when a
hostname and port is specified has to try various different files.
This is the appropriate shell script, which is most conveniently
maintained here in osstest. It can be used by putting something like
this in inetd.conf:
Ian Jackson [Thu, 2 Oct 2014 11:56:06 +0000 (12:56 +0100)]
sg-report-flight: Add a succcess/failure column header.
There are so many rows and columns that scrolling around trying to find the
column associated with a failure is getting pretty tricky. Finding all failing
columns is even harder.
Add a clue near the top: replicate the worst or most interesting step
result box contents and colour.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson [Tue, 19 Aug 2014 10:31:17 +0000 (11:31 +0100)]
Fix mg-hosts mkpxedir
* Have selecthost no longer demand that $job is set; if it is not,
simply do not set Suite (which of course ultimately comes from
runvars and hence the job).
* Add missing use Osstest::TestSupport.
* Mention the need to run it in README.dev.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Thu, 31 Jul 2014 14:42:56 +0000 (15:42 +0100)]
rump kernel tests: Get upstream from rumprun-xen
Replace https://github.com/rumpkernel/rumpuser-xen
with https://github.com/rumpkernel/rumprun-xen
as the source for the pre-test version. The repo has been renamed
upstream.
We do not change, at this time:
* The name of the post-test output repo on xenbits,
git://xenbits.xen.org/rumpuser-xen.git#xen-tested-master
Changing those would be a bit fiddly.
And we don't change the following, which contain `rumpuserxen':
* osstest `branch', `recipe', and `jobs' which are used to correlate
and specify aspects of the different production test runs of
different versions of the code. The branch name appears in the
Subject lines of email reports, and the job names in the body (and
as column names in the html test report matrix).
* The osstest `tree' names (which are used to specify and correlate
the different versions and urls of the `same' codebase in the
various tests). Also, we leave unchanged the corresponding
osstest tree plumbing environment variable names.
* The osstest build output identifiers from ts-rumpuser-xen-build
including the runvar name and the dist part name. (These are used
to communicate between the build and test jobs.)
Changing those would break cross-change comparison of test results and
osstest's automatic bisection across the change.
We also don't change:
* The names of the ts-rumpuserxen-* test scripts.
While this could be changed by renaming the files and editing
sg-run-job to match, it doesn't seem sensible to rename these while
leaving everything else unchanged.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Campbell [Wed, 16 Jul 2014 11:45:05 +0000 (12:45 +0100)]
make-flight: Enable qemu tests for 4.5 onwards
Stefano is about to enable qemu support on ARM, we should test it.
standalone-generate-dump-flight-runvars verifies that the affect of this change
is to add armhf jobs/runvars to qemu-mainline and qemu-upstream-unstable as
desired.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Cc: Stefano.Stabellini@eu.citrix.com Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Mon, 14 Jul 2014 12:08:58 +0000 (13:08 +0100)]
rump kernel tests: Run xenstore-ls demo
We can use the newly-generalised ts-rumpuserxen-demo-setup to set up
the config file etc. to run xenstore-ls from the xen.git build.
We have a bespoke script to actually execute it: we extract the actual
program output from the guest console log, and compare the results to
those we expect (based on running xenstore-ls in dom0).
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
v2: Fix copyright date in new script.
Ian Jackson [Tue, 15 Jul 2014 14:18:00 +0000 (15:18 +0100)]
ts-guest-destroy-hard: New script
This just calls xl destroy; it doesn't check that the guest is
running, or on the network, or anything. (It also doesn't check that
the guest is down afterwards.)
We are going to want this to be able to destroy the rumpuserxen
xenstorels domain, which is short-lived (and will normally have
terminated by the time we want to destroy it).
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v2: Fix copyright date in new script.
Ian Jackson [Mon, 14 Jul 2014 11:58:57 +0000 (12:58 +0100)]
rump kernel tests: Specify cmdline via runvar
The runvar <gn>_cmdline is used for the rumpuserxen guest command
line. Previously "3" (a binary flags word meaningful to the test/demo
program included in rumpuser-xen.git) was hardcoded.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson [Tue, 15 Jul 2014 14:40:08 +0000 (15:40 +0100)]
rump kernel tests: Specify kernel via runvar
The runvar <gn>_builtimage is set to <distpart>:<distpath>.
Previously we hardcoded `/usr/local/lib/xen/rump-kernel' in
`rumpuserxen'.
The ffs filesystem images, and the demo config template, still come
from a single particular build, which we find from the
`guests_rumpuserxenbuildjob' runvar. This runvar is then doing double
duty as the build job for the ffs filesystem images and example
configs, and the `default' build job for guest kernels.
The subpath /usr/local/lib/xen/rump-kernel is duplicated. It
now appears:
1. To find the demo guest's kernel: in make-flight, setting a runvar,
with the actual rump-kernel kernel image leafname appended.
2. For the filesystem images and config file: hardcoded in
ts-rumpuserxen-demo-setup.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v2: Fix demo application kernel image filename.
Disentangle wording of last para of commit message.
Ian Jackson [Mon, 14 Jul 2014 12:04:35 +0000 (13:04 +0100)]
rump kernel tests: Do a xen.git tools build
Having laid the groundwork, we can now:
* In make-flight, for the rumpuserxen builds, set the runvars that
specify which xen.git bits etc. to build. (We pass qemuu now,
because we intend that eventually we will want to build that too.)
* In ts-rumpuserxen-build, record the locations of the app-tools
configure and make wrappers in the runvars which will be honoured by
ts-xen-build. Thus ts-xen-build in rumpuserxen build jobs will use
these app-tools and do a rump cross build.
* Add a ts-xen-build step to the build-rumpuserxen recipe, passing
`tools' as the target to build.
ts-rumpuserxen-build does not generate a formal set of build outputs.
Instead, the ts-xen-build step references the files generated by
ts-rumpuserxen-build directly (and is told where to find them via the
new command prefix runvars).
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v2: Improve commit message.
Ian Jackson [Mon, 14 Jul 2014 11:37:43 +0000 (12:37 +0100)]
building: Delete when cloning, not when preparing $builddir
Make prepbuilddirs no longer delete $builddir; instead:
- have prepbuilddirs delete *-stamp files and dist/;
- have build_clone delete the specific source directories;
- any individual ts-*-build scripts that use other things
would have to delete them (but searching for $builddir
reveals none).
This makes it possible for later steps within a build job to look at
the build trees from previous steps. This is helpful if for any
reason it isn't easy or desirable to package up a formal set of build
outputs for reuse.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
v2: Provide motivation in the commit message.
Ian Jackson [Mon, 14 Jul 2014 12:07:30 +0000 (13:07 +0100)]
sg-run-job: Support toggling `adding' by repeating `+'
Specifying `+' as an argument to spawn-ts suppresses the automatic
addition of subsequent arguments to the testid. Make it possible to
toggle this by repeating `+'.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson [Fri, 11 Jul 2014 14:27:11 +0000 (15:27 +0100)]
ts-rumpuserxen-build: Cope with new layout, disabling bodges
The rumpsrc psuedo-submodule has been integrated with the nblibs
submodule and made into a proper submodule.
Also, in the intervening time, rumpuserxen/Config.mk now uses ?= when
setting XEN_HEADERS.
So if we detect that we have a new rumpuser-xen, we can disable:
- ad-hoc arrangements for rumpsrc
- grievous-bodge
- patching Config.mk
We consider a rumpuserxen old iff it has nblibs as a submodule and
lacks rumpsrc as a submodule.
We have to have a different name for the rumpsrc submodule (rather
than using rumpsrc), because in the new arrangements it needs to be
fetched from the location specified in rumpuserxen's .gitmodules
rather than our own configured location for the old rumpsrc. (And the
local name is in the runvars etc.)
The old runvars etc. for rumpsrc need to stay around to cope with old
rumpuserxen, for now, but are going to be obsolete. Leave some
comments about this in relevant places.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ian Jackson [Tue, 8 Jul 2014 10:27:59 +0000 (11:27 +0100)]
adhoc-revtuple-generator: Explicitly specify refs to fetch
Some of the git trees in $c{Repos} may have been created (eg by
earlier versions of git or earlier versions of osstest) without the
usual
remote.origin.fetch = +refs/heads/*:refs/remotes/origin/*
git config setting.
Pass this refspec explicitly, instead of relying on it being set as
the default.
Amongst other things, this will prevent a possible error during
bisection
git fetch -p origin
[stuff]
fatal: bad object 3e3d5f6146c3f684f4b7bd71feffc31e1c668754
which is currently appearing in our cron mail.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Mon, 30 Jun 2014 10:13:37 +0000 (11:13 +0100)]
executive-postgresql-schema: Schema dump
We really ought to have a proper schema script. But for now, include
the output of pg_dump --schema-only on the osstestdb, for the benefit
of anyone trying to set up a hosted (rather than standalone) instance
(or just trying to see how the code works).
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
v2: New patch in v2 of the series.
Ian Jackson [Fri, 27 Jun 2014 10:20:19 +0000 (11:20 +0100)]
make-flight: Test rump kernels only when needed
Introduce a new subroutine branch_wants_rumpkernel_tests and call it
before making the rumpuserxen build and test jobs.
The default is not to include a rumpuserxen test. We include it for
rumpuserxen itself, xen, linux (in case there is a frontend/backend
incompatibility), and osstest.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Fri, 27 Jun 2014 11:45:11 +0000 (12:45 +0100)]
mg-list-all-branches: New script.
The approach here is a little unpleasant but it has the virtue of
getting the ones out of crontab too, and of not depending on the
production database (which will in any case contain obsolete flights).
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
projects / people / dariof / osstest.git / log