Add a weekly coverity flight

This primarily consists of ts-coverity-{build,upload} and
make-coverity-flight which constructs the sole job.

The branch is named "xen-unstable-coverity" which matches various xen*
in the cr-* scripts. Places which needed special treatement are
handled by matching xen-*-coverity, which leaves the possibility of
xen-4.7-testing-coverity etc in the future, but note that care would
be needed so that coverity's tracking of new vs existing issues would
likely be confused by uploading different branches without
differentiating somehow (I don't know how this is supposed to work).

The most recently scanned revision is pushed to a new
coverity-scanned/master branch in the usual xen.git, tests are run on
the master branch.

I initially thoughts that $c{CoverityEmail} would need to be an actual
account registered with scan, however a manual experiment using
email=security@xen.org was accepted by the service. An "analysis
complete" message was sent to security@ while individual results mails
were sent to each member of the coverity project who was configured to
receive them. I think this is what we want. The "analysis complete"
mail contained no sensitive data, but also no real information other
than "success" (or presumably "failure" if that were to be the case).
I think going to security@ is probably OK.

The upload URL defaults to a dummy local URL, which will fail (it
would be possible in principal to put a stunt CGI there though). When
run with "cr-daily-branch --real" (i.e. in full on production mode)
then this is set instead to the value of CoverityUploadUrl from the
config (production-config etc). This means that adhoc and play runs
still exercise all the code (but the curl will fail) while --real runs
upload to a site-configurable location. (Note that the URL includes
the coverity project name, which would likely differ for different
instances).

I have run this via cr-daily-branch --real on the production infra
and it did upload as expected (flight 80516). Since
master==coverity-tested/master at this point it came out as a baseline
test which didn't attempt ap-push, which I would have expected to fail
anyway since it was running as my user in the colo which cannot push
to osstest@xenbits.

In my experiments the curl command took ~35 minutes to complete (rate
in the 100-200k range). Not sure if this is a problem, but use curl
--max-time passing it an hour to bound things. Note that curl is run
on the controller (via system_checked).  timeout etc.

Note that the token must be supplied with </path/to/token and not
@/path/to/token. The latter appears to the server as a file upload
rather than a text field in a form which doesn't work. In early
attempts I thought that the trailing \n in /path/to/token might be an
issue and hence wrote a big comment. However having discovered < vs @
I am no longer 100% sure that is the case, but I left the comment
anyway since I can observe on the wire that the \n is included in the
upload (but each test takes ~35 mins and there is a ratelimit on the
server side too).

A final niggle is that the descripton field in the web ui ends up as:
    80516:\ git://xenbits.xen.org/xen.git\ 9937763265d9597e5f2439249b16d995842cdf0
(i.e. spaces are \ escaped). I've confirmed with curl --trace-ascii
the the uploaded data is not escaped (this is from an earlier attempt
which did not include the flight number):

009a: Content-Disposition: form-data; name="description"
00ce:
00d0: git://xenbits.xen.org/xen.git 9937763265d9597e5f2439249b16d99584
0110: 2cdf0f

Due to the limitations on the numbers of uploads I've not experimented
with possible fixes yet (e.g. URL escaping the upload). Worst case we
either live with it or adjust the syntax to avoid the problematic
characters.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>

diff --git a/ap-fetch-version b/ap-fetch-version
index a7b658bc1704f0842402c92edb8c8c51cc896c76..03ee7545cb62f3dd27b337f22606d2193bb210ec 100755 (executable)
--- a/ap-fetch-version
+++ b/ap-fetch-version
@@ -53,6 +53,9 @@ xen-4.*-testing)
        repo_tree_rev_fetch_git xen \
                $TREE_XEN staging-$branchcore $LOCALREV_XEN
        ;;
+xen-unstable-coverity)
+       repo_tree_rev_fetch_git xen $TREE_XEN master $LOCALREV_XEN
+       ;;
 qemu-mainline)
        repo_tree_rev_fetch_git $branch \
                $TREE_QEMU_MAINLINE master $LOCALREV_QEMU_UPSTREAM

diff --git a/ap-fetch-version-old b/ap-fetch-version-old
index e2c6b3bc30140c2baef831f088d4c5049271178a..920b1056fa442e21b8e53d002cd40816fadd8424 100755 (executable)
--- a/ap-fetch-version-old
+++ b/ap-fetch-version-old
@@ -59,6 +59,9 @@ xen-4.*-testing)
        repo_tree_rev_fetch_git xen \
                $TREE_XEN stable-$branchcore $LOCALREV_XEN
        ;;
+xen-unstable-coverity)
+       repo_tree_rev_fetch_git xen $TREE_XEN coverity-tested/master $LOCALREV_XEN
+       ;;
 qemu-mainline)
         repo_tree_rev_fetch_git $branch \
                $BASE_TREE_QEMU_UPSTREAM upstream-tested $LOCALREV_QEMU_UPSTREAM

diff --git a/ap-push b/ap-push
index 8def652001ece31a0d9ee3922907051eb9af9422..eeff4a940b02f0ee13e63f3781ed06dcfac6bb92 100755 (executable)
--- a/ap-push
+++ b/ap-push
@@ -68,6 +68,10 @@ xen-*-testing)
        xenversion=${xenversion#xen-}
        git push $TREE_XEN $revision:refs/heads/stable-$xenversion
        ;;
+xen-unstable-coverity)
+       cd $repos/xen
+       git push $TREE_XEN $revision:refs/heads/coverity-tested/master
+       ;;
 qemu-mainline)
        cd $repos/qemu-mainline
        git push $TREE_QEMU_UPSTREAM $revision:refs/heads/upstream-tested

diff --git a/cr-daily-branch b/cr-daily-branch
index 364238c9be0c5416fe03acb8dac5c29a271a449c..8b7c789c5643c531c9b743863c66a24a6fef7565 100755 (executable)
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -259,6 +259,7 @@ fi
 
 case $branch in
 distros-*) makeflight=./make-distros-flight ;;
+xen-*-coverity)makeflight=./make-coverity-flight ;;
 *)         makeflight=./make-flight ;;
 esac
 

diff --git a/cri-common b/cri-common
index 6dfe8dffe00458946c11c754f18d550cede5a3e0..cdee48daae0be1296f6df8472c8813492db86de6 100644 (file)
--- a/cri-common
+++ b/cri-common
@@ -67,6 +67,8 @@ select_prevxenbranch () {
 select_xenbranch () {
        case "$branch" in
        xen-unstable-smoke)     tree=xen;       xenbranch=$branch; qemuubranch=qemu-upstream-unstable;;
+       xen-*-coverity)         tree=xen;       xenbranch=$branch; qemuubranch=qemu-upstream-${branch#xen-}
+                                                                  qemuubranch=${qemuubranch%-coverity};;
        xen-*)                  tree=xen;       xenbranch=$branch ;;
        qemu-mainline)          tree=qemuu;     xenbranch=xen-unstable; qemuubranch=qemu-mainline;;
         qemu-upstream-*)    tree=qemuu; xenbranch=xen-${branch#qemu-upstream-};;

diff --git a/crontab b/crontab
index 09b8d148f0e3a201dd2c72e03ddc0b9e02f2332f..f83e4948ccfa3a73d21bf4be9b7c34c5ed8d001b 100755 (executable)
--- a/crontab
+++ b/crontab
@@ -8,6 +8,7 @@ MAILTO=ian.jackson@citrix.com,ian.campbell@eu.citrix.com
 0              *       * * *           cd testing.git && BRANCHES=xen-unstable-smoke   ./cr-for-branches branches -q "./cr-daily-branch --real"
 4-59/30                *       * * *           cd testing.git &&                               ./cr-for-branches branches -q "./cr-daily-branch --real"
 18             9       * * 1,3,5       cd testing.git && BRANCHES=linux-next           ./cr-for-branches branches -w "./cr-daily-branch --real"
+18             9       * * 7           cd testing.git && BRANCHES=xen-unstable-coverity ./cr-for-branches branches -w "./cr-daily-branch --real"
 18             4       * * *           cd testing.git && BRANCHES='linux-linus linux-mingo-tip-master linux-3.0 libvirt rumpuserxen' ./cr-for-branches branches -w "./cr-daily-branch --real"
 6-59/15        *       * * *           cd testing.git && EXTRA_BRANCHES='linux-linus linux-3.0 rumpuserxen libvirt' ./cr-for-branches bisects -w "./cr-try-bisect --real"
 #8-59/5                *       * * *           cd bisects/adhoc.git && with-lock-ex -q data-tree-lock bash -c "./cr-try-bisect-adhoc; exit $?"

diff --git a/daily-cron-settings-real b/daily-cron-settings-real
index b5032afdcc4f9aa67316109b2395220286bcb8c1..f1b4fbc3719dcb1d4433af9d582363e50a5f7814 100644 (file)
--- a/daily-cron-settings-real
+++ b/daily-cron-settings-real
@@ -1,2 +1,4 @@
 : ${OSSTEST_HTML_SUBDIR:=results${OSSTEST_HTML_SUFFIX}}
 : ${OSSTEST_PUSH:=true}
+: ${OSSTEST_COVERITY_URL:=`getconfig CoverityUploadUrl`}
+export OSSTEST_COVERITY_URL

diff --git a/make-coverity-flight b/make-coverity-flight
new file mode 100755 (executable)
index 0000000..9cba9b7
--- /dev/null
+++ b/make-coverity-flight
@@ -0,0 +1,56 @@
+#!/bin/bash
+
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+set -e -o posix
+
+branch=$1
+xenbranch=$2
+blessing=$3
+buildflight=$4
+
+flight=`./cs-flight-create $blessing $branch`
+
+. ./cri-common
+. ./ap-common
+. ./mfi-common
+
+# daily-cron-settings-real will have set this to $c{CoverityUploadUrl}
+# in real runs. .
+: ${OSSTEST_COVERITY_URL:=http://localhost/xen-osstest/stunt-coverity}
+
+defsuite=`getconfig DebianSuite`
+
+arch=amd64
+suite=$defsuite
+
+build_hostflags=share-build-$suite-$arch,arch-$arch,suite-$suite,purpose-build
+
+./cs-job-create $flight coverity-$arch coverity \
+       arch=$arch host_hostflags=$build_hostflags \
+       tree_xen=$TREE_XEN \
+       revision_xen=$REVISION_XEN \
+       coverity_submit_url=${OSSTEST_COVERITY_URL}
+
+echo $flight
+
+# Local variables:
+# mode: sh
+# sh-basic-offset: 2
+# indent-tabs-mode: nil
+# End:

diff --git a/production-config b/production-config
index f2f05845c5f9d866d5be006c7e3e6fe6e9f79b57..410893e5dd8a08af3588e0978f0abeb20400866e 100644 (file)
--- a/production-config
+++ b/production-config
@@ -100,6 +100,13 @@ TftpGrubVersion XXXX-XX-XX
 XenUsePath /usr/groups/xencore/systems/bin/xenuse
 XenUseUser osstest
 
+# Results might include potential vulnerabilities.
+CoverityEmail security@xen.org
+# This is only read from daily-cron-settings-real, everything else
+# gets the default/dummy path
+CoverityUploadUrl https://scan.coverity.com/builds?project=XenProject
+CoverityTools cov-analysis-linux64-7.7.0.4.tar.gz
+
 # We use the IP address because Citrix can't manage reliable nameservice
 #DebianMirrorHost debian.uk.xensource.com
 #DebianMirrorHost 10.80.16.196

diff --git a/sg-run-job b/sg-run-job
index 20ebb6484476143cdaec366633f7f20c699ce587..3e0f966ce0f929739e84700cf7efbf8e1030d0a2 100755 (executable)
--- a/sg-run-job
+++ b/sg-run-job
@@ -445,6 +445,12 @@ proc prepare-build-host {} {
     run-ts . host-build-prep ts-xen-build-prep
 }
 
+proc need-hosts/coverity {} { return BUILD }
+proc run-job/coverity {} {
+    run-ts . = ts-coverity-build + host
+    run-ts . = ts-coverity-upload + host
+}
+
 #---------- main program ----------
 
 jobdb::set-flight

diff --git a/ts-coverity-build b/ts-coverity-build
new file mode 100755 (executable)
index 0000000..1d8bd0c
--- /dev/null
+++ b/ts-coverity-build
@@ -0,0 +1,81 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use DBI;
+use Osstest;
+use File::Path;
+use POSIX;
+use Osstest::TestSupport;
+use Osstest::BuildSupport;
+
+tsreadconfig();
+selectbuildhost(\@ARGV);
+# remaining arguments are passed as targets to "make"
+builddirsprops();
+
+sub checkout () {
+    prepbuilddirs();
+
+    build_clone($ho, 'xen', $builddir, 'xen');
+}
+
+sub covtools () {
+    target_putfile($ho, 100, "$c{Images}/$c{CoverityTools}", "$builddir/covtools.tar.gz");
+    target_cmd($ho, <<END, 100);
+set -xe
+c=$builddir/covtools
+mkdir -p \$c
+cd \$c
+tar --strip-components=1 -xaf $builddir/covtools.tar.gz
+END
+}
+
+sub build () {
+    my $make = "make $makeflags";
+
+    # Pre build things we don't want coverity to scan, but which are
+    # normally built by some other command.
+    target_cmd_build($ho, 1000, $builddir, <<END);
+cd $builddir/xen
+./configure
+$make -C tools/firmware/etherboot all
+$make mini-os-dir
+END
+
+    # Now the stuff we want coverity to look at. The resulting tarball
+    # must have "cov-int/" as a root directory.
+    target_cmd_build($ho, 9000, $builddir, <<END);
+cd $builddir/xen
+export PATH=$builddir/covtools/bin:\$PATH
+cov-build --dir $builddir/coverity/cov-int $make -C extras/mini-os/
+cov-build --dir $builddir/coverity/cov-int $make xen tools
+END
+
+    built_stash($ho, $builddir, 'coverity', 'coverity');
+
+    my $xen_version = target_cmd_output($ho, <<END, 30);
+    cd $builddir/xen
+    make xenversion
+END
+    store_runvar("built_version_xen", $xen_version);
+}
+
+checkout();
+covtools();
+build();
+collect_xen_built_versions();

diff --git a/ts-coverity-upload b/ts-coverity-upload
new file mode 100755 (executable)
index 0000000..6d43cf8
--- /dev/null
+++ b/ts-coverity-upload
@@ -0,0 +1,54 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use DBI;
+use Osstest;
+use File::Path;
+use POSIX;
+use Osstest::TestSupport;
+use Osstest::BuildSupport;
+
+tsreadconfig();
+selectbuildhost(\@ARGV);
+
+# This must contain exactly and only the token, for example there must
+# be no trailing "\n", otherwise it is included in the literal token,
+# which is then invalid.
+my $tokenfile = "$ENV{HOME}/.xen-osstest/coverity-secret";
+
+sub upload() {
+    my $dist = get_stashed("path_coverity", $job);
+
+    my @form_args;
+    push @form_args, "token=\<$tokenfile";
+    push @form_args, "email=$c{CoverityEmail}";
+    push @form_args, "file=\@$dist";
+    push @form_args, "version=$r{built_version_xen}";
+    push @form_args, "description=$flight: $r{tree_xen} $r{built_revision_xen}";
+
+    my @args = map { ("--form", $_) } @form_args;
+
+    push @args, qw(--max-time 3600);
+    push @args, qw(--fail); # turn 404 etc into a failure.
+    push @args, $r{coverity_submit_url};
+
+    logm("curl args: ".(join " ", map { qq("$_") } @args));
+    system_checked("curl", @args);
+}
+
+upload();