make-flight: Trim the matrix of disk format flights

We don't need to test every combination of toolstack, architecture,
and disk format.  We don't expect many architecture-specific bugs in
the per-disk-format code in the toolstack layers.

We _do_ want to test every combination of toolstack and disk format
(since the format configuration machinery is toolstack specific) and a
reasonable selection of architectures for each disk format (since
arch-specific bugs in actual underlying disk drivers are a
possibility).

The implementation strategy is for do_pv_debian_tests to select a
particular architecture for each combination of toolstack and format.
(Because the architecture is actually in an outer loop, we recalculate
that selection multiple times, and skip inner iterations for the other
architectures.  This is all in bash code so the wasted computation is
not particularly important.)

We have a safety catch which spots if any architecture is entirely
untested in any of these combinations; this would happen if a new
architecture is introduced elsewhere and not added to the list.  We do
not have a safety catch which spots when a (toolstack,format)
combination becomes untested due to deletion of an architecture.
(That would be more fiddly to implement without restructuring.)

We list armhf twice because we would like to do at least as many ARM
as x86 tests (particularly given our current workload and capacity).

The result is that the set of generated jobs is adjusted as follows:

   keep  test-amd64-i386-xl-raw
     -   test-amd64-i386-xl-vhd
     -   test-amd64-i386-xl-qcow2
     -   test-amd64-i386-libvirt-raw
     -   test-amd64-i386-libvirt-vhd
     -   test-amd64-i386-libvirt-qcow2
     -   test-amd64-amd64-xl-raw
     -   test-amd64-amd64-xl-vhd
   keep  test-amd64-amd64-xl-qcow2
     -   test-amd64-amd64-libvirt-raw
   keep  test-amd64-amd64-libvirt-vhd
     -   test-amd64-amd64-libvirt-qcow2
     -   test-armhf-armhf-xl-raw
   keep  test-armhf-armhf-xl-vhd
     -   test-armhf-armhf-xl-qcow2
   keep  test-armhf-armhf-libvirt-raw
     -   test-armhf-armhf-libvirt-vhd
   keep  test-armhf-armhf-libvirt-qcow2

(Where `-' means `drop this job'.)

Or to look at it another way:

                         i386   amd64   armhf

                 raw     keep     -       -
                 vhd       -      -     keep
                 qcow2     -    keep      -

         libvirt raw       -      -     keep
         libvirt vhd       -    keep      -
         libvirt qcow2     -      -     keep

I have diffed standalone-generate-dump-flight-runvars output before
and after and it looks plausible.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>

diff --git a/make-flight b/make-flight
index 8c75a9c6f72a9b8b55f2b470ba7ce384d6e0df07..0291b5febe1145ee1ee431b3b3eb463c665b6291 100755 (executable)
--- a/make-flight
+++ b/make-flight
@@ -460,10 +460,29 @@ do_pv_debian_tests () {
     do_pv_debian_test_one libvirt '' libvirt '' enable_xsm=$xsm
   done
 
+  # We compute a desired architecture for each combination of ts and fmt
+  # Within each fmt we rotate through the list of arches
+  # The starting list rotates once per ts, so that we try to
+  # exercise each fmt on each arch family.
+  local fmtarches_outer="i386 armhf amd64 armhf"
+  local endfmt="do_pv_debian_tests-missing-ts-fmt-for-dom0arch="
+
   for ts in xl libvirt ; do
 
+    local fmtarches=$fmtarches_outer
+    fmtarches_outer="${fmtarches_outer#* } ${fmtarches_outer%% *}"
+
     for fmt in raw vhd qcow2 ; do
 
+      local fmtarch="${fmtarches%% *}"
+      fmtarches="${fmtarches#* } $fmtarch"
+
+      if [ "x$fmtarch" != "x$dom0arch" ]; then
+        continue
+      else
+        endfmt=": "
+      fi
+
       fmt_runvar="debian_diskfmt=$fmt"
 
       do_pv_debian_test_one $ts-$fmt '-di' $ts '' \
@@ -475,6 +494,8 @@ do_pv_debian_tests () {
 
     done
   done
+
+  $endfmt$dom0arch
 }
 
 test_matrix_do_one () {