cr-ensure-disk-space: Look at referring flights

Previously the flight to delete was simply the one with the lowest
flight number.  Now we sort flights not by their own flight number,
but by the highest flight number of any referencing flight.

This means that flights whose builds are being reused are kept as long
as the reusing flights.

This almost-entirely fixes a largely-theoretical race in the way
cs-bisection-step works (where the flight's logs and build outputs
might be deleted between the setup and execution of the referring
flight).

A smaller race still exists because the stash check in
cs-bisection-step occurs before the being-created flight is visible to
other db clients.  We will have to fix this by taking the flights
lock.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>

diff --git a/cr-ensure-disk-space b/cr-ensure-disk-space
index ced977725135bd624f3cc05316305e483e83c52c..0314e7a0d4f093288cb99dde5a95e760362d5166 100755 (executable)
--- a/cr-ensure-disk-space
+++ b/cr-ensure-disk-space
@@ -55,7 +55,17 @@ flock LOCK, LOCK_EX or die $!;
 $|=1;
 
 my $chkq= db_prepare("SELECT * FROM flights WHERE flight=?");
+
+my $refq= db_prepare(<<END);
+    SELECT flight, val
+      FROM runvars
+     WHERE name like '%job'
+       AND val like '%.%'
+       AND flight >= ?
+END
+
 our @flights;
+our %latestref;
 
 for (;;) {
     open P, "-|", onloghost "df --block-size=1M -P $logdir" or die $!;
@@ -69,23 +79,46 @@ for (;;) {
     last if $space >= logcfg('MinSpaceMby');
 
     if (!@flights) {
+       %latestref = ();
        open P, "-|", onloghost "ls -1 $logdir" or die $!;
+
+       my $minflight = undef;
        while (<P>) {
            next unless m/^(\d+)\n$/;
-           push @flights, $1;
+           $latestref{$1} = $1;
+           $minflight //= $1;
+           $minflight = $1 if $1 < $minflight;
        }
        $!=$?=0; close P or die "ls: $? $!";
-       @flights = sort { $b <=> $a } @flights;
+
+       print DEBUG "MINFLIGHT $minflight\n";
+
+       $refq->execute($minflight);
+       while (my $rr= $refq->fetchrow_hashref) {
+           my $testflight = $rr->{flight};
+           next unless $rr->{val} =~ m/^(\d+)\./;
+           my $buildflight = $1;
+           next unless exists $latestref{$buildflight};
+           if ($testflight > $latestref{$buildflight}) {
+               print DEBUG "REF $buildflight <- $testflight\n";
+               $latestref{$buildflight} = $testflight;
+           }
+       }
+
+       @flights =
+           sort { $latestref{$b} <=> $latestref{$a} }
+           keys %latestref;
         printf "(%d flights) ", scalar @flights;
         die unless @flights;
     }
     my $flight = pop @flights;
-    printf "selected %s ", $flight;
+    my $latestref = $latestref{$flight};
+    printf "selected %s (latest ref %s) ", $flight, $latestref;
 
-    $chkq->execute($flight);
+    $chkq->execute($latestref);
     my $row= $chkq->fetchrow_hashref();
     $chkq->finish();
-    die $flight unless defined $row;
+    die "$flight $latestref" unless defined $row;
     my $age= time - $row->{started};
 
     printf "(age %dd) ", $age / 86400;