]> xenbits.xensource.com Git - osstest.git/log
osstest.git
8 months agoosstest: update Debian Bookworm installer version
Anthony PERARD [Mon, 2 Sep 2024 08:42:45 +0000 (08:42 +0000)]
osstest: update Debian Bookworm installer version

Signed-off-by: Anthony PERARD <anthony@xenproject.org>
9 months agopreseed_base: Use "keep" NIC NamePolicy when "force-mac-address"
Anthony PERARD [Fri, 14 Jun 2024 11:28:30 +0000 (11:28 +0000)]
preseed_base: Use "keep" NIC NamePolicy when "force-mac-address"

We have a few machine (arndale-*) that have a nic without mac address,
so the kernel assign a random one. For those there's a flags
"force-mac-address" which tells osstest to make it so that the machine
changes the mac address to a predefined one at boot. This normally
tells systemd rules to not use the mac address to rename the network
interface as it a temporary mac, but that doesn't always work.
(Machine installed by osstest should use the mac namepolicy otherwise,
since 367166c32329 ("preseed_base, ts-host-install: Change NIC
NamePolicy to "mac"")).

Often, on the branch "linux-linus", so with more recent version of
Linux, the network interface gets renamed sometime with the "mac"
namepolicy which break networking. These are the kernel messages when
the rename happen:

> usb 1-3.2.4: new high-speed USB device number 4 using exynos-ehci
> asix 1-3.2.4:1.0 (unnamed net_device) (uninitialized): invalid hw address, using random
> asix 1-3.2.4:1.0 (unnamed net_device) (uninitialized): PHY [usb-001:004:10] driver [Asix Electronics AX88772A] (irq=POLL)
> asix 1-3.2.4:1.0 eth0: register 'asix' at usb-12110000.usb-3.2.4, ASIX AX88772 USB 2.0 Ethernet, 06:85:e5:95:f0:7c
> usbcore: registered new device driver onboard-usb-dev
> usb 1-3.2.4: USB disconnect, device number 4
> asix 1-3.2.4:1.0 eth0: unregister 'asix' usb-12110000.usb-3.2.4, ASIX AX88772 USB 2.0 Ethernet
> hub 1-3.2:1.0: USB hub found
> hub 1-3.2:1.0: 4 ports detected
> hub 1-3.2:1.0: USB hub found
> hub 1-3.2:1.0: 4 ports detected
> usb 1-3.2.4: new high-speed USB device number 5 using exynos-ehci
> asix 1-3.2.4:1.0 (unnamed net_device) (uninitialized): PHY [usb-001:005:10] driver [Asix Electronics AX88772A] (irq=POLL)
> Asix Electronics AX88772A usb-001:005:10: attached PHY driver (mii_bus:phy_addr=usb-001:005:10, irq=POLL)
> asix 1-3.2.4:1.0 eth0: register 'asix' at usb-12110000.usb-3.2.4, ASIX AX88772 USB 2.0 Ethernet, 06:85:e5:95:f0:7c
> asix 1-3.2.4:1.0 enx0685e595f07c: renamed from eth0

The "xenbr0" bridge is setup to use "eth0", because that was the name
of the nic during setup, so with a new name for the main interface the
bridge doesn't work.

In order to avoid the issue, we will use the NamePolicy "keep" when
there is a flag "force-mac-address", which keep the original name of
the interface (eth0). That flags only works if there's a single
network interface, so we can expect "eth0" to always be the same
interface.

Even if the problem so far exhibit only at runtime after rebooting
under Xen (which is fixed by a change in preseed_base()), we will also
add the policy change to the installer (change in ts-host-install), to
be future proof.

(The filename of the policy is to have it apply before
"73-usb-net-by-mac.link" that is installed on the system.)

Signed-off-by: Anthony PERARD <anthony.perard@vates.tech>
9 months agocr-for-branches: Add Xen and QEMU 4.19 branch
osstest service owner [Thu, 18 Jul 2024 13:57:44 +0000 (13:57 +0000)]
cr-for-branches: Add Xen and QEMU 4.19 branch

12 months agomake-hosts-flight: default to bookworm
Anthony PERARD [Sat, 28 Oct 2023 12:09:32 +0000 (13:09 +0100)]
make-hosts-flight: default to bookworm

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
12 months agoSwitch to Debian Bookworm as default suite
Anthony PERARD [Wed, 7 Jun 2023 16:35:24 +0000 (17:35 +0100)]
Switch to Debian Bookworm as default suite

Xen 4.16 doesn't build with Debian Bookworm. It fails to build
IPXE/etherboot, on "amd64". So we keep using Debian Buster on Xen 4.17
and earlier branches. Xen 4.17 builds 4.16 via job "build-amd64-prev".

Xen 4.16 would needs 18a36b4a9b08 ("tools: ipxe: update for fixing
build with GCC12").

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
12 months agoproduction-config: Set Bookworm's debian-installer version
Anthony PERARD [Tue, 2 Apr 2024 09:14:25 +0000 (10:14 +0100)]
production-config: Set Bookworm's debian-installer version

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
12 months agoproduction-config: override mirror url for buster, use archive
Anthony PERARD [Sun, 14 Apr 2024 16:27:29 +0000 (17:27 +0100)]
production-config: override mirror url for buster, use archive

buster-backport isn't available on the main mirror anymore.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
12 months agoosstest: increase boot timeout for Debian PV guests
Roger Pau Monne [Fri, 12 Apr 2024 14:11:21 +0000 (16:11 +0200)]
osstest: increase boot timeout for Debian PV guests

The current timeout of 40s seems to be too low for AMD boxes (pinots and
rimavas) in the lab after XSA-455, see:

http://logs.test-lab.xenproject.org/osstest/logs/185303/test-amd64-coresched-amd64-xl/info.html

Increase the timeout to 60s.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
13 months agocr-daily-branch: Actually use Linux 6.1 by default on Arm
Anthony PERARD [Fri, 5 Apr 2024 20:44:28 +0000 (21:44 +0100)]
cr-daily-branch: Actually use Linux 6.1 by default on Arm

Commit 95ee1714750b wasn't effective at changing the default version
of Linux, osstest kept on using "linux-arm-xen".

This time, make osstest use the same revision for both LINUX and
LINUX_ARM, like it was done for "linux*" branches only before.

If for some reason $BASE_TAG_LINUX and $BASE_TAG_LINUX_ARM are
different, this change would be wrong, but there's shouldn't be a need
for them to be different.

Fixes: 95ee1714750b ("ap-common: Switch to Linux 6.1 by default on Arm")
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
13 months agoap-common: Fix libvirt's git repo URL
Anthony PERARD [Tue, 26 Mar 2024 10:30:07 +0000 (10:30 +0000)]
ap-common: Fix libvirt's git repo URL

The current URL doesn't work anymore and just timeout, switch to the
new main repo URL.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
13 months agomake-flight: Keep using buster for L2 guest in nested tests
Anthony PERARD [Fri, 1 Mar 2024 17:46:05 +0000 (17:46 +0000)]
make-flight: Keep using buster for L2 guest in nested tests

When starting the installation of the L2 guest, L0 kills L1. Switching
the L2 guest back to Debian Buster works fine, so do that to prevent
regression in the test.

Part of the logs from the host L0:

> domain_crash called from arch/x86/hvm/vmx/vvmx.c:2770
> Domain 3 (vcpu#0) crashed on cpu#4:
> d3v0 vmentry failure (reason 0x80000021): Invalid guest state (2)

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agots-xen-install: Fix bridge setup, ask to copy MAC addr
Anthony PERARD [Mon, 29 Jan 2024 12:06:25 +0000 (12:06 +0000)]
ts-xen-install: Fix bridge setup, ask to copy MAC addr

This ask to copy the MAC address from $physif on the bridge.

On Debian Bookworm, when running as a Xen guest for nested tests, the
bridge does get a random MAC address and a different IP address from
DHCP than before setting up the bridge, so the test fails.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agots-debian-*-install: Replace dots in hostnames by dashs
Anthony PERARD [Fri, 27 Oct 2023 17:03:15 +0000 (18:03 +0100)]
ts-debian-*-install: Replace dots in hostnames by dashs

When running ./ts-debian-di-install, hostname on the command line is
interpreted by the debian installer. As the installer find it to be a
FQDN, it uses part of the hostname as the domain, thus overwriting the
value from the DHCP and from d-i netcfg/get_domain setting (maybe).

But the result is that /etc/resolv.conf contains "search
bookworm.guest.osstest" and can't find an IP for "cache". So the
installation fails.

Also replace ".guest.osstest" in a few other places, even if it
may not be an issue.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agobookworm: Extend guest bootloader workaround
Anthony PERARD [Sat, 28 Oct 2023 12:00:09 +0000 (13:00 +0100)]
bookworm: Extend guest bootloader workaround

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agots-debian-hvm-install: Increase min guest ram size
Anthony PERARD [Mon, 30 Oct 2023 17:09:48 +0000 (17:09 +0000)]
ts-debian-hvm-install: Increase min guest ram size

768 MB for the guest isn't enough anymore for Debian Bookworm, at boot
it print this message:

    Low memory
    ----------

    Entering low memory mode

    This system has relatively little free memory, so it will enter low memory
    mode. Among other things, this means that this program will proceed in English.
    You should set up swap space as soon as possible.
    [Press enter to continue]

So, automatic installation fails. An empiric test shows that the min
was 817M for one particular situation (test-amd64-amd64-xl-qemuu-ovmf-amd64)
but we probably need some leg room, so increase to 1 GB.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agots-debian-install: keep avoiding to use pygrub
Anthony PERARD [Wed, 18 Oct 2023 17:35:05 +0000 (18:35 +0100)]
ts-debian-install: keep avoiding to use pygrub

xen-tools commit 83c37b476a75 ("Start all Debian releases since
Stretch (9) with pygrub by default") started to use pygrub by default.
Revert this.

With "pygrub" setting, xen-create-guest fails on armhf, the
80-install-kernel hook fails because it doesn't know about "armhf".

https://github.com/xen-tools/xen-tools/commit/83c37b476a7534c432ecc9941817aeb989677da6

There's "--nopygrub" but that doesn't work due to several issues, so
removing "pygrub" from "distributions.conf" is the only way.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agots-debian-fixup: Fix nic names for bookworm
Anthony PERARD [Tue, 3 Oct 2023 16:22:17 +0000 (17:22 +0100)]
ts-debian-fixup: Fix nic names for bookworm

`xen-create-image` doesn't create image for bookworm with a working
network, we need to fix the interface name.

For reference, there's a bug report upstream:
    "UnPredictableNetworkInterfaceNames 'fun' with Bookworm domU: eth0 -> enX0"
    https://github.com/xen-tools/xen-tools/issues/65

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agots-debian-hvm-install: Allow udev failure in install media
Anthony PERARD [Sat, 2 Mar 2024 15:08:15 +0000 (15:08 +0000)]
ts-debian-hvm-install: Allow udev failure in install media

Kernel in "debian-12.1.0-amd64-netinst.iso" prevent debian installer
from booting. Early on, it does `udevadm trigger --action=add`, which
fails, the same way as the following runes fails:

$ cat /sys/devices/virtual/input/input2/name
Xen Virtual Keyboard
$ echo add > /sys/devices/virtual/input/input2/uevent
[   25.884403] synth uevent: /devices/virtual/input/input2: failed to send uevent
[   25.916498] input input2: uevent: failed to send synthetic uevent: -12
sh: write error: Cannot allocate memory
$ uname -a
Linux (none) 6.1.0-10-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.38-1 (2023-07-14) x86_64 GNU/Linux

This doesn't looks like a new issue, Debian Buster ISO seems to do the
same thing (early boot command, and error in Linux logs), so it's
probable that now `udevadm trigger --action=add` return an error when
there's a failure.

Bug report in the kernel and in Debian:
    https://bugzilla.kernel.org/show_bug.cgi?id=207695
    https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=983357

One way to workaround the issue is to remove the vkb device, with xl's
"vkb_device=0", but libvirt doesn't have support for this config
option.

The second option is to edit the installation media, and allow the
failure.

Once installed, the guest boot fine.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agots-leak-check: add new name for udevd workers
Anthony PERARD [Fri, 18 Aug 2023 10:17:51 +0000 (11:17 +0100)]
ts-leak-check: add new name for udevd workers

udevd on Bookworm shows as "(udev-worker)" in the process list.
Suppress them.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agots-nested-setup, setup l1 lvm volume groupe in guest
Anthony PERARD [Thu, 26 Oct 2023 16:42:09 +0000 (17:42 +0100)]
ts-nested-setup, setup l1 lvm volume groupe in guest

LVM in l0 doesn't let us run pvcreate on the host LV, `pvcreate
$outer_lvdev` fails with:
    Cannot use /dev/$l0-vg/l1_gueststorage_outer_lv: device is an LV

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agobookworm: Extend ARM clock workaround
Anthony PERARD [Wed, 18 Oct 2023 13:50:01 +0000 (14:50 +0100)]
bookworm: Extend ARM clock workaround

Still broken on arndale, serial stop working early, then the machine
timeout when working on creating a xen guest with xen-create-guest.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agosetupboot_grub2: Parse arm64 uefi grub verbes
Anthony PERARD [Thu, 12 Oct 2023 10:48:27 +0000 (11:48 +0100)]
setupboot_grub2: Parse arm64 uefi grub verbes

20_linux_xen now uses "xen_hypervisor" and "xen_module" in place of
"multiboot2?" and "module2?" when generating a config file for
arm64-uefi.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agots-xtf-install: Install python symlink
Anthony PERARD [Tue, 15 Aug 2023 12:37:43 +0000 (13:37 +0100)]
ts-xtf-install: Install python symlink

ts-xtf-run does run ./xtf-runner, which run `python` in its shebang.
So install a `python` symlink to `python3`.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agooverlay-bookworm: 20_linux_xen: Fix XSM entries generation
Anthony PERARD [Tue, 15 Aug 2023 11:55:59 +0000 (12:55 +0100)]
overlay-bookworm: 20_linux_xen: Fix XSM entries generation

It turns out that setting $xen_version in linux_entry_xsm() override
$xen_version in the loop over $xen_list. This means that only one
entry per Xen version is going to enable XSM, but all further entries
are going to have "(XSM enabled)" in their titles without enabling
XSM.

When a "xenpolicy-$xen_version" file was found for the current
$xen_version, it would be overwrite $xen_version to add "(XSM
enabled)" to the menu entry title. Once change, the next call to
linux_entry_xsm() would also have this modified $xen_version and would
look for the file "xenpolicy-*(XSM enabled)" and fail.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
(upstream commit: https://git.savannah.gnu.org/cgit/grub.git/commit/?id=db1faedccdce3cf83336155a95c04a8db03744c5)
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agooverlay-bookworm: Import grub's 20_linux_xen from Debian Bookworm
Anthony PERARD [Tue, 15 Aug 2023 11:53:50 +0000 (12:53 +0100)]
overlay-bookworm: Import grub's 20_linux_xen from Debian Bookworm

This is a copy of the file installed, from grub-common package.

We are going to edit it shortly.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agots-xen-install: remove "libc6-xen" package installation
Anthony PERARD [Mon, 14 Aug 2023 15:59:05 +0000 (16:59 +0100)]
ts-xen-install: remove "libc6-xen" package installation

libc6-xen packaged have been removed from Debian Bookworm.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agotarget_editfile: Use the same user to retrieve and send
Anthony PERARD [Tue, 24 Oct 2023 10:51:06 +0000 (11:51 +0100)]
target_editfile: Use the same user to retrieve and send

The file "/boot/grub/grub.cfg" on Debian Bookworm isn't accessible
from the "osstest" user, but target_editfile_root() tries to grab the
file as "osstest" then edit it as "root.

Change teditfileex() to use the same $user also to get the file. This
will fix ts-examine-serial-pre step.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agobl_getmenu_open: Read grub.cfg as root
Anthony PERARD [Fri, 16 Jun 2023 18:38:06 +0000 (19:38 +0100)]
bl_getmenu_open: Read grub.cfg as root

On bookworm, "/boot/grub/grub.cfg" isn't accessible by user "osstest",
so read the file as user "root".

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agots-xen-build-prep: Change package selection for Bookworm
Anthony PERARD [Thu, 8 Jun 2023 15:04:41 +0000 (16:04 +0100)]
ts-xen-build-prep: Change package selection for Bookworm

python-dev:
    It doesn't exist on bookworm, and python2 shouldn't be needed
    anymore.

libsdl-dev:
    On buster this already select "libsdl1.2-dev", but to not change
    buster installation we will only use the new package name on
    Bookworm.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agopreseed_base, ts-host-install: Change NIC NamePolicy to "mac"
Anthony PERARD [Wed, 25 Oct 2023 16:18:31 +0000 (17:18 +0100)]
preseed_base, ts-host-install: Change NIC NamePolicy to "mac"

On "italia?" machine, the two network interfaces are competing to have
"eno1", base on the "onboard" naming policy. So the name of the
network interface can change between "eno1" and "eth0".

Switching to "mac" should avoid the unpredictable name based on
"onboard" or "slot" policy.

The "mac" naming policy break `vif-bridge` for the "vif*.*" network
interfaces. So we will avoid the "mac" policy if the driver is "vif".
This also have an impact on guest created with ./ts-debian-install, as
they use the initrd from dom0, so the interface in the guest will be
renamed according to default policy (which rename eth0 to enX0).

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agoDisable persistent net generator on Bookworm
Anthony PERARD [Fri, 18 Aug 2023 10:55:11 +0000 (11:55 +0100)]
Disable persistent net generator on Bookworm

This schema doesn't work. Even if the udev rule is there, the name of
the different NIC are different from one boot to the next. On a
machine (sabro*) with 3 different NIC, the name of each interface is
basically random and could take on of three name, "eth[0-2]".

net.ifnames=0 does still mean that we have "eth*" names, but the
generated file "75-persistent-net-generator" doesn't seems to name the
interfaces as expected.

The new predictable network interface names schema seems to work well
enough. The names are still the same after rebooting into Xen, at
least on the few machine tested so far. But, we will change to a "mac"
name policy in a follow-up commit.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agodi_installcmdline_core: Add link_wait_timeout to install cmdline
Anthony PERARD [Thu, 3 Aug 2023 16:42:04 +0000 (17:42 +0100)]
di_installcmdline_core: Add link_wait_timeout to install cmdline

The default wait time of 3 seconds isn't always enough get an IP from
the DHCP, give more time to the installer to find a NIC that works.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agots-host-install,preseed_create: Do lvm vgextend at install time
Anthony PERARD [Wed, 25 Oct 2023 12:55:00 +0000 (13:55 +0100)]
ts-host-install,preseed_create: Do lvm vgextend at install time

When there's more than one disk, like the "pinot?" machine, the name
assigned to e.g. "sda" may change after a reboot, at least when
installing Debian Bookworm, which is using Linux 6.1.

I believe Linux probes disk controller in parallel and assign "sda"
to the first controller to respond, or something like that, so disk
aren't assigned a name in a predictable order.

So, instead of extending lvm volume group to a new disk on reboot, do
that at install time when we know that lvm is on sda.

Alternatively, we could try to set "d-i partman-auto/disk" with a list
of all disk, but that would mean knowing all the disk before starting
the machine.

This new shell script in /lib/partman/finish.d/ should work in all
debian, so replace the script in ts-host-install by this new one.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agopreseed_create: Workaround fail grub-install on arndale
Anthony PERARD [Mon, 16 Oct 2023 15:05:42 +0000 (16:05 +0100)]
preseed_create: Workaround fail grub-install on arndale

grub-installer on arndale-* machine fails with Debian Bookworm. It
tries to install "grub-pc" which doesn't exist. Skip installation.

Somehow, cubietruck-* installation works fine.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agopreseed_create: osstest-erase-other-disks: workaround creating /dev/sdXD files
Anthony PERARD [Wed, 11 Oct 2023 11:02:55 +0000 (11:02 +0000)]
preseed_create: osstest-erase-other-disks: workaround creating /dev/sdXD files

Sometime, it can happen that the erase-other-disks script fails and
indicate that /dev/sda1 isn't a block device anymore. This happened
when /dev/sda was erased before /dev/sda1. So to avoid this, we will
zero the partitions of ${disk} first, and hopefully the error won't
happen again.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agopreseed_create: Use new "d-i grub-installer/update-nvram" for UEFI installation
Anthony PERARD [Tue, 20 Jun 2023 10:21:21 +0000 (11:21 +0100)]
preseed_create: Use new "d-i grub-installer/update-nvram" for UEFI installation

Instead of "grub-installer/no-nvram" proposed in Debian bug #789798,
we have "grub-installer/update-nvram". Make use of it, and remove
workaround.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agots-host-install: Restart ntp service
Anthony PERARD [Thu, 2 Nov 2023 13:24:04 +0000 (13:24 +0000)]
ts-host-install: Restart ntp service

Otherwise, the change to the config file isn't taken into account
until the next reboot.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agots-host-install: fix ntp server setting
Anthony PERARD [Thu, 8 Jun 2023 15:16:20 +0000 (16:16 +0100)]
ts-host-install: fix ntp server setting

The Debian #778564 bug report is still open, the ntp.conf file doesn't
contain the setting from NtpServer after installation, so we still
need to edit ntp.conf on Bookworm.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agots-host-install: fix ntp.conf path on bookworm
Anthony PERARD [Wed, 7 Jun 2023 18:35:17 +0000 (19:35 +0100)]
ts-host-install: fix ntp.conf path on bookworm

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agomg-debian-installer-update: Download non-free firmware from new repo.
Anthony PERARD [Wed, 7 Jun 2023 16:36:05 +0000 (17:36 +0100)]
mg-debian-installer-update: Download non-free firmware from new repo.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agomgi-common: Fix fetch_debian_package error message
Anthony PERARD [Wed, 7 Jun 2023 16:30:04 +0000 (16:30 +0000)]
mgi-common: Fix fetch_debian_package error message

$@ expand to two or more words, but fail() only take one argument.
So if there's more than one argument left, fail() only shows the first
one, and don't event display "not found".

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agots-xen-build-prep: Only force git protocol v2 on buster
Anthony PERARD [Wed, 4 Oct 2023 15:17:11 +0000 (16:17 +0100)]
ts-xen-build-prep: Only force git protocol v2 on buster

Newer version of Debian and thus git would use this automatically, no
need to force it.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agoproduction-config: Add bookworm debian install media filename
Anthony PERARD [Tue, 19 Sep 2023 14:54:42 +0000 (15:54 +0100)]
production-config: Add bookworm debian install media filename

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agoap-common: Switch to Linux 6.1 by default on x86 + drop dom0 i386
Anthony PERARD [Mon, 5 Feb 2024 12:13:43 +0000 (12:13 +0000)]
ap-common: Switch to Linux 6.1 by default on x86 + drop dom0 i386

linux-4.19 branch in xenbits is outdated, it haven't been updated and
tested since 2020 as it has been disabled in osstest. Also, this 4.19
branch doesn't build on Bookworm.

So we will start to use a newer version of Linux. We switch to 6.1 for
the Arm* tests recently, so will use that same version for x86.

Also, following commit 3a3089c94913 ("mfi-common: Drop Linux dom0 i386
tests for newer Linux branches"), 32bit dom0 isn't tested on newer
Linux, so we need to drop all dom0 i386 tests wherever the default
linux branch is used. That is, this changes in jobs will apply to
"xen-unstable" branch but also all xen stable branches, seabios, qemu,
osstest, libvirt, so every branch that aren't "linux-*".

Here is the list jobs that changes, and whether they are replace, or
have existing equivalents, on the "xen-unstable" branch. Changes
compared with:
    OSSTEST_CONFIG=standalone-config-example nice eatmydata ./standalone-generate-dump-flight-runvars

Gone, without exiting or new test-amd64-amd64-*:
- test-amd64-i386-freebsd10-amd64
- test-amd64-i386-freebsd10-i386
- test-amd64-i386-qemut-rhel6hvm-amd
- test-amd64-i386-qemut-rhel6hvm-intel
- test-amd64-i386-qemuu-rhel6hvm-amd
- test-amd64-i386-qemuu-rhel6hvm-intel

Gone, but with exiting test-amd64-amd64-* equivalent:
- test-amd64-coresched-i386-xl
- test-amd64-i386-examine
- test-amd64-i386-examine-bios
- test-amd64-i386-examine-uefi
- test-amd64-i386-libvirt
- test-amd64-i386-libvirt-pair
- test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm
- test-amd64-i386-libvirt-xsm
- test-amd64-i386-livepatch
- test-amd64-i386-migrupgrade
- test-amd64-i386-pair
- test-amd64-i386-xl
- test-amd64-i386-xl-pvshim
- test-amd64-i386-xl-qemut-debianhvm-amd64
- test-amd64-i386-xl-qemut-debianhvm-i386-xsm
- test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm
- test-amd64-i386-xl-qemut-win7-amd64
- test-amd64-i386-xl-qemut-ws16-amd64
- test-amd64-i386-xl-qemuu-debianhvm-amd64
- test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow
- test-amd64-i386-xl-qemuu-debianhvm-i386-xsm
- test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict
- test-amd64-i386-xl-qemuu-ovmf-amd64
- test-amd64-i386-xl-qemuu-win7-amd64
- test-amd64-i386-xl-qemuu-ws16-amd64
- test-amd64-i386-xl-shadow
- test-amd64-i386-xl-simplat-amd64-buster
- test-amd64-i386-xl-xsm

Gone, but replaced by a new test-amd64-amd64-*:
- test-amd64-i386-libvirt-qcow2
- test-amd64-i386-libvirt-raw
- test-amd64-i386-xl-vhd
+ test-amd64-amd64-libvirt-qcow2
+ test-amd64-amd64-libvirt-raw
+ test-amd64-amd64-xl-vhd

In any case, the list of test would be the same as for the existing
branch "linux-linus" or "linux-6.1" branches.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Juergen Gross <jgross@suse.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agomfi-common: Rework toolstack-disk_format test matrix
Anthony PERARD [Thu, 14 Mar 2024 17:50:23 +0000 (17:50 +0000)]
mfi-common: Rework toolstack-disk_format test matrix

There is 6 potenteil test of toolstack disk_format combination:
{xl,libvirt}-{raw,vhd,qcow2}. Commit f536e834f673 ("make-flight: Trim
the matrix of disk format flights") introduced a way to avoid testing
on every architecture and actually do only 6 tests accross all
arches, 3 on x86, 3 on armhf (no arm64 at the time).

But, commit 5c70735f177f ("fmtarches: Use dom0arches, not hardcoded
arch list") broke that. There's an extra space ' ' added at the
begining of the string in variable $fmtarches_outer which break the
algorithm. This mean that the combination of xl+raw is never tested
and both armhf and amd64 are testing the same combination of
toolstack+disk_format.

We could just remove the extra ' ', but that would mean 3 extra tests
on armhf and 3 extra tests on arm64. At least x86 would have a test
for each combination.

I'd like to avoid adding more arm* tests due to limited capacity, so
the new goal is to have the 6 combination of toolstack-disk_format
spread on both arm*, where each get to test all possible disk_format.
On x86, we are also going to tests all 6 combinations, which might all
happen on dom0=amd64 if we don't test dom0=i386, or they will be
spread on both kind of dom0.

To avoid too many changes in jobs on x86 with the current list of
jobs, we reverse the order of "i386" and "amd64".

Jobs changes on xen-unstable and most branches:
x86:
+ test-amd64-amd64-xl-raw
+ test-amd64-i386-libvirt-qcow2
arm*:
- test-armhf-armhf-libvirt-qcow2
- test-armhf-armhf-libvirt-raw
- test-armhf-armhf-xl-vhd
+ test-armhf-armhf-libvirt-vhd
+ test-armhf-armhf-xl-qcow2
+ test-armhf-armhf-xl-raw

Jobs changes on linux-{linus,next,6.1}:
x86:
+ test-amd64-amd64-libvirt-vhd
+ test-amd64-amd64-xl-qcow2
+ test-amd64-amd64-xl-raw
arm*:
- test-armhf-armhf-libvirt-qcow2
- test-armhf-armhf-libvirt-raw
- test-armhf-armhf-xl-vhd
+ test-armhf-armhf-libvirt-vhd
+ test-armhf-armhf-xl-qcow2
+ test-armhf-armhf-xl-raw

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
13 months agomake-fligh: Fix freebsd guest test test-id
Anthony PERARD [Wed, 13 Mar 2024 14:28:47 +0000 (14:28 +0000)]
make-fligh: Fix freebsd guest test test-id

Current freebsd as guest tests rely on the variable $qemuu_suffix, but
that one may or may not be set yet, and can't be rely upon. It isn't
set on the first iteration which call test_matrix_do_one(), with
xenarch=amd64 dom0arch=i386, but it is on the second call with
xenarch=amd64 dom0arch=amd64.

Only flights on branch linux-linus and linux-6.1 are changes (and the
disabled linux-next branch). This is because the first call of
test_matrix_do_one() is with dom0arch=amd64 as we skip "i386". On
those branches, we have this changes in jobs names:

-test-amd64-amd64-freebsd11-amd64
-test-amd64-amd64-freebsd12-amd64
+test-amd64-amd64-qemuu-freebsd11-amd64
+test-amd64-amd64-qemuu-freebsd12-amd64

This patch will have the benefit to avoid the opposite changes in all the other
branches as we are going to make some changes to job generation.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
14 months agots-kernel-build: Add driver for Intel X722
Anthony PERARD [Fri, 23 Feb 2024 19:10:27 +0000 (19:10 +0000)]
ts-kernel-build: Add driver for Intel X722

This card is used on "septiner*".

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
14 months agots-kernel-build: Add CONFIG_IXGBE, for X550 NIC
Anthony PERARD [Thu, 30 Nov 2023 15:36:36 +0000 (15:36 +0000)]
ts-kernel-build: Add CONFIG_IXGBE, for X550 NIC

The machine rubintos* have:
    Intel Corporation Ethernet Controller X550
which want ixgbe kernel module.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
14 months agoap-common: Switch to Linux 6.1 by default on Arm
Anthony PERARD [Tue, 20 Feb 2024 14:12:17 +0000 (14:12 +0000)]
ap-common: Switch to Linux 6.1 by default on Arm

The "linux-arm-xen" tree currently point to Linux 5.4.17 + 3 commits:
  69117aaf07f7 ("xen/privcmd: replace kcalloc() by kvcalloc() when allocating empty pages")
  bca5bc0229a6 ("xen/privcmd: fix error handling in mmap-resource processing")
  f0f0e602f7c9 ("xen/privcmd: drop "pages" parameter from xen_remap_pfn()")

This stable version is old and doesn't build on Debian Bookworm.

The 3 extra commits are present in at least Linux 5.15, so there's
currently no reason to have a separate tree when we could use a stable
tree that osstest can update regulary.

So switch to 6.1. This will change the default Linux tree for all Arm
jobs that aren't on "linux-*" osstest branches.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
15 months agocr-for-branches: Add Linux 6.1
Anthony PERARD [Wed, 31 Jan 2024 16:13:39 +0000 (16:13 +0000)]
cr-for-branches: Add Linux 6.1

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
15 months agoosstest: bump /boot partition to 1G
Roger Pau Monne [Mon, 15 Jan 2024 10:05:11 +0000 (11:05 +0100)]
osstest: bump /boot partition to 1G

Increase the size of the boot partition, as arm64 initrd is already exhausting
all the space:

update-initramfs: Generating /boot/initrd.img-6.7.0+
gzip: stdout: No space left on device

Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
16 months agots-kernel-build: Remove use of $XEN_TARGET_ARCH
Anthony PERARD [Fri, 8 Dec 2023 15:38:35 +0000 (15:38 +0000)]
ts-kernel-build: Remove use of $XEN_TARGET_ARCH

osstest doesn't set $XEN_TARGET_ARCH, and this is a Xen build
variable, not a Linux build one. So, this code have been running with
an empty $XEN_TARGET_ARCH for a while which mean none of those
`setopt` has run in a while.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
16 months agocreate_build_jobs: Enable X86_GENERIC for i386 kernel pvops jobs.
Anthony PERARD [Mon, 4 Dec 2023 16:46:24 +0000 (16:46 +0000)]
create_build_jobs: Enable X86_GENERIC for i386 kernel pvops jobs.

This is following a failure to build "arch/x86/xen/enlighten.c" in
build-i386-pvops jobs with linux 968f35f4ab1c ("Merge tag
'v6.7-rc3-smb3-client-fixes' of git://git.samba.org/sfrench/cifs-2.6")
in linux-linus branch.

Today the cpu type for a 32-bit x86 kernel is not specified in the
used kernel config, resulting in the M686 to be used.

Instead of using the M686 which isn't even a 64-bit cpu (thus not
capable to run a Xen guest), use the X86_GENERIC variant which is
more appropriate.

Suggested-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
16 months agocreate_build_jobs: Set reset pvops_kconfig_overrides
Anthony PERARD [Mon, 4 Dec 2023 16:46:08 +0000 (16:46 +0000)]
create_build_jobs: Set reset pvops_kconfig_overrides

Those two variables, and especially "pvops_kconfig_overrides", aren't
reset for the next loop of `for arch ...`. So when
"pvops_kconfig_overrides" is for "armhf", it is also set for "arm64"
as this is the next "arch" in the loop.

Avoid this by setting default values for the variables before the
`case` command.

This changes Linux arm64 builds which will not have "CONFIG_EXT4_FS=y"
anymore, but should get "CONFIG_EXT4_FS=m" instead from
`ts-kernel-build`. But resulting build still have "CONFIG_EXT4_FS=y",
so no change.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
17 months agopreseed_base: Remove reportbug-gtk if present
Anthony PERARD [Mon, 27 Nov 2023 16:11:11 +0000 (16:11 +0000)]
preseed_base: Remove reportbug-gtk if present

The Debian installer have started to install "reportbug-gtk" which
depends on "dconf-service" and "dbus-user-session" and "systemd".

And when we try to install "sysvinit-core" we are met with the
following error message:
>    The following packages have unmet dependencies:
>     dconf-service : Depends: default-dbus-session-bus or
>        dbus-session-bus

So, remove that new package and its dependencies before moving on.

While here, add `set -x` to help debugging.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
18 months agocr-for-branches: Add Xen and QEMU 4.18 branch
osstest service owner [Fri, 27 Oct 2023 12:37:34 +0000 (12:37 +0000)]
cr-for-branches: Add Xen and QEMU 4.18 branch

21 months agots-kernel-build: add kernel kconfig for the Arndale
Luca Fancellu [Fri, 21 Jul 2023 12:16:27 +0000 (13:16 +0100)]
ts-kernel-build: add kernel kconfig for the Arndale

The Linux commit 33629d35090f5ce2b1b4ce78aa39954c603536d5 has
removed the 'snps,dwc-ahci' compatible from the generic
AHCI-platform driver control module selected by
CONFIG_SATA_AHCI_PLATFORM.

A new driver, the DWC AHCI SATA platform driver is now implemented
and handles the above compatible when CONFIG_AHCI_DWC is selected.

The module is needed for the Arndale board to have the SATA controller
working, so enable the CONFIG_AHCI_DWC as additional kconfig parameter
in ts-kernel-build.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
21 months agots-xen-build-prep: force use of git protocol v2
Anthony PERARD [Thu, 22 Jun 2023 09:09:18 +0000 (09:09 +0000)]
ts-xen-build-prep: force use of git protocol v2

Recent version of QEMU (to be 8.1) started to use meson subproject to
clone extra repo. With the example of the subproject "dtc", they do a
shallow clone with a sha1. Meson end up running:

    git fetch --depth 1 origin b6910bec11614980a21e46fbccc35934b671bd81

This command fails. I think the error message is something like "the
remote end hung up unexpectedly", on Debian Buster. A more useful
message with more recent version of git seems to be "couldn't find
remote ref".

If we allow git to communicate with the protocol v2, then the shallow
clone works. But git on buster still use v1 by default. Force it to
use v2.

This needs a git-cache-proxy version that can allow to switch to v2 of
the protocol. A bug is open upstream to track this change:
    https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1040476

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
22 months agoUse updated mirror for buster armhf, and update debian-installer
Anthony PERARD [Mon, 19 Jun 2023 14:37:14 +0000 (14:37 +0000)]
Use updated mirror for buster armhf, and update debian-installer

This reverts commit b838a9daeb3b ("production-config: Use a snapshot
for buster armhf")

Installation now fails with "Invalid Release signature", while
downloading the "Release" file of a repo.

But, using the main mirror for armhf boxes seems to work fine now.

To use the live mirror, we need to update the debian-installer as
there's a newer version of the kernel. Otherwise, installation fails
with the installer not able to find the disk.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
23 months agots-xen-build-prep: Install python3-venv for QEMU
Anthony PERARD [Mon, 22 May 2023 13:54:48 +0000 (13:54 +0000)]
ts-xen-build-prep: Install python3-venv for QEMU

Since QEMU commit 81e2b198a8cb ("configure: create a python venv
unconditionally"), a python venv is always created and this new package
is needed on Debian.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
2 years agoUpdate cache proxy certificate
Anthony PERARD [Mon, 3 Apr 2023 14:46:07 +0000 (15:46 +0100)]
Update cache proxy certificate

2 years agoPDU/IPMI: Be less aggressive with IPMI commands
Anthony PERARD [Fri, 10 Mar 2023 17:07:21 +0000 (17:07 +0000)]
PDU/IPMI: Be less aggressive with IPMI commands

The machine rimava0 doesn't power off or power on when we keep doing
"power status" then "power off" ipmi command without waiting.

Currently, osstest does:
    loop
ipmitool power status
ipmitool power off
sleep 1

This loop goes on forever, the machine never change power state.
Adding some wait in between both command works.

While here, test for timeout just after a status check instead of
sending the `power off` command again which we would never check the
result of.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoap-common: Fix qemu.git URL
Anthony PERARD [Mon, 6 Mar 2023 13:57:31 +0000 (13:57 +0000)]
ap-common: Fix qemu.git URL

Cloning from "git://git.qemu.org/qemu.git" haven't work for maybe a
month.

While "https://git.qemu.org/qemu.git" seems to work, it only redirects
to gitlab.com.

qemu.org website point to a repo hosted by gitlab.com.

So, switch to "https://gitlab.com/qemu-project/qemu.git".

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
2 years agocr-for-branches: Add Xen and QEMU 4.17 branch
Julien Grall [Thu, 1 Dec 2022 10:32:33 +0000 (10:32 +0000)]
cr-for-branches: Add Xen and QEMU 4.17 branch

Signed-off-by: Julien Grall <julien@xen.org>
2 years agolibvirt: disable Werror for non-libvirt flights
Roger Pau Monne [Tue, 13 Sep 2022 08:39:48 +0000 (10:39 +0200)]
libvirt: disable Werror for non-libvirt flights

Current usage of Werror=switch-enum by default for libvirt builds out
of the git tree causes issues when new items are added to libxl public
API enums if those are used in a switch statement in libvirt code.
This leads to libvirt build failures for seemingly unrelated libxl
changes.

In order to prevent those errors from blocking the push gate, disable
Werror for libvirt builds when not in a libvirt specific flight.

The errors will be reported on the libvirt flight, and block the
pushes there.  So the author of the changes in libxl is still expected
to send a fix to libvirt code.  This is no ideal, but the other option
is to just disable Werror for all libvirt builds and let libvirt
developers fix the breakage when they notice it.

runvar differences for a xen-unstable flight are:

--- /dev/fd/63  2022-09-15 15:57:44.340581730 +0000
+++ /dev/fd/62  2022-09-15 15:57:44.340581730 +0000
@@ -574,6 +574,10 @@
 test-xtf-amd64-amd64-3                                arch                            amd64
 test-xtf-amd64-amd64-4                                arch                            amd64
 test-xtf-amd64-amd64-5                                arch                            amd64
+build-amd64-libvirt                                   autogen_options                 --disable-werror
+build-arm64-libvirt                                   autogen_options                 --disable-werror
+build-armhf-libvirt                                   autogen_options                 --disable-werror
+build-i386-libvirt                                    autogen_options                 --disable-werror
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm    bios                            seabios
 test-amd64-amd64-qemuu-nested-amd                     bios                            seabios
 test-amd64-amd64-qemuu-nested-intel                   bios                            seabios
@@ -1217,6 +1221,10 @@
 build-arm64-libvirt                                   make_njobs                      1
 build-armhf-libvirt                                   make_njobs                      1
 build-i386-libvirt                                    make_njobs                      1
+build-amd64-libvirt                                   meson_options                   -Dwerror=false
+build-arm64-libvirt                                   meson_options                   -Dwerror=false
+build-armhf-libvirt                                   meson_options                   -Dwerror=false
+build-i386-libvirt                                    meson_options                   -Dwerror=false
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict recipe_dmrestrict               true
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict  recipe_dmrestrict               true
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict recipe_nomigrate                true

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
---
Changes since v1:
 - Use -Dwerror=false instead of -Dgit_werror=disabled
---
Cc: Ian Jackson <iwj@xenproject.org>
Cc: Anthony PERARD <anthony.perard@citrix.com>
Cc: Julien Grall <julien@xen.org>
2 years agots-libvirt-build: Rework to build with meson
Anthony PERARD [Fri, 29 Jul 2022 13:39:52 +0000 (14:39 +0100)]
ts-libvirt-build: Rework to build with meson

libvirt have switch build system to Meson + Ninja, there is no more
autogen.sh script.

Add missing dependency "libtirpc-dev" and "meson".

libvirt needs at least meson 0.54, the version avaiable in Debian
Buster is too old, but we can install it from backports.

In order to find out if Meson is used or not, we'll check if the
"autogen.sh" script is missing. And in the build dir, we will look for
a "Makefile".

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoTestSupport: Add support for installing from backport repo
Anthony PERARD [Mon, 1 Aug 2022 16:36:41 +0000 (17:36 +0100)]
TestSupport: Add support for installing from backport repo

We are going to need to install package from the debian backport
repository in order to do a build.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agots-xen-build-prep: Install newer NASM version, to build OVMF
Anthony PERARD [Thu, 19 May 2022 10:55:25 +0000 (11:55 +0100)]
ts-xen-build-prep: Install newer NASM version, to build OVMF

Recent versions of OVMF now need a version of NASM that is newer
than the one available on Debian oldstable/buster. They want to use
NASM 2.15.05 [1], which is available in Debian stable/bullseye. The
need to use a newer version started with d3febfd9ade3 ("MdePkg:
Replace Opcode with the corresponding instructions.").

There is no backport package available but the nasm package from
Debian Bullseye can easily be installed on Buster as it has few
dependencies and are already satisfied.

[1] https://github.com/tianocore/edk2/commit/6a890db161cd6d378bec3499a1e774db3f5a27a7
    ("BaseTools: Upgrade the version of NASM tool")

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoosstest: update Debian Buster install CD media to 10.12
Roger Pau Monne [Mon, 2 May 2022 15:13:32 +0000 (17:13 +0200)]
osstest: update Debian Buster install CD media to 10.12

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
3 years agoosstest: install irqbalance
Roger Pau Monne [Tue, 12 Apr 2022 11:20:58 +0000 (13:20 +0200)]
osstest: install irqbalance

Or else all interrupts will get bound to (v)CPU 0.

This doesn't cause issues on small boxes, but boxes with a non-trivial
amount of CPUs can struggle without interrupts being balanced across
available vCPUs, as the number of vCPUs offered to dom0 matches the
number of physical CPUs.

For example sabro boxes (Xeon Silver 4114 x 2 sockets) would sometimes
report timeouts which seem to be solved by using irqbalance in dom0.

irqbalance is also available on Arm, so install unconditionally.

Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
3 years agoosstest: stop anacron service
Roger Pau Monne [Mon, 4 Apr 2022 09:58:47 +0000 (11:58 +0200)]
osstest: stop anacron service

Just disabling cron in rc.d is not enough. There's also anacron which
will get invoked during startup, and since apt-compat has a delay of
up to 30min it can be picked up by the leak detector if the test
finishes fast enough:

LEAKED [process 14563 sleep] process: root     14563 14556  0 07:49 ?        00:00:00 sleep 1163
LEAKED [process 14550 /bin/sh] process: root     14550  2264  0 07:49 ?        00:00:00 /bin/sh -c run-parts --report /etc/cron.daily
LEAKED [process 14551 run-parts] process: root     14551 14550  0 07:49 ?        00:00:00 run-parts --report /etc/cron.daily
LEAKED [process 14556 /bin/sh] process: root     14556 14551  0 07:49 ?        00:00:00 /bin/sh /etc/cron.daily/apt-compat

From:

http://logs.test-lab.xenproject.org/osstest/logs/169015

To prevent this disable anacron like it's done for cron.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
---
Changes since v1:
 - Don't fail if the service is not present.

3 years agoosstest: update Debian installer version
Roger Pau Monne [Mon, 28 Mar 2022 08:20:09 +0000 (10:20 +0200)]
osstest: update Debian installer version

Update the configuration file Debian version after having run
mg-debian-installer-update-all.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
3 years agocoverity: disable flight from crontab
Roger Pau Monne [Mon, 21 Feb 2022 09:46:38 +0000 (10:46 +0100)]
coverity: disable flight from crontab

We are currently doing the Coverity Scans using a github workflow.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
3 years agodaily-cron-email-real-*: Temporarily drop osstest-admin
Ian Jackson [Tue, 14 Dec 2021 14:51:05 +0000 (14:51 +0000)]
daily-cron-email-real-*: Temporarily drop osstest-admin

Roger has agreed to take on osstest admin for the moment.  Someone who
intents to take on the role long term will probably want to get CC's
of these flight reports, but it's fairly noisy.  So for now, send them
only to the lists.

Signed-off-by: Ian Jackson <iwj@xenproject.org>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
3 years agodaily-cron-email-{play,adhoc}-*: Drop Citrix email
Ian Jackson [Tue, 14 Dec 2021 14:45:46 +0000 (14:45 +0000)]
daily-cron-email-{play,adhoc}-*: Drop Citrix email

Any such adhoc flights run from cron should be reported to
osstest-admin, not my (soon to be deleted) Citrix adddress.

Now the only remaining occurrences are
  - examples
  - authorship annotation of the init script
  - crontab-cambridge, which would need updating

Signed-off-by: Ian Jackson <iwj@xenproject.org>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
3 years agots-xen-install: enable timestamp on guests logs
Roger Pau Monne [Thu, 25 Nov 2021 15:06:46 +0000 (16:06 +0100)]
ts-xen-install: enable timestamp on guests logs

Enable the timestamp feature of xenconsoled so guests logs have a time
reference. Can be helpful when debugging boot related slowness.

This requires using the XENCONSOLED_ARGS option and setting both the
log and the timestamp options. Note that setting XENCONSOLED_TRACE
will override any options in XENCONSOLED_ARGS, so they can not be used
in conjunction.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Ian Jackson <iwj@xenproject.org>
3 years agots-memdisk-try-append: Fix for fact that broken() does not return
Ian Jackson [Fri, 5 Nov 2021 17:07:49 +0000 (17:07 +0000)]
ts-memdisk-try-append: Fix for fact that broken() does not return

Signed-off-by: Ian Jackson <iwj@xenproject.org>
3 years agoTestSupport: sub broken: document that it does not return.
Ian Jackson [Fri, 5 Nov 2021 17:07:38 +0000 (17:07 +0000)]
TestSupport: sub broken: document that it does not return.

Signed-off-by: Ian Jackson <iwj@xenproject.org>
3 years agoTestSupport: sub broken: exit with status 0 for skip
Ian Jackson [Fri, 5 Nov 2021 17:06:01 +0000 (17:06 +0000)]
TestSupport: sub broken: exit with status 0 for skip

Otherwise, whatever calls us will set the step to fail (or something).

Signed-off-by: Ian Jackson <iwj@xenproject.org>
3 years agoTestSupport: sub broken: Add a newline
Ian Jackson [Fri, 5 Nov 2021 17:05:06 +0000 (17:05 +0000)]
TestSupport: sub broken: Add a newline

This suppresses printing of the the line number, which is not useful.

Signed-off-by: Ian Jackson <iwj@xenproject.org>
3 years agoTestSupport: sub broken: break out "$msg"
Ian Jackson [Fri, 5 Nov 2021 17:01:04 +0000 (17:01 +0000)]
TestSupport: sub broken: break out "$msg"

No functional change.

Signed-off-by: Ian Jackson <iwj@xenproject.org>
3 years agosg-run-job: Do not set a step status to pass unless it's running
Ian Jackson [Fri, 5 Nov 2021 18:04:47 +0000 (18:04 +0000)]
sg-run-job: Do not set a step status to pass unless it's running

This avoids skip etc. being treated as pass.

Signed-off-by: Ian Jackson <iwj@xenproject.org>
3 years agostarvation: Only count "pass" and "fail" as done jobs
Ian Jackson [Tue, 12 Oct 2021 17:25:59 +0000 (18:25 +0100)]
starvation: Only count "pass" and "fail" as done jobs

The main effect is to treat "broken" and "blocked" jobs as not part of
the flight for starvation calculations.

This is correct for commissioning flights and OK for other flights.

Signed-off-by: Ian Jackson <iwj@xenproject.org>
Release-Acked-by: Ian Jackson <iwj@xenproject.org>
3 years agoexamination: skip memdisk on non-BIOS hosts, run per-firmware on x86
Ian Jackson [Fri, 22 Oct 2021 15:21:27 +0000 (16:21 +0100)]
examination: skip memdisk on non-BIOS hosts, run per-firmware on x86

This is a combination of two changes:

ts-memdisk-try-append: skip memdisk test on non-BIOS hosts

make-flight: examine: Insist on -bios and -uefi tests on x86

  This will let us skip ts-memdisk-try-append on non-bios platforms
  without risking regression.  It will also definitely spot
  regressions which will occur on any uefi host.

  standalone-generate-dump-flight-runvars reveals the changes are as
  follows:

  New jobs
     test-amd64-i386-examine-bios     test-amd64-i386-examine-uefi
     test-amd64-amd64-examine-bios    test-amd64-amd64-examine-uefi
  added everywhere that has the corresponding plain job, namely
     osstest
     linux-*
     xen-unstable

  These jobs are just like the plain jobs, except that one of
     ,PropEq:Firmware:bios:bios
     ,PropEq:Firmware:bios:uefi
  has been added to the end of all_hostflags.

Signed-off-by: Ian Jackson <iwj@xenproject.org>
Release-Acked-by: Ian Jackson <iwj@xenproject.org>
3 years agots-memdisk-try-append: More defensive case test
Ian Jackson [Fri, 22 Oct 2021 15:05:19 +0000 (16:05 +0100)]
ts-memdisk-try-append: More defensive case test

Signed-off-by: Ian Jackson <iwj@xenproject.org>
Release-Acked-by: Ian Jackson <iwj@xenproject.org>
3 years agots-memdisk-try-append: Enable perl warnings
Ian Jackson [Fri, 22 Oct 2021 14:57:40 +0000 (15:57 +0100)]
ts-memdisk-try-append: Enable perl warnings

Signed-off-by: Ian Jackson <iwj@xenproject.org>
Release-Acked-by: Ian Jackson <iwj@xenproject.org>
3 years agots-memdisk-try-append: Reindent (nfc)
Ian Jackson [Fri, 22 Oct 2021 14:55:26 +0000 (15:55 +0100)]
ts-memdisk-try-append: Reindent (nfc)

Signed-off-by: Ian Jackson <iwj@xenproject.org>
Release-Acked-by: Ian Jackson <iwj@xenproject.org>
3 years agomake-flight: do_examine_one: add firmware argument
Ian Jackson [Fri, 22 Oct 2021 15:35:39 +0000 (16:35 +0100)]
make-flight: do_examine_one: add firmware argument

Nothing sets this yet.

No functional change (checked with standalone-generate-dump-flight-runvars)

Signed-off-by: Ian Jackson <iwj@xenproject.org>
Release-Acked-by: Ian Jackson <iwj@xenproject.org>
3 years agoproduction-config: renumbering the test lab
Ian Jackson [Sat, 4 Dec 2021 22:16:32 +0000 (22:16 +0000)]
production-config: renumbering the test lab

Our dhcp pool exhausted.  I am going to force push this.

Signed-off-by: Ian Jackson <iwj@xenproject.org>
3 years agomg-repro-setup: Make ordinary alloc: work again
Ian Jackson [Wed, 24 Nov 2021 14:34:47 +0000 (14:34 +0000)]
mg-repro-setup: Make ordinary alloc: work again

In e7febe5f6edc, we hosted an error earlier in the script.  But in the
old location, OSSTEST_TASK would always be set if statictask was.  In
the new one, that hasn't been done yet.

No release implications since it touches a by-hand utility only.

Fixes: e7febe5f6edc "mg-repro-setup: Promote an error test to before builds (nfc)"
CC: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Ian Jackson <iwj@xenproject.org>
Release-Acked-by: Ian Jackson <iwj@xenproject.org>
3 years agocr-for-branches: Xen 4.16 branch
Ian Jackson [Tue, 23 Nov 2021 16:27:04 +0000 (16:27 +0000)]
cr-for-branches: Xen 4.16 branch

Signed-off-by: Ian Jackson <iwj@xenproject.org>
3 years agoallow.all: Treat freeBSD memdisk-try-append failures as nonblocking
Ian Jackson [Mon, 22 Nov 2021 11:51:01 +0000 (11:51 +0000)]
allow.all: Treat freeBSD memdisk-try-append failures as nonblocking

In practice this currently fails on some of our hosts.  We don't seem
to have resources to investigate properly, and this will be blocking
recomissioning of hosts if we don't get it out of the way.

Signed-off-by: Ian Jackson <iwj@xenproject.org>
Release-Acked-by: Ian Jackson <iwj@xenproject.org>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
3 years agoREADME.dev: Document to use updated osstest for commissioning
Ian Jackson [Mon, 22 Nov 2021 11:50:18 +0000 (11:50 +0000)]
README.dev: Document to use updated osstest for commissioning

Signed-off-by: Ian Jackson <iwj@xenproject.org>
Release-Acked-by: Ian Jackson <iwj@xenproject.org>
3 years agots-xen-build: Pass --enable if --disable found in usage, and v.v.
Ian Jackson [Wed, 27 Oct 2021 10:53:50 +0000 (11:53 +0100)]
ts-xen-build: Pass --enable if --disable found in usage, and v.v.

The existing code works in practice if the usage message always lists
the non-default, since the unlisted-in-usage options that would be
supported, but are elided, are in any case the default.

But configure might *compute* its defaults.  In which case it will
list only one of them in the usage message.  If the computed default
is not the same as the usual default (the one implied by listing the
opposite in the usage message) we would wrongly not pass the option.

So grep for both enable and disable.

Signed-off-by: Ian Jackson <iwj@xenproject.org>
Reviewed-by: Juergen Gross <jgross@suse.com>
3 years agots-xen-build: Refactor enable/disable configure options
Ian Jackson [Wed, 27 Oct 2021 10:49:04 +0000 (11:49 +0100)]
ts-xen-build: Refactor enable/disable configure options

Replace the repeated pattern with a function to generate that code.
No significant functional change.

Signed-off-by: Ian Jackson <iwj@xenproject.org>
Reviewed-by: Juergen Gross <jgross@suse.com
>

3 years agoosstest: explicitly set either enable or disable qemu-traditional
Juergen Gross [Tue, 26 Oct 2021 13:56:45 +0000 (15:56 +0200)]
osstest: explicitly set either enable or disable qemu-traditional

Instead of setting "--enable-qemu-traditional" or not, switch to
setting "--enable-qemu-traditional" or "--disable-qemu-traditional".

This avoids a latent bug in the disable case, as the availability
of the option is tested via grep, which will otherwise grep for
nothing.

Reported-by: Ian Jackson <iwj@xenproject.org>
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Ian Jackson <iwj@xenproject.org>
Release-Acked-by: Ian Jackson <iwj@xenproject.org>
3 years agoosstest: explicitly enable building qemu-traditional
Juergen Gross [Mon, 25 Oct 2021 09:54:38 +0000 (11:54 +0200)]
osstest: explicitly enable building qemu-traditional

It is planned to no longer build qemu-traditional per default.

In order to be able to continue running tests with ioemu-stubdom run
configure with --enable-qemu-traditional.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Ian Jackson <iwj@xenproject.org>
3 years agomg-repro-setup: Check allocation feasibility at the start
Ian Jackson [Mon, 4 Oct 2021 16:55:48 +0000 (17:55 +0100)]
mg-repro-setup: Check allocation feasibility at the start

Signed-off-by: Ian Jackson <iwj@xenproject.org>
3 years agomg-allocate: feasibility check: print a reassuring message
Ian Jackson [Mon, 4 Oct 2021 16:48:54 +0000 (17:48 +0100)]
mg-allocate: feasibility check: print a reassuring message

Signed-off-by: Ian Jackson <iwj@xenproject.org>
3 years agomg-allocate: Add --dry-run mode
Ian Jackson [Mon, 4 Oct 2021 16:44:48 +0000 (17:44 +0100)]
mg-allocate: Add --dry-run mode

Signed-off-by: Ian Jackson <iwj@xenproject.org>