]> xenbits.xensource.com Git - libvirt.git/log
libvirt.git
4 years agovirhook: support hooks placed in $driver.d/
Dmitry Nesterenko [Tue, 23 Jun 2020 14:45:34 +0000 (17:45 +0300)]
virhook: support hooks placed in $driver.d/

It is easier for management software (and subsequently
distributions) to install hook script under
/etc/libvirt/hooks/$driver.d/ and have libvirt execute them in
alphabetical order. To maintain backwards compatibility,
/etc/libvirt/hooks/$driver hook script is executed the first
followed by scripts from the $driver.d directory.

The stdio is chained between the scripts. The output of the first
script is input of the second and so on.

Signed-off-by: Dmitry Nesterenko <dmitry.nesterenko@virtuozzo.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agovirhook: Separate hook script invocation into a function
Dmitry Nesterenko [Tue, 23 Jun 2020 14:45:33 +0000 (17:45 +0300)]
virhook: Separate hook script invocation into a function

This refactor is needed to support support hooks placed in
several files.

Signed-off-by: Dmitry Nesterenko <dmitry.nesterenko@virtuozzo.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agoNEWS.rst: update for the new TPM Proxy device
Daniel Henrique Barboza [Wed, 10 Jun 2020 18:11:52 +0000 (15:11 -0300)]
NEWS.rst: update for the new TPM Proxy device

Tested-by: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
4 years agotests/qemuxml2argvtest.c: add TPM Proxy command line tests
Daniel Henrique Barboza [Wed, 10 Jun 2020 18:11:51 +0000 (15:11 -0300)]
tests/qemuxml2argvtest.c: add TPM Proxy command line tests

Add tests for both supported scenarios: a single TPM Proxy and
a TPM Proxy with a regular TPM device in the same domain.

Tested-by: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
4 years agoqemu: build command line for the TPM Proxy device
Daniel Henrique Barboza [Wed, 10 Jun 2020 18:11:50 +0000 (15:11 -0300)]
qemu: build command line for the TPM Proxy device

This patch wraps it up all the wiring done in previous patches,
enabling a PPC64 guest to launch a guest using a TPM Proxy
device.

Note that device validation is already being done in qemu_validate.c,
qemuValidateDomainDeviceDefTPM(), on domain define time. We don't
need to verify QEMU capabilities for this device again inside
qemu_command.c.

Tested-by: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
4 years agotests: add XML schema tests for the TPM Proxy device
Daniel Henrique Barboza [Wed, 10 Jun 2020 18:11:49 +0000 (15:11 -0300)]
tests: add XML schema tests for the TPM Proxy device

This tests aims to exercise how a TPM Proxy device can be
added in the domain, either alone or with a regular TPM
device. It also ensures that we do not allow bogus scenarios
to slip by.

Tested-by: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
4 years agoqemu: add validations after TPM Proxy model introduction
Daniel Henrique Barboza [Wed, 10 Jun 2020 18:11:48 +0000 (15:11 -0300)]
qemu: add validations after TPM Proxy model introduction

Previous patch handled the conversion of def->tpm to the
array def->tpms and the XML parsing logic. This patch handles
the validations needed to ensure the intended behavior.

The existing qemuValidateDomainDeviceDefTPM() function was updated
to guarantee that the VIR_DOMAIN_TPM_MODEL_SPAPR_PROXY model is
exclusive to PPC64 guests and to the VIR_DOMAIN_TPM_TYPE_PASSTHROUGH
backend.

A new function called qemuDomainDefTPMsPostParse() was added to guarantee
that the following combinations in the same domain are valid:

- a single TPM device
- a single TPM Proxy device
- a single TPM + single TPM Proxy devices

And these combinations in the same domain are NOT valid:

- 2 or more TPM devices
- 2 or more TPM Proxy devices

Tested-by: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
4 years agoconf, qemu, security, tests: introducing 'def->tpms' array
Daniel Henrique Barboza [Wed, 10 Jun 2020 18:11:47 +0000 (15:11 -0300)]
conf, qemu, security, tests: introducing 'def->tpms' array

A TPM Proxy device can coexist with a regular TPM, but the
current domain definition supports only a single TPM device
in the 'tpm' pointer. This patch replaces this existing pointer
in the domain definition to an array of TPM devices.

All files that references the old pointer were adapted to
handle the new array instead. virDomainDefParseXML() TPM related
code was adapted to handle the parsing of an extra TPM device.
TPM validations after this new scenario will be updated in
the next patch.

Tested-by: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
4 years agoqemu_tpm, security, tests: change 'switch' clauses for 'if'
Daniel Henrique Barboza [Wed, 10 Jun 2020 18:11:46 +0000 (15:11 -0300)]
qemu_tpm, security, tests: change 'switch' clauses for 'if'

This trivial rework is aimed to reduce the amount of line changes
made by the next patch, when 'def->tpm' will become a 'def->tpms'
array.

Instead of using a 'switch' where only the VIR_DOMAIN_TPM_TYPE_EMULATOR
label does something, use an 'if' clause instead.

Tested-by: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
4 years agoqemu_extdevice.c: remove unneeded 'ret' variable
Daniel Henrique Barboza [Wed, 10 Jun 2020 18:11:45 +0000 (15:11 -0300)]
qemu_extdevice.c: remove unneeded 'ret' variable

qemuExtDevicesInitPaths() does not need 'ret'.

Tested-by: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
4 years agoqemu: Extend QEMU capabilities with 'spapr-tpm-proxy'
Daniel Henrique Barboza [Wed, 10 Jun 2020 18:11:44 +0000 (15:11 -0300)]
qemu: Extend QEMU capabilities with 'spapr-tpm-proxy'

Expose the TPM Proxy support for PPC64 guests by creating a new
cap called QEMU_CAPS_DEVICE_SPAPR_TPM_PROXY.

This device is part of the machinery the guest need to orchestrate
with the PPC64 Ultravisor the transition to the Secure VM (SVM)
mode. Inside QEMU, this device will be used with the H_TPM_COMM
hypercall to connect with the TPM Resource Manager, enabling
the guest to open and close TPM sessions with the host TPM.

Tested-by: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
4 years agodocs: documentation and schema for the new TPM Proxy model
Daniel Henrique Barboza [Wed, 10 Jun 2020 18:11:43 +0000 (15:11 -0300)]
docs: documentation and schema for the new TPM Proxy model

QEMU 4.1.0 introduced a new device type called TPM Proxy, currently
implemented by PPC64 guests via a new virtual device called
'spapr-tpm-proxy' (see QEMU 0fb6bd073230 for more info).

The TPM Proxy device interacts with a TPM Resource Manager, a host
device capable of multiplexing the host TPM with multiple processes.
This allows multiple guests to access some TPM features at the
same time. Note that this mode of operation does not provide
full TPM features to be available for the guest - for that case
the guest still needs to assign a vTPM device (tpm-spapr for
PPC64 guests). Although redundant, there is currently no technical
limitation for a guest to assign both a vTPM and a TPM Proxy at the
same time.

This patch adds documentation and schema for a new TPM model
type called 'spapr-tpm-proxy' that creates this new TPM Proxy
device. This model is valid only for the 'passthrough' backend.
An example of a TPM Proxy device connected to a TPM Resource Manager
'/dev/tpmrm0' will look like this:

<tpm model='spapr-tpm-proxy'>
  <backend type='passthrough'>
    <device path='/dev/tpmrm0'/>
  </backend>
</tpm>

Tested-by: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
4 years agoqemu_capabilities.c: modernize virQEMUCapsCacheLookupDefault
Daniel Henrique Barboza [Mon, 22 Jun 2020 13:32:33 +0000 (10:32 -0300)]
qemu_capabilities.c: modernize virQEMUCapsCacheLookupDefault

Use g_autoptr() in qemuCaps to get rid of a virObjectUnref call,
a 'cleanup' label and the 'ret' pointer.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
4 years agoqemu_command: Fix IOMMU validator name in two comments
Michal Privoznik [Tue, 23 Jun 2020 13:55:42 +0000 (15:55 +0200)]
qemu_command: Fix IOMMU validator name in two comments

When building command line for IOMMU or machine, there are two
comments which mention function that validate IOMMU. But they
both refer to old name which was changed in v6.3.0-rc1~246.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
4 years agoqemu: format address wdith on intel-iommu command line
Menno Lageman [Thu, 4 Jun 2020 07:42:43 +0000 (09:42 +0200)]
qemu: format address wdith on intel-iommu command line

Format the address width attribute. Depending on the version of
QEMU it is named 'aw-bits' or 'x-aw-bits'.

Signed-off-by: Menno Lageman <menno.lageman@oracle.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agoconf: add address width attribute to iommu
Menno Lageman [Thu, 4 Jun 2020 07:42:42 +0000 (09:42 +0200)]
conf: add address width attribute to iommu

Add a new aw_bits attribute to the iommu device to control
the address width of the intel-iommu

Signed-off-by Menno Lageman <menno.lageman@oracle.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agoqemu: Correct the log name for qemu_security.c
Liao Pingfang [Tue, 23 Jun 2020 07:58:12 +0000 (15:58 +0800)]
qemu: Correct the log name for qemu_security.c

Correct the log name for qemu_security.c to qemu.qemu_security
instead of qemu.qemu_process.

Signed-off-by: Liao Pingfang <liao.pingfang@zte.com.cn>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agoqemuxml2argvtest: hostdev-scsi-virtio-scsi: Integrate iSCSI authentication cases
Peter Krempa [Wed, 17 Jun 2020 13:24:05 +0000 (15:24 +0200)]
qemuxml2argvtest: hostdev-scsi-virtio-scsi: Integrate iSCSI authentication cases

Integrate both 'disk-hostdev-scsi-virtio-iscsi-auth-AES' and
'hostdev-scsi-virtio-iscsi-auth' as the new test infrastructure tests
both legacy and 'secret' object cases.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agoqemuxml2argvtest: hostdev-scsi-virtio-scsi: Integrate 'hostdev-scsi-virtio-iscsi...
Peter Krempa [Wed, 17 Jun 2020 13:19:36 +0000 (15:19 +0200)]
qemuxml2argvtest: hostdev-scsi-virtio-scsi: Integrate 'hostdev-scsi-virtio-iscsi' case

We can add the iSCSI hostdevs to the same test file.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agoqemuxml2argvtest: hostdev-scsi-virtio-scsi: Integrate 'hostdev-scsi-readonly'
Peter Krempa [Wed, 17 Jun 2020 13:57:13 +0000 (15:57 +0200)]
qemuxml2argvtest: hostdev-scsi-virtio-scsi: Integrate 'hostdev-scsi-readonly'

This can be tested along with other stuff.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agoqemuxml2argvtest: hostdev-scsi-virtio-scsi: Add qemu-2.8 variant
Peter Krempa [Wed, 17 Jun 2020 13:10:51 +0000 (15:10 +0200)]
qemuxml2argvtest: hostdev-scsi-virtio-scsi: Add qemu-2.8 variant

qemu-2.8 didn't yet support QEMU_CAPS_ISCSI_PASSWORD_SECRET. This
version will allow integrating multiple test cases into one.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agoqemuxml2argvtest: hostdev-scsi-virtio-scsi: Add "latest" caps version
Peter Krempa [Wed, 17 Jun 2020 13:51:57 +0000 (15:51 +0200)]
qemuxml2argvtest: hostdev-scsi-virtio-scsi: Add "latest" caps version

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agoqemuxml2argvtest: hostdev-scsi-virtio-scsi: Modernize to qemu-4.1
Peter Krempa [Wed, 17 Jun 2020 13:42:54 +0000 (15:42 +0200)]
qemuxml2argvtest: hostdev-scsi-virtio-scsi: Modernize to qemu-4.1

Modernize the current state to the pre-blockdev version of qemu to
minimize changes. Later patch will add a 'latest' case too.

Additionally this removes duplicated call of the same test.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agoqemuxml2argvtest: hostdev-scsi-lsi: Integrate 'hostdev-scsi-lsi-iscsi-auth' case
Peter Krempa [Wed, 17 Jun 2020 13:24:05 +0000 (15:24 +0200)]
qemuxml2argvtest: hostdev-scsi-lsi: Integrate 'hostdev-scsi-lsi-iscsi-auth' case

We can add the authenticated iSCSI hostdevs to the same test file.
Additionally this now covers passing secret via the 'secret' object
rather than on the command line.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agoqemuxml2argvtest: hostdev-scsi-lsi: Integrate 'hostdev-scsi-lsi-iscsi' case
Peter Krempa [Wed, 17 Jun 2020 13:19:36 +0000 (15:19 +0200)]
qemuxml2argvtest: hostdev-scsi-lsi: Integrate 'hostdev-scsi-lsi-iscsi' case

We can add the iSCSI hostdevs to the same test file.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agoqemuxml2argvtest: hostdev-scsi-lsi: Add test of readonly variant
Peter Krempa [Wed, 17 Jun 2020 13:15:17 +0000 (15:15 +0200)]
qemuxml2argvtest: hostdev-scsi-lsi: Add test of readonly variant

"hostdev-scsi-readonly" case tests the readonly disk with a virtio-scsi
controller. Add it for the 'lsi' controller test as well.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agoqemuxml2argvtest: hostdev-scsi-lsi: Add "latest" caps version
Peter Krempa [Wed, 17 Jun 2020 12:58:49 +0000 (14:58 +0200)]
qemuxml2argvtest: hostdev-scsi-lsi: Add "latest" caps version

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agoqemuxml2argvtest: hostdev-scsi-lsi: Add qemu-2.8 variant
Peter Krempa [Wed, 17 Jun 2020 13:10:51 +0000 (15:10 +0200)]
qemuxml2argvtest: hostdev-scsi-lsi: Add qemu-2.8 variant

qemu-2.8 didn't yet support QEMU_CAPS_ISCSI_PASSWORD_SECRET. This
version will allow integrating multiple test cases into one.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agoqemuxml2argvtest: hostdev-scsi-lsi: Modernize to qemu-4.1
Peter Krempa [Wed, 17 Jun 2020 12:55:04 +0000 (14:55 +0200)]
qemuxml2argvtest: hostdev-scsi-lsi: Modernize to qemu-4.1

Modernize the current state to the pre-blockdev version of qemu to
minimize changes. Later patch will add a 'latest' case too.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agoci: Drop mips container for Debian sid
Andrea Bolognani [Tue, 23 Jun 2020 09:22:49 +0000 (11:22 +0200)]
ci: Drop mips container for Debian sid

The build job for this container has been failing every single
time, and as it turns out the explanation for that is very simple:
Debian is just not going to support the mips architecture going
forward.

Reported-by: Pino Toscano <ptoscano@redhat.com>
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
4 years agoqemuBackupDiskDataCleanupOne: Free 'incrementalBitmap'
Peter Krempa [Tue, 23 Jun 2020 04:28:44 +0000 (06:28 +0200)]
qemuBackupDiskDataCleanupOne: Free 'incrementalBitmap'

The bitmap name used for the incremental backup would be leaked
otherwise.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
4 years agoqemuBackupDiskDataCleanupOne: Don't exit early when the job has started
Peter Krempa [Tue, 23 Jun 2020 04:22:54 +0000 (06:22 +0200)]
qemuBackupDiskDataCleanupOne: Don't exit early when the job has started

Originally the function was cleaning up a failed job only but now
there's other stuff that needs to be cleared too.

Make only steps which clean up after a failed job depend on the
'started' field and execute the rest of the code always.

This fixes a leak of the backup job tracking object and the blockdev-add
helper data.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
4 years agoqemuBackupDiskStarted: Fix improper dereference of array
Peter Krempa [Tue, 23 Jun 2020 05:44:45 +0000 (07:44 +0200)]
qemuBackupDiskStarted: Fix improper dereference of array

The code would repeatedly mark the first disk's blockjob as started
rather than accessing all the blockjobs. Fix the dereferencing operator.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
4 years agoqemu: backup: Initialize 'store' source properly and just once
Peter Krempa [Tue, 23 Jun 2020 04:32:04 +0000 (06:32 +0200)]
qemu: backup: Initialize 'store' source properly and just once

Two functions called in sequence both initialized the virStorageSource
backing 'store' leading to a memleak.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
4 years agoqemuBackupBegin: Don't leak 'def' on early failures
Peter Krempa [Tue, 23 Jun 2020 05:33:41 +0000 (07:33 +0200)]
qemuBackupBegin: Don't leak 'def' on early failures

The cleanup path expects that 'def' is assigned to 'priv->backup', but
that's not the case for early failures. Add a check to stop overwriting
of 'def' so that it can be freed.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
4 years agoutil: remove unused virKModConfig method
Daniel P. Berrangé [Tue, 16 Jun 2020 11:14:01 +0000 (12:14 +0100)]
util: remove unused virKModConfig method

Using virKModConfig would not simplify any existing code.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
4 years agosrc: remove redundant arg to virKModLoad
Daniel P. Berrangé [Tue, 16 Jun 2020 11:07:30 +0000 (12:07 +0100)]
src: remove redundant arg to virKModLoad

All callers except for the test suite pass the same value
for the second arg, so it can be removed, simplifying the
code.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
4 years agovirsh: cmdBlock*: Remove 'error:' prefix for an empty line
Peter Krempa [Wed, 17 Jun 2020 12:08:22 +0000 (14:08 +0200)]
virsh: cmdBlock*: Remove 'error:' prefix for an empty line

When a block copy job fails prior to reaching the synchronized phase
while we are waiting for the job to finish virsh would print the
following:

 $ virsh blockcopy backup-test vda /tmp/dst.qcow2 --wait --reuse-external --transient-job
 error:
 Copy failed

The above message looks like we've forgot to print the error message
itself as the line ends after 'error:'. Unfortunately with the current
API design clients have no way of actually getting the error message as
the VIR_DOMAIN_EVENT_ID_BLOCK_JOB(_2) event only reports the status but
not an error and the job then vanishes.

Fix the expectations by using vshPrintExtra instead of vshError:

 $ virsh blockcopy backup-test vda /tmp/dst.qcow2 --wait --reuse-external --transient-job

 Copy failed

Note that the newline is required to avoid printing the 'Copy failed'
message on the same line when printing the job progress percentage.

Inspired by https://bugzilla.redhat.com/show_bug.cgi?id=1847867

Fix the same issue also for block pull and block commit job

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agokbase: Add document outlining internals of incremental backup in qemu
Peter Krempa [Thu, 11 Jun 2020 12:50:31 +0000 (14:50 +0200)]
kbase: Add document outlining internals of incremental backup in qemu

Outline the basics and how to integrate with externally created
overlays. Other topics will continue later.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemublocktest: Add test cases for handling bitmaps during block-copy
Peter Krempa [Mon, 1 Jun 2020 15:37:14 +0000 (17:37 +0200)]
qemublocktest: Add test cases for handling bitmaps during block-copy

Test both 'basic' and 'snapshots' cases on shallow and deep copy modes.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemu: Rewrite bitmap handling for block copy
Peter Krempa [Fri, 29 May 2020 08:34:11 +0000 (10:34 +0200)]
qemu: Rewrite bitmap handling for block copy

Reuse qemuBlockGetBitmapMergeActions which allows the removal of the
ad-hoc implementation of bitmap merging for block copy.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemu: blockjob: Remove 'disabledBitmapsBase' field from commit job private data
Peter Krempa [Fri, 29 May 2020 08:52:19 +0000 (10:52 +0200)]
qemu: blockjob: Remove 'disabledBitmapsBase' field from commit job private data

New semantics of the bitmap handling don't need this. Remove the field
and all uses of it including the status XML.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemublocktest: Add 'snapshots' tests for block commit bitmap handling
Peter Krempa [Fri, 29 May 2020 11:03:53 +0000 (13:03 +0200)]
qemublocktest: Add 'snapshots' tests for block commit bitmap handling

Simulate commit between all the combinations of layers in the
'snapshots' case to see whether the code merges the correct bitmaps with
the correct depth of temporary bitmaps.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemublocktest: Add 'basic' tests for commit bitmap handling
Peter Krempa [Fri, 29 May 2020 11:01:55 +0000 (13:01 +0200)]
qemublocktest: Add 'basic' tests for commit bitmap handling

In the 'basic' case we have few bitmaps in only the top layer. Simulate
commit into the backing of the top layer and also 2 levels deep.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemu: Rewrite bitmap handling for block commit
Peter Krempa [Fri, 17 Apr 2020 07:51:40 +0000 (09:51 +0200)]
qemu: Rewrite bitmap handling for block commit

Reuse qemuBlockGetBitmapMergeActions which allows removing the ad-hoc
implementation of bitmap merging for block commit. The new approach is
way simpler and more robust and also allows us to get rid of the
disabling of bitmaps done prior to the start as we actually do want to
update the bitmaps in the base.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemublocktest: Add 'snapshots' tests for backup bitmap handling
Peter Krempa [Fri, 29 May 2020 10:55:02 +0000 (12:55 +0200)]
qemublocktest: Add 'snapshots' tests for backup bitmap handling

The 'snapshots' case has multiple layers so we need to make sure that
the bitmaps are merged with the appropriate temporary bitmaps formatted
from the allocation bitmap for any backing chain layer above.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemublocktest: Add 'basic' tests for backup bitmap handling
Peter Krempa [Fri, 29 May 2020 10:52:58 +0000 (12:52 +0200)]
qemublocktest: Add 'basic' tests for backup bitmap handling

The 'basic' case is just a single backing store layer containing the
bitmaps so we just copy the bitmaps over to the backup bitmap.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemu: backup: Rewrite backup bitmap handling to the new bitmap semantics
Peter Krempa [Fri, 22 May 2020 12:48:46 +0000 (14:48 +0200)]
qemu: backup: Rewrite backup bitmap handling to the new bitmap semantics

Reuse qemuBlockGetBitmapMergeActions which allows removal of the ad-hoc
implementation of bitmap merging for backup. The new approach is simpler
and also more robust in case some of the bitmaps break as they remove
the dependency on the whole chain of bitmaps working.

The new approach also allows backups if a snapshot is created outside of
libvirt.

Additionally the code is greatly simplified.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemu: block: Add universal helper for merging dirty bitmaps for all scenarios
Peter Krempa [Mon, 1 Jun 2020 10:05:49 +0000 (12:05 +0200)]
qemu: block: Add universal helper for merging dirty bitmaps for all scenarios

Add a function which allows merging bitmaps according to the new
semantics and will allow replacing all the specific ad-hoc functions
currently in use for 'backup', 'block commit', 'block copy' and will
also be usable in the future for 'block pull' and non-shared storage
migration.

The semantics are a bit quirky for the 'backup' case but these quirks
are documented and will prevent us from having two slightly different
algorithms.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemublocktest: Re-introduce testing of checkpoint deletion
Peter Krempa [Mon, 1 Jun 2020 08:41:16 +0000 (10:41 +0200)]
qemublocktest: Re-introduce testing of checkpoint deletion

Exercise the now arguably simpler checkpoint deletion code on the
'basic', 'snapshots', and 'synthetic' test data sets.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemublocktest: Rename TEST_CHECKPOINT_DELETE_MERGE to TEST_CHECKPOINT_DELETE
Peter Krempa [Mon, 1 Jun 2020 08:40:04 +0000 (10:40 +0200)]
qemublocktest: Rename TEST_CHECKPOINT_DELETE_MERGE to TEST_CHECKPOINT_DELETE

Also rename the helper struct and function.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemu: checkpoint: Don't merge checkpoints during deletion
Peter Krempa [Tue, 21 Apr 2020 06:17:58 +0000 (08:17 +0200)]
qemu: checkpoint: Don't merge checkpoints during deletion

Now that we've switched to the simple handling, the first thing that can
be massively simplified is checkpoint deletion. We now need to only go
through the backing chain and find the appropriately named bitmaps and
delete them, no complex lookups or merging.

Note that compared to other functions this deletes the bitmap in all
layers compared to others where we expect only exactly 1 bitmap of a
name in the backing chain to prevent potential problems.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemublocktest: Add new 'synthetic' bitmap detection and validation test case
Peter Krempa [Tue, 2 Jun 2020 13:11:25 +0000 (15:11 +0200)]
qemublocktest: Add new 'synthetic' bitmap detection and validation test case

Based on the 'snapshots' example with manual tweaks to introduce
inactive, transient, inconsistent and duplicate bitmaps in various parts
of the chain to exercise detection and new validation code.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemublocktest: Re-add bitmap validation for 'basic' and 'snapshots' cases
Peter Krempa [Tue, 2 Jun 2020 14:47:35 +0000 (16:47 +0200)]
qemublocktest: Re-add bitmap validation for 'basic' and 'snapshots' cases

Now that we've updated both the test data and the validator to new
semantics we can start testing again.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemuBlockBitmapChainIsValid: Adjust to new semantics of bitmaps
Peter Krempa [Tue, 21 Apr 2020 08:25:09 +0000 (10:25 +0200)]
qemuBlockBitmapChainIsValid: Adjust to new semantics of bitmaps

Reject duplicates and other problematic bitmaps according to the new
semantics of bitmap use in libvirt.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemublocktest: Replace 'snapshots' bitmap detection test case data
Peter Krempa [Fri, 29 May 2020 10:46:27 +0000 (12:46 +0200)]
qemublocktest: Replace 'snapshots' bitmap detection test case data

Use test data which conforms to the new semantics which changed in the
previous patch.

The test data was created by the same set of commands as originally in
commit 0b27b655b1bac480186ce80457113cd5dc34e6a1

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemublocktest: Replace 'basic' bitmap detection test case data
Peter Krempa [Fri, 29 May 2020 10:40:32 +0000 (12:40 +0200)]
qemublocktest: Replace 'basic' bitmap detection test case data

Use test data which conforms to the new semantics which changed in the
previous patch.

The test data was created by the same set of commands as originally in
commit 9aac9d5bdab039a50de2d8c627b3a1f1578ed471

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemu: checkpoint: Don't chain bitmaps for checkpoints
Peter Krempa [Fri, 17 Apr 2020 07:18:49 +0000 (09:18 +0200)]
qemu: checkpoint: Don't chain bitmaps for checkpoints

Chaining bitmaps for checkpoints (disabling the active one and creating
a new) severely overcomplicated all operations in regards to bitmaps.

Specifically it requires us re-matching the on-disk state to the
internal metadata and in case of merging during block jobs it makes it
almost impossible to cover all corner cases.

Since the checkpoints and incremental backups were not yet enabled,
let's change the design to keep one bitmap per checkpoint. In case of
layered snapshots this will be filled in by using dirty-bitmap-populate.

Finally the main reason for this unnecessary complexity was the fear
that qemu's performance could degrade. In the end I think that
addressing the performance issue will be better done in qemu (e.g by
keeping an internal bitmap updated with changes and merging it
periodically back to the real bitmaps. QEMU writes out changes to disk
at shutdown so consistency is not a problem).

Removing the relationships between bitmaps frees us from complex
handling and also makes all the surrounding code more robust as one
broken bitmap doesn't necessarily invalidate whole chains of backups.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemublocktest: Extract printing of nodename list
Peter Krempa [Tue, 2 Jun 2020 07:11:36 +0000 (09:11 +0200)]
qemublocktest: Extract printing of nodename list

There will be multiple places where we'll need to print nodenames from a
GSList of virStorageSource for testing purposes. Extract the code into a
function.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemublocktest: Delete 'synthetic' bitmap test cases
Peter Krempa [Tue, 2 Jun 2020 13:02:30 +0000 (15:02 +0200)]
qemublocktest: Delete 'synthetic' bitmap test cases

They will be replaced by a different set which will test scenarios
relevant for the new semantics.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemublocktest: Disable testcases for all bitmap handling
Peter Krempa [Thu, 28 May 2020 15:02:18 +0000 (17:02 +0200)]
qemublocktest: Disable testcases for all bitmap handling

Upcoming patches are going to rewrite and semantically modify how
bitmaps are handled during blockjobs. This is possible as incremental
backup is not yet fully enabled.

As the changes are going to be incompatible with any current test data
remove all test cases for bitmap handling during checkpoint deletion,
incremental backups, block commit, block copy, and bitmap validation
operations.

The tests will be gradually added back later after the code and
test-data is refactored.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemublocktest: Add 'empty' case for checkpoint bitmap handling
Peter Krempa [Thu, 28 May 2020 14:34:40 +0000 (16:34 +0200)]
qemublocktest: Add 'empty' case for checkpoint bitmap handling

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemublocktest: Add 'empty' case for blockcopy bitmap handling test
Peter Krempa [Thu, 28 May 2020 14:33:06 +0000 (16:33 +0200)]
qemublocktest: Add 'empty' case for blockcopy bitmap handling test

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemublocktest: Add 'empty' case for checkpoint deletion
Peter Krempa [Thu, 28 May 2020 14:30:06 +0000 (16:30 +0200)]
qemublocktest: Add 'empty' case for checkpoint deletion

Use the new test data for checkpoint deletion testing. This test also
requires modification of the internals to allow checking for test
failure.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemublocktest: Add 'empty' case for incremental backup test
Peter Krempa [Thu, 28 May 2020 14:25:55 +0000 (16:25 +0200)]
qemublocktest: Add 'empty' case for incremental backup test

Use the new test data when calculating incremental backup operations. As
incremental backup fails with no bitmap the test code is modified to
allow testing this case too.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemublocktest: Add 'empty' test case for bitmaps
Peter Krempa [Thu, 28 May 2020 14:19:07 +0000 (16:19 +0200)]
qemublocktest: Add 'empty' test case for bitmaps

Add test data for an image without bitmaps.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemu: backup: Move fetching of checkpoint list for incremental backup
Peter Krempa [Fri, 22 May 2020 07:44:22 +0000 (09:44 +0200)]
qemu: backup: Move fetching of checkpoint list for incremental backup

Fetch the checkpoint list for every disk specifically based on the new
per-disk 'incremental' field.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoconf: backup: Store incremental backup checkpoint name per-disk
Peter Krempa [Fri, 22 May 2020 07:32:23 +0000 (09:32 +0200)]
conf: backup: Store incremental backup checkpoint name per-disk

In preparation to allow heterogenous backups store the 'incremental'
field per-disk and fill it by default from the per-backup field.

Having this will be important once we'll want to allow incremental
backup working while hotplugging a new disk.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemu: backup: Fix backup of disk skipped in an intermediate checkpoint
Peter Krempa [Thu, 21 May 2020 11:23:40 +0000 (13:23 +0200)]
qemu: backup: Fix backup of disk skipped in an intermediate checkpoint

If a disk is not captured by one of the intermediate checkpoints the
code would fail, but we can easily calculate the bitmaps to merge
correctly by skipping over checkpoints which don't describe the disk.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agoqemu: backup: Split up code traversing checkpoint list looking for bitmaps
Peter Krempa [Tue, 28 Apr 2020 13:03:11 +0000 (15:03 +0200)]
qemu: backup: Split up code traversing checkpoint list looking for bitmaps

The algorithm is getting quite complex. Split out the lookup of range of
backing chain storage sources and bitmaps contained in them which
correspond to one checkpoint.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
4 years agovirsh: Fix typo of metadata
Han Han [Mon, 22 Jun 2020 02:51:10 +0000 (10:51 +0800)]
virsh: Fix typo of metadata

Signed-off-by: Han Han <hhan@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agocpu_map: Add missing AMD SVM features
Jiri Denemark [Wed, 17 Jun 2020 09:48:16 +0000 (11:48 +0200)]
cpu_map: Add missing AMD SVM features

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
4 years agocpu_map: Add missing x86 features in 0x80000008 CPUID leaf
Jiri Denemark [Wed, 17 Jun 2020 12:50:42 +0000 (14:50 +0200)]
cpu_map: Add missing x86 features in 0x80000008 CPUID leaf

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
4 years agocpu_map: Add missing x86 features in 0x7 CPUID leaf
Jiri Denemark [Wed, 17 Jun 2020 12:08:43 +0000 (14:08 +0200)]
cpu_map: Add missing x86 features in 0x7 CPUID leaf

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
4 years agocpu_map: Request test files update when adding x86 features
Jiri Denemark [Wed, 17 Jun 2020 12:42:59 +0000 (14:42 +0200)]
cpu_map: Request test files update when adding x86 features

The CPUID data in *-{disabled,enabled}.xml convert feature names from
the corresponding *.json file into raw CPUID and MSR data and thus some
of them may need to be updated when new features are added into the CPU
map.

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
4 years agonodedev: Makefile: Use AM_LDFLAGS_MOD_NOUNDEF with the installed library
Erik Skultety [Fri, 19 Jun 2020 12:43:24 +0000 (14:43 +0200)]
nodedev: Makefile: Use AM_LDFLAGS_MOD_NOUNDEF with the installed library

Rather than using the AM_LDFLAGS_MOD_NOUNDEF options with the noinstall
library that will come out of libtool from
libvirt_driver_nodedev_impl_la, use it with the installed version
libvirt_driver_nodedev_la.

Broken-by-commit: c44bffb9
Fixes: https://ci.centos.org/job/libvirt-rpm/systems=libvirt-fedora-32/1155/
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
4 years agonews: mediated devices can be created
Jonathon Jongsma [Thu, 18 Jun 2020 21:06:05 +0000 (16:06 -0500)]
news: mediated devices can be created

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agodocs: note node device fields that are read-only
Jonathon Jongsma [Thu, 18 Jun 2020 21:06:04 +0000 (16:06 -0500)]
docs: note node device fields that are read-only

As noted by Erik Skultety, we use the same XML schema to report
existing devices and to define new devices. However, some schema
elements are "read-only". In other words, they are used to report
information from the node device driver and cannot be used to define a
new device. Note these in the documentation.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agonodedev: Add testing for 'mdevctl stop'
Jonathon Jongsma [Thu, 18 Jun 2020 21:06:03 +0000 (16:06 -0500)]
nodedev: Add testing for 'mdevctl stop'

Test that we run 'mdevctl' with the proper arguments when we destroy
mediated devices with virNodeDeviceDestroy()

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agonodedev: add mdev support to virNodeDeviceDestroy()
Jonathon Jongsma [Thu, 18 Jun 2020 21:06:02 +0000 (16:06 -0500)]
nodedev: add mdev support to virNodeDeviceDestroy()

Add the ability to destroy mdev node devices via the mdevctl utility.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agonodedev: Add testing for 'mdevctl start'
Jonathon Jongsma [Thu, 18 Jun 2020 21:06:01 +0000 (16:06 -0500)]
nodedev: Add testing for 'mdevctl start'

Test that we run 'mdevctl' with the proper arguments when creating new
mediated devices with virNodeDeviceCreateXML().

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agonodedev: Build a non-loadable driver lib
Jonathon Jongsma [Thu, 18 Jun 2020 21:06:00 +0000 (16:06 -0500)]
nodedev: Build a non-loadable driver lib

In order to test the nodedev driver, we need to link against a
non-loadable module. Similar to other loadable modules already in the
repository, create an _impl library that can be linked against the unit
tests and then create a loadable module from that.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agonodedev: add mdev support to virNodeDeviceCreateXML()
Jonathon Jongsma [Thu, 18 Jun 2020 21:05:59 +0000 (16:05 -0500)]
nodedev: add mdev support to virNodeDeviceCreateXML()

With recent additions to the node device xml schema, an xml schema can
now describe a mdev device sufficiently for libvirt to create and start
the device using the mdevctl utility.

Note that some of the the configuration for a mediated device must be
passed to mdevctl as a JSON-formatted file. In order to avoid creating
and cleaning up temporary files, the JSON is instead fed to stdin and we
pass the filename /dev/stdin to mdevctl. While this may not be portable,
neither are mediated devices, so I don't believe it should cause any
problems.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agonodedev: store mdev UUID in mdev caps
Jonathon Jongsma [Thu, 18 Jun 2020 21:05:58 +0000 (16:05 -0500)]
nodedev: store mdev UUID in mdev caps

In order to allow libvirt to create and start new mediated devices, we
need to be able to verify that the device has been started. In order to
do this, we'll need to save the UUID of newly-discovered devices within
the virNodeDevCapMdev structure. This allows us to search the device
list by UUID and verify whether the expected device has been started.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agonodedev: refactor nodeDeviceFindNewDevice()
Jonathon Jongsma [Thu, 18 Jun 2020 21:05:57 +0000 (16:05 -0500)]
nodedev: refactor nodeDeviceFindNewDevice()

In preparation for creating mediated devices in libvirt, we will need to
wait for new mediated devices to be created as well. Refactor
nodeDeviceFindNewDevice() so that we can re-use the main logic from this
function to wait for different device types by passing a different
'find' function.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agonodedev: add support for mdev attributes
Jonathon Jongsma [Thu, 18 Jun 2020 21:05:56 +0000 (16:05 -0500)]
nodedev: add support for mdev attributes

Mediated devices support arbitrary vendor-specific attributes that can
be attached to a mediated device. These attributes are ordered, and are
written to sysfs in order after a device is created. This patch adds
support for these attributes to the mdev data types and XML schema.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agonodedev: factor out nodeDeviceHasCapability()
Jonathon Jongsma [Thu, 18 Jun 2020 21:05:55 +0000 (16:05 -0500)]
nodedev: factor out nodeDeviceHasCapability()

Currently nodeDeviceCreateXML() and nodeDeviceDestroy() only support
NPIV HBAs, but we want to be able to create mdev devices as well. This
is a first step to enabling that support.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agonodedev: make iommuGroup optional for mdevs
Jonathon Jongsma [Thu, 18 Jun 2020 21:05:54 +0000 (16:05 -0500)]
nodedev: make iommuGroup optional for mdevs

When parsing a nodedev xml file, the iommuGroup element should be
optional. This element should be read-only and is determined by the
device driver. While this is a change to existing behavior, it doesn't
break backwards-compatibility because it makes the parser less strict.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agoFix allocation of virDomainFSDef
Ján Tomko [Thu, 18 Jun 2020 09:58:29 +0000 (11:58 +0200)]
Fix allocation of virDomainFSDef

Some less commonly used drivers were omitted when we switched
the allocator from a plain VIR_ALLOC to virDomainFSDefNew.

https://bugzilla.redhat.com/show_bug.cgi?id=1846450

Fixes: da665fbd4858890fbb3bbf5da2a7b6ca37bb3220
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
4 years agoutil: remove outdated comment from virLogFormatString
Ján Tomko [Thu, 18 Jun 2020 13:07:28 +0000 (15:07 +0200)]
util: remove outdated comment from virLogFormatString

Introduced by commit 72ab0b6dc8129a388aff260a6e444d48495b0e93 which
added some code depending on libvirt's log format string into
qemuProcessReadLogOutput. This function was deleted by commit
932534e85f34a479c7eac174e997bfd9c85bd22d later.

Drop the comment.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Closes: https://gitlab.com/libvirt/libvirt/-/issues/35
4 years agoqemuSecurityDomainRestorePathLabel: Introduce @ignoreNS argument
Michal Privoznik [Wed, 17 Jun 2020 10:37:06 +0000 (12:37 +0200)]
qemuSecurityDomainRestorePathLabel: Introduce @ignoreNS argument

In a few cases we might set seclabels on a path outside of
namespaces. For instance, when restoring a domain from a file,
the file is opened, relabelled and only then the namespace is
created and the FD is passed to QEMU (see v6.3.0-rc1~108 for more
info). Therefore, when restoring the label on the restore file,
we must ignore domain namespaces and restore the label directly
in the host.

This bug demonstrates itself when restoring a domain from a block
device. We don't create the block device inside the domain
namespace and thus the following error is reported at the end of
(otherwise successful) restore:

error : virProcessRunInFork:1236 : internal error: child reported (status=125): unable to stat: /dev/sda: No such file or directory
error : virProcessRunInFork:1240 : unable to stat: /dev/sda: No such file or directory

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
4 years agoqemu: Rename qemuSecurityRestoreSavedStateLabel()
Michal Privoznik [Wed, 17 Jun 2020 10:40:28 +0000 (12:40 +0200)]
qemu: Rename qemuSecurityRestoreSavedStateLabel()

The function calls virSecurityManagerDomainRestorePathLabel()
after all.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
4 years agosecurity: Rename virSecurityManagerRestoreSavedStateLabel()
Michal Privoznik [Wed, 17 Jun 2020 09:32:53 +0000 (11:32 +0200)]
security: Rename virSecurityManagerRestoreSavedStateLabel()

The new name is virSecurityManagerDomainRestorePathLabel().

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
4 years agosecurity: Drop unused virSecurityManagerSetSavedStateLabel()
Michal Privoznik [Wed, 17 Jun 2020 09:12:30 +0000 (11:12 +0200)]
security: Drop unused virSecurityManagerSetSavedStateLabel()

After previous commit this function is used no more.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
4 years agoqemu: Drop unused qemuSecuritySetSavedStateLabel()
Michal Privoznik [Wed, 17 Jun 2020 09:10:49 +0000 (11:10 +0200)]
qemu: Drop unused qemuSecuritySetSavedStateLabel()

After previous commit this function is used no more.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
4 years agoqemu: Use qemuSecurityDomainSetPathLabel() to set seclabes on not saved state files
Michal Privoznik [Wed, 17 Jun 2020 09:01:05 +0000 (11:01 +0200)]
qemu: Use qemuSecurityDomainSetPathLabel() to set seclabes on not saved state files

There are two places within qemu driver that misuse
qemuSecuritySetSavedStateLabel() to set seclabels on tempfiles
that are not state files: qemuDomainScreenshot() and
qemuDomainMemoryPeek(). They are doing so because of lack of
qemuSecurityDomainSetPathLabel() at the time of their
introduction.

In all three secdrivers (well, four if you count NOP driver) the
implementation of .domainSetSavedStateLabel and
.domainSetPathLabel callbacks is the same anyway.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
4 years agoNEWS.rst: document the 'auto-fill' feature
Daniel Henrique Barboza [Thu, 18 Jun 2020 11:12:25 +0000 (08:12 -0300)]
NEWS.rst: document the 'auto-fill' feature

Although this can be considered a new feature, from the user
standpoint is more of a QoL improvement.

Suggested-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agoformatdomain.html.in: document the NUMA cpus auto fill feature
Daniel Henrique Barboza [Wed, 10 Jun 2020 18:35:53 +0000 (15:35 -0300)]
formatdomain.html.in: document the NUMA cpus auto fill feature

We're not mentioning that we're replicating QEMU behavior on purpose.
First because QEMU will one day, maybe, change the behavior and
start to refuse incomplete NUMA setups, and then our documentation
is now deprecated. Second, auto filling the CPUs in the first
cell will work regardless of QEMU changes in the future.

The idea is to encourage the user to provide a complete NUMA CPU topology,
not relying on the CPU auto fill mechanic.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agoqemuxml2xmltest.c: add NUMA vcpus auto fill tests
Daniel Henrique Barboza [Wed, 10 Jun 2020 18:35:52 +0000 (15:35 -0300)]
qemuxml2xmltest.c: add NUMA vcpus auto fill tests

Add a unit test to verify the NUMA vcpus autocomplete implemented
in the previous patch.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
4 years agoqemu_domain.c: NUMA CPUs auto-fill for incomplete topologies
Daniel Henrique Barboza [Wed, 10 Jun 2020 18:35:51 +0000 (15:35 -0300)]
qemu_domain.c: NUMA CPUs auto-fill for incomplete topologies

Libvirt allows the user to define an incomplete NUMA topology, where
the sum of all CPUs in each cell is less than the total of VCPUs.
What ends up happening is that QEMU allocates the non-enumerated CPUs
in the first NUMA node. This behavior is being flagged as 'to be
deprecated' at least since QEMU commit ec78f8114bc4 ("numa: use
possible_cpus for not mapped CPUs check").

In [1], Maxiwell suggested that we forbid the user to define such
topologies. In his review [2], Peter Krempa pointed out that we can't
break existing guests, and suggested that Libvirt should emulate the
QEMU behavior of putting the remaining vCPUs in the first NUMA node
in these cases.

This patch implements Peter Krempa's suggestion. Since we're going
to most likely end up with disjointed NUMA configuration in node 0
after the auto-fill, we're making auto-fill dependent on QEMU_CAPS_NUMA.

A following patch will update the documentation not just to inform
about the auto-fill mechanic with incomplete NUMA topologies, but also
to discourage the user to create such topologies in the future. This
approach also makes Libvirt independent of whether QEMU changes
its current behavior since we're either auto-filling the CPUs in
node 0 or the user (hopefully) is aware that incomplete topologies,
although supported in Libvirt, are to be avoided.

[1] https://www.redhat.com/archives/libvir-list/2019-June/msg00224.html
[2] https://www.redhat.com/archives/libvir-list/2019-June/msg00263.html

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>