Integrate both 'disk-hostdev-scsi-virtio-iscsi-auth-AES' and
'hostdev-scsi-virtio-iscsi-auth' as the new test infrastructure tests
both legacy and 'secret' object cases.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Peter Krempa [Wed, 17 Jun 2020 13:24:05 +0000 (15:24 +0200)]
qemuxml2argvtest: hostdev-scsi-lsi: Integrate 'hostdev-scsi-lsi-iscsi-auth' case
We can add the authenticated iSCSI hostdevs to the same test file.
Additionally this now covers passing secret via the 'secret' object
rather than on the command line.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Andrea Bolognani [Tue, 23 Jun 2020 09:22:49 +0000 (11:22 +0200)]
ci: Drop mips container for Debian sid
The build job for this container has been failing every single
time, and as it turns out the explanation for that is very simple:
Debian is just not going to support the mips architecture going
forward.
Reported-by: Pino Toscano <ptoscano@redhat.com> Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Peter Krempa [Tue, 23 Jun 2020 05:33:41 +0000 (07:33 +0200)]
qemuBackupBegin: Don't leak 'def' on early failures
The cleanup path expects that 'def' is assigned to 'priv->backup', but
that's not the case for early failures. Add a check to stop overwriting
of 'def' so that it can be freed.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
The above message looks like we've forgot to print the error message
itself as the line ends after 'error:'. Unfortunately with the current
API design clients have no way of actually getting the error message as
the VIR_DOMAIN_EVENT_ID_BLOCK_JOB(_2) event only reports the status but
not an error and the job then vanishes.
Fix the expectations by using vshPrintExtra instead of vshError:
Peter Krempa [Fri, 29 May 2020 11:03:53 +0000 (13:03 +0200)]
qemublocktest: Add 'snapshots' tests for block commit bitmap handling
Simulate commit between all the combinations of layers in the
'snapshots' case to see whether the code merges the correct bitmaps with
the correct depth of temporary bitmaps.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Peter Krempa [Fri, 17 Apr 2020 07:51:40 +0000 (09:51 +0200)]
qemu: Rewrite bitmap handling for block commit
Reuse qemuBlockGetBitmapMergeActions which allows removing the ad-hoc
implementation of bitmap merging for block commit. The new approach is
way simpler and more robust and also allows us to get rid of the
disabling of bitmaps done prior to the start as we actually do want to
update the bitmaps in the base.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Peter Krempa [Fri, 29 May 2020 10:55:02 +0000 (12:55 +0200)]
qemublocktest: Add 'snapshots' tests for backup bitmap handling
The 'snapshots' case has multiple layers so we need to make sure that
the bitmaps are merged with the appropriate temporary bitmaps formatted
from the allocation bitmap for any backing chain layer above.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Peter Krempa [Fri, 22 May 2020 12:48:46 +0000 (14:48 +0200)]
qemu: backup: Rewrite backup bitmap handling to the new bitmap semantics
Reuse qemuBlockGetBitmapMergeActions which allows removal of the ad-hoc
implementation of bitmap merging for backup. The new approach is simpler
and also more robust in case some of the bitmaps break as they remove
the dependency on the whole chain of bitmaps working.
The new approach also allows backups if a snapshot is created outside of
libvirt.
Additionally the code is greatly simplified.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Peter Krempa [Mon, 1 Jun 2020 10:05:49 +0000 (12:05 +0200)]
qemu: block: Add universal helper for merging dirty bitmaps for all scenarios
Add a function which allows merging bitmaps according to the new
semantics and will allow replacing all the specific ad-hoc functions
currently in use for 'backup', 'block commit', 'block copy' and will
also be usable in the future for 'block pull' and non-shared storage
migration.
The semantics are a bit quirky for the 'backup' case but these quirks
are documented and will prevent us from having two slightly different
algorithms.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Peter Krempa [Tue, 21 Apr 2020 06:17:58 +0000 (08:17 +0200)]
qemu: checkpoint: Don't merge checkpoints during deletion
Now that we've switched to the simple handling, the first thing that can
be massively simplified is checkpoint deletion. We now need to only go
through the backing chain and find the appropriately named bitmaps and
delete them, no complex lookups or merging.
Note that compared to other functions this deletes the bitmap in all
layers compared to others where we expect only exactly 1 bitmap of a
name in the backing chain to prevent potential problems.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Peter Krempa [Tue, 2 Jun 2020 13:11:25 +0000 (15:11 +0200)]
qemublocktest: Add new 'synthetic' bitmap detection and validation test case
Based on the 'snapshots' example with manual tweaks to introduce
inactive, transient, inconsistent and duplicate bitmaps in various parts
of the chain to exercise detection and new validation code.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Peter Krempa [Fri, 17 Apr 2020 07:18:49 +0000 (09:18 +0200)]
qemu: checkpoint: Don't chain bitmaps for checkpoints
Chaining bitmaps for checkpoints (disabling the active one and creating
a new) severely overcomplicated all operations in regards to bitmaps.
Specifically it requires us re-matching the on-disk state to the
internal metadata and in case of merging during block jobs it makes it
almost impossible to cover all corner cases.
Since the checkpoints and incremental backups were not yet enabled,
let's change the design to keep one bitmap per checkpoint. In case of
layered snapshots this will be filled in by using dirty-bitmap-populate.
Finally the main reason for this unnecessary complexity was the fear
that qemu's performance could degrade. In the end I think that
addressing the performance issue will be better done in qemu (e.g by
keeping an internal bitmap updated with changes and merging it
periodically back to the real bitmaps. QEMU writes out changes to disk
at shutdown so consistency is not a problem).
Removing the relationships between bitmaps frees us from complex
handling and also makes all the surrounding code more robust as one
broken bitmap doesn't necessarily invalidate whole chains of backups.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Peter Krempa [Tue, 2 Jun 2020 07:11:36 +0000 (09:11 +0200)]
qemublocktest: Extract printing of nodename list
There will be multiple places where we'll need to print nodenames from a
GSList of virStorageSource for testing purposes. Extract the code into a
function.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Peter Krempa [Thu, 28 May 2020 15:02:18 +0000 (17:02 +0200)]
qemublocktest: Disable testcases for all bitmap handling
Upcoming patches are going to rewrite and semantically modify how
bitmaps are handled during blockjobs. This is possible as incremental
backup is not yet fully enabled.
As the changes are going to be incompatible with any current test data
remove all test cases for bitmap handling during checkpoint deletion,
incremental backups, block commit, block copy, and bitmap validation
operations.
The tests will be gradually added back later after the code and
test-data is refactored.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Peter Krempa [Thu, 28 May 2020 14:25:55 +0000 (16:25 +0200)]
qemublocktest: Add 'empty' case for incremental backup test
Use the new test data when calculating incremental backup operations. As
incremental backup fails with no bitmap the test code is modified to
allow testing this case too.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Peter Krempa [Thu, 21 May 2020 11:23:40 +0000 (13:23 +0200)]
qemu: backup: Fix backup of disk skipped in an intermediate checkpoint
If a disk is not captured by one of the intermediate checkpoints the
code would fail, but we can easily calculate the bitmaps to merge
correctly by skipping over checkpoints which don't describe the disk.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Peter Krempa [Tue, 28 Apr 2020 13:03:11 +0000 (15:03 +0200)]
qemu: backup: Split up code traversing checkpoint list looking for bitmaps
The algorithm is getting quite complex. Split out the lookup of range of
backing chain storage sources and bitmaps contained in them which
correspond to one checkpoint.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Jiri Denemark [Wed, 17 Jun 2020 12:42:59 +0000 (14:42 +0200)]
cpu_map: Request test files update when adding x86 features
The CPUID data in *-{disabled,enabled}.xml convert feature names from
the corresponding *.json file into raw CPUID and MSR data and thus some
of them may need to be updated when new features are added into the CPU
map.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Erik Skultety [Fri, 19 Jun 2020 12:43:24 +0000 (14:43 +0200)]
nodedev: Makefile: Use AM_LDFLAGS_MOD_NOUNDEF with the installed library
Rather than using the AM_LDFLAGS_MOD_NOUNDEF options with the noinstall
library that will come out of libtool from
libvirt_driver_nodedev_impl_la, use it with the installed version
libvirt_driver_nodedev_la.
Broken-by-commit: c44bffb9 Fixes: https://ci.centos.org/job/libvirt-rpm/systems=libvirt-fedora-32/1155/ Signed-off-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Jonathon Jongsma [Thu, 18 Jun 2020 21:06:04 +0000 (16:06 -0500)]
docs: note node device fields that are read-only
As noted by Erik Skultety, we use the same XML schema to report
existing devices and to define new devices. However, some schema
elements are "read-only". In other words, they are used to report
information from the node device driver and cannot be used to define a
new device. Note these in the documentation.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Jonathon Jongsma [Thu, 18 Jun 2020 21:06:00 +0000 (16:06 -0500)]
nodedev: Build a non-loadable driver lib
In order to test the nodedev driver, we need to link against a
non-loadable module. Similar to other loadable modules already in the
repository, create an _impl library that can be linked against the unit
tests and then create a loadable module from that.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Jonathon Jongsma [Thu, 18 Jun 2020 21:05:59 +0000 (16:05 -0500)]
nodedev: add mdev support to virNodeDeviceCreateXML()
With recent additions to the node device xml schema, an xml schema can
now describe a mdev device sufficiently for libvirt to create and start
the device using the mdevctl utility.
Note that some of the the configuration for a mediated device must be
passed to mdevctl as a JSON-formatted file. In order to avoid creating
and cleaning up temporary files, the JSON is instead fed to stdin and we
pass the filename /dev/stdin to mdevctl. While this may not be portable,
neither are mediated devices, so I don't believe it should cause any
problems.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Jonathon Jongsma [Thu, 18 Jun 2020 21:05:58 +0000 (16:05 -0500)]
nodedev: store mdev UUID in mdev caps
In order to allow libvirt to create and start new mediated devices, we
need to be able to verify that the device has been started. In order to
do this, we'll need to save the UUID of newly-discovered devices within
the virNodeDevCapMdev structure. This allows us to search the device
list by UUID and verify whether the expected device has been started.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Jonathon Jongsma [Thu, 18 Jun 2020 21:05:57 +0000 (16:05 -0500)]
nodedev: refactor nodeDeviceFindNewDevice()
In preparation for creating mediated devices in libvirt, we will need to
wait for new mediated devices to be created as well. Refactor
nodeDeviceFindNewDevice() so that we can re-use the main logic from this
function to wait for different device types by passing a different
'find' function.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Jonathon Jongsma [Thu, 18 Jun 2020 21:05:56 +0000 (16:05 -0500)]
nodedev: add support for mdev attributes
Mediated devices support arbitrary vendor-specific attributes that can
be attached to a mediated device. These attributes are ordered, and are
written to sysfs in order after a device is created. This patch adds
support for these attributes to the mdev data types and XML schema.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Jonathon Jongsma [Thu, 18 Jun 2020 21:05:55 +0000 (16:05 -0500)]
nodedev: factor out nodeDeviceHasCapability()
Currently nodeDeviceCreateXML() and nodeDeviceDestroy() only support
NPIV HBAs, but we want to be able to create mdev devices as well. This
is a first step to enabling that support.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Jonathon Jongsma [Thu, 18 Jun 2020 21:05:54 +0000 (16:05 -0500)]
nodedev: make iommuGroup optional for mdevs
When parsing a nodedev xml file, the iommuGroup element should be
optional. This element should be read-only and is determined by the
device driver. While this is a change to existing behavior, it doesn't
break backwards-compatibility because it makes the parser less strict.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
In a few cases we might set seclabels on a path outside of
namespaces. For instance, when restoring a domain from a file,
the file is opened, relabelled and only then the namespace is
created and the FD is passed to QEMU (see v6.3.0-rc1~108 for more
info). Therefore, when restoring the label on the restore file,
we must ignore domain namespaces and restore the label directly
in the host.
This bug demonstrates itself when restoring a domain from a block
device. We don't create the block device inside the domain
namespace and thus the following error is reported at the end of
(otherwise successful) restore:
error : virProcessRunInFork:1236 : internal error: child reported (status=125): unable to stat: /dev/sda: No such file or directory
error : virProcessRunInFork:1240 : unable to stat: /dev/sda: No such file or directory
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com>
Michal Privoznik [Wed, 17 Jun 2020 09:01:05 +0000 (11:01 +0200)]
qemu: Use qemuSecurityDomainSetPathLabel() to set seclabes on not saved state files
There are two places within qemu driver that misuse
qemuSecuritySetSavedStateLabel() to set seclabels on tempfiles
that are not state files: qemuDomainScreenshot() and
qemuDomainMemoryPeek(). They are doing so because of lack of
qemuSecurityDomainSetPathLabel() at the time of their
introduction.
In all three secdrivers (well, four if you count NOP driver) the
implementation of .domainSetSavedStateLabel and
.domainSetPathLabel callbacks is the same anyway.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com>
Although this can be considered a new feature, from the user
standpoint is more of a QoL improvement.
Suggested-by: Michal Privoznik <mprivozn@redhat.com> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
formatdomain.html.in: document the NUMA cpus auto fill feature
We're not mentioning that we're replicating QEMU behavior on purpose.
First because QEMU will one day, maybe, change the behavior and
start to refuse incomplete NUMA setups, and then our documentation
is now deprecated. Second, auto filling the CPUs in the first
cell will work regardless of QEMU changes in the future.
The idea is to encourage the user to provide a complete NUMA CPU topology,
not relying on the CPU auto fill mechanic.
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
qemu_domain.c: NUMA CPUs auto-fill for incomplete topologies
Libvirt allows the user to define an incomplete NUMA topology, where
the sum of all CPUs in each cell is less than the total of VCPUs.
What ends up happening is that QEMU allocates the non-enumerated CPUs
in the first NUMA node. This behavior is being flagged as 'to be
deprecated' at least since QEMU commit ec78f8114bc4 ("numa: use
possible_cpus for not mapped CPUs check").
In [1], Maxiwell suggested that we forbid the user to define such
topologies. In his review [2], Peter Krempa pointed out that we can't
break existing guests, and suggested that Libvirt should emulate the
QEMU behavior of putting the remaining vCPUs in the first NUMA node
in these cases.
This patch implements Peter Krempa's suggestion. Since we're going
to most likely end up with disjointed NUMA configuration in node 0
after the auto-fill, we're making auto-fill dependent on QEMU_CAPS_NUMA.
A following patch will update the documentation not just to inform
about the auto-fill mechanic with incomplete NUMA topologies, but also
to discourage the user to create such topologies in the future. This
approach also makes Libvirt independent of whether QEMU changes
its current behavior since we're either auto-filling the CPUs in
node 0 or the user (hopefully) is aware that incomplete topologies,
although supported in Libvirt, are to be avoided.
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com> Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
numa_conf.c: add helper functions for cpumap operations
These helpers will be used in an auto-fill feature for incomplete
NUMA topologies in the next patch.
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com> Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Erik Skultety [Wed, 17 Jun 2020 14:17:52 +0000 (16:17 +0200)]
ci: Makefile: Mention CI_MAKE_ARGS and CI_CONFIGURE_ARGS in ci-help
Document the CI_MAKE_ARGS and CI_CONFIGURE_ARGS so that users don't have
to skim through the Makefile to be able to pass arbitrary recognized
make targets to the build system.
Signed-off-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>
qemu: do not add model when actual iface type is hostdev
No default model should be added to the interface
entry at post parse when its actual network type is hostdev
as doing so might cause a mismatch between the interface
definition and its actual device type.
Signed-off-by: Paulo de Rezende Pinatti <ppinatti@linux.ibm.com> Reviewed-by: Laine Stump <laine@redhat.com>
Michal Privoznik [Tue, 16 Jun 2020 11:01:38 +0000 (13:01 +0200)]
virsysinfo: Don't leak fw_cfg
In v6.4.0-72-g3dda889a44 I've introduced parsing and formatting
of new sysinfo type 'fwcfg'. However, I've forgot to introduce
code that would free parsed data.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
projects / libvirt.git / log