Peter Krempa [Wed, 1 Jul 2020 10:25:42 +0000 (12:25 +0200)]
conf: backup: Add 'tls' attribute for 'server' element
Allow enabling TLS for the NBD server used to do pull-mode backups. Note
that documentation already mentions 'tls', so this just implements the
schema and XML bits.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Peter Krempa [Wed, 1 Jul 2020 14:58:29 +0000 (16:58 +0200)]
conf: checkpoint: Add a flag storing whether disk 'size' is valid
Avoid printing '0' size in case when we weren't able to determine the
backup size by adding a flag whether the size is valid and interlock
printing of the field according to the flag.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Peter Krempa [Thu, 25 Jun 2020 10:49:45 +0000 (12:49 +0200)]
conf: backup: Don't explicitly forbid backup of read-only disk
Users may want to use this to create a full backup or even incremental
if the checkpoints are pre-existing. We still will not allow to create a
checkpoint on a read-only disk as that makes no sense.
Peter Krempa [Tue, 30 Jun 2020 15:56:08 +0000 (17:56 +0200)]
virQEMUDriverConfigLoadSpecificTLSEntry: Split up fetching of server-only config options
The '*_tls_x509_verify' options are relevant only when we are going to
expose a server socket as client sockets always enable verification.
Split up the macro to separate the common bits from the server bits so
that when we'll later extend support of 'nbd' and 'vxhs' disks which are
client only we can reuse the existing macros.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Laine Stump [Fri, 19 Jun 2020 02:33:28 +0000 (22:33 -0400)]
util: remove OOM error log from virGetHostnameImpl()
The strings allocated in virGetHostnameImpl() are all allocated via
g_strdup(), which will exit on OOM anyway, so the call to
virReportOOMError() is redundant, and removing it allows slight
modification to the code, in particular the cleanup label can be
eliminated.
Signed-off-by: Laine Stump <laine@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Laine Stump [Fri, 19 Jun 2020 21:40:17 +0000 (17:40 -0400)]
network: fix memory leak in networkBuildDhcpDaemonCommandLine()
hostsfilestr was not being freed. This will be turned into g_autofree
in an upcoming patch converting a lot more of the same file to using
g_auto*, but I wanted to make a separate patch for this first so the
other patch is simpler to review (and to make backporting easier).
Laine Stump [Thu, 18 Jun 2020 23:16:33 +0000 (19:16 -0400)]
use g_autoptr for all xmlBuffers
AUTOPTR_CLEANUP_FUNC is set to xmlBufferFree() in util/virxml.h (This
is actually new - added accidentally (but fortunately harmlessly!) in
commit 257aba2dafe. I had added it along with the hunks in this patch,
then decided to remove it and submit separately, but missed taking out
the hunk in virxml.h)
Signed-off-by: Laine Stump <laine@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Laine Stump [Thu, 18 Jun 2020 16:49:09 +0000 (12:49 -0400)]
conf, vmx: check for OOM after calling xmlBufferCreate()
Although libvirt itself uses g_malloc0() and friends, which exit when
there isn't enouogh memory, libxml2 uses standard malloc(), which just
returns NULL on OOM - this means we must check for NULL on return from
any libxml2 functions that allocate memory.
xmlBufferCreate(), for example, might return NULL, and we don't always
check for it. This patch adds checks where it isn't already done.
(NB: Although libxml2 has a provision for changing behavior on OOM (by
calling xmlMemSetup() to change what functions are used to
allocating/freeing memory), we can't use that, since parts of libvirt
code end up in libvirt.so, which is linked and called directly by
applications that may themselves use libxml2 (and may have already set
their own alternate malloc()), e.g. drivers like esx which live totally
in the library rather than a separate process.)
Signed-off-by: Laine Stump <laine@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Andrea Bolognani [Mon, 29 Jun 2020 17:00:36 +0000 (19:00 +0200)]
cirrus: Generate jobs dynamically
Instead of having static job definitions for FreeBSD and macOS,
use a generic template for both and fill in the details that are
actually different, such as the list of packages to install, in
the GitLab CI job, right before calling cirrus-run.
The target-specific information are provided by lcitool, so that
keeping them up to date is just a matter of running the refresh
script when necessary.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com>
The root cause flaw is that non-root libvirtd is using /etc/libvirt for
its hooks. Traditionally that has been harmless though since we checked
whether we could access the hook file and degraded gracefully. We need
the same access check for iterating over the hook directory.
Long term we should make it possible to have an unprivileged hook dir
under $HOME.
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
With the recent update of Fedora rawhide I've noticed
virnettlssessiontest and virnettlscontexttest failing with:
Our own certificate servercertreq-ctx.pem failed validation
against cacertreq-ctx.pem: The certificate uses an insecure
algorithm
This is result of Fedora changes to support strong crypto [1]. RSA
with 1024 bit key is viewed as legacy and thus insecure. Generate
a new private key then. Moreover, switch to EC which is not only
shorter but also not deprecated that often as RSA. Generated
using the following command:
Andrea Bolognani [Fri, 26 Jun 2020 18:11:58 +0000 (20:11 +0200)]
ci: Run all jobs, for all branches, all the time
After recent changes (increasing the parallelism of the pipeline
by reducing the number of stages, introducing FreeBSD builds that
take longer than any other job), the difference between running
the full pipeline or a reduced one has basically disappeared: in
both cases, the completion time is around 25-35 minutes depending
on whether containers need to be rebuilt and how many shared
runners are available.
Reduce the complexity of our .gitlab-ci.yml and make things
simpler for contributors by simply always running all jobs.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Michal Privoznik [Sat, 27 Jun 2020 07:09:39 +0000 (09:09 +0200)]
domain_conf: Remove zPCI validation from formatter
In 076591009ad a validation code was added to
virDomainDeviceInfoFormat() which reports an error if zPCI
address entered in was incomplete. But, there are two problems
with this approach.
The first problem is the placement of the code - it doesn't
belong into XML formatter rather than XML validator.
The second one is that at the point of formatting XML the post
parse callback has run and thus filled in required info.
Therefore this check can never do something useful and instead of
moving it into validator, it's removed completely.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Michal Privoznik [Sat, 27 Jun 2020 08:49:37 +0000 (10:49 +0200)]
qemu_validate: Fix how qemuValidateDomainDeviceDefZPCIAddress() is called
To make the code future proof, the rest of the
qemuValidateDomainDeviceDefAddress() has to be executed (even
though there is nothing there yet) instead of returning directly.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>
If initializing test monitor in testQemuHotplugCpuPrepare()
fails, the control jumps to error label where
testQemuHotplugCpuDataFree() is called. But since the data->mon
is NULL due to aforementioned failure,
qemuMonitorTestGetMonitor() dereferences a NULL pointer leading
to a SIGSEGV.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>
domain_conf.c: skip checking ZPCI address is incomplete if not present
Commit 076591009ad1 ("conf: fix zPCI address auto-generation on
s390") is doing a check for virZPCIDeviceAddressIsIncomplete()
prior to checking if the device has a ZPCI address at all. This
results in errors like these when starting libvirt:
error : virDomainDeviceInfoFormat:7527 : internal error:
Missing uid or fid attribute of zPCI address
Fix it by moving virZPCIDeviceAddressIsIncomplete() after the
check done by virZPCIDeviceAddressIsPresent().
Fixes: 076591009ad11ec108521b52a4945d0f895fa160 Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>
1. Test for auto-generating uids while specifying valid fids
2. Test for auto-generating fids while specifying valid uids
3. Test for parse error while specifying a valid fid and an invalid
uid
4. Test for parse error while specifying two ZPCI devices with same
uid and fid addresses
5. Test for parse error when both uid and fid are set to zero
6. Test for error while specifying uid and not providing ZPCI
capability.
Signed-off-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Signed-off-by: Bjoern Walk <bwalk@linux.ibm.com> Signed-off-by: Shalini Chellathurai Saroja <shalini@linux.ibm.com> Reviewed-by: Bjoern Walk <bwalk@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>
we expect uid='0x0001' (or the next available uid for the domain).
However, we get a parsing error:
$ virsh define zpci.xml
error: XML error: Invalid PCI address uid='0x0000', must be > 0x0000
and <= 0xffff
Secondly, when the uid is specified explicitly with the invalid
numerical value '0x0000', we actually expect the parsing error above.
However, the domain is being defined and the uid value is silently
changed to a valid value.
The first issue is a bug and the second one is undesired behaviour, and
both issues are related to how we (in-band) signal invalid values for
uid and fid. So let's fix the XML parsing to do validation based on what
is actually specified in the XML.
The first issue is also related to the current code behaviour, which
is, if either uid or fid is specified by the user, it is incorrectly
assumed that both uid and fid are specified. This bug is fixed by
identifying when the user specified ZPCI address is incomplete and
auto-generating the missing ZPCI address.
Signed-off-by: Bjoern Walk <bwalk@linux.ibm.com> Signed-off-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Signed-off-by: Shalini Chellathurai Saroja <shalini@linux.ibm.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Michal Privoznik [Thu, 25 Jun 2020 14:34:30 +0000 (16:34 +0200)]
qemuxml2xmltest: Set dummy non-hypervisor drivers
When parsing domain XML post parse callbacks are run and one of
them might try and call API from a non-hypervisor driver (e.g.
just like qemuDomainDeviceNetDefPostParse() is doing - it calls a
network API). To avoid this in the test suite, set dummy drivers,
which renders all non-hypervisor APIs return error.
This mimics what qemuxml2argvtest does.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Jonathon Jongsma [Thu, 25 Jun 2020 20:18:23 +0000 (15:18 -0500)]
tests: ensure failure if input file doesn't exist
When using the DO_TEST_PARSE_ERROR() macro, a failure to parse the input
file is considered a successful test. However, if the input file is
totally missing, that should be distinguished from a parsing error and
not be treated as a test success.
The function virDomainDefParseFile() simply returns NULL for any parse
failure, including a missing file. So we need to explicitly check
whether the file exists first, and fail the test if it is missing.
Jonathon Jongsma [Thu, 25 Jun 2020 20:18:22 +0000 (15:18 -0500)]
qemu: ramfb video device doesn't support PCI address
Although a ramfb video device is not a PCI device, we don't currently
report an error for ramfb device definitions containing a PCI address.
However, a guest configured with such a device will fail to start:
# virsh start test1
error: Failed to start domain test1
error: internal error: qemu unexpectedly closed the monitor: 2020-06-16T05:23:02.759221Z qemu-kvm: -device ramfb,id=video0,bus=pcie.0,addr=0x1: Device 'ramfb' can't go on PCIE bus
A better approach is to reject any device definitions that contain PCI
addresses. While this is a change in behavior, any existing
configurations were non-functional.
Michal Privoznik [Thu, 25 Jun 2020 07:27:30 +0000 (09:27 +0200)]
qemuDomainDeviceNetDefPostParse: Switch order of conditions
A few commits back (in v6.4.0-131-gbdb8f2e418) the post parse
function for domain interface was changed so that it doesn't fill
in model for hostdev types of interfaces (including network type
interfaces which would end up hostdevs).
While the idea is sound, the execution can be a bit better:
virDomainNetResolveActualType() which is used to determine
runtime type of given interface is heavy gun - it connects to
network driver, fetches network XML, parses it. This all is
followed by check whether the interface doesn't already have
model set (from domain XML).
If we switch the order of these two checks then the short circuit
evaluation will ensure the expensive check is done only if really
needed.
This commit in fact fixes qemuxml2xmltest which due to lacking
fake network driver tries to connect to network:///session and
start the virtnetworkd. Fortunately, because of v6.3.0-25-gf28fbb05d3 it fails to do so and
virDomainNetResolveActualType() returns -1. The only reason we
don't see the test failing is because our input XMLs have model
and thus we are saved by the latter (now former) check.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Laine Stump <laine@redhat.com>
Andrea Bolognani [Thu, 25 Jun 2020 12:28:25 +0000 (14:28 +0200)]
ci: Refresh Dockerfiles
All targets get pip3, which is now part of the base system, and
the number of native packages included in the MinGW containers is
significantly reduced.
The corresponding libvirt-ci commit is 4ff697ba0b5d.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Peter Krempa [Wed, 24 Jun 2020 09:41:28 +0000 (11:41 +0200)]
kbase: incrementalbackupinternals: Replace bash with pseudocode
Simplify the docs and reduce maintenance burden by just describing the
algorithm by a pseudo-language. Users are encouraged to use libvirt
anyways and projects such as oVirt which do some management of storage
themselves are unlikely to use bash anyways.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Peter Krempa [Wed, 24 Jun 2020 09:26:29 +0000 (11:26 +0200)]
kbase: incrementalbackupinternals: Add secion on bitmap handling in shell
Add a section that outlines usage of tools to handle bitmaps and
introduce terms corresponding to the output of qemu-img to be used in
further sections.
With this we can simplify the section about checking bitmap health as we
don't have to explain the qemu-img output but can refer to the newly
defined terms.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Laine Stump [Wed, 17 Jun 2020 20:18:23 +0000 (16:18 -0400)]
qemu: auto-assign hostdev devices to PCIe
Until recently, an <interface type='network'> would automatically be
assigned model "rtl8139", which in turn would lead to the device being
assigned a PCI address on a conventional PCI controller (i.e. a
pcie-to-pci-bridge). If the network was a typical Linux host
bridge-based network that used an emulated device, this would be
appropriate, since the guest actually would get an emulated rtl8139
NIC, and that device is a conventional PCI device.
However, if the network being used was a pool of hostdev devices, the
guest would get an actual PCIe network device assigned from the host
via VFIO; while the interface model in that case is irrelevant for the
QEMU commandline to assign the device, the PCI address would have
already been assigned prior to runtime, so the address assignment
would be done based on the model='rtl8139' - a conventional PCI
device. VFIO assignment of a PCIe device to a conventional PCI slot
works, but we would rather have these devices in a PCIe slot.
Since commit bdb8f2e4186, if <interface type='network'> points to a
etwork that is a pool of hostdev devices, the interface model will be
_unset_ by default. This patch uses that information when deciding
what type of slot to assign to the device: since all hostdev network
interfaces are SR-IOV VFs, and *all* SR-IOV network cards are PCIe, it
is safe to assume that the VFs are PCIe and we should assign then to a
PCIe slot in the guest.
Signed-off-by: Laine Stump <laine@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>
qemu_domainjob: moved domain job APIs to a separate file
All the domain job related APIs were present in `qemu_domain.c`
along with the other domain APIs. In this patch, we move all the
qemu domain job APIs into a separate file.
Also, in this process, `qemuDomainTrackJob()`,
`qemuDomainFreeJob()`, `qemuDomainInitJob()` and
`qemuDomainObjSaveStatus()` were converted to a non-static
funciton and exposed using `qemu_domain.h`.
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
qemu_domain: Avoid using qemuDomainObjPrivatePtr as parameter
In functions `qemuDomainObjInitJob`, `qemuDomainObjResetJob`,
`qemuDomainObjResetAgentJob`, `qemuDomainObjResetAsyncJob`,
`qemuDomainObjFreeJob`, `qemuDomainJobAllowed`,
`qemuDomainNestedJobAllowed` we avoid sending the complete
qemuDomainObjPrivatePtr as parameter and instead just send
qemuDomainJobObjPtr.
This is done in a effort to separating the qemu-job APIs into
a spearate file.
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Mon, 15 Jun 2020 10:53:48 +0000 (12:53 +0200)]
leaseshelper: Report more errors
Some functions or code paths that may fail don't report error
(e.g. when acquiring PID file fails) leading to a silent quit
of the leaseshelper. This makes it super hard for us and users
to debug what is happening. Fortunately, dnsmasq captures both
stdout and stderr so we can write an error message there.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Michal Privoznik [Thu, 11 Jun 2020 14:43:22 +0000 (16:43 +0200)]
leaseshelper: Wait to acquire PID file
On a DHCP transaction, dnsmasq runs our leases helper which
updates corresponding JSON files. While one dnsmasq won't run the
leaseshelper in parallel, two dnsmasqs (from two distinct
networks) might. To avoid corrupting JSON file, the leaseshelper
acquires PID file first. Well, the way it's acquiring it is not
ideal - it calls virPidFileAcquirePath(wait = false); which
means, that either it acquires the PID file instantly or returns
an error and does not touch the JSON at all. This in turn means
that there might be a leases record missing. With wait = true,
this won't happen.
qemu: fix missing error reports in capabilities probing
The "virsh domcapabilities --arch ppc64" command will fail with no
error message set if qemu-system-ppc64 is not currently installed.
This is because virQEMUCapsCacheLookup() does not report any error
message if not capabilities can be obtained from the cache. Almost
all methods calling this expected an error to be set on failure.
Once that's fixed though, we see a further bug which is that
virQEMUCapsCacheLookupDefault() is passing a NULL binary path to
virQEMUCapsCacheLookup(), so we need to catch that too.
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Jonathon Jongsma [Tue, 23 Jun 2020 18:29:56 +0000 (13:29 -0500)]
qemu: format 'ramfb' attribute for mediated devices
It's possible to use ramfb as the boot display of an assigned vgpu
device. This was introduced in 4b95738c, but unfortunately the attribute
was not formatted into the xml output for such a device. This patch
fixes that oversight and adds a xml2xml test to verify proper behavior.
The XML format used for QEMU capabilities is not required to be
stable across releases, as we invalidate the cache whenever the
libvirt binary changes.
We none the less always try to parse te entire XML file before
we do any validity checks. Thus if we change the format of any
part of the data, or change permitted values for enums, then
libvirtd logs will be spammed with errors.
These are not in fact errors, but an expected scenario.
This change makes the loading code validate the cache timestamp
against the libvirtd timestamp immediately. If they don't match
then we stop loading the rest of the XML file.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
projects / libvirt.git / log