John Ferlan [Mon, 2 Apr 2018 14:03:02 +0000 (10:03 -0400)]
lxc: Fix possible leaked @vm in lxcDomainCreateXMLWithFiles
In error paths, if we call virDomainObjListRemove we will leak
the @vm because we have called with a reffed and locked @vm.
So rather than set it to NULL, relock the @vm and allow the
virDomainObjEndAPI to perform the magic of Unlock/Unref.
John Ferlan [Mon, 23 Oct 2017 15:00:42 +0000 (11:00 -0400)]
bhyve: Use virDomainObjListFindBy{UUID|ID}Ref
For bhyveDomObjFromDomain, bhyveDomainLookupByUUID, and
bhyveDomainLookupByID let's return a locked and referenced
@vm object so that callers can then use the common and more
consistent virDomainObjEndAPI in order to handle cleanup rather
than needing to know that the returned object is locked and
calling virObjectUnlock.
The LookupByName already returns the ref counted and locked object,
so this will make things more consistent.
For bhyveDomainUndefine and bhyveDomainDestroy since the
virDomainObjListRemove will return an unlocked object, we need to
relock before making the EndAPI call.
Andrea Bolognani [Thu, 29 Mar 2018 11:38:30 +0000 (13:38 +0200)]
qemu: Format gic-version=2 on the command line
Up until now we have only formatted non-default GIC versions on
the command line, in order to maintain compatibility with older
QEMU versions that didn't implement the gic-version option to
begin with; however, doing so is entirely unnecessary for newer
QEMU versions, where the option is available. Moreover, having
the GIC version formatted on the command line at all times
ensures that QEMU changing its own defaults doesn't affect the
ABI of libvirt guests.
A few test cases are removed to avoid extra churn. It doesn't
matter for coverage, as those scenarios are already covered by
other parts of the test suite.
This patch is better viewed with 'git show -w'.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
Vincent Bernat [Tue, 10 Apr 2018 06:27:15 +0000 (08:27 +0200)]
util: don't check for parallel iteration in hash-related functions
This is the responsability of the caller to apply the correct lock
before using these functions. Moreover, the use of a simple boolean
was still racy: two threads may check the boolean and "lock" it
simultaneously.
Users of functions from src/util/virhash.c have to be checked for
correctness. Lookups and iteration should hold a RO
lock. Modifications should hold a RW lock.
Most important uses seem to be covered. Callers have now a greater
responsability, notably the ability to execute some operations while
iterating were reliably forbidden before are now accepted.
For patches pushed upstream we want builds run on master branch, but
don't want them run on the -maint branches, as we are not keeping the
.travis.yml file on stable branches updated wrt latest needs of travis
CI platforms.
We can't just whitelist 'master' though, because that will prevent
developers triggering their own private travis builds. So we just
blacklist *-maint, since developers will typically use named feature
branches for any work.
Reviewed-by: Pavel Hrdina <phrdina@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Wim ten Have [Mon, 9 Apr 2018 18:14:35 +0000 (20:14 +0200)]
storage: extend preallocation flags support for qemu-img
This patch adds support to qcow2 formatted filesystem object storage by
instructing qemu-img to build them with preallocation=falloc whenever the
XML described storage <allocation> matches its <capacity>. For all other
cases the filesystem stored objects are built with preallocation=metadata.
Signed-off-by: Wim ten Have <wim.ten.have@oracle.com> Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
John Ferlan [Fri, 6 Apr 2018 16:14:44 +0000 (12:14 -0400)]
qemu: Obtain reference on monConfig
Because we allow a QEMU_JOB_DESTROY to occur while we're starting
up and we drop the @vm lock prior to qemuMonitorOpen, it's possible
that a domain destroy operation "wins" the race, calls qemuProcessStop
which will free and reinitialize priv->monConfig. Depending on the
exact timing either qemuMonitorOpen will be passed a NULL @config
variable or it will be using free'd (and possibly reclaimed) memory
as the @config parameter - neither of which is good.
Resolve this by localizing the @monConfig, taking an extra reference,
and then once we get the @vm lock again removing our reference since
we are done with it.
Signed-off-by: John Ferlan <jferlan@redhat.com> Reviewed-by: Marc Hartmayer <mhartmay@linux.vnet.ibm.com>
John Ferlan [Fri, 6 Apr 2018 16:05:38 +0000 (12:05 -0400)]
conf: Convert virDomainChrSourceDefNew to return object
Let's use object referencing to handle the ChrSourceDef. A subsequent
patch then can allow the monConfig to take an extra reference before
dropping the domain lock to then ensure nothing free's the memory that
needs to be used.
Signed-off-by: John Ferlan <jferlan@redhat.com> Reviewed-by: Marc Hartmayer <mhartmay@linux.vnet.ibm.com>
John Ferlan [Tue, 27 Mar 2018 15:53:41 +0000 (11:53 -0400)]
qemu: Fix qemuProcessAutoDestroy
Upon entry from virCloseCallbacksRun, the @dom will have a
Ref and Lock from virDomainObjListFindByUUIDRef, so there's
no need to take an extra reference nor should the code call
virDomainObjEndAPI when done since that both Unref's and
Unlock's the @dom which means the callers call to EndAPI
would be unlocking an unlocked object. At least the Ref
saved the code from referencing something already freed.
Signed-off-by: John Ferlan <jferlan@redhat.com> Reviewed-by: Marc Hartmayer <mhartmay@linux.vnet.ibm.com>
Jim Fehlig [Wed, 28 Mar 2018 22:50:04 +0000 (16:50 -0600)]
Remove the xend driver
xend was deprecated in Xen 4.2 and removed from the Xen sources
before the Xen 4.5 release. The last Xen release to contain xend
was Xen 4.4, which was retired upstream in March 2017.
Remove xend support from libvirt since it is unrealistic to use
modern libvirt with ancient Xen.
Signed-off-by: Jim Fehlig <jfehlig@suse.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Jim Fehlig [Fri, 30 Dec 2016 21:43:43 +0000 (14:43 -0700)]
tests: move xmconfig tests to WITH_LIBXL
In preparation of removing the legacy Xen driver, move the
xmconfig tests from WITH_XEN to WITH_LIBXL. Even though the
legacy driver will be removed, we'll want to maintain the ability
to convert xm config to XML. Requires fixing up the tests to account
for different behavior of Xen vs libxl post parse functions. For
consistency with other Xen config tests, change the <os> arch to
x86_64.
There is some test file fallout due to differences in handling of
default values between xend and libxl.
Signed-off-by: Jim Fehlig <jfehlig@suse.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Jim Fehlig [Tue, 27 Dec 2016 23:19:43 +0000 (16:19 -0700)]
tests: move sexpr2xml tests to WITH_LIBXL
In preparation of removing the legacy Xen driver, move the
sexpr2xml tests from WITH_XEN to WITH_LIBXL. Even though the
legacy driver will be removed, we'll want to maintain the ability
to convert sexpr to XML. Requires fixing up the tests to account
for different behavior of Xen vs libxl post parse functions.
There is some test file fallout due to differences in handling
of default values between xend and libxl.
Signed-off-by: Jim Fehlig <jfehlig@suse.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Jim Fehlig [Tue, 27 Dec 2016 22:27:08 +0000 (15:27 -0700)]
tests: move xml2sexpr tests to WITH_LIBXL
In preparation of removing the legacy Xen driver, move the
xml2sexpr tests from WITH_XEN to WITH_LIBXL. Even though the
legacy driver will be removed, we'll want to maintain the ability
to convert sexpr <-> XML.
There is some test file fallout due to differences in handling
of default values between xend and libxl.
Signed-off-by: Jim Fehlig <jfehlig@suse.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
remote: fix path for locating libvirtd in source tree during autostart
Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
rpc: refactor way connection object is generated for remote dispatch
Calling a push_privconn method to directly push the connection object
name into the arg list is inconvenient. Refactor so that we acquire
the connection variable name upfront, and push it to the arg list
separately. This allows various hardcoded usage of "priv->conn" to
be parameterized.
Reviewed-by: John Ferlan <jferlan@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
docs: add page describing goals for host platform version support
Described how we decide which host platforms to support for libvirt,
which in turn makes it easier to decide when a platform / software
version can be dropped.
Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com> Reviewed-by: Pavel Hrdina <phrdina@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Andrea Bolognani [Tue, 27 Mar 2018 12:23:29 +0000 (14:23 +0200)]
conf: Remove dubious code from virDomainPCIAddressSetGrow()
I haven't been able to come up with a single scenario in which
the code in question would be executed; even if there was one,
it would be due to the user specifying a *partial* PCI topology
in the guest XML, which is of course entirely unsupportable and
thus providing even the slightest hint that doing so is in any
way a good idea is actively harmful.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
Erik Skultety [Tue, 3 Apr 2018 07:52:32 +0000 (09:52 +0200)]
util: virlog: Introduce wildcard to log filters
Since the introduction of log tuning capabilities to virt-admin by
@06b91785, this has been a much needed missing improvement on the way to
deprecate the global 'log_level'.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Erik Skultety [Tue, 3 Apr 2018 07:56:09 +0000 (09:56 +0200)]
libvirtd.conf: Document that we do a 'first' match on log filters
When applying the log filters, one has to define the more specific
filters before the generic ones, because the first filter that matches
will be applied. However, we've been missing this information in the
config, so it always has been a trial-error scenario figuring out that
e.g. '4:util 1:util.pci' doesn't actually enable verbose logging on the
src/util/virpci.c module because 4:util will be matched first.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
John Ferlan [Mon, 26 Mar 2018 22:21:59 +0000 (18:21 -0400)]
conf: Fix virDomainObjParseFile object handling
When virDomainObjParseFile runs, it returns a locked @obj with
one reference. Rather than just use virObjectUnref to clean that
up, use virObjectEndAPI.
John Ferlan [Mon, 26 Mar 2018 23:14:30 +0000 (19:14 -0400)]
conf: Fix error path logic in virDomainObjListLoadStatus
If the virHashAddEntry fails, then we need to "careful" about
how we free the @obj. When virDomainObjParseFile returns there
is one reference and the object is locked, so use virDomainObjEndAPI
when done.
Add a virObjectRef in the error path for the second virHashAddEntry
call since it doesn't call virObjectRef, but virHashRemoveEntry
will call virObjectUnref because virObjectFreeHashData is called
when the element is removed from the hash table as set up in
virDomainObjListNew.
John Ferlan [Mon, 26 Mar 2018 22:29:30 +0000 (18:29 -0400)]
conf: Fix error path logic in virDomainObjListAddLocked
If the virHashAddEntry fails, then we need to "careful" about
how we free the @vm. When virDomainObjNew returns there is one
reference and the object is locked, so use virDomainObjEndAPI
when done.
Add a virObjectRef in the error path for the second virHashAddEntry
call since it doesn't call virObjectRef, but virHashRemoveEntry
will call virObjectUnref because virObjectFreeHashData is called
when the element is removed from the hash table as set up in
virDomainObjListNew.
Eventually these paths should goto error and error should be changed
to use EndAPI as well, but that requires more adjustments to other
paths in the code to have a locked and ref counted @vm.
Andrea Bolognani [Wed, 28 Mar 2018 10:48:38 +0000 (12:48 +0200)]
tests: Use pcie-to-pci-bridge for aarch64-traditional-pci
Now that support for the pcie-to-pci-bridge controller has
been implemented, adding the QEMU_CAPS_DEVICE_PCIE_PCI_BRIDGE
capability to the existing test is enough to cause the guest
to use pcie-to-pci-bridge instead of dmi-to-pci-bridge.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
Andrea Bolognani [Wed, 28 Mar 2018 10:48:21 +0000 (12:48 +0200)]
conf: Prefer pcie-to-pci-bridge to dmi-to-pci-bridge
Both pcie-to-pci-bridge and dmi-to-pci-bridge can be used to
create a traditional PCI topology in a pure PCIe guest such as
those using the x86_64/q35 or aarch64/virt machine type;
however, the former should be preferred, as it doesn't need to
obey limitation of real hardware and is completely
architecture-agnostic.
Just like the existing areMultipleRootsSupported, this will
allow us to change the results of the driver-agnostic PCI
address allocation logic based on whether the QEMU binary
supports certain features.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
Andrea Bolognani [Wed, 28 Mar 2018 13:12:53 +0000 (15:12 +0200)]
docs: Tweak PCI controller model documentation
Instead of first listing the models on their own, and then
listing them again grouped by the libvirt release they were
introduced in, have a single list.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
Jim Fehlig [Mon, 26 Mar 2018 18:53:05 +0000 (12:53 -0600)]
util: introduce virSocketAddrParseAny
When preparing for migration, the libxl driver creates a new TCP listen
socket for the incoming migration by calling virNetSocketNewListenTCP,
passing the destination host name. virNetSocketNewListenTCP calls
virSocketAddrParse to check if the host name is a wildcard address, in
which case it avoids adding the AI_ADDRCONFIG flag to the hints passed to
getaddrinfo. If the host name is not an IP address, virSocketAddrParse
reports an error
error : virSocketAddrParseInternal:121 : Cannot parse socket address
'myhost.example.com': Name or service not known
But virNetSocketNewListenTCP succeeds regardless and the overall migration
operation succeeds.
Introduce virSocketAddrParseAny and use it when simply testing if a host
name/addr is parsable.
Signed-off-by: Jim Fehlig <jfehlig@suse.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
Problem with device mapper targets is that there can be several
other devices 'hidden' behind them. For instance, /dev/dm-1 can
consist of /dev/sda, /dev/sdb and /dev/sdc. Therefore, when
setting up devices CGroup and namespaces we have to take this
into account.
This bug was exposed after Linux kernel was fixed. Initially,
kernel used different functions for getting block device in
open() and ioctl(). While CGroup permissions were checked in the
former case, due to a bug in kernel they were not checked in the
latter case. This changed with the upstream commit of 519049afead4f7c3e6446028c41e99fde958cc04 (v4.16-rc5~11^2~4).
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Mon, 26 Mar 2018 12:48:07 +0000 (14:48 +0200)]
util: Introduce virDevMapperGetTargets
This helper fetches dependencies for given device mapper target.
At the same time, we need to provide a dummy log function because
by default libdevmapper prints out error messages to stderr which
we need to suppress.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
John Ferlan [Tue, 27 Mar 2018 17:16:38 +0000 (13:16 -0400)]
test: Use virDomainObjListFindByIDRef
Rather than using virDomainObjListFindByID, let's be more consistent
and return a reffed and locked object. Since we're using the Ref API,
use virDomainObjEndAPI on @dom and not just virObjectUnlock.
Signed-off-by: John Ferlan <jferlan@redhat.com> Reviewed-by: Marc Hartmayer <mhartmay@linux.vnet.ibm.com>
John Ferlan [Fri, 9 Mar 2018 15:37:52 +0000 (10:37 -0500)]
test: Use virDomainObjListFindByUUIDRef
Rather than using virDomainObjListFindByUUID, let's be more consistent
and return a reffed and locked object. Since we're using the Ref API,
use virDomainObjEndAPI on @dom and not just virObjectUnlock.
Signed-off-by: John Ferlan <jferlan@redhat.com> Reviewed-by: Marc Hartmayer <mhartmay@linux.vnet.ibm.com>
John Ferlan [Tue, 27 Mar 2018 17:39:52 +0000 (13:39 -0400)]
test: Need to relock afer virDomainObjListRemove
For all @dom's fetched from a testDomObjFromDomain because
virDomainObjListRemove will return an unlocked domain object
we should relock it prior to the cleanup label which will use
virDomainObjEndAPI which would Unlock and Unref the passed
object (and we should avoid unlocking an unlocked object).
Signed-off-by: John Ferlan <jferlan@redhat.com> Reviewed-by: Marc Hartmayer <mhartmay@linux.vnet.ibm.com>
Peter Krempa [Tue, 27 Mar 2018 09:11:26 +0000 (11:11 +0200)]
qemu: domain: Move initialization of disk cachemode for <shareable> disks
The qemu command line generator code set disk caching of shareable disks
to 'none' when formatting the command line silently. Move this code to a
common place when preparing the domain definition for startup so that it
does not have to be duplicated.
The new test case shows that the actual cache mode will now be recorded
in the live XML definition.
Peter Krempa [Tue, 27 Mar 2018 09:11:01 +0000 (11:11 +0200)]
tests: qemuxml2argv: Allow testing of config processed at startup
Add a new kind of XML output test for the files in qemuxml2argvtest
where we can validate setup and defaults applied when starting up the
VM.
This is achieved by formatting of the definition processed by the
qemuxml2argvtest into a XML and it's compared against files in
qemuxml2startupxmloutdata. This test is automatically executed if the
output file is present and it's skipped otherwise.
The first example test case is created from 'disk-drive-shared' test
case.
Peter Krempa [Wed, 28 Mar 2018 10:21:31 +0000 (12:21 +0200)]
qemu: command: Add support for qcow2 + luks
The old qcow2 encryption format was buggy, so the new approach is to use
luks inside qcow2. As it turns out, it didn't require that many changes.
It was necessary to fix the command line formatter to stop mangling the
format when secrets are present and specify the encryption format and
secret in correct format.
Peter Krempa [Thu, 29 Mar 2018 06:49:08 +0000 (08:49 +0200)]
qemu: domain: Forbid VIR_STORAGE_FILE_ISO as a disk format
This format is used by the storage driver and other hypervisors but qemu
does not have notion of the 'iso' format and libvirt does not translate
it to anything useful, so it would not work anyways. Users should use
'raw' instead.
Peter Krempa [Tue, 5 Dec 2017 15:40:27 +0000 (16:40 +0100)]
qemu: domain: Add helper to initialize detected parts of the backing chain
It will be necessary to initialize various aspects for the detected
members of the backing chain. Add a function that will handle it and
call it from qemuDomainPrepareDiskSource and qemuDomainDetermineDiskChain
Peter Krempa [Wed, 29 Nov 2017 14:17:10 +0000 (15:17 +0100)]
conf: Extract logic for updating 'detect_zeroes' mode
For some reason we've decided to silently translate the disk
detect_zeroes mode if it would be invalid. Extract the
logic so that it does not need to be copypasta'd across the code base.
Peter Krempa [Fri, 16 Mar 2018 14:32:09 +0000 (15:32 +0100)]
conf: Don't require 'def' in virDomainDiskDefParse
In some use cases (mostly in tests) it is not required to check the
seclabel definition validity. Add possibility to call
virDomainDiskDefParse without the domain definition.
Peter Krempa [Fri, 16 Mar 2018 11:43:23 +0000 (12:43 +0100)]
conf: Refactor/rename virDomainDiskDefSourceParse
Make the function more usable by returning the full disk definition and
fix the only caller for the new semantics. The new name for the function
is virDomainDiskDefParse.
Jiri Denemark [Wed, 28 Mar 2018 10:45:21 +0000 (12:45 +0200)]
qemu_monitor_json: Properly check "return" type
My commit 2e0d6cdec41 claimed qemuMonitorJSONCheckError guarantees
"return" object exists in the JSON reply. But it only makes sure the key
is there, while the type of the value is not checked. A lot of callers
do not care since they only want to see whether their QMP command failed
or not, but any caller which needs to read some data from the reply
wants to make sure the correct data type was returned.
This patch adds a new API called qemuMonitorJSONCheckReply which calls
qemuMonitorJSONCheckError and checks "return" contains a value of the
specified type.
Peter Krempa [Fri, 30 Mar 2018 10:35:52 +0000 (12:35 +0200)]
qemu: monitor: Don't resist stealing 'actions' in qemuMonitorJSONTransaction
Rather than trying to prevent stealing of the 'actions' virJSONValue
into the monitor command replace the code so that it does the same
thing, since 'actions' was actually not really used after calling the
monitor.
Peter Krempa [Thu, 29 Mar 2018 18:38:35 +0000 (20:38 +0200)]
util: json: Add accessor for geting a VIR_JSON_TYPE_NUMBER as string
Sometimes it's desired to get a JSON number as string. Add a helper.
This will help in cases where we'd want to convert the internal type from
string to something else.
Peter Krempa [Fri, 30 Mar 2018 09:12:57 +0000 (11:12 +0200)]
util: json: Fix freeing of objects appended to virJSONValue
It was not possible to determine whether virJSONValueObjectAddVArgs and
the functions using it would consume a virJSONValue or not when used
with the 'a' or 'A' modifier depending on when the loop failed.
Fix this by passing in a pointer to the pointer so that it can be
cleared once it's successfully consumed and the callers don't have to
second-guess leaving a chance of leaking or double freeing the value
depending on the ordering.
docs: formatdomain: Clarify CPU feature policy option "require"
Using the 'require' value for the 'policy' attribute indicates
that the guest will have the feature so the host CPU does not
need to support it if the hypervisor can emulate it.
E.g. 'x2apic' is emulated by QEMU even if the host does not support it:
projects / libvirt.git / log