The blockdev-add code provides a mechanism to sanely provide user
and password-secret arguments for iscsi without placing them on the
command line to be viewable by a 'ps -ef' type command or needing
to create separate -iscsi devices for each disk/volume found.
So modify the iSCSI command line building to check for the presence
of the capability in order properly setup and use the domain master
secret object to encrypt the password in a secret object and alter
the parameters for the command line to utilize.
Modify the xml2argvtest to exhibit the syntax for both disk and
hostdev configurations.
John Ferlan [Fri, 1 Sep 2017 19:09:29 +0000 (15:09 -0400)]
qemu: Get capabilities to use iscsi password-secret argument
Detect the capability via the query-qmp-schema for blockdev-add
to find the 'password-secret' parameter that will allow the iSCSI
code to use the master secret object to encrypt the secret for an
and only need to provide the object id of the secret on the command
line thus obsfuscating the passphrase.
Jason J. Herne [Fri, 24 Nov 2017 08:02:02 +0000 (09:02 +0100)]
s390: qemu-capabilities: Avoid error message when missing non-kvm host cpu info
Libvirt prints an error on startup when it is missing host cpu model
information for any queried qemu binary. On s390 we only have host cpu model
information for kvm enabled qemu instances. So when virt type is not kvm, this
is actually not an error on s390.
This patch adds virt type as a parameter to virQEMUCapsInitCPUModelS390, and a
new return code 2 for virQEMUCapsInitCPUModel and virQEMUCapsInitCPUModelS390.
If the virt type is not kvm then we skip printing the scary error message
and return 2 because this case is actually expected behavior. The new return
code is meant to differentiate between the failure case and the case where we
simply expect the cpu model information to be unattainable.
Signed-off-by: Jason J. Herne <jjherne@linux.vnet.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.vnet.ibm.com> Reviewed-by: Marc Hartmayer <mhartmay@linux.vnet.ibm.com> Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
John Ferlan [Sun, 8 Oct 2017 12:44:31 +0000 (08:44 -0400)]
storage: Introduce virStoragePoolObjListSearch
Create an API to search through the storage pool objects looking for
a specific truism from a callback API in order to return the specific
storage pool object that is desired.
John Ferlan [Sun, 8 Oct 2017 12:14:56 +0000 (08:14 -0400)]
storage: Introduce virStoragePoolObjListForEach
Create an API to walk the pools->objs[] list in order to perform a
callback function for each element of the objs array that doesn't care
about whether the action succeeds or fails as the desire is to run the
code over every element in the array rather than fail as soon as or if
one fails.
John Ferlan [Sun, 8 Oct 2017 13:09:09 +0000 (09:09 -0400)]
storage: Introduce virStoragePoolObjEndAPI
For now it'll just call the virStoragePoolObjUnlock, but a future
adjustment will do something different. Since the new API will check
for a NULL object before the Unlock call, callers no longer need to
check for NULL before calling.
The virStoragePoolObjUnlock is now private/static to virstorageobj.c
with a short term forward reference.
Peter Krempa [Thu, 23 Nov 2017 16:15:17 +0000 (17:15 +0100)]
qemu: command: Properly format disk 'debug' attribute
Move the setup of the disk attribute to the disk source prepare function
which will allow proper usage with JSON props and move the fallback
(legacy) generating code into the block which is executed with legacy
options.
As a side-effect of this change we can clean up propagation of 'cfg'
into the command generator.
Also it's nice to see that the test output is the same even when the
value is generated in a different place.
Peter Krempa [Thu, 23 Nov 2017 15:06:33 +0000 (16:06 +0100)]
util: storage: Add fields for debug options for disk drivers
Some drive backends allow output of debugging information which can be
configured using properties of the image. Add fields to virStorageSource
which will allow configuring them.
Peter Krempa [Thu, 23 Nov 2017 14:37:37 +0000 (15:37 +0100)]
qemu: command: Inject password-secret only when not using JSON props
The 'file.password-secret' injection should be used only if we are using
the old formatter. When formatting the source string from the JSON
properties, the property should be added there.
Also drop the comment which refers to stuff that will not be used in
libvirt since -blockdev is the way to go.
Michal Privoznik [Thu, 23 Nov 2017 17:00:33 +0000 (18:00 +0100)]
tools: Work around ancient readline
My latest commit of a785186446de785d uncovered a problem we fixed
in 9eb23fe2 but then reverted in 834c5720e443. Turns out, some
systems (I'm looking at you OS X) have ancient readline with
broken header file.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Peter Krempa [Wed, 15 Nov 2017 14:21:14 +0000 (15:21 +0100)]
qemu: command: Mark <shared/> disks as such in qemu
Qemu has now an internal mechanism for locking images to fix specific
cases of disk corruption. This requires libvirt to mark the image as
shared so that qemu lifts certain restrictions.
Peter Krempa [Tue, 14 Nov 2017 14:37:09 +0000 (15:37 +0100)]
qemu: domain: Reject shared disk access if backing format does not support it
Disk sharing between two VMs may corrupt the images if the format driver
does not support it. Check that the user declared use of a supported
storage format when they want to share the disk.
Peter Krempa [Tue, 14 Nov 2017 14:34:46 +0000 (15:34 +0100)]
qemu: block: Add function to check if storage source allows concurrent access
Storage source format backing a shared device (e.g. running a cluster
filesystem) needs to support the sharing so that metadata are not
corrupted. Add a central function for checking this.
Julio Faracco [Fri, 17 Nov 2017 21:27:40 +0000 (19:27 -0200)]
tests: changing network interface types when backend tag is defined.
Some test cases have the backend tag inside wrong interfaces. The backend xml
tag does not support <interface type='user|direct|hostdev'>. So this commit
changes some network types inside the interfaces that have backend defined.
Signed-off-by: Julio Faracco <jcfaracco@gmail.com>
Michal Privoznik [Tue, 14 Nov 2017 13:29:20 +0000 (14:29 +0100)]
qemu_capabilities: Introcude QEMU_CAPS_NUMA_DIST
This capability says if qemu is capable of specifying distances
between NUMA nodes on the command line. Unfortunately, there's no
real way to check this and thus we have to go with version check.
QEMU introduced this in 0f203430dd8 (and friend) which was
released in 2.10.0.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
There's no point in checking if numa->mem_nodes[node].ndistances
is set if we check for numa->mem_nodes[node].distances. However,
it makes sense to check if the sibling node (@cellid) caller
passed falls within boundaries.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
Jiri Denemark [Wed, 22 Nov 2017 12:12:00 +0000 (13:12 +0100)]
qemu: Use the end of QEMU log for reporting errors
When QEMU dies, we read its output stored in a log file and use it for
reporting a hopefully useful error. However, virReportError will trim
the message to (VIR_ERROR_MAX_LENGTH - 1) characters, which means the
end of the log (which likely contains the error message we want to
report) may get lost. We should trim the beginning of the log instead.
Jiri Denemark [Wed, 22 Nov 2017 09:19:12 +0000 (10:19 +0100)]
qemu: Properly skip "char device redirected to" in QEMU log
When reading QEMU log for reporting it as an error message, we want to
skip "char device redirected to" line. However, this string is not
printed at the beginning of a line, which means STRPREFIX will never
find it.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com> Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
xinhua.Cao [Mon, 13 Nov 2017 13:07:58 +0000 (21:07 +0800)]
remote: Move the call to remoteClientFreePrivateCallbacks from FreeFunc to CloseFunc
Still because of commit id 'fe8f1c8b' where we generate a REF for the
Register and that's transparent to the consumer (e.g. how would they
know they need to ensure that Deregister is called), thus the purpose of
this patch is to find a way to Deregister if it's determined that the
consumer hasn't by the time of the "last" REF we'd have.
This solution to this problem is to alter the processing to have the
remoteClientCloseFunc handle performing the Deregister calls instead of
the remoteClientFreeFunc because there's no way FreeFunc would be called
unless the Deregister was already called.
We have a check for backslash alignment that checks for two blanks preceding a
backslash. However there can be alignment done using a tabulator and in some
cases one might be enough. There are none currently, but I found out that was
the case before. at some point. so let's check for \t preceding the final
backslash as well.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Peter Krempa [Wed, 1 Nov 2017 15:31:55 +0000 (16:31 +0100)]
qemu: command: Split out geometry frontend attribute formatting from -drive
Historically we've formatted a lot of the attributes of a disk (disk
geometry, etc) with -drive. Since we use -device now, they should be
formatted there. Extract them to a separate function for keeping
compatibility with SDcards which still use only -drive.
Start this by moving the geometry into a separate function.
Peter Krempa [Wed, 22 Nov 2017 15:42:53 +0000 (16:42 +0100)]
qemu: domain: Don't call namespace setup for storage already accessed by vm
When doing block commit we need to allow write for members of the
backing chain so that we can commit the data into them.
qemuDomainDiskChainElementPrepare was used for this which since commit 786d8d91b4 calls qemuDomainNamespaceSetupDisk which has very adverse
side-effects, namely it relabels the nodes to the same label it has in
the main namespace. This was messing up permissions for the commit
operation since its touching various parts of a single backing chain.
Since we are are actually not introducing new images at that point add a
flag for qemuDomainDiskChainElementPrepare which will refrain from
calling to the namespace setup function.
Calls from qemuDomainSnapshotCreateSingleDiskActive and
qemuDomainBlockCopyCommon do introduce new members all calls from
qemuDomainBlockCommit do not, so the calls are anotated accordingly.
Since we don't pack symlinks we cannot have recursive loops in them. Since we
need one directory to be in tests/vircaps2xmldata/linux-caches/, instead of
creating a symlink, just move the files in that directory and adjust tests.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Michal Privoznik [Thu, 16 Nov 2017 13:35:12 +0000 (14:35 +0100)]
tools: Enable warnings for more binaries/libs
Because WARN_CFLAGS and COVERAGE_CFLAGS are not set globally, we
rely on each binary built to include WARN_CFLAGS/COVERAGE_CFLAGS.
But it is easy to forget those - e.g. libvirt_shell.la. However,
don't enable WARN_FLAGS (i.e. don't include AM_CFLAGS) for
wireshark plugin - parts of that code are generated and trigger
some warnings.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com>
Michal Privoznik [Thu, 16 Nov 2017 12:38:27 +0000 (13:38 +0100)]
vsh: Make self-test more robust
There are couple of limitations when it comes to option types and
flags for the options. For instance, VSH_OT_STRING cannot have
VSH_OFLAG_REQ set (commit c7543a728). For some reason this is
checked in vshCmddefHelp() but not in vshCmddefCheckInternals().
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com>
Just like in 9324f67a572f9b32 we need to put default sata alias
(which is hardcoded to "ide", obvious, right?) onto the command
line instead of the one provided by user.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
The compiler can warn us if we add a value to the
virDomainChrSerialTargetType enumeration but forget to handle
it properly in the code. Let's take advantage of that.
This commit is best viewed with 'git diff -w'.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
John Ferlan [Mon, 6 Nov 2017 20:20:55 +0000 (15:20 -0500)]
libvirtd: Fix order of cleanup processing
Current cleanup processing is ad-hoc at best - it's led to a couple of
strange and hard to diagnose timing problems and crashes.
So rather than perform cleanup in a somewhat random order, let's
perform cleanup in the exact opposite order of startup.
NB: It is possible that virNetlinkEventServerStart fails and we jump
to cleanup before driversInitialized has been set. That could leave
things inconsistent; however, resolution of that possibility is perhaps
more trouble than it's worth to handle.
John Ferlan [Mon, 6 Nov 2017 21:55:15 +0000 (16:55 -0500)]
libvirtd: Alter order of virNetDaemonNew
Let's be sure we can get a Daemon object before the server object.
This is a more "orderly" way to do things since the svr object would
be added to the dmn object afterwards.
Starting from qemu 2.11, the `-device vmcoreinfo` will create a fw_cfg
entry for a guest to store dump details, necessary to process kernel
dump with KASLR enabled and providing additional kernel details.
In essence, it is similar to -fw_cfg name=etc/vmcoreinfo,file=X but in
this case it is not backed by a file, but collected by QEMU itself.
Since the device is a singleton and shouldn't use additional hardware
resources, it is presented as a <feature> element in the libvirt
domain XML.
The device is arm/x86 only for now (targets that support fw_cfg+dma).
Related to:
https://bugzilla.redhat.com/show_bug.cgi?id=1395248
tests: Add resctrl-skx-twocaches test case to vircaps2xmltest
This doesn't add very much value for now, but future test for virresctrl will
take information from vircaps2xmldata (since it is dependent on the same info
then why duplicate it) and this particular use case helps us cover bit more of
the code regarding proper formatting and handling errors. And one more test for
vircaps2xmltest doesn't hurt either.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
This patch modifies some not yet used test data so that the adding a test using
this data is a clean patch and not an addition of huge file with some
adjustments in small files that will be hidden in the middle of that commit.
These changes include:
- Add system dir in vircaps2xmldata/linux-caches
Back when data for systems with resctrl support were added they had the
/sys/fs/system directory put into a system/ subdir of the test and
/sys/fs/resctrl in a resctrl/ subdir of that test. However, if we also want a
negative test for the resctrl (requesting allocation on a system that does not
support resctrl), we need one a test case with any sensible (with cache info)
system/ subdir and no resctrl/ one. Easiest way is to add a
system -> . symlink into existing test case.
- Change default group schemata for linux-resctrl and linux-resctrl-cdp
That way we can fit some allocation in.
- Remove one cache from resctrl-skx's schemata and make some room for
allocations
That system already has only one cache, so that file was wrong anyway. We
have a version with 2 caches already (linux-resctrl-skx-twocaches), so this
will also add variety to future tests.
- Add some empty allocation for resctrl-skx
Just to have slightly more coverage and variety. We can be sure nothing bad
happens if such allocation exists in case we have that in the tests.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Because the cache banks are initialized based on the order in which their
respective directories exist on the filesystem, they can appear in different
order. This is here mainly for tests because the cache directory might have
different order of children nodes and tests would fail otherwise. It should not
be the case with sysfs, but one can never be sure. And this does not take
almost any extra time, mainly because it gets initialized once per driver.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
Sometimes the size of the bitmap matters and it might not be guessed correctly
when parsing from some type of input. For example virBitmapNewData() has Byte
granularity, virBitmapNewString() has nibble granularity and so on.
virBitmapParseUnlimited() can be tricked into creating huge bitmap that's not
needed (e.g.: "0-2,^99999999"). This function provides a way to shrink the
bitmap. It is not supposed to free any memory.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
Our bitmaps can be represented as data (raw bytes for which we have
virBitmapNewData() and virBitmapToData()), human representation (list
of numbers in a string for which we have virBitmapParse() and
virBitmapFormat()) and hexadecimal string (for which we have only
virBitmapToString()). So let's add the missing complement for the
last one so that we can parse hexadecimal strings.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
util: Don't output too many zeros from virBitmapToString
Truncate the output so that it is only as big as is needed to fit all
the bits, not all the units from the map. This will be needed in the
future in order to properly format bitmaps for kernel's sysfs files.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
util: Rename virBitmapDataToString to virBitmapDataFormat
It is literally only a wrapper around virBitmapNewData() and
virBitmapFormat(), only the naming was wrong since it was introduced.
And because we have virBitmap*String functions where the meaning of
the 'String' is constant, this might confuse someone.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
This follows the virBitmapToData() function and, similarly to
virBitmapNewData(), we'll be able to have virBitmapNewString() later
on without name confusion.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
We can't output better memory sizes if we want to be compatible with libvirt
older than the one which introduced /memory/unit, but for new things we can just
output nicer capacity to the user if available. And this function enables that.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
Only output initialized capabilities with VIR_TEST_DEBUG > 1
Currenty virTestInit() outputs all capabilities that it created when running
with VIR_TEST_DEBUG=1. Since this is quite a lot of output for every call of
this function (and it is not needed until debugging a really deep-down issue)
let's just output the info when VIR_TEST_DEBUG is strictly greater than 1.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Since update to glibc-2.26 removed the /usr/include/rpc/rpc.h we used until now,
it showed us a problem with not using XDR_CFLAGS properly. On linux that
variable has usually -I/usr/include/tirpc because we already probe for it
properly, we just don't use it everywhere we need. It is needed by wireshark
dissector as well as testutilsqemu.c (through includes) so the build fails with:
wireshark/src/packet-libvirt.c:33:10: fatal error: rpc/xdr.h: No such file or directory
#include <rpc/xdr.h>
^~~~~~~~~~~
and
In file included from ../src/logging/log_manager.h:29:0,
from ../src/qemu/qemu_domain.h:40,
from testutilsqemu.c:11:
../src/logging/log_protocol.h:9:10: fatal error: rpc/rpc.h: No such file or directory
#include <rpc/rpc.h>
^~~~~~~~~~~
Since lot of tests use testutilsqemu.c it is easier to add XDR_CFLAGS to
AM_CFLAGS than adding it to all $binary_CFLAGS. It's just for tests and we
already have bunch of CFLAGS there anyway.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
docs: add a page describing support guarantees for libvirt features
While we have collective knowledge about the support status of various
parts of libvirt, this has never been formally documented, leaving our
users to guess.
Note, this document makes one change to our previous policy. It explicitly
declares the RPC protocol of libvirtd as being a supported interface. THis
accepts the reality that we can a) never change it without breaking compat
with old libvirt.so, b) there are both rust + go impls that are written
against the RPC protocol already.
Reviewed-by: John Ferlan <jferlan@redhat.com> Reviewed-by: Jim Fehlig <jfehlig@suse.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
John Ferlan [Mon, 6 Nov 2017 20:22:07 +0000 (15:22 -0500)]
storage: Resolve storage driver crash
Resolve a storage driver crash as a result of a long running
storageVolCreateXML when the virStorageVolPoolRefreshThread is
run as a result of when a storageVolUpload completed and ran the
virStoragePoolObjClearVols without checking if the creation
code was currently processing a buildVol after incrementing
the driver->asyncjob count.
The refreshThread will now check the pool asyncjob count before
attempting to pursue the pool refresh. Adjust the documentation
to describe the condition.
Crash from valgrind is as follows (with a bit of editing):
==21309== Invalid read of size 8
==21309== at 0x153E47AF: storageBackendUpdateVolTargetInfo
==21309== by 0x153E4C30: virStorageBackendUpdateVolInfo
==21309== by 0x153E52DE: virStorageBackendVolRefreshLocal
==21309== by 0x153DE29E: storageVolCreateXML
==21309== by 0x562035B: virStorageVolCreateXML
==21309== by 0x147366: remoteDispatchStorageVolCreateXML
...
==21309== Address 0x2590a720 is 64 bytes inside a block of size 336 free'd
==21309== at 0x4C2F2BB: free
==21309== by 0x54CB9FA: virFree
==21309== by 0x55BC800: virStorageVolDefFree
==21309== by 0x55BF1D8: virStoragePoolObjClearVols
==21309== by 0x153D967E: virStorageVolPoolRefreshThread
...
==21309== Block was alloc'd at
==21309== at 0x4C300A5: calloc
==21309== by 0x54CB483: virAlloc
==21309== by 0x55BDC1F: virStorageVolDefParseXML
==21309== by 0x55BDC1F: virStorageVolDefParseNode
==21309== by 0x55BE5A4: virStorageVolDefParse
==21309== by 0x153DDFF1: storageVolCreateXML
==21309== by 0x562035B: virStorageVolCreateXML
==21309== by 0x147366: remoteDispatchStorageVolCreateXML
...
projects / libvirt.git / log