John Ferlan [Mon, 7 May 2018 14:17:43 +0000 (10:17 -0400)]
storage_util: Split backing_fmt set in storageBackendCreateQemuImgOpts
The only way backing_fmts could be set is if the info->format was
not RAW (see storageBackendCreateQemuImgSetBacking), so let's just
extract it from the if/else surrounding the application of the
encryption options.
John Ferlan [Wed, 18 Apr 2018 18:18:14 +0000 (14:18 -0400)]
storage_util: Move secretPath generation
Move generation of secretPath to storageBackendGenerateSecretData
and simplify a bit since we know vol->target.encryption is set plus
we have a local @enc.
John Ferlan [Wed, 18 Apr 2018 18:08:51 +0000 (14:08 -0400)]
storage_util: Generate the qcow secret earlier
Rather than having storageBackendCreateQemuImgCheckEncryption
perform the virStorageGenerateQcowEncryption, let's just do that
earlier during storageBackendCreateQemuImg so that the check
helper is just a check helper rather doing something different
based on whether the format is qcow[2] or raw based encryption.
This fixes an issue in the storageBackendResizeQemuImg processing
for qcow encryption where if a secret was not available for a
volume, a new secret will not be generated and instead an error
message will be generated.
John Ferlan [Wed, 18 Apr 2018 16:43:43 +0000 (12:43 -0400)]
storage_util: Remove unnecessary check
Commit id 'a48c71411' altered the logic a bit and didn't
remove an unnecessary check as info.encryption is true when
vol->target.encryption != NULL, so if we enter the if segment
with info.format == VIR_STORAGE_FILE_RAW && vol->target.encryption
!= NULL, then there's no way info.encryption could be false.
Jim Fehlig [Mon, 14 May 2018 20:45:05 +0000 (14:45 -0600)]
libxl: don't set hasManagedSave when performing save
libxlDoDomainSave() is used in both the save and managedsave code
paths but was unconditionally setting hasManagedSave to true on
success. As a result, undefine would fail after a non-managed
save/restore operation. E.g.
virsh define; virsh start
virsh save; virsh restore
virsh shutdown
virsh undefine
error: Refusing to undefine while domain managed save image exists
Modify libxlDoDomainSave() to take an additional parameter to
specify managed vs non-managed save, and change callers to use it.
Signed-off-by: Jim Fehlig <jfehlig@suse.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Michal Privoznik [Tue, 15 May 2018 06:58:33 +0000 (08:58 +0200)]
tests: Link mocks with libvirt.so
In a lot of our mocks (if not all of them) we use our internal
APIs (e.g. VIR_ALLOC). So far, we're relying on test binary that
links with the mock to drag in libvirt.so. Well, this works only
partially. Firstly, whatever binary we execute from tests will
fail (e.g. as Martin reported on the list ./qemucapsprobe fails
to execute qemu). Secondly, if there's a program that tries to
validate linking (like valgrind is doing) it fails because of
unresolved symbols.
Because of that we have to link our mocks with libvirt.so.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
Generates the QEMU command line for the vfio-ccw device.
Adds various functionality testing for vfio-ccw in libvirt:
1. Generation of QEMU command line from domain xml file
2. Generation of dump xml from domain xml file
3. Checks duplicate/invalid addresses for vfio-ccw devices.
Signed-off-by: Shalini Chellathurai Saroja <shalini@linux.vnet.ibm.com> Reviewed-by: Bjoern Walk <bwalk@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Reviewed-by: Marc Hartmayer <mhartmay@linux.vnet.ibm.com> Reviewed-by: Stefan Zimmermann <stzi@linux.ibm.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
Introduces the vfio-ccw model for mediated devices and prime vfio-ccw
devices such that CCW address will be generated.
Alters the qemuxml2xmltest for testing a basic mdev device using vfio-ccw.
Signed-off-by: Shalini Chellathurai Saroja <shalini@linux.vnet.ibm.com> Reviewed-by: Bjoern Walk <bwalk@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Reviewed-by: Marc Hartmayer <mhartmay@linux.vnet.ibm.com> Reviewed-by: Stefan Zimmermann <stzi@linux.ibm.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
Let us introduce the capability vfio-ccw for supporting the basic
channel I/O passthrough, which have been introduced in QEMU 2.10. The
current focus is to support dasd-eckd (cu_type/dev_type = 0x3990/0x3390)
as the target device.
Let us also introduce the capability QEMU_CAPS_CCW_CSSID_UNRESTRICTED
for virtual-css-bridge. This capability is based on the
cssid-unrestricted property which exists if QEMU no longer enforces
cssid restrictions based on ccw device types.
Vfio-ccw capability is dependent on the hidden virtual-css-bridge, so
that we are able to probe for the cssid-unrestriced property to make
sure the devices are visible to non-mcss-e enabled guests.
Signed-off-by: Shalini Chellathurai Saroja <shalini@linux.vnet.ibm.com> Reviewed-by: Bjoern Walk <bwalk@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
Let us introduce the capability QEMU_CAPS_CCW for virtual-css-bridge
and replace QEMU_CAPS_VIRTIO_CCW with QEMU_CAPS_CCW in code segments
which identify support for ccw devices.
The virtual-css-bridge is part of the ccw support introduced in QEMU 2.7.
The QEMU_CAPS_CCW capability is based on the existence of the QEMU type.
Let us also add the capability QEMU_CAPS_CCW to the tests which
require support for ccw devices.
Signed-off-by: Shalini Chellathurai Saroja <shalini@linux.vnet.ibm.com> Reviewed-by: Bjoern Walk <bwalk@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
Andrea Bolognani [Mon, 14 May 2018 07:45:10 +0000 (09:45 +0200)]
travis: Uninstall packages before upgrade
numpy (needed by cgal) started having the same issue with
linking as python, which makes upgrade and thus the entire
build fail on macOS.
Instead of playing more tricks with linking/unlinking, just
uninstall the problematic packages (and those dragging them
in) before doing anything else.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
QEMU has this new feature memory-backend-file.discard-data=yes
which is a nifty optimization. Basically, when qemu is quitting
or on memory hotplug it calls munmap() and close() on the file
that is backing the memory. However, this does not mean kernel
won't stop touching that part of memory. It still might. With
this feature enabled we tell kernel: "we don't need this memory
nor data stored in it". This makes kernel drop the memory
immediately without trying to sync memory with the mapped file.
Unfortunately, this cannot be turned on by default because we
can't be sure when users really don't care about what happens to
data after qemu dies. So it has to be opt-in. As usual, there are
three places where one can configure memory attributes. This
patch adds the feature to all of them.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Michal Privoznik [Thu, 12 Apr 2018 13:04:07 +0000 (15:04 +0200)]
qemu_monitor: Introduce qemuMonitorGetObjectProps
Now that we've gotten rid of misleading names we can introduce
qemuMonitorGetObjectProps() function which queries -object
properties. Again, some parts of code can be reused.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
log: update docs for daemons to improve user understanding
Strongly recommend against use of the log_levels setting since it
creates overly verbose logs and has a serious performance impact.
Describe the log filter syntax better and mention use of shell
glob syntax. Also provide more realistic example of good settings
to use. The libvirtd example is biased towards QEMU, but when the
drivers split off each daemon can get its own more appropriate
example.
Reviewed-by: John Ferlan <jferlan@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Rather than specialcasing handling of the '*' character, use fnmatch()
to get normal shell wildcard syntax, as described in 'man glob(7)'.
To get an indication of the performance impact of using globs instead
of plain string matches, a test program was written. The list of all
260 log categories was extracted from the source. Then a typical log
filters setup was picked by creating an array of the strings "qemu",
"security", "util", "cgroup", "event", "object". Every filter string
was matched against every log category. Timing information showed that
using strstr() this took 8 microseconds, while fnmatch() took 114
microseconds.
IOW, fnmatch is 14 times slower than our existing strstr check. These
numbers show a worst case scenario that will never be hit, because it
is rare that every log category would have data output. The log category
matches are cached, so each category is only checked once no matter how
many log statements are emitted. IOW despite being slower, this will
be lost in the noise and have no consequence on real world logging
performance.
Reviewed-by: John Ferlan <jferlan@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Michal Privoznik [Fri, 11 May 2018 12:05:53 +0000 (14:05 +0200)]
virbuffer: Set child buffer indent properly
There's this macro virBufferSetChildIndent which sets offset of
child buffer from given parent buffer. However, it is calling
virBufferAdjustIndent() which only adds adjustment instead of
calling virBufferSetIndent() which clears out any adjustment
previously set.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Julio Faracco [Thu, 10 May 2018 20:38:57 +0000 (17:38 -0300)]
rpc: replacing ssh_get_publickey() by ssh_get_server_publickey().
After version 0.7.5, libssh deprecated the function scope
ssh_get_publickey() and moved to ssh_get_server_publickey(). So, Libvirt
is failing to compile using this new function name.
Signed-off-by: Julio Faracco <jcfaracco@gmail.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Julio Faracco [Thu, 10 May 2018 20:38:56 +0000 (17:38 -0300)]
m4: checking if ssh_get_server_publickey() exists.
This commit adds some checks inside libssh m4 checking to verify if
ssh_get_server_publickey is available. This new function scope replaces
the old ssh_get_publickey() from libssh 0.7.5 and below. Assuming that
some distros are not showing the right version of libssh. This is a
simple way to check which function is available.
Signed-off-by: Julio Faracco <jcfaracco@gmail.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Michal Privoznik [Thu, 19 Apr 2018 08:00:36 +0000 (10:00 +0200)]
qemu: Start PR daemon on domain startup
Before we exec() qemu we have to spawn pr-helper processes for
all managed reservations (well, technically there can only one).
The only caveat there is that we should place the process into
the same namespace and cgroup as qemu (so that it shares the same
view of the system). But we can do that only after we've forked.
That means calling the setup function between fork() and exec().
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
Michal Privoznik [Tue, 10 Apr 2018 06:00:59 +0000 (08:00 +0200)]
qemu_cgroup: Allow /dev/mapper/control for PR
Just like in previous commit, qemu-pr-helper might want to open
/dev/mapper/control under certain circumstances. Therefore we
have to allow it in cgroups.
The change virdevmapper.c might look spurious but it isn't. After 6dd84f6850ca437 any path that we're allowing in deivces CGroup is
subject to virDevMapperGetTargets() inspection. And libdevmapper
returns ENXIO for the path from subject.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
If qemu-pr-helper is compiled with multipath support the first
thing it does is open /dev/mapper/control. Since we're going
to be running it inside qemu namespace we need to create it
there. Unfortunately, we don't know if it was compiled with or
without multipath so we have to create it anyway.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
In -object pr-manager-helper we tell qemu which socket to connect
to, then in -drive file-pr-manager we just reference the object
the drive in question should use.
For managed PR helper the alias is always "pr-helper0" and socket
path "${vm->priv->libDir}/pr-helper0.sock".
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
Michal Privoznik [Mon, 19 Feb 2018 06:41:25 +0000 (07:41 +0100)]
qemu: Introduce pr-manager-helper capability
The capability tracks if qemu has pr-manager-helper object. At
this time don't actually detect if qemu has the capability. Not
just yet. Only after the code is written the feature will be
enabled.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
a) there's no monitor command to change path where the pr-helper
connects to, or
b) there's no monitor command to introduce a new pr-helper for a
disk that already exists.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
John Ferlan [Thu, 19 Apr 2018 21:29:02 +0000 (17:29 -0400)]
util: Clean up consumers of virJSONValueArraySize
Rather than have virJSONValueArraySize return a -1 when the input
is not an array and then splat an error message, let's check for
an array before calling and then change the return to be a size_t
instead of ssize_t.
That means using the helper virJSONValueIsArray as well as using a
more generic error message such as "Malformed <something> array".
In some cases we can remove stack variables and when we cannot,
those variables should be size_t not ssize_t. Alter a few references
of if (!value) to be if (value == 0) instead as well.
Some callers can already assume an array is being worked on based
on the previous call, so there's less to do.
Signed-off-by: John Ferlan <jferlan@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Roland Schulz [Thu, 10 May 2018 14:16:30 +0000 (16:16 +0200)]
qemu: Remove parameter 'driver' from qemuBlockJobUpdate
The pointer to the qemu driver is already included in domain object's
private data, so does not need to be passed as yet another parameter
when the domain object is already passed.
Also removes parameter 'driver' from functions which had it just because of
qemuBlockJobUpdate.
Signed-off-by: Roland Schulz <schullzroll@gmail.com> Reviewed-by: Ján Tomko <jtomko@redhat.com> Signed-off-by: Ján Tomko <jtomko@redhat.com>
Lin Ma [Tue, 8 May 2018 14:20:34 +0000 (22:20 +0800)]
vshReadlineParse: Ignore vshReadlineOptionsGenerator for VSH_OT_ARGV options
Currently the VSH_OT_ARGV options don't support complete, But some of
VSH_OT_ARGV options are gonna support complete in upcoming patches.
Once applied the upcoming completion patches for VSH_OT_ARGV options, If
we don't ignore VSH_OT_ARGV here, The vshReadlineOptionsGenerator will
be called, Hence complete output will consist of the result by command
completer + the result by option completer, It's confusing.
e.g.
$ virsh domstats --domain <TAB><TAB>
--backing --interface --list-paused --perf --vcpu
--balloon leap42.3 --list-persistent --raw win10
--block --list-active --list-running sles12sp3
--cpu-total --list-inactive --list-shutoff sles15
--enforce --list-other --list-transient --state
After this patch and the upcoming completion patches:
$ virsh domstats --domain <TAB><TAB>
leap42.3 sles12sp3 sles15 win10
Lin Ma [Tue, 8 May 2018 14:20:31 +0000 (22:20 +0800)]
virsh: Conditionally Ignore the first entry in list of completions
The first entry in the returned array is the substitution for TEXT. It
causes unnecessary output if other commands or options share the same
prefix, e.g.
Peter Krempa [Wed, 28 Mar 2018 08:27:14 +0000 (10:27 +0200)]
tests: qemublock: Add tests for basic backing chain formats
Formats supporting backing chain such as qed, vmdk, don't have any other
parameters than the backing store and 'qcow' has only encryption params
which will be tested extra. Add this test case so they are covered since
any further test cases will mainly care about 'qcow2' and 'raw'.
The top level disk image would generate the following '-drive' cmdline:
Peter Krempa [Wed, 28 Mar 2018 07:40:03 +0000 (09:40 +0200)]
tests: qemublock: Add basic 'raw' file test
Test the JSON props generator with a very simple 'raw' image with no
other options. The node-names for the image are 31 bytes long so that we
validate our node name detector.
The top level disk image would generate the following '-drive' cmdline:
projects / libvirt.git / log