]> xenbits.xensource.com Git - libvirt.git/log
libvirt.git
5 years agovirprocess: Passthru error from virProcessRunInForkHelper
Michal Privoznik [Wed, 18 Mar 2020 11:59:08 +0000 (12:59 +0100)]
virprocess: Passthru error from virProcessRunInForkHelper

When running a function in a forked child, so far the only thing
we could report is exit status of the child and the error
message. However, it may be beneficial to the caller to know the
actual error that happened in the child.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Pavel Mores <pmores@redhat.com>
5 years agovirfile: Handle directories in virFileBindMountDevice()
Michal Privoznik [Fri, 4 Oct 2019 19:01:49 +0000 (21:01 +0200)]
virfile: Handle directories in virFileBindMountDevice()

The @src is not always a file. It may also be a directory (for
instance qemuDomainCreateDeviceRecursive() assumes that) - even
though it doesn't happen usually. Anyway, mount() can mount only
a dir onto a dir and a file onto a file.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Pavel Mores <pmores@redhat.com>
5 years agoqemuDomainBuildNamespace: Make @devPath const
Michal Privoznik [Fri, 4 Oct 2019 19:56:25 +0000 (21:56 +0200)]
qemuDomainBuildNamespace: Make @devPath const

The @devPath variable is not modifiable. It merely just points to
string containing path where private devtmpfs is being
constructed. Make it const so it doesn't look weird that it's not
freed.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Pavel Mores <pmores@redhat.com>
5 years agoqemuDomainBuildNamespace: Try harder to remove temp directories
Michal Privoznik [Fri, 4 Oct 2019 19:01:29 +0000 (21:01 +0200)]
qemuDomainBuildNamespace: Try harder to remove temp directories

If building namespace fails somewhere in the middle (that is some
files exists under devMountsSavePath[i]), then plain rmdir() is
not enough to remove dir. Umount the temp location and use
virFileDeleteTree() to remove the directory.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Pavel Mores <pmores@redhat.com>
5 years agoqemuDomainCreateDeviceRecursive: Report error if mkdir() fails
Michal Privoznik [Fri, 4 Oct 2019 18:59:10 +0000 (20:59 +0200)]
qemuDomainCreateDeviceRecursive: Report error if mkdir() fails

The virFileMakePathWithMode() which is our recursive version of
mkdir() fails, it simply just returns a negative value with errno
set. No error is reported (as compared to virFileTouch() for
instance).

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Pavel Mores <pmores@redhat.com>
5 years agotests: virstoragetest: validate that array deflattening works for gluster
Peter Krempa [Thu, 14 Dec 2017 17:01:05 +0000 (18:01 +0100)]
tests: virstoragetest: validate that array deflattening works for gluster

Validate that we are able to parse back the dotted syntax arrays we were
generating in the pre-blockdev era.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agojsontest: Add test cases for deflattening of arrays
Peter Krempa [Wed, 18 Mar 2020 16:02:42 +0000 (17:02 +0100)]
jsontest: Add test cases for deflattening of arrays

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agovirjson: Deflatten arrays generated by the json->commandline generator
Peter Krempa [Thu, 14 Dec 2017 17:03:04 +0000 (18:03 +0100)]
virjson: Deflatten arrays generated by the json->commandline generator

For the few instances where we'd generate an array in dotted syntax we
should be able to parse it back. Add another step in deflattening of the
dotted syntax which reconstructs the arrays so that the backing store
parser can parse it.

https://bugzilla.redhat.com/show_bug.cgi?id=1466177

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agoutil: json: Extract deflattening of keys into a separate function
Peter Krempa [Thu, 19 Mar 2020 17:11:48 +0000 (18:11 +0100)]
util: json: Extract deflattening of keys into a separate function

Extract the code so that there's a clean separation once we'll want do
do other steps.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agovirJSONValueObjectDeflattenWorker: Refactor cleanup
Peter Krempa [Wed, 18 Mar 2020 15:38:16 +0000 (16:38 +0100)]
virJSONValueObjectDeflattenWorker: Refactor cleanup

Use automatic memory handling to remove the cleanup section.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agovirBitmapNewEmpty: Use g_new0 to allocate and remove error checking
Peter Krempa [Wed, 18 Mar 2020 14:51:14 +0000 (15:51 +0100)]
virBitmapNewEmpty: Use g_new0 to allocate and remove error checking

virBitmapNewEmpty can't fail now so we can make it obvious and fix all
callers.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agovirStorageEncryptionSecretCopy: Properly copy internals
Peter Krempa [Thu, 19 Mar 2020 14:38:06 +0000 (15:38 +0100)]
virStorageEncryptionSecretCopy: Properly copy internals

virStorageEncryptionSecretPtr may have a string inside it, thus we must
copy the string too. Use virSecretLookupDefCopy to do that.

Caused by non-obvious code introduced in 756b46ddd24 and later 47e88b33b
which added a string that needed to be copied.

https://bugzilla.redhat.com/show_bug.cgi?id=1814923

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agovirSecretLookupDefCopy: Remove return value
Peter Krempa [Thu, 19 Mar 2020 14:27:40 +0000 (15:27 +0100)]
virSecretLookupDefCopy: Remove return value

The function always returns succes so there's no need for a return
value.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agoqemu: block: Split up formatting of JSON props for 'raw' and 'luks' drivers
Peter Krempa [Thu, 19 Mar 2020 16:23:33 +0000 (17:23 +0100)]
qemu: block: Split up formatting of JSON props for 'raw' and 'luks' drivers

qemuBlockStorageSourceGetFormatRawProps aggregated both formats but
since we now have props specific for either of those formats it's
unwanted to aggregate the code such way. Split out the 'luks' props
formatter into qemuBlockStorageSourceGetFormatLUKSProps.

The wrong separation demonstrates istself on formatting of the 'size'
and 'offset' attributes for the 'luks' driver which does not conform
to the qapi schema.

https://bugzilla.redhat.com/show_bug.cgi?id=1814975

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemuBlockStorageSourceNeedsStorageSliceLayer: Deal with 'luks' files
Peter Krempa [Thu, 19 Mar 2020 15:54:52 +0000 (16:54 +0100)]
qemuBlockStorageSourceNeedsStorageSliceLayer: Deal with 'luks' files

The 'luks' driver in qemu is as any other non-raw format driver and thus
doesn't support the properties for 'slice'. Since libvirt considers
luks files to be raw+encryption we need to special case them when
dealing with the slice.

https://bugzilla.redhat.com/show_bug.cgi?id=1814975

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemu: block: Extract logic decision when to use a separate 'raw' layer for slice
Peter Krempa [Thu, 19 Mar 2020 15:43:49 +0000 (16:43 +0100)]
qemu: block: Extract logic decision when to use a separate 'raw' layer for slice

Introduce qemuBlockStorageSourceNeedsStorageSliceLayer which will hold
the decision logic and fix all places that open-code it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemuxml2argvdata/disk-slices: Add test case for 'luks' encryption
Peter Krempa [Thu, 19 Mar 2020 15:26:53 +0000 (16:26 +0100)]
qemuxml2argvdata/disk-slices: Add test case for 'luks' encryption

Since libvirt handles the luks encryption in a weird special way
(raw+encryption) we should really test that case with slices as well.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemu: reset await_event in all error paths in qemuAgentCommand
Nikolay Shirokovskiy [Fri, 20 Mar 2020 07:16:23 +0000 (10:16 +0300)]
qemu: reset await_event in all error paths in qemuAgentCommand

A fixup to patch [1]. We need to reset await_event in all
error paths.

[1] 52532073d : qemu: remove redundant needReply argument of qemuAgentCommand

Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agoqemu: Don't crash when getting targets for a multipath
Michal Privoznik [Thu, 19 Mar 2020 11:51:55 +0000 (12:51 +0100)]
qemu: Don't crash when getting targets for a multipath

In one of my previous commits I've introduced code that creates
all devices for given (possible) multipath target. But I've made
a mistake there - the code accesses 'next->path' without checking
if the disk source is local. Note that the 'next->path' is
NULL/doesn't make sense for VIR_STORAGE_TYPE_NVME.

Fixes: a30078cb832646177defd256e77c632905f1e6d0
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1814947

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agodocs: Use <h1> to make sure kbase.html has page title
Sebastian Mitterle [Thu, 19 Mar 2020 17:59:27 +0000 (18:59 +0100)]
docs: Use <h1> to make sure kbase.html has page title

Signed-off-by: Sebastian Mitterle <smitterl@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agodocs: formatbackup: Fix link to knowledge base article
Sebastian Mitterle [Thu, 19 Mar 2020 17:59:20 +0000 (18:59 +0100)]
docs: formatbackup: Fix link to knowledge base article

Signed-off-by: Sebastian Mitterle <smitterl@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agotests: switch away from HAVE_SOCKETPAIR
Pino Toscano [Thu, 19 Mar 2020 11:02:45 +0000 (12:02 +0100)]
tests: switch away from HAVE_SOCKETPAIR

Since the removal of gnulib, HAVE_SOCKETPAIR is no more defined, making
these two tests effectively skipped.

Use the same strategy used in other generic library bits, i.e. exclude
the socketpair usage on Windows.

Semi-related change in virnetdaemontest.c to make it build: since
virutil.h does not include unistd.h anymore, we need to include it.

Signed-off-by: Pino Toscano <ptoscano@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agovmx: make 'fileName' optional for CD-ROMs
Pino Toscano [Wed, 18 Mar 2020 09:46:55 +0000 (10:46 +0100)]
vmx: make 'fileName' optional for CD-ROMs

It seems like CD-ROMs may have no 'fileName' property specified in case
there is nothing configured as attachment for the drive. Hence, make
sure that virVMXParseDisk() do not consider it mandatory anymore,
considering it an empty block cdrom device. Sadly virVMXParseDisk() is
used also to parse disk and floppies, so make sure that a NULL fileName
is handled in cdrom- and floppy-related paths.

https://bugzilla.redhat.com/show_bug.cgi?id=1808610

Signed-off-by: Pino Toscano <ptoscano@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Tested-by: Richard W.M. Jones <rjones@redhat.com>
5 years agovmx: shortcut earlier few 'ignore' cases in virVMXParseDisk()
Pino Toscano [Wed, 18 Mar 2020 09:29:51 +0000 (10:29 +0100)]
vmx: shortcut earlier few 'ignore' cases in virVMXParseDisk()

Move earlier the checks for skipping a hard disk when parsing a CD-DROM,
and for skipping a CD-ROM when parsing a hard disk. This should have no
behaviour changes, and avoids to add repeated checks in following
commits.

Signed-off-by: Pino Toscano <ptoscano@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Tested-by: Richard W.M. Jones <rjones@redhat.com>
5 years agoqemu: Suppress error reporting from qemuMonitorDelObject in cleanup paths
Peter Krempa [Wed, 18 Mar 2020 11:24:40 +0000 (12:24 +0100)]
qemu: Suppress error reporting from qemuMonitorDelObject in cleanup paths

Many calls of qemuMonitorDelObject don't actually check the return value
or report the error from the object deletion itself since they are on
cleanup paths. In some cases this can lead to reporting of spurious
errors e.g. when qemuMonitorDelObject is used to clean up a possibly
pre-existing objects.

Add a new argument for qemuMonitorDelObject which controls whether
the internals report errors from qemu and fix all callers accordingly.

Note that some of the cases on device unplug which check the error code
don't in fact propagate the error to the user, but in this case it is
important to add the log entry anyways for tracing that the device
deletion failed.

https://bugzilla.redhat.com/show_bug.cgi?id=1784040

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agoqemuMonitorJSONCheckError: Allow suppressing of error reporting
Peter Krempa [Wed, 18 Mar 2020 10:08:58 +0000 (11:08 +0100)]
qemuMonitorJSONCheckError: Allow suppressing of error reporting

In some cases we'll need to check whether there was an error but avoid
reporting an actual libvirt error. Rename qemuMonitorJSONCheckError to
qemuMonitorJSONCheckErrorFull with a new flag to suppress the error
reporting and add a wrapper with the original name so that callers don't
need to be fixed.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agoqemuMonitorJSONCheckError: Use g_autofree
Peter Krempa [Wed, 18 Mar 2020 09:34:32 +0000 (10:34 +0100)]
qemuMonitorJSONCheckError: Use g_autofree

Eliminate cleanup code by using g_autofree.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agoqemuMonitorJSON(Add|Del)Object: Refactor cleanup
Peter Krempa [Wed, 18 Mar 2020 09:29:41 +0000 (10:29 +0100)]
qemuMonitorJSON(Add|Del)Object: Refactor cleanup

Use 'g_autoptr' and remove the cleanup label and ret variable.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agoqemuDomainChangeEjectableMedia: Don't always remove managed PR daemon
Peter Krempa [Wed, 18 Mar 2020 10:44:01 +0000 (11:44 +0100)]
qemuDomainChangeEjectableMedia: Don't always remove managed PR daemon

When changing media we'd attempt to remove the managed pr daemon even if
neither of the images involved in the media change used it. This caused
libvirtd to log a spurious error:

2020-03-18 01:41:19.832+0000: 643207: error : qemuMonitorJSONCheckError:412 : internal error: unable to execute QEMU command 'object-del': object 'pr-helper0' not found

With this patch we completely avoid calling the deletion code.

https://bugzilla.redhat.com/show_bug.cgi?id=1814486

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agoqemuDomainVcpuValidateConfig: Properly initialize 'firstcpu' variable
Peter Krempa [Tue, 17 Mar 2020 17:33:31 +0000 (18:33 +0100)]
qemuDomainVcpuValidateConfig: Properly initialize 'firstcpu' variable

The loop which checks whether the vcpus are in proper configuration for
the requested hot(un)plug skips the first modified vcpu. This means
that 'firstvcpu' which is used to print the error message in case the
configuration is not suitable would never point to the first modified
vcpu.

In cases such as:

  <vcpu placement='auto' current='5'>8</vcpu>
  <vcpus>
    <vcpu id='0' enabled='yes' hotpluggable='no'/>
    <vcpu id='1' enabled='yes' hotpluggable='no'/>
    <vcpu id='2' enabled='yes' hotpluggable='no'/>
    <vcpu id='3' enabled='yes' hotpluggable='no'/>
    <vcpu id='4' enabled='yes' hotpluggable='no'/>
    <vcpu id='5' enabled='no' hotpluggable='yes'/>
    <vcpu id='6' enabled='no' hotpluggable='yes'/>
    <vcpu id='7' enabled='no' hotpluggable='yes'/>
  </vcpus>

 # virsh setvcpu --config --disable  upstream 1
 error: invalid argument: vcpu '-1' can't be modified as it is followed by non-hotpluggable online vcpus

After this fix the proper vcpu is reported in the error message:

 # virsh setvcpu --config --disable  upstream 1
 error: invalid argument: vcpu '1' can't be modified as it is followed by non-hotpluggable online vcpu

https://bugzilla.redhat.com/show_bug.cgi?id=1611061

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
5 years agoconf: Don't generate clashing machine names for embed driver
Michal Privoznik [Wed, 11 Mar 2020 16:56:12 +0000 (17:56 +0100)]
conf: Don't generate clashing machine names for embed driver

So far, when using the qemu:///embed driver, management
applications can't chose whether they want to register their
domains in machined or not. While having that option is certainly
desired, it will require more work. What we can do meanwhile is
to generate names that include part of hash of the root
directory. This is to ensure that if two applications using
different roots but the same domain name (and ID) start the
domain no clashing name for machined is generated.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
5 years agovirDomainGenerateMachineName: Use g_autofree for @username
Michal Privoznik [Wed, 18 Mar 2020 14:45:07 +0000 (15:45 +0100)]
virDomainGenerateMachineName: Use g_autofree for @username

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
5 years agoqemu_conf: Track embed root dir
Michal Privoznik [Wed, 11 Mar 2020 17:33:34 +0000 (18:33 +0100)]
qemu_conf: Track embed root dir

When initializing virQEMUDriverConfig structure we are given the
root directory for possible embed connection. Save it for future
use. While we could get it later from @uri member, it's not as
easy as dereferencing a pointer (virURIParse() +
virURIGetParam() + error reporting).

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
5 years agocpu: Honor check='full' for host-passthrough CPUs
Jiri Denemark [Mon, 9 Mar 2020 13:14:04 +0000 (14:14 +0100)]
cpu: Honor check='full' for host-passthrough CPUs

The check attribute was completely ignored for host-passthrough CPUs
even if they explicitly requested some features to be enabled. For
example, a domain with the following CPU definition

  <cpu mode='host-passthrough' check='full'>
    <feature policy='require' name='svm'/>
  </cpu>

would happily start even when 'svm' cannot be enabled.

Let's call virCPUArchUpdateLive for host-passthrough CPUs with
VIR_CPU_CHECK_FULL to make sure the architecture specific code can
validate the provided virtual CPU against the desired definition.

https://bugzilla.redhat.com/show_bug.cgi?id=1515677

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agocpu_x86: Prepare virCPUx86UpdateLive for easier extension
Jiri Denemark [Mon, 9 Mar 2020 13:12:04 +0000 (14:12 +0100)]
cpu_x86: Prepare virCPUx86UpdateLive for easier extension

Adding more checks into the existing if statements would turn them into
an unreadable mess.

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agocpu: Change control flow in virCPUUpdateLive
Jiri Denemark [Mon, 9 Mar 2020 09:30:39 +0000 (10:30 +0100)]
cpu: Change control flow in virCPUUpdateLive

The updateLive CPU sub-driver function is supposed to be called only for
a subset of CPU definitions. Let's make it more obvious by turning a
negative test and return into a positive check.

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agolxc: Add HPET device into allowed devices
Julio Faracco [Mon, 2 Mar 2020 00:54:13 +0000 (21:54 -0300)]
lxc: Add HPET device into allowed devices

This commit is related to RTC timer device too. HPET is being shared
from host device through `localtime` clock. This timer is available
creating a new timer using `hpet` name.

Signed-off-by: Julio Faracco <jcfaracco@gmail.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agolxc: Add Real Time Clock device into allowed devices
Julio Faracco [Mon, 2 Mar 2020 00:54:12 +0000 (21:54 -0300)]
lxc: Add Real Time Clock device into allowed devices

This commit share host Real Time Clock device (rtc) into LXC containers
to support hardware clock. This should be available setting up a `rtc`
timer under clock section. Since this option is not emulated, it should
be available only for `localtime` clock. This option should be readonly
due to security reasons.

Before:
    root# hwclock --verbose
    hwclock from util-linux 2.32.1
    System Time: 1581877557.598365
    Trying to open: /dev/rtc0
    Trying to open: /dev/rtc
    Trying to open: /dev/misc/rtc
    No usable clock interface found.
    hwclock: Cannot access the Hardware Clock via any known method.

Now:
    root# hwclock
    2020-02-16 18:23:55.374134+00:00
    root# hwclock -w
    hwclock: ioctl(RTC_SET_TIME) to /dev/rtc to set the time failed:
    Permission denied

Signed-off-by: Julio Faracco <jcfaracco@gmail.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agoqemublocktest: Add tests for re-enabling of bitmaps after commit
Peter Krempa [Tue, 17 Mar 2020 14:51:57 +0000 (15:51 +0100)]
qemublocktest: Add tests for re-enabling of bitmaps after commit

Some branches were not covered and thus we didn't catch that the bitmaps
are not re-enabled if nothing is merged into them. Two bitmaps are
necessary to reliably test the case due to hash table ordering.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Tested-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agoqemuBlockBitmapsHandleCommitFinish: Use proper variable to iterate
Peter Krempa [Tue, 17 Mar 2020 14:31:12 +0000 (15:31 +0100)]
qemuBlockBitmapsHandleCommitFinish: Use proper variable to iterate

The function repeatedly checked the first element rather than iterating
through the array.

Reported-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Tested-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agoqemuBlockBitmapsHandleCommitStart: Fix allocation of string list
Peter Krempa [Tue, 17 Mar 2020 14:10:23 +0000 (15:10 +0100)]
qemuBlockBitmapsHandleCommitStart: Fix allocation of string list

Allocate space also for the terminating NULL.

Reported-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Tested-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agoqemu: lookup node device against nodedev driver before getting XML
Daniel P. Berrangé [Tue, 10 Mar 2020 17:03:54 +0000 (17:03 +0000)]
qemu: lookup node device against nodedev driver before getting XML

Some of the node device APIs are a little odd because they accept a
virNodeDevicePtr object but are still implemented by the virt drivers.
The first thing the virt drivers need to do is get the XML config
associated with the node device, and that means talking to the node
device driver.

This worked previously because with monolithic libvirtd, both the
virt driver and node device driver were in the same daemon and thus
a single virConnectPtr can talk to both drivers.

With the split daemon world though, the virNodeDevicePtr passed into
the APIs is associated with the QEMU driver virConnectPtr, which has
no ability to invoke APIs against the node device driver. We must thus
get a duplicate virNodeDevicePtr object which is associated with a
virConnectPtr for the node device driver.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agorpc: avoid name lookup when dispatching node device APIs
Daniel P. Berrangé [Tue, 10 Mar 2020 16:40:47 +0000 (16:40 +0000)]
rpc: avoid name lookup when dispatching node device APIs

The node device APIs are a little unusual because we don't use a
"remote_nonnull_node_device" object on the wire, instead we just
have a "remote_string" for the device name. This meant dispatcher
code generation needed special cases. In doing so we mistakenly
used the virNodeDeviceLookupByName() API which gets dispatched
into the driver, instead of get_nonnull_node_device() which
directly populates a virNodeDevicePtr object.

This wasn't a problem with monolithic libvirtd, as the
virNodeDeviceLookupByName() API call was trivially satisfied
by the registered driver, albeit with an extra (undesirable)
authentication check. With the split daemons, the call to
virNodeDeviceLookupByName() fails in virtqemud, because the
node device driver obviously doesn't exist in that daemon.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agorpc: fix dispatch for node device APIs for virt drivers
Daniel P. Berrangé [Mon, 9 Mar 2020 11:59:35 +0000 (11:59 +0000)]
rpc: fix dispatch for node device APIs for virt drivers

Despite their names, the following APIs:

    virNodeDeviceDettach
    virNodeDeviceDetachFlags
    virNodeDeviceReAttach
    virNodeDeviceReset

are all handled by the virt drivers, not the node device driver.
A bug in the RPC generator meant that these APIs were sent to
the nodedev driver for handling. This caused breakage with the
split daemons, since nothing was available to process them.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agovirhostcpu.c: fix 'die_id' parsing for Power hosts
Daniel Henrique Barboza [Tue, 17 Mar 2020 00:01:34 +0000 (21:01 -0300)]
virhostcpu.c: fix 'die_id' parsing for Power hosts

Commit 7b79ee2f78 makes assumptions about die_id parsing in
the sysfs that aren't true for Power hosts. In both Power8
and Power9, running 5.6 and 4.18 kernel respectively,
'die_id' is set to -1:

$ cat /sys/devices/system/cpu/cpu0/topology/die_id
-1

This breaks virHostCPUGetDie() parsing because it is trying to
retrieve an unsigned integer, causing problems during VM start:

virFileReadValueUint:4128 : internal error: Invalid unsigned integer
value '-1' in file '/sys/devices/system/cpu/cpu0/topology/die_id'

This isn't necessarily a PowerPC only behavior. Linux kernel commit
0e344d8c70 added in the former Documentation/cputopology.txt, now
Documentation/admin-guide/cputopology.rst, that:

  To be consistent on all architectures, include/linux/topology.h
  provides default definitions for any of the above macros that are
  not defined by include/asm-XXX/topology.h:

  1) topology_physical_package_id: -1
  2) topology_die_id: -1
  (...)

This means that it might be expected that an architecture that
does not implement the die_id element will mark it as -1 in
sysfs.

It is not required to change die_id implementation from uInt to
Int because of that. Instead, let's change the parsing of the
die_id in virHostCPUGetDie() to read an integer value and, in
case it's -1, default it to zero like in case of file not found.
This is enough to solve the issue Power hosts are experiencing.

Fixes: 7b79ee2f78bbf2af76df2f6466919e19ae05aeeb
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agonodedev: fix race in API usage vs initial device enumeration
Daniel P. Berrangé [Fri, 13 Mar 2020 11:53:25 +0000 (11:53 +0000)]
nodedev: fix race in API usage vs initial device enumeration

During startup the udev node device driver impl uses a background thread
to populate the list of devices to avoid blocking the daemon startup
entirely. There is no synchronization to the public APIs, so it is
possible for an application to start calling APIs before the device
initialization is complete.

This was not a problem in the old approach where libvirtd was started
on boot, as initialization would easily complete before any APIs were
called.

With the use of socket activation, however, APIs are invoked from the
very moment the daemon starts. This is easily seen by doing a

  'virsh -c nodedev:///system list'

the first time it runs it will only show one or two devices. The second
time it runs it will show all devices. The solution is to introduce a
flag and condition variable for APIs to synchronize against before
returning any data.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agoqemuDomainGetGuestInfo: don't assign NULL hostname
Peter Krempa [Fri, 13 Mar 2020 07:57:25 +0000 (08:57 +0100)]
qemuDomainGetGuestInfo: don't assign NULL hostname

Don't rely on error check and assign hostname only when non-NULL.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemu: blockjob: Re-enable bitmaps after failed block-commit
Peter Krempa [Tue, 3 Mar 2020 14:58:09 +0000 (15:58 +0100)]
qemu: blockjob: Re-enable bitmaps after failed block-commit

If a block-commit fails we should at least re-enable the bitmaps so that
the operation can be re-tried.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoqemu: blockjob: Handle bitmaps after finish of normal block-commit
Peter Krempa [Tue, 3 Mar 2020 14:46:15 +0000 (15:46 +0100)]
qemu: blockjob: Handle bitmaps after finish of normal block-commit

Merge the bitmaps into base of the block commit after the job finishes.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoqemuDomainBlockPivot: Handle merging of bitmaps when pivoting an active block-commit
Peter Krempa [Tue, 3 Mar 2020 13:14:28 +0000 (14:14 +0100)]
qemuDomainBlockPivot: Handle merging of bitmaps when pivoting an active block-commit

Active layer block commit makes the 'base' image the new top image of
the disk after it finishes. This means that all bitmap operations need
to be handled prior to this happening as we'd lose writes otherwise.

The ideal place is to handle it when pivoting to the new image as only
guest-writes would be happening after this point.

Use qemuBlockBitmapsHandleCommitFinish to calculate the merging
transaction.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoqemuDomainBlockCommit: Handle bitmaps on start of commit
Peter Krempa [Tue, 3 Mar 2020 12:59:48 +0000 (13:59 +0100)]
qemuDomainBlockCommit: Handle bitmaps on start of commit

On start of the commit job, we need to disable any active bitmap in the
base. Use qemuBlockBitmapsHandleCommitStart to calculate which and call
the appropriate QMP APIs. We use blockdev-reopen to make the 'base'
writable to disable the bitmaps.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoqemuBlockJobDiskNewCommit: Propagate 'disabledBitmapsBase'
Peter Krempa [Tue, 3 Mar 2020 12:31:57 +0000 (13:31 +0100)]
qemuBlockJobDiskNewCommit: Propagate 'disabledBitmapsBase'

Add an argument to qemuBlockJobDiskNewCommit to propagate the list of
disabled bitmaps into the job data structure.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoqemublocktest: Add tests of broken bitmap chain handling during block-commit
Peter Krempa [Wed, 4 Mar 2020 16:25:03 +0000 (17:25 +0100)]
qemublocktest: Add tests of broken bitmap chain handling during block-commit

Use the 'snapshots-synthetic-broken' test data for block-commit.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoqemublocktest: Add more tests for block-commit bitmap handling with snapshots
Peter Krempa [Mon, 2 Mar 2020 16:26:05 +0000 (17:26 +0100)]
qemublocktest: Add more tests for block-commit bitmap handling with snapshots

Test handling of more complex cases of merging bitmaps accross
snapshots.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoqemublocktest: Add tests for handling of bitmaps during block-commit
Peter Krempa [Mon, 2 Mar 2020 16:16:17 +0000 (17:16 +0100)]
qemublocktest: Add tests for handling of bitmaps during block-commit

Add code for testing the two necessary steps of handling bitmaps during
block commit and exercise the code on the test data which we have for
bitmap handling.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoqemu: block: Implement helpers for dealing with bitmaps during block commit
Peter Krempa [Mon, 2 Mar 2020 13:40:18 +0000 (14:40 +0100)]
qemu: block: Implement helpers for dealing with bitmaps during block commit

qemuBlockBitmapsHandleCommitStart prepares for disabling the bitmaps in
the 'base' of the commit job so that the bitmaps are not dirtied by the
commit job. This needs to be done prior to start of the commit job.

qemuBlockBitmapsHandleCommitFinish then calculates the necessary merges
that agregate all the bitmaps between the commited images and write them
into the base bitmap.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoqemublocktest: Fix and optimize fake image chain
Peter Krempa [Mon, 2 Mar 2020 14:03:07 +0000 (15:03 +0100)]
qemublocktest: Fix and optimize fake image chain

Set the 'id' field of the backing chain properly so that we can look
up images, and initialize 6 images instead of 10 as we don't use more
currently.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoqemu: blockjob: Store list of bitmaps disabled prior to commit
Peter Krempa [Tue, 25 Feb 2020 06:28:44 +0000 (07:28 +0100)]
qemu: blockjob: Store list of bitmaps disabled prior to commit

Starting a commit job will require disabling bitmaps in the base image
so that they are not dirtied by the commit job. We need to store a list
of the bitmaps so that we can later re-enable them.

Add a field and status XML handling code as well as a test.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoqemu: domain: Extract parsing of 'commit' blockjob data into a function
Peter Krempa [Tue, 25 Feb 2020 06:22:05 +0000 (07:22 +0100)]
qemu: domain: Extract parsing of 'commit' blockjob data into a function

I'll be adding more fields to care about so splitting the code out will
be better long-term.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Pavel Mores <pmores@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoqemu: domain: Extract formatting of 'commit' blockjob data into a function
Peter Krempa [Tue, 25 Feb 2020 06:14:13 +0000 (07:14 +0100)]
qemu: domain: Extract formatting of 'commit' blockjob data into a function

I'll be adding more fields to care about so splitting the code out will
be better long-term.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Pavel Mores <pmores@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoqemuDomainBlockCommit: Move checks depending on capabilities after liveness check
Peter Krempa [Tue, 3 Mar 2020 15:42:23 +0000 (16:42 +0100)]
qemuDomainBlockCommit: Move checks depending on capabilities after liveness check

Since capabilities are not present for inactive VMs we'd report that we
don't support '--delete' or committing while checkpoints exist rather
than the proper error.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoqemuCheckpointDiscardBitmaps: Use correct field for checkpoint bitmap name
Peter Krempa [Mon, 24 Feb 2020 16:30:26 +0000 (17:30 +0100)]
qemuCheckpointDiscardBitmaps: Use correct field for checkpoint bitmap name

The code deleting checkpoints needs the name of the parent checkpoint's
disk's bitmap but was using the disk alias instead. This would create
wrong bitmaps after deleting some checkpoints.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoqemuCheckpointDiscardBitmaps: Reopen images for bitmap modifications
Peter Krempa [Thu, 13 Feb 2020 11:49:14 +0000 (12:49 +0100)]
qemuCheckpointDiscardBitmaps: Reopen images for bitmap modifications

Qemu's bitmap APIs don't reopen the appropriate images read-write for
modification. It's libvirt's duty to reopen them via blockdev-reopen
if we wish to modify the bitmaps.

Use the new helpers to reopen the images for bitmap manipulation.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoqemu: block: implement helpers for blockdev-reopen
Peter Krempa [Thu, 13 Feb 2020 08:24:36 +0000 (09:24 +0100)]
qemu: block: implement helpers for blockdev-reopen

Introduce a set of helpers to call blockdev-reopen in certain scenarios

Libvirt will use the QMP command to turn certain members of the backing
chain read-write for bitmap manipulation and we'll also want to use it
to replace/install the backing chain of a qcow2 format node.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoqemu: monitor: Add handler for blockdev-reopen
Peter Krempa [Thu, 13 Feb 2020 08:00:37 +0000 (09:00 +0100)]
qemu: monitor: Add handler for blockdev-reopen

Introduce the monitor code for using blockdev-reopen.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoqemu: capabilities: Add QEMU_CAPS_BLOCKDEV_REOPEN
Peter Krempa [Thu, 27 Feb 2020 11:23:29 +0000 (12:23 +0100)]
qemu: capabilities: Add QEMU_CAPS_BLOCKDEV_REOPEN

This capability will be asserted once qemu stabilizes 'blockdev-reopen'.
For now we just add the capability so that we can introduce some code
that will use the reopening call. This will show our willingness to
adopt use of reopen and help qemu developers stabilize it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoqemu: convert DomainLogContext class to use GObject
Gaurav Agrawal [Mon, 16 Mar 2020 12:10:24 +0000 (17:40 +0530)]
qemu: convert DomainLogContext class to use GObject

Signed-off-by: Gaurav Agrawal <agrawalgaurav@gnome.org>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
5 years agotests: validate parsing of CPUs with dies > 1
Daniel P. Berrangé [Mon, 16 Mar 2020 12:55:52 +0000 (12:55 +0000)]
tests: validate parsing of CPUs with dies > 1

Add sample data files for validating handling of a QEMU guest started
with:

  -smp 7,maxcpus=16,sockets=2,dies=2,cores=2,threads=2

Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agoqemu: fix detection of vCPU pids when multiple dies are present
Daniel P. Berrangé [Fri, 13 Mar 2020 16:43:26 +0000 (16:43 +0000)]
qemu: fix detection of vCPU pids when multiple dies are present

The logic for querying hotpluggable CPUs needs to sort the list
of CPUs returned by QEMU. Unfortunately our sorting method failed
to use the die_id field, so CPUs were not correctly sorted.

This is seen when configuring a guest with partially populated
CPUs

  <vcpu placement='static' current='1'>16</vcpu>
  <cpu...>
    <topology sockets='4' dies='2' cores='1' threads='2'/>
  </cpu>

Then trying to start it would fail:

  # virsh -c qemu:///system start demo
  error: Failed to start domain demo
  error: internal error: qemu didn't report thread id for vcpu '0'

Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agodocs: virtiofs: add missing aposthrophe
Ján Tomko [Mon, 16 Mar 2020 14:51:17 +0000 (15:51 +0100)]
docs: virtiofs: add missing aposthrophe

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reported-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
5 years agoqemu: Pass through arguments of 'ssh' block driver used by libguestfs
Peter Krempa [Mon, 9 Mar 2020 14:05:58 +0000 (15:05 +0100)]
qemu: Pass through arguments of 'ssh' block driver used by libguestfs

We currently don't model the 'ssh' protocol properties properly and
since it seems impossible for now (agent path passed via environment
variable). To allow libguestfs to work as it used in pre-blockdev era we
must carry the properties over to the command line. For this instance we
just store it internally and format it back.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemublocktest: Add JSON->JSON test cases for block device backends
Peter Krempa [Mon, 9 Mar 2020 16:06:10 +0000 (17:06 +0100)]
qemublocktest: Add JSON->JSON test cases for block device backends

Add testing of the interpretation of the JSON pseudo-protocol backing
store into JSON structs for blockdev. This will be used to test JSON
pseudo-URIs used by libguestfs while actually also validating the output
against the QMP schema. Since libguestfs uses obsolete/undocumented
values the outputs will differ and a benefit is that modern output is
used now.

The example test case covers the fields and values used by libguestfs
when using the https driver.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemublocktest: XMLjsonXML: Test formatting/parsing of modern JSON
Peter Krempa [Mon, 9 Mar 2020 14:39:04 +0000 (15:39 +0100)]
qemublocktest: XMLjsonXML: Test formatting/parsing of modern JSON

The test was invoking the JSON formatter with the 'legacy' flag thus
formatting bunch of obsolete JSON blockdev definitions. We also should
test the modern ones. Add a boolean and re-run all the tests in both
cases.

Additionally for any modern invocation we should also validate that the
output conforms to the QAPI schema.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemublocktest: Extract schema root for blockdev-add validation
Peter Krempa [Mon, 9 Mar 2020 14:33:58 +0000 (15:33 +0100)]
qemublocktest: Extract schema root for blockdev-add validation

Move lookup of the schema root earlier so that multiple functions
can use it for validation.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemublocktest: Load QMP schema earlier
Peter Krempa [Mon, 9 Mar 2020 14:24:56 +0000 (15:24 +0100)]
qemublocktest: Load QMP schema earlier

Multiple tests require the schema. Extract the loading into a separate
variable to avoid issues with ownership of the pointer.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agovirStorageSourceParseBackingJSONUri: Handle undocumented value 'off' for sslverify
Peter Krempa [Fri, 6 Mar 2020 07:29:59 +0000 (08:29 +0100)]
virStorageSourceParseBackingJSONUri: Handle undocumented value 'off' for sslverify

libguestfs abuses a quirk of qemu's parser to accept also other variants
of the 'sslverify' field which would be valid on the command line but
are not documented in the QMP schema.

If we encounter the 'off' string instead of an boolean handle it rather
than erroring out to continue support of pre-blockdev configurations.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agovirstoragefile: Add JSON parser for 'sslverify', 'readahead', 'cookies' and 'timeout'
Peter Krempa [Fri, 6 Mar 2020 07:13:06 +0000 (08:13 +0100)]
virstoragefile: Add JSON parser for 'sslverify', 'readahead', 'cookies' and 'timeout'

Add support for parsing the recently added fields from backing file
pseudo-protocol strings.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemu: block: Implement readahead and timeout properties for 'curl' driver
Peter Krempa [Fri, 6 Mar 2020 06:09:22 +0000 (07:09 +0100)]
qemu: block: Implement readahead and timeout properties for 'curl' driver

Pass in the correct fields.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemu: block: Add support for HTTP cookies
Peter Krempa [Thu, 4 May 2017 10:55:49 +0000 (12:55 +0200)]
qemu: block: Add support for HTTP cookies

Pass the alias of the secret object holding the cookie data as
'cookie-secret' to qemu.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemu: Handle hotplug and commandline for secret objects for http cookies
Peter Krempa [Mon, 9 Mar 2020 08:14:07 +0000 (09:14 +0100)]
qemu: Handle hotplug and commandline for secret objects for http cookies

Implement both commandline support and hotplug by adding the http cookie
handling to 'qemuBlockStorageSourceAttachData' handling functions for
it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemuDomainSecretStorageSourcePrepare: Setup secret for http cookies
Peter Krempa [Mon, 9 Mar 2020 08:04:33 +0000 (09:04 +0100)]
qemuDomainSecretStorageSourcePrepare: Setup secret for http cookies

QEMU's curl driver requires the cookies concatenated and allows themi to
be passed in via a secret. Prepare the value for the secret and encrypt
it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemu: domain: Store data for 'secret' object representing http cookies
Peter Krempa [Mon, 9 Mar 2020 07:19:02 +0000 (08:19 +0100)]
qemu: domain: Store data for 'secret' object representing http cookies

The http cookies can have potentially sensitive values and thus should
not be leaked into the command line. This means that we'll need to
instantiate a 'secret' object in qemu to pass the value encrypted.

This patch adds infrastructure for storing of the alias in the status
XML.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemu: block: Implement ssl verification configuration
Peter Krempa [Fri, 28 Apr 2017 10:58:17 +0000 (12:58 +0200)]
qemu: block: Implement ssl verification configuration

Allow disabling of SSL certificate validation for HTTPS and FTPS drives
in qemu.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemuxml2argvtest: Add test case for disks with http(s) source
Peter Krempa [Mon, 9 Mar 2020 11:38:21 +0000 (12:38 +0100)]
qemuxml2argvtest: Add test case for disks with http(s) source

Upcoming patches will implement the support for sslverify, cookies,
readahead, and timeout properties. Add a test file which will collect
the cases.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemuDomainValidateStorageSource: Validate new network storage parameters
Peter Krempa [Thu, 5 Mar 2020 16:59:04 +0000 (17:59 +0100)]
qemuDomainValidateStorageSource: Validate new network storage parameters

Ensure that the new fields are allowed only when -blockdev is used or
when they are in the detected part of the backing chain where qemu will
handle them internally.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoconf: Add support for setting timeout and readahead size for network disks
Peter Krempa [Thu, 5 Mar 2020 15:50:46 +0000 (16:50 +0100)]
conf: Add support for setting timeout and readahead size for network disks

Some disk backends support configuring the readahead buffer or timeout
for requests. Add the knobs to the XML.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoconf: Add support for cookies for HTTP based disks
Peter Krempa [Tue, 9 May 2017 12:52:40 +0000 (14:52 +0200)]
conf: Add support for cookies for HTTP based disks

Add possibility to specify one or more cookies for http based disks.
This patch adds the config parser, storage and validation of the
cookies.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoconf: Add support for modifying ssl validation for https/ftps disks
Peter Krempa [Fri, 28 Apr 2017 10:24:46 +0000 (12:24 +0200)]
conf: Add support for modifying ssl validation for https/ftps disks

To allow turning off verification of SSL cerificates add a new element
<ssl> to the disk source XML which will allow configuring the validation
process using the 'verify' attribute.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemuDomainGetSecretAESAlias: Replace outstanding uses with qemuAliasForSecret
Peter Krempa [Mon, 9 Mar 2020 07:03:34 +0000 (08:03 +0100)]
qemuDomainGetSecretAESAlias: Replace outstanding uses with qemuAliasForSecret

There are two last callers of this function. Replace them by
qemuAliasForSecret and delete qemuDomainGetSecretAESAlias.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemuDomainSecretStorageSourcePrepare: Change aliases for disk secrets
Peter Krempa [Mon, 9 Mar 2020 05:58:57 +0000 (06:58 +0100)]
qemuDomainSecretStorageSourcePrepare: Change aliases for disk secrets

Originally there was only the secret for authentication so we didn't use
any suffix to tell it apart. With the introduction of encryption we
added a 'luks' suffix for the encryption secrets. Since encryption is
really generic and authentication is not the only secret modify the
aliases for the secrets to better describe what they are used for.

This is possible as we store the disk secrets in the status XML thus
only new machines will use the new secrets.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemuDomainSecretAESSetupFromSecret: Use 'qemuAliasForSecret'
Peter Krempa [Mon, 9 Mar 2020 05:56:04 +0000 (06:56 +0100)]
qemuDomainSecretAESSetupFromSecret: Use 'qemuAliasForSecret'

Replace qemuDomainGetSecretAESAlias by the new function so that we can
reuse qemuDomainSecretAESSetupFromSecret also for setting up other kinds
of objects.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemu: Split out initialization of secrets for 'iscsi' hostdevs
Peter Krempa [Mon, 9 Mar 2020 06:31:53 +0000 (07:31 +0100)]
qemu: Split out initialization of secrets for 'iscsi' hostdevs

Currently we don't have infrastructure to remember the secret aliases
for hostdevs. Since an upcoming patch is going to change aliases for
the disks, initialize the iscsi hostdevs separately so that we can keep
the alias. At the same time let's use qemuAliasForSecret instead of
qemuDomainGetSecretAESAlias when unplugging the iscsi hostdev.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemuDomainDeviceDiskDefPostParseRestoreSecAlias: Hardcode restored aliases
Peter Krempa [Fri, 6 Mar 2020 14:28:18 +0000 (15:28 +0100)]
qemuDomainDeviceDiskDefPostParseRestoreSecAlias: Hardcode restored aliases

In order to be able to change the function generating the alias and thus
also the aliases itself, we must hardcode the old format for the case of
upgrading form libvirt which didn't record them in the status XML yet.

Note that this code path is tested by
'tests/qemustatusxml2xmldata/disk-secinfo-upgrade-in.xml'

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemuDomainSecretStorageSourcePrepare: Fix naming of alias variables
Peter Krempa [Fri, 6 Mar 2020 14:13:21 +0000 (15:13 +0100)]
qemuDomainSecretStorageSourcePrepare: Fix naming of alias variables

The naming of the variables was tied to what they are used for not what
the alias represents. Since we'll need to use some of the aliases for
another type of secrets fix the name so that it makes sense.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemu: Introduce another helper for creating alias for a 'secret' object
Peter Krempa [Fri, 6 Mar 2020 14:36:42 +0000 (15:36 +0100)]
qemu: Introduce another helper for creating alias for a 'secret' object

qemuAliasForSecret is meant as a replacement qemuDomainGetSecretAESAlias
with saner API. The sub-type we are creating the alias for is passed in
as a string rather than the unflexible 'isLuks' boolean.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agobuild: workaround behaviour regression in gnu make 4.3
Daniel P. Berrangé [Fri, 13 Mar 2020 18:39:25 +0000 (18:39 +0000)]
build: workaround behaviour regression in gnu make 4.3

We need the "$(space)" variable to contain a single whitespace
character. We do this by assigning and then appending an empty
string to the variable. Variable appends get separated by a
single whitespace historically, but GNU make 4.3 introduced a
behaviour regression.

  https://lists.gnu.org/archive/html/bug-make/2020-01/msg00057.html

[quote]
* WARNING: Backward-incompatibility!
  Previously appending using '+=' to an empty variable would
  result in a value starting with a space.  Now the initial
  space is only added if the variable already contains some
  value.  Similarly, appending an empty string does not
  add a trailing space.
[/quote]

This patch tries a new trick to get a single whitespace by
getting make to expand two non-existant variables separated
by a space.

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agoRemove qemuDomainSecretInfoNew
Peter Krempa [Mon, 16 Mar 2020 09:42:36 +0000 (10:42 +0100)]
Remove qemuDomainSecretInfoNew

Replace it by a direct call to qemuDomainSecretAESSetupFromSecret.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemuDomainSecretAESSetup: Split out lookup of secret data
Peter Krempa [Mon, 16 Mar 2020 09:37:26 +0000 (10:37 +0100)]
qemuDomainSecretAESSetup: Split out lookup of secret data

Split out the lookup of the secret from the secret driver into
qemuDomainSecretAESSetupFromSecret so that we can also instantiate
secret objects in qemu with data from other sources.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemuDomainSecretAESSetup: Allocate and return 'secinfo' here
Peter Krempa [Mon, 16 Mar 2020 09:23:24 +0000 (10:23 +0100)]
qemuDomainSecretAESSetup: Allocate and return 'secinfo' here

Rather than passing in an empty qemuDomainSecretInfoPtr allocate it
in this function and return it. This is done by absorbing the check from
qemuDomainSecretInfoNew and removing the internals of
qemuDomainSecretInfoNew.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemuDomainSecretAESSetup: Automatically free non-secret locals
Peter Krempa [Mon, 16 Mar 2020 09:13:38 +0000 (10:13 +0100)]
qemuDomainSecretAESSetup: Automatically free non-secret locals

Use g_autofree for the ciphertext and init vector as they are not
secret and thus don't have to be cleared and use g_new0 to allocate the
iv for parity.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>