Laine Stump [Thu, 2 Jul 2020 22:03:45 +0000 (18:03 -0400)]
bhyve: use g_auto() for all virBuffers
In most cases this eliminates one or more calls to
virBufferClearAndReset(), but even when it doesn't it's better because:
1) it makes the code more consistent, making it more likely that new
contributors who are "learning by example" will to the right thing.
2) it protects against future modifications that might have otherwise
needed to add a virBufferClearAndReset()
3) Currently some functions don't call virBufferClearAndReset() only
because they're relying on some subordinate function to call it for
them (e.g. bhyveConnectGetSysinfo() in this patch relies on
virSysinfoFormat() to clear out the buffer when there is an
error). I think this is sloppy behavior, and that the toplevel
function that defines and initializes the buffer should be the
function clearing it at the end.
Signed-off-by: Laine Stump <laine@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
conf, qemu: consider available CPUs in vcpupin/emulatorpin output
The output of vcpupin and emulatorpin for a domain with vcpu
placement='static' is based on a default bitmap that contains
all possible CPUs in the host, regardless of the CPUs being offline
or not. E.g. for a Linux host with this CPU setup (from lscpu):
This is benign by its own, but can make the user believe that all
CPUs from the 0-191 range are eligible for pinning. Which can lead
to situations like this:
$ sudo ./run tools/virsh vcpupin vcpupin_test 0 1
error: Invalid value '1' for 'cpuset.cpus': Invalid argument
This is exarcebated by the fact that 'virsh vcpuinfo' considers only
available host CPUs in the 'CPU Affinity' field:
$ sudo ./run tools/virsh vcpuinfo vcpupin_test
(...)
CPU Affinity: y-------y-------y-------(...)
This patch changes the default bitmap of vcpupin and emulatorpin, in
the case of domains with static vcpu placement, to all available CPUs
instead of all possible CPUs. Aside from making it consistent with
the behavior of 'vcpuinfo', users will now have one less incentive to
try to pin a vcpu in an offline CPU.
The idea is to have a function that calls virHostCPUGetOnlineBitmap()
but, instead of returning NULL if the host does not have CPU
offlining capabilities, fall back to a bitmap containing all
present CPUs.
Next patch will use this helper in two other places.
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
This function reads the string in sysfspath/cpu/present and
parses it manually to retrieve the number of present CPUs.
virHostCPUGetPresentBitmap() reads and parses the same file,
using a more robust parser via virBitmapParseUnlimited(),
but returns a bitmap. Let's drop all the manual parsing done
here and simply return the size of the resulting bitmap
from virHostCPUGetPresentBitmap().
Given that no more parsing is being done manually in the function,
rename it to virHostCPUCountLinux().
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Use g_auto* pointers to avoid the need for the cleanup label. The
type of the pointer 'virDomainPtr dom' was changed to its alias
'virshDomainPtr' to allow the use of g_autoptr().
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Since the ESX virtual hardware version 4.0, virtual machines support up
to 10 virtual NICs instead of 4 previously. This changes the limit
accordingly based on the provided `virtualHW.version`.
Signed-off-by: Bastien Orivel <bastien.orivel@diateam.net> Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
A few commits ago, in aeecbc87b73, I've implemented command line
generation for ACPI HMAT. For this, we need to know if at least
one guest NUMA node has vCPUs. This is tracked in
@masterInitiator variable, which is initialized to -1, then we
iterate through guest NUMA nodes and break the loop if we find a
node with a vCPU. After the loop, if masterInitiator is still
negative then no NUMA node has a vCPU and we error out. But this
exact check was missing comparison for negativeness.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Thu, 28 May 2020 15:23:00 +0000 (17:23 +0200)]
conf: Validate NUMA HMAT configuration
There are several restrictions, for instance @initiator and
@target have to refer to existing NUMA nodes (daa), @cache has to
refer to a defined cache level and so on.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Michal Privoznik [Wed, 27 May 2020 09:46:33 +0000 (11:46 +0200)]
conf: Parse and format HMAT
To cite ACPI specification:
Heterogeneous Memory Attribute Table describes the memory
attributes, such as memory side cache attributes and bandwidth
and latency details, related to the System Physical Address
(SPA) Memory Ranges. The software is expected to use this
information as hint for optimization.
According to our upstream discussion [1] this is exposed under
<numa/> as <cache/> under NUMA <cell/> and <latency> or
<bandwidth/> under numa/latencies.
Michal Privoznik [Wed, 27 May 2020 11:42:22 +0000 (13:42 +0200)]
Allow NUMA nodes without vCPUs
QEMU allows creating NUMA nodes that have memory only.
These are somehow important for HMAT.
With check done in qemuValidateDomainDef() for QEMU 2.7 or newer
(checked via QEMU_CAPS_NUMA), we can be sure that the vCPUs are
fully assigned to NUMA nodes in domain XML.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Michal Privoznik [Wed, 27 May 2020 09:49:51 +0000 (11:49 +0200)]
numa_conf: Make virDomainNumaSetNodeCpumask() return void
There is only one caller of virDomainNumaSetNodeCpumask() which
checks for the return value but because the function will return
NULL iff the @cpumask was NULL in the first place. But in that
place @cpumask can't be NULL because it was just allocated by
virBitmapParse().
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Michal Privoznik [Fri, 29 May 2020 12:57:20 +0000 (14:57 +0200)]
qemu_command: Rename qemuBuildNumaArgStr()
The function doesn't just build the argument for -numa. Since the
-numa can be repeated multiple times, it also puts -numa onto the
cmd line. Also, the rest of the functions has 'Command' infix.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Michal Privoznik [Thu, 28 May 2020 14:52:22 +0000 (16:52 +0200)]
numa_conf: Drop CPU from name of two functions
There are two functions virDomainNumaDefCPUFormatXML() and
virDomainNumaDefCPUParseXML() which format and parse domain's
<numa/>. There is nothing CPU specific about them. Drop the
infix.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Michal Privoznik [Thu, 28 May 2020 10:38:43 +0000 (12:38 +0200)]
conf: Move and rename virDomainParseScaledValue()
There is nothing domain specific about the function, thus it
should not have virDomain prefix. Also, the fact that it is a
static function makes it impossible to use from other files.
Move the function to virxml.c and drop the 'Domain' infix.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Michal Privoznik [Wed, 27 May 2020 12:24:56 +0000 (14:24 +0200)]
qemuxml2xmltest: Add "numatune-distance" test case
This test case checks that expanding NUMA distance works. On
input we accept if only distance from A to B is specified. On the
output we format the B to A distance too.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Peter Krempa [Wed, 8 Jul 2020 07:13:42 +0000 (09:13 +0200)]
qemuBuildMemoryBackendProps: Use boolean type for 'pmem' property
Commit 82576d8f35e used a string "on" to enable the 'pmem' property.
This is okay for the command line visitor, but the property is declared
as boolean in qemu and thus it will not work when using QMP.
Modify the type to boolean. This changes the command line, but
fortunately the command line visitor in qemu parses both 'yes' and 'on'
as true for the property.
Erik Skultety [Fri, 3 Jul 2020 12:26:13 +0000 (14:26 +0200)]
qemu: Use virQEMUCapsCacheLookupDefault instead of lookup by arch
Firstly, SEV is present only on AMD, so we can safely assume x86.
Secondly, the problem with looking up capabilities in the cache by arch
is that it's using virHashSearch with a callback to find the right
capabilities and get the binary name from it as well, but since the
cache is empty, it will return NULL and we won't get the corresponding
binary name out of the lookup either. Then, during the cache validation
we try to create a new cache entry for the emulator, but since we don't
have the binary name, nothing gets created.
Therefore, virQEMUCapsCacheLookupDefault is used to fix this issue,
because it doesn't rely on the capabilities cache to construct the
emulator binary name.
The semantics of the backup operation don't strictly require that all
disks being backed up are part of the same incremental part (when a disk
was checkpointed/backed up separately or in a different VM), or even
they may not have a previous checkpoint at all (e.g. when the disk
was freshly hotplugged to the vm).
In such cases we can still create a common checkpoint for all of them
and backup differences according to configuration.
This patch adds a per-disk configuration of the checkpoint to do the
incremental backup from via the 'incremental' attribute and allows
perform full backups via the 'backupmode' attribute.
Note that no changes to the qemu driver are necessary to take advantage
of this as we already obey the per-disk 'incremental' field.
Peter Krempa [Tue, 7 Jul 2020 14:38:00 +0000 (16:38 +0200)]
backupxml2xmltest: Call 'virDomainBackupAlignDisks' before formatting output
Call the post-processing function so that we can validate that it does
the correct thing.
virDomainBackupAlignDisks requires disk definitions to be present so
let's fake them by copying disks from the backup definition and add one
extra disk 'vdextradisk'.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Fixes: 193ad364062407c3fcd3267f0f135d8960b53020 Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Peter Krempa [Wed, 1 Jul 2020 10:25:42 +0000 (12:25 +0200)]
conf: backup: Add 'tls' attribute for 'server' element
Allow enabling TLS for the NBD server used to do pull-mode backups. Note
that documentation already mentions 'tls', so this just implements the
schema and XML bits.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Peter Krempa [Wed, 1 Jul 2020 14:58:29 +0000 (16:58 +0200)]
conf: checkpoint: Add a flag storing whether disk 'size' is valid
Avoid printing '0' size in case when we weren't able to determine the
backup size by adding a flag whether the size is valid and interlock
printing of the field according to the flag.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Peter Krempa [Thu, 25 Jun 2020 10:49:45 +0000 (12:49 +0200)]
conf: backup: Don't explicitly forbid backup of read-only disk
Users may want to use this to create a full backup or even incremental
if the checkpoints are pre-existing. We still will not allow to create a
checkpoint on a read-only disk as that makes no sense.
Peter Krempa [Tue, 30 Jun 2020 15:56:08 +0000 (17:56 +0200)]
virQEMUDriverConfigLoadSpecificTLSEntry: Split up fetching of server-only config options
The '*_tls_x509_verify' options are relevant only when we are going to
expose a server socket as client sockets always enable verification.
Split up the macro to separate the common bits from the server bits so
that when we'll later extend support of 'nbd' and 'vxhs' disks which are
client only we can reuse the existing macros.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Laine Stump [Fri, 19 Jun 2020 02:33:28 +0000 (22:33 -0400)]
util: remove OOM error log from virGetHostnameImpl()
The strings allocated in virGetHostnameImpl() are all allocated via
g_strdup(), which will exit on OOM anyway, so the call to
virReportOOMError() is redundant, and removing it allows slight
modification to the code, in particular the cleanup label can be
eliminated.
Signed-off-by: Laine Stump <laine@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Laine Stump [Fri, 19 Jun 2020 21:40:17 +0000 (17:40 -0400)]
network: fix memory leak in networkBuildDhcpDaemonCommandLine()
hostsfilestr was not being freed. This will be turned into g_autofree
in an upcoming patch converting a lot more of the same file to using
g_auto*, but I wanted to make a separate patch for this first so the
other patch is simpler to review (and to make backporting easier).
Laine Stump [Thu, 18 Jun 2020 23:16:33 +0000 (19:16 -0400)]
use g_autoptr for all xmlBuffers
AUTOPTR_CLEANUP_FUNC is set to xmlBufferFree() in util/virxml.h (This
is actually new - added accidentally (but fortunately harmlessly!) in
commit 257aba2dafe. I had added it along with the hunks in this patch,
then decided to remove it and submit separately, but missed taking out
the hunk in virxml.h)
Signed-off-by: Laine Stump <laine@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Laine Stump [Thu, 18 Jun 2020 16:49:09 +0000 (12:49 -0400)]
conf, vmx: check for OOM after calling xmlBufferCreate()
Although libvirt itself uses g_malloc0() and friends, which exit when
there isn't enouogh memory, libxml2 uses standard malloc(), which just
returns NULL on OOM - this means we must check for NULL on return from
any libxml2 functions that allocate memory.
xmlBufferCreate(), for example, might return NULL, and we don't always
check for it. This patch adds checks where it isn't already done.
(NB: Although libxml2 has a provision for changing behavior on OOM (by
calling xmlMemSetup() to change what functions are used to
allocating/freeing memory), we can't use that, since parts of libvirt
code end up in libvirt.so, which is linked and called directly by
applications that may themselves use libxml2 (and may have already set
their own alternate malloc()), e.g. drivers like esx which live totally
in the library rather than a separate process.)
Signed-off-by: Laine Stump <laine@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Andrea Bolognani [Mon, 29 Jun 2020 17:00:36 +0000 (19:00 +0200)]
cirrus: Generate jobs dynamically
Instead of having static job definitions for FreeBSD and macOS,
use a generic template for both and fill in the details that are
actually different, such as the list of packages to install, in
the GitLab CI job, right before calling cirrus-run.
The target-specific information are provided by lcitool, so that
keeping them up to date is just a matter of running the refresh
script when necessary.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com>
The root cause flaw is that non-root libvirtd is using /etc/libvirt for
its hooks. Traditionally that has been harmless though since we checked
whether we could access the hook file and degraded gracefully. We need
the same access check for iterating over the hook directory.
Long term we should make it possible to have an unprivileged hook dir
under $HOME.
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
With the recent update of Fedora rawhide I've noticed
virnettlssessiontest and virnettlscontexttest failing with:
Our own certificate servercertreq-ctx.pem failed validation
against cacertreq-ctx.pem: The certificate uses an insecure
algorithm
This is result of Fedora changes to support strong crypto [1]. RSA
with 1024 bit key is viewed as legacy and thus insecure. Generate
a new private key then. Moreover, switch to EC which is not only
shorter but also not deprecated that often as RSA. Generated
using the following command:
Andrea Bolognani [Fri, 26 Jun 2020 18:11:58 +0000 (20:11 +0200)]
ci: Run all jobs, for all branches, all the time
After recent changes (increasing the parallelism of the pipeline
by reducing the number of stages, introducing FreeBSD builds that
take longer than any other job), the difference between running
the full pipeline or a reduced one has basically disappeared: in
both cases, the completion time is around 25-35 minutes depending
on whether containers need to be rebuilt and how many shared
runners are available.
Reduce the complexity of our .gitlab-ci.yml and make things
simpler for contributors by simply always running all jobs.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Michal Privoznik [Sat, 27 Jun 2020 07:09:39 +0000 (09:09 +0200)]
domain_conf: Remove zPCI validation from formatter
In 076591009ad a validation code was added to
virDomainDeviceInfoFormat() which reports an error if zPCI
address entered in was incomplete. But, there are two problems
with this approach.
The first problem is the placement of the code - it doesn't
belong into XML formatter rather than XML validator.
The second one is that at the point of formatting XML the post
parse callback has run and thus filled in required info.
Therefore this check can never do something useful and instead of
moving it into validator, it's removed completely.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Michal Privoznik [Sat, 27 Jun 2020 08:49:37 +0000 (10:49 +0200)]
qemu_validate: Fix how qemuValidateDomainDeviceDefZPCIAddress() is called
To make the code future proof, the rest of the
qemuValidateDomainDeviceDefAddress() has to be executed (even
though there is nothing there yet) instead of returning directly.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>
If initializing test monitor in testQemuHotplugCpuPrepare()
fails, the control jumps to error label where
testQemuHotplugCpuDataFree() is called. But since the data->mon
is NULL due to aforementioned failure,
qemuMonitorTestGetMonitor() dereferences a NULL pointer leading
to a SIGSEGV.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>
domain_conf.c: skip checking ZPCI address is incomplete if not present
Commit 076591009ad1 ("conf: fix zPCI address auto-generation on
s390") is doing a check for virZPCIDeviceAddressIsIncomplete()
prior to checking if the device has a ZPCI address at all. This
results in errors like these when starting libvirt:
error : virDomainDeviceInfoFormat:7527 : internal error:
Missing uid or fid attribute of zPCI address
Fix it by moving virZPCIDeviceAddressIsIncomplete() after the
check done by virZPCIDeviceAddressIsPresent().
Fixes: 076591009ad11ec108521b52a4945d0f895fa160 Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>
1. Test for auto-generating uids while specifying valid fids
2. Test for auto-generating fids while specifying valid uids
3. Test for parse error while specifying a valid fid and an invalid
uid
4. Test for parse error while specifying two ZPCI devices with same
uid and fid addresses
5. Test for parse error when both uid and fid are set to zero
6. Test for error while specifying uid and not providing ZPCI
capability.
Signed-off-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Signed-off-by: Bjoern Walk <bwalk@linux.ibm.com> Signed-off-by: Shalini Chellathurai Saroja <shalini@linux.ibm.com> Reviewed-by: Bjoern Walk <bwalk@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>
we expect uid='0x0001' (or the next available uid for the domain).
However, we get a parsing error:
$ virsh define zpci.xml
error: XML error: Invalid PCI address uid='0x0000', must be > 0x0000
and <= 0xffff
Secondly, when the uid is specified explicitly with the invalid
numerical value '0x0000', we actually expect the parsing error above.
However, the domain is being defined and the uid value is silently
changed to a valid value.
The first issue is a bug and the second one is undesired behaviour, and
both issues are related to how we (in-band) signal invalid values for
uid and fid. So let's fix the XML parsing to do validation based on what
is actually specified in the XML.
The first issue is also related to the current code behaviour, which
is, if either uid or fid is specified by the user, it is incorrectly
assumed that both uid and fid are specified. This bug is fixed by
identifying when the user specified ZPCI address is incomplete and
auto-generating the missing ZPCI address.
Signed-off-by: Bjoern Walk <bwalk@linux.ibm.com> Signed-off-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Signed-off-by: Shalini Chellathurai Saroja <shalini@linux.ibm.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>
projects / libvirt.git / log