Michal Privoznik [Sat, 13 Apr 2019 20:38:55 +0000 (22:38 +0200)]
virhostcpu: Make virHostCPUGetMSR() work only on x86
Model specific registers are a thing only on x86. Also, the
/dev/cpu/0/msr path exists only on Linux and the fallback
mechanism (asking KVM) exists on Linux and FreeBSD only.
Therefore, move the function within #ifdef that checks all
aforementioned constraints and provide a dummy stub for all
other cases.
This fixes the build on my arm box, mingw-* builds, etc.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
PPC64 support for NVIDIA V100 GPU with NVLink2 passthrough
The NVIDIA V100 GPU has an onboard RAM that is mapped into the
host memory and accessible as normal RAM via an NVLink2 bridge. When
passed through in a guest, QEMU puts the NVIDIA RAM window in a
non-contiguous area, above the PCI MMIO area that starts at 32TiB.
This means that the NVIDIA RAM window starts at 64TiB and go all the
way to 128TiB.
This means that the guest might request a 64-bit window, for each PCI
Host Bridge, that goes all the way to 128TiB. However, the NVIDIA RAM
window isn't counted as regular RAM, thus this window is considered
only for the allocation of the Translation and Control Entry (TCE).
For more information about how NVLink2 support works in QEMU,
refer to the accepted implementation [1].
This memory layout differs from the existing VFIO case, requiring its
own formula. This patch changes the PPC64 code of
@qemuDomainGetMemLockLimitBytes to:
- detect if we have a NVLink2 bridge being passed through to the
guest. This is done by using the @ppc64VFIODeviceIsNV2Bridge function
added in the previous patch. The existence of the NVLink2 bridge in
the guest means that we are dealing with the NVLink2 memory layout;
- if an IBM NVLink2 bridge exists, passthroughLimit is calculated in a
different way to account for the extra memory the TCE table can alloc.
The 64TiB..128TiB window is more than enough to fit all possible
GPUs, thus the memLimit is the same regardless of passing through 1 or
multiple V100 GPUs.
Further reading explaining the background
[1] https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg03700.html
[2] https://www.redhat.com/archives/libvir-list/2019-March/msg00660.html
[3] https://www.redhat.com/archives/libvir-list/2019-April/msg00527.html
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com> Reviewed-by: Erik Skultety <eskultet@redhat.com>
qemu_domain: NVLink2 bridge detection function for PPC64
The NVLink2 support in QEMU implements the detection of NVLink2
capable devices by verifying the attributes of the VFIO mem region
QEMU allocates for the NVIDIA GPUs. To properly allocate an
adequate amount of memLock, Libvirt needs this information before
a QEMU instance is even created, thus querying QEMU is not
possible and opening a VFIO window is too much.
An alternative is presented in this patch. Making the following
assumptions:
- if we want GPU RAM to be available in the guest, an NVLink2 bridge
must be passed through;
- an unknown PCI device can be classified as a NVLink2 bridge
if its device tree node has 'ibm,gpu', 'ibm,nvlink',
'ibm,nvlink-speed' and 'memory-region'.
This patch introduces a helper called @ppc64VFIODeviceIsNV2Bridge
that checks the device tree node of a given PCI device and
check if it meets the criteria to be a NVLink2 bridge. This
new function will be used in a follow-up patch that, using the
first assumption, will set up the rlimits of the guest
accordingly.
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
This does not cause a problem in usual scenarios thanks to us allowing
CAP_DAC_OVERRIDE for the qemu process, however in some scenarios this might be
an issue because the directory is created with mkdtemp(3) which explicitly
creates that with 0700 permissions and qemu running as non-root cannot access
that.
The scenarios include:
- Builds without CAPNG
- Running libvirtd in certain container configurations [1]
- and possibly others.
Jiri Denemark [Fri, 22 Mar 2019 15:44:02 +0000 (16:44 +0100)]
vircpuhost: Add support for reading MSRs
The new virHostCPUGetMSR internal API will try to read the MSR from
/dev/cpu/0/msr and if it is not possible (the device does not exist or
libvirt is running unprivileged), it will fallback to asking KVM for the
MSR using KVM_GET_MSRS ioctl.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Jiri Denemark [Tue, 26 Mar 2019 09:18:10 +0000 (10:18 +0100)]
cputest: Add support for MSR features to cpu-gather.sh
This patch adds an inline python code for reading MSR features. Since
reading MSRs is a privileged operation, we have to read them from
/dev/cpu/*/msr if it is readable (i.e., the script runs as root) or
fallback to using KVM ioctl which can be done by any user that can start
virtual machines.
The python code is inlined rather than provided in a separate script
because whenever there's an issue with proper detection of CPU features,
we ask the reporter to run cpu-gather.sh script to give us all data we
need to know about the host CPU. Asking them to run several scripts
would likely result in one of them being ignored or forgotten.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
cputest: Fix comparison in checkCPUIDFeature in cpu-cpuid.py
leaf["eax"] & eax > 0 check works correctly only if there's at most 1
bit set in eax. Luckily that's been always the case, but fixing this
could save us from future surprises.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
cputest: Drop support for old QEMU from cpu-parse.sh
We don't really need to parse CPU data from QEMU older than 2.9 (i.e.,
before query-cpu-model-expansion) at this point. But even if there's a
need to do so, we can always use an older version of this script to do
the conversion.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Jiri Denemark [Fri, 15 Mar 2019 17:36:58 +0000 (18:36 +0100)]
cpu_x86: Introduce virCPUx86DataCmp
virCPUx86DataSorter already compares two virCPUx86DataItem structs.
Let's add a tiny wrapper around it called virCPUx86DataCmp and use it
instead of open coded comparisons.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
The following patches introduce CPU features read from MSR in addition
to those queried via CPUID instruction. Let's introduce a container
struct which will be able to describe either feature type.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Andrea Bolognani [Fri, 12 Apr 2019 12:41:10 +0000 (14:41 +0200)]
maint: Add filetype annotations to Makefile.inc.am
Vim has trouble figuring out the filetype automatically because
the name doesn't follow existing conventions; annotations like
the ones we already have in Makefile.ci help it out.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Unfortunately the data reported by pkg-config is not completely
accurate, so until the issue has been fixed in readline we need
to work around it in libvirt.
The good news is that we only need the fix to land in FreeBSD
ports and macOS homebrew before we can drop the kludge, so
we're talking months rather than years.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With the 7.0 release, readline has finally started shipping
pkg-config support in the form of a readline.pc file.
Unfortunately, most downstreams have yet to catch up with this
change: among Linux distributions in particular, Fedora Rawhide
seems to be the only one installing it at the moment.
Non-Linux operating systems have been faring much better in
this regard: both FreeBSD (through ports) and macOS (through
homebrew) include pkg-config support in their readline package.
This is great news for us, since those are the platforms where
pkg-config is more useful on account of them installing headers
and libraries outside of the respective default search paths.
Our implementation checks whether readline is registered as a
pkg-config package, and if so obtains CFLAGS and LIBS using the
tool; if not, we just keep using the existing logic.
This commit is best viewed with 'git show -w'.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The first implementation of this logic was introduced with
commit 2ec759fc58fe all the way back in 2007; looking at the
build logs from our CI environment, however, it's apparent
that none of the platforms we currently target are actually
using it, so we can assume whatever issue it was working
around has been fixed at some point in the last 12 years.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The current code is a bit awkward, and we're going to need
to share it later anyway. We can drop the call to AC_SUBST()
while we're at it, since LIBVIRT_CHECK_LIB() already marks
READLINE_CFLAGS for substitution.
The new code goes to some extra length to avoid setting
-D_FUNCTION_DEF twice: this is mostly for cosmetic reasons,
and it's necessary because LIBVIRT_CHECK_READLINE() is called
twice: once on its own, and then once more as part of
LIBVIRT_CHECK_BASH_COMPLETION().
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The check was added in 74416b1d4849 without offering any
explanation outside of the commit message. Introduce a comment
to make digging through the git history unnecessary.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
lib: Don't use virReportSystemError() if virCommandRun() fails
Firstly, virCommandRun() does report an error on failure (which
in most cases is more accurate than what we overwrite it with).
Secondly, usually errno is not set (or gets overwritten in the
cleanup code) which makes virReportSystemError() report useless
error messages. Drop all virReportSystemError() calls in cases
like this (I've found three occurrences).
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Michal Privoznik [Fri, 12 Apr 2019 08:16:06 +0000 (10:16 +0200)]
qemu_capabilities: Introduce QEMU_CAPS_OVERCOMMIT
Added in QEMU commit of v3.0.0-rc0~48^2~9 (then fixed by
v3.1.0-rc0~119^2~37) QEMU is replacing '-realtime mlock' with
'-overcommit mem-lock'. Add a capability to tell if we're dealing
new new enough qemu to use the replacement.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Michal Privoznik [Fri, 12 Apr 2019 08:03:36 +0000 (10:03 +0200)]
qemu: Always assume QEMU_CAPS_REALTIME_MLOCK
The '-realtime mlock' cmd line argument was introduced in QEMU
commit v1.5.0-rc0~190 which matches minimal QEMU version we
require. Therefore, the capability will always be present.
Apparently, nearly none of our xml2argv test cases had the
capability hence slightly bigger change under qemuxml2argvdata/.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Pavel Hrdina [Thu, 11 Apr 2019 15:13:16 +0000 (17:13 +0200)]
virresctrl: fix MBA memory leak
The 'bandwidths' variable is allocated using VIR_RESIZE_N so it has to
be freed as well.
==118315== 8 bytes in 1 blocks are definitely lost in loss record 299 of 2,401
==118315== at 0x4C29DAD: malloc (vg_replace_malloc.c:308)
==118315== by 0x4C2C100: realloc (vg_replace_malloc.c:836)
==118315== by 0x52C3FAF: virReallocN (viralloc.c:245)
==118315== by 0x52C4079: virExpandN (viralloc.c:294)
==118315== by 0x532BBA8: virResctrlAllocParseProcessMemoryBandwidth (virresctrl.c:1156)
==118315== by 0x532BBA8: virResctrlAllocParseMemoryBandwidthLine (virresctrl.c:1211)
==118315== by 0x532BBA8: virResctrlAllocParse (virresctrl.c:1414)
==118315== by 0x532BBA8: virResctrlAllocGetGroup (virresctrl.c:1446)
==118315== by 0x532C11D: virResctrlAllocGetDefault (virresctrl.c:1464)
==118315== by 0x532D15E: virResctrlAllocAssign (virresctrl.c:1923)
==118315== by 0x532D15E: virResctrlAllocCreate (virresctrl.c:2042)
==118315== by 0x31E1ABEE: qemuProcessResctrlCreate (qemu_process.c:2596)
==118315== by 0x31E1ABEE: qemuProcessLaunch (qemu_process.c:6444)
==118315== by 0x31E1E341: qemuProcessStart (qemu_process.c:6721)
==118315== by 0x31E81315: qemuDomainObjStart.constprop.50 (qemu_driver.c:7288)
==118315== by 0x31E81A65: qemuDomainCreateWithFlags (qemu_driver.c:7341)
==118315== by 0x54DDB4B: virDomainCreate (libvirt-domain.c:6534)
Andrea Bolognani [Thu, 11 Apr 2019 15:12:02 +0000 (17:12 +0200)]
m4: sasl: Use pkg-config
Since commit 4e75b0a00fb4 we support SASL 2.1.26 and newer
releases only, all of which ship a .pc file. Using pkg-config
allows FreeBSD builds to pick up the dependency automatically.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Andrea Bolognani [Thu, 11 Apr 2019 15:58:24 +0000 (17:58 +0200)]
src: Include SASL_CFLAGS where appropriate
A bunch of files include src/rpc/virnetsaslcontext.h, which
in turn includes <sasl/sasl.h>, and without the corresponding
CFLAGS the compiler can't locate the latter if it happens to
be installed outside of the default include path as is the
case, for example, on FreeBSD.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Laine Stump [Thu, 11 Apr 2019 16:53:54 +0000 (12:53 -0400)]
build: set --without-firewalld-zone in configure commandline for Fedora 30
The firewalld package in Fedora 30 didn't get support for rich rule
priorities, which is required by the libvirt zonefile that's installed
when the build is configured with --with-firewalld-zone, so we need to
set --without-firewalld-zone for that version of Fedora. The needed
feature is already upstream in firewalld, so it just needs another
upstream release to be there. Let's be optimistic and assume that will
happen prior to F31.
Resolves: https://bugzilla.redhat.com/1699051 Signed-off-by: Laine Stump <laine@laine.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Acked-by: Eric Garver <eric@garver.life>
travis: put macOS script inline in the macOS matrix entry
Now that we don't have separate scripts defined for native and mingw
builds, there is no point having one for macOS. It can just be inlined
at the one place it is needed.
Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
travis: convert Ubuntu, CentOS & MinGW builds to use new make rules
Change the Travis CI configuration to invoke the new ci-build@$IMAGE
target instead of directly running Docker. This guarantees that when a
developer runs ci-build@$IMAGE locally, the container build setup is
identical to that used in Travis CI, with exception of the host kernel
and Docker version.
Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
tests: add targets for building libvirt inside Docker containers
The Travis CI system uses Docker containers for its build environment.
These are pre-built and hosted under quay.io/libvirt so that developers
can use them for reproducing problems locally.
Getting the right Docker command syntax to use them, however, is not
entirely easy. This patch addresses that usability issue by introducing
some make targets. To run a simple build (aka 'make all') using the
Fedora 28 container:
make ci-build@fedora-28
To also run unit tests
make ci-check@fedora-28
This is just syntax sugar for calling the previous command with a
custom make target
make ci-build@fedora-28 CI_MAKE_ARGS="check"
To do a purely interactive build it is possible to request a shell
make ci-shell@fedora-28
To do a MinGW build, it is currently possible to use the fedora-rawhide
image and request a different configure script
make ci-build@fedora-rawhide CI_CONFIGURE=mingw32-configure
It is also possible to do cross compiled builds via the Debian containers
make ci-build@debian-9-cross-s390x
In all cases the GIT source tree is cloned locally into a 'ci-tree/src'
sub-directory which is then exposed to the container at '/src'. It is
setup to use a separate build directory so the build takes place in a
subdir '/src/build'. A source tree build can be requested instead
by passing an empty string CI_VPATH= arg to make.
The make rules are kept in a standalone file that is included into the
main Makefile.am, so that it is possible to run them without having to
invoke autotools first.
It is neccessary to disable the gnulib submodule commit check because
this fails due to the way we have manually cloned submodule repos as
primary git repos with their own .git directory, instead of letting
git treat them as submodules in the top level .git directory.
make[1]: Entering directory '/src/build'
fatal: Not a valid object name origin
fatal: run_command returned non-zero status for .gnulib
.
maint.mk: found non-public submodule commit
make: *** [/src/maint.mk:1448: public-submodule-commit] Error 1
Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Cole Robinson [Sat, 16 Mar 2019 18:20:32 +0000 (14:20 -0400)]
Always put _LAST enums on second line of VIR_ENUM_IMPL
Standardize on putting the _LAST enum value on the second line
of VIR_ENUM_IMPL invocations. Later patches that add string labels
to VIR_ENUM_IMPL will push most of these to the second line anyways,
so this saves some noise.
remote: enforce ACL write permission for getting guest time & hostname
Getting the guest time and hostname both require use of guest agent
commands. These must not be allowed for read-only users, so the
permissions check must validate "write" permission not "read".
Fixes CVE-2019-3886 Reviewed-by: Jim Fehlig <jfehlig@suse.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
api: disallow virDomainGetHostname for read-only connections
The virDomainGetHostname API is fetching guest information and this may
involve use of an untrusted guest agent. As such its use must be
forbidden on a read-only connection to libvirt.
Fixes CVE-2019-3886 Reviewed-by: Jim Fehlig <jfehlig@suse.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Allow targetting the search scope to the website, wiki or mailing lists
only. When javascript is disabled this should gracefully fallback to
only searching the website.
Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
build-aux: ensure all scripts are included in EXTRA_DIST
Few of the scripts in build-aux are included in EXTRA_DIST. This is not
a serious problem since they are primarily tools intended for developers
upstream, and downstream builds won't need them. Having them missing,
however, complicates downstream patching because it means patches that
are auto-exported from git will fail to apply if they include a change
to a file in build-aux/. By bundling all these scripts in the dist we
make patching more straightforward.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Cole Robinson [Sun, 31 Mar 2019 15:32:18 +0000 (11:32 -0400)]
tests: qemuxml2argv: Rename testInfo* to testQemuInfo*
In preparation for moving these bits to a shared place, rename them
to match one of the testutilsqemu.c function prefixes. Rename
info->flags handling too as it will need to be moved
testInfoSetPaths isn't renamed because it will stay local
Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Cole Robinson <crobinso@redhat.com>
Cole Robinson [Sun, 31 Mar 2019 15:08:19 +0000 (11:08 -0400)]
tests: qemuxml2argv: add testInfoSetPaths
This moves infile and outfile building outside the test case,
which better fits the pattern of qemuxml2xmltest. It also lets us
drop the qemuxml2argtest-specific 'suffix' from testInfo
Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Cole Robinson <crobinso@redhat.com>
Cole Robinson [Sun, 31 Mar 2019 14:59:30 +0000 (10:59 -0400)]
tests: qemuxml2argv: Add info->{in,out}file
Track infile and outfile in testInfo. This is step towards moving path
creation out of the test case, which will eventually help sharing more
code with qemuxml2xmltest.c
Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Cole Robinson <crobinso@redhat.com>
Peter Krempa [Fri, 5 Apr 2019 15:29:40 +0000 (17:29 +0200)]
internal: Introduce VIR_RETURN_PTR
With the introduction of more and more internal data types which support
VIR_AUTOPTR it's becoming common to see the following pattern:
VIR_AUTOPTR(virSomething) some = NULL
virSomethingPtr ret = NULL;
... (ret is not touched ) ...
VIR_STEAL_PTR(ret, some);
return ret;
This patch introduces a macro named VIR_RETURN_PTR which returns the
pointer directly without the need for an explicitly defined return
variable and use of VIR_STEAL_PTR. Internally obviously a temporary
pointer is created to allow setting the original pointer to NULL so that
the VIR_AUTOPTR function does not free the memory which we want to
actually return.
The name of the temporary variable is deliberately long and complex to
minimize the possibility of collision.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Andrea Bolognani [Wed, 10 Apr 2019 13:08:46 +0000 (15:08 +0200)]
tests: Fix MinGW build for domaincapstest
Commit 5b9819eedc71 started using the virFileWrapper APIs in
the test program, and correctly called them only in the section
of code guarded by WITH_QEMU; however, a single call to the
virFileWrapperClearPrefixes() function ended up in the
hypervisor-agnostic section, causing a build failure on MinGW.
Move the call to the QEMU-only section; while at it, also drop
the virFileWrapperRemovePrefix() calls, which are entirely
redundant since we'd drop all prefixes immediately afterwards
anyway.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Acked-by: Michal Privoznik <mprivozn@redhat.com>
domain capabilities: Expose firmware auto selection feature
If a management application wants to use firmware auto selection
feature it can't currently know if the libvirtd it's talking to
support is or not. Moreover, it doesn't know which values that
are accepted for the @firmware attribute of <os/> when parsing
will allow successful start of the domain later, i.e. if the mgmt
application wants to use 'bios' whether there exists a FW
descriptor in the system that describes bios.
This commit then adds 'firmware' enum to <os/> element in
<domainCapabilities/> XML like this:
We can see both 'bios' and 'efi' listed which means that there
are descriptors for both found in the system (matched with the
machine type and architecture reported in the domain capabilities
earlier and not shown here).
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Acked-by: Laszlo Ersek <lersek@redhat.com>
projects / libvirt.git / log