Peter Krempa [Tue, 23 May 2023 14:52:24 +0000 (16:52 +0200)]
docs: go: Add 'go-import' metadata via rST
The '.. meta::' rST directive allows adding header metadata. Move the
specific metadata from page.xsl into the individual files and pass them
through into the header from page.xsl.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
qemu_passt: Format portForward device even without address
It's almost like we've anticipated this. Our XML parser and
formatter handles @address and @dev attributes of <portForward/>
element completely independent of each other. And as of commit
2023_03_29.b10b983~3 passt allows handling these two separately
too. All that's left is generate the cmd line according to this
new fact.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2210287 Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Michal Privoznik [Thu, 25 May 2023 13:43:56 +0000 (15:43 +0200)]
conf: Reject invalid device's <seclabel relabel='yes'/> with no <label/>
We allow (some) domain devices to have a different <seclabel/>
than the top level domain one (this is mostly to allow access to
a resource for multiple domains). Now, we do couple of sanity
checks for such <seclabel/>, e.g. when the <label/> is specified,
but '@relabel' is set to no. But what we are missing is the
opposite: when '@relabel' is set, but no <label/> was provided.
Our schema already denies such combination. Make our parser
behave the same.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2160356 Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Michal Privoznik [Wed, 31 May 2023 09:02:21 +0000 (11:02 +0200)]
include: Fix 'Since' for new VIR_MIGRATE_PARAM_COMPRESSION_* macros
In v9.3.0-98-g150ae3e62b two new macros were introduced:
VIR_MIGRATE_PARAM_COMPRESSION_ZLIB_LEVEL and
VIR_MIGRATE_PARAM_COMPRESSION_ZSTD_LEVEL. But both list 9.1.0 as
the version they were introduced in (this is because the patch
was sent in that release time frame). Change the version to the
current release.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Michal Privoznik [Tue, 28 Mar 2023 08:44:15 +0000 (10:44 +0200)]
qemu_command: Generate .memaddr for virtio-mem and virtio-pmem
This is fairly trivial. Just set .memaddr attribute if a value
was set in the XML.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2180679 Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Michal Privoznik [Tue, 28 Mar 2023 08:41:39 +0000 (10:41 +0200)]
qemu: Fill virtio-mem/virtio-pmem .memaddr at runtime
After a QEMU domain is started, among other thing we query memory
device information. And while memory address is returned by QEMU
for all models, we store it only for DIMMs and NVDIMMs. Do store
it for VIRTIO_MEM and VIRTIO_PMEM too.
This effectively reports the address the virtio-mem/virtio-pmem
is mapped to in live XML.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Michal Privoznik [Tue, 28 Mar 2023 08:39:55 +0000 (10:39 +0200)]
conf: Introduce <address/> for virtio-mem and virtio-pmem
Both virtio-mem and virtio-pmem devices have '.memaddr' attribute
which controls the address where they are mapped in the guest
memory. Ideally, users do not need to specify this as QEMU does
the right thing and computes addresses automatically on startup.
But soon, we will need to record this address as it is part of
guest ABI. And also, there might be some users that want to
control this value. Now, we are in a bit of a pickle, because
both these device types already have a PCI address, therefore we
can't just use <address/> blindly. But what we can do, is
introduce <address/> under the <target/> element. This is also
more conceptual, as knobs under <target/> control guest visible
config of memory device (and .memaddr surely falls into that
category).
NB, SgxEPCDeviceInfo struct in QMP definition also has .memaddr
attribute, but because of the way we build cmd line there's no
(easy) way to set the attribute. So ignore that for now.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Michal Privoznik [Fri, 26 May 2023 14:15:56 +0000 (16:15 +0200)]
conf: Run virDomainInputDefPostParse() only for VIR_DOMAIN_DEVICE_INPUT
Due to missed break; statement the virDomainInputDefPostParse()
is called not only for VIR_DOMAIN_DEVICE_INPUT but also
VIR_DOMAIN_DEVICE_LEASE and VIR_DOMAIN_DEVICE_NET, which can lead
to all sort of unpredictable results.
Fixes: c4bc4d3b82fbe22e03c986ca896090f481df5c10 Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Michal Privoznik [Wed, 24 May 2023 08:53:03 +0000 (10:53 +0200)]
docs: Clarify values reported by virDomainMemoryStats()
I was approached by a KubeVirt developer to clarify what value
does VIR_DOMAIN_MEMORY_STAT_DISK_CACHES report, whether it's from
the guest or the host POV. And since I didn't know the answer
even after reading the docs I think we can do better. Clarify the
POV then.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Michal Privoznik [Tue, 23 May 2023 16:46:58 +0000 (18:46 +0200)]
docs: newapi.xsl: Generate overall docs for enums too
There are/can be overall docs for enums (e.g.
virDomainModificationImpact) not just individual values. But
these never make it into the generated HTML which is a bit
unfortunate as they can contain valuable information for users.
Generate a block with overall enum documentation, just like we do
for functions.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Michal Privoznik [Wed, 24 May 2023 08:22:47 +0000 (10:22 +0200)]
docs: newapi.xsl: Generate docs on generic typedefs
We have plenty of generic typedefs (that basically just alias a
struct, or our popular virXXXPtr). Because we do not generate
HTML docs for it, the documentation is placed at random places,
e.g.: comment from virDomainPtr typedef ("a virDomainPtr is
pointer to a virDomain private structure ...") ends up after
virDomainProcessSignal enum block.
There are some less weird occurrences of this problem (e.g.
virBlkioParameterPtr), but yet - the typedef appears in TOC.
Therefore, generate a block for each typedef and put its
description there.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Michal Privoznik [Mon, 22 May 2023 13:55:10 +0000 (15:55 +0200)]
qemu_hotplug: Temporarily allow emulator thread to access other NUMA nodes during mem hotplug
Again, this fixes the same problem as one of previous commits,
but this time for memory hotplug. Long story short, if there's a
domain running and the emulator thread is restricted to a subset
of host NUMA nodes, but the memory that's about to be hotplugged
requires memory from a host NUMA node that's not in the set we
need to allow emulator thread to access the node, temporarily.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
What this means is the emulator thread is pinned onto host NUMA
node #0 (by setting corresponding cpuset.mems to "0"), and two
memory-backend-* objects are created:
Note, the emulator thread is pinned well before QEMU is even
exec()-ed.
Now, the way memory allocation works in QEMU is: the emulator
thread calls mmap() followed by mbind() (which is sane, that's
how everybody should do it). BUT, because the thread is already
restricted by CGroups to just NUMA node #0, calling:
mbind(host-nodes:[1]); /* made up syntax (TM) */
fails. This is expected though. Kernel was instructed to place
the memory at NUMA node "0" and yet, process is trying to place
it elsewhere.
We used to solve this by not restricting emulator thread at all
initially, and only after it's done initializing (i.e. we got the
QMP greeting) we placed it onto desired nodes. But this had its
own problems (e.g. QEMU might have locked pieces of its memory
which were then unable to migrate onto different NUMA nodes).
Therefore, in v5.1.0-rc1~282 we've changed this and set cgroups
upfront (even before exec()-ing QEMU). And this used to work, but
something has changed (I can't really put my finger on it).
Therefore, for the initialization start the thread with union of
all configured host NUMA nodes ("0-1" in our example) and fix the
placement only after QEMU is started.
NB, the memory hotplug suffers the same problem, but that will
be fixed in the next commit.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2138150 Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Michal Privoznik [Fri, 19 May 2023 12:02:01 +0000 (14:02 +0200)]
qemuProcessSetupPid: Use @numatune variable more
Inside of qemuProcessSetupPid() there's @numatune variable which
is set to vm->def->numa, but it lives only in one block. In the
rest of places the expanded form (vm->def->numa) is used instead.
Move the variable declaration at the beginning of the function
and use it instead of the expanded form.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
qemu: Use thread-context even with numatune's restrictive mode
We cannot use host-nodes attribute for it, but there is no reason for us
to skip the preallocation optimisation using thread-context in such
case. Thankfully returning the proper nodemask from
qemuBuildMemoryBackendProps is enough to trigger this.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Tim Wiederhake [Tue, 28 Mar 2023 14:14:55 +0000 (16:14 +0200)]
cpu-data.py: Filter out apic current logical processor
Commit 10b5e789c5 attempts to filter out the logical processor id
in the generated data to remove noise and irrelevant changes in the
output.
cpuid-leaf 0x0B may have more than two sub-leaves though. Filter out
logical processor id from all sub-leaves of 0x0B and 0x1F (superset
of the information in 0x0B).
Signed-off-by: Tim Wiederhake <twiederh@redhat.com> Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
Andrea Bolognani [Mon, 22 May 2023 07:50:20 +0000 (09:50 +0200)]
Revert "conf: Introduce MTE domain feature"
The QEMU interface is still in a state of flux, and KVM support
has been pulled shortly after having been merged. Let's not
commit to a stable interface in libvirt just yet.
Reverts: 720e8f13ff71377580cd37b118cee8a1f982d1d8 Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com> Reviewed-by: Cornelia Huck <cohuck@redhat.com>
The QEMU interface is still in a state of flux, and KVM support
has been pulled shortly after having been merged. Let's not
commit to a stable interface in libvirt just yet.
Reverts: 1347a19f75a23b4d92e6a7b549fcde52b23f0258 Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com> Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Andrea Bolognani [Mon, 22 May 2023 07:49:51 +0000 (09:49 +0200)]
Revert "qemu: Validate MTE feature"
The QEMU interface is still in a state of flux, and KVM support
has been pulled shortly after having been merged. Let's not
commit to a stable interface in libvirt just yet.
Reverts: c6c9b5d251de215ed378aa0bc31daa2e1170409e Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com> Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Andrea Bolognani [Mon, 22 May 2023 07:49:33 +0000 (09:49 +0200)]
Revert "qemu: Generate command line for MTE feature"
The QEMU interface is still in a state of flux, and KVM support
has been pulled shortly after having been merged. Let's not
commit to a stable interface in libvirt just yet.
Reverts: b10bc8f7ab6f9986ccc54ba04fc5b3bad7576be6 Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com> Reviewed-by: Cornelia Huck <cohuck@redhat.com>
It's not used as part of the build process or searched for at
build time, and the QEMU driver detects its path at runtime,
so one could think that the BuildRequires is unnecessary. But
we actually need it to be present at build time in order to
run the full test suite.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Michal Privoznik [Thu, 18 May 2023 08:31:12 +0000 (10:31 +0200)]
numa_conf: Deny other memory modes than 'restrictive' if a memnode is 'restrictive'
We already do check that if there's <memory mode='restrictive'/>
then all <memnode/> have to be of 'restrictive' mode too. But
what we are missing the reverse: if there is <memnode/> with
'restrictive' mode, then the <memory/> has to be of the same mode
too.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2208946 Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Michal Privoznik [Thu, 18 May 2023 08:27:52 +0000 (10:27 +0200)]
numa_conf: Move memnode mode validation into virDomainNumaDefValidate()
When parsing a <memnode/> we also check whether the @mode
argument fulfills some requirements wrt 'restrictive' mode. This
is not the right place though. There's virDomainNumaDefValidate()
which contains other checks.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
The virDomainNumatuneNodeSpecified() function does not write into
passed @numatune pointer, it just reads from it. Therefore, the
argument should be const, which allows this function to be called
from places where virDomainNuma is already const (e.g. domain
validation code).
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Laszlo Ersek [Thu, 18 May 2023 12:59:39 +0000 (14:59 +0200)]
docs: make isa-debugcon example more useful / directly applicable
The type='pty' attribute in the <serial> element causes a Pseudo TTY to be
allocated on the host side via "/dev/ptmx", which is meant to be
interacted with via "virsh console" or similar.
That's not how a firmware log is typically viewed or saved. Replace
type='pty' with type='file', and also provide an example <source> element
(with the pathname of the logfile), similarly to how the <serial> example
just above provides a <source> element too.
Cc: "Daniel P. Berrangé" <berrange@redhat.com> Cc: Andrea Bolognani <abologna@redhat.com>
Updates: 654968381df0256c047d2ecd4542ccc90dc57ad0 Signed-off-by: Laszlo Ersek <lersek@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Laszlo Ersek [Thu, 18 May 2023 12:59:38 +0000 (14:59 +0200)]
docs: fix typo in isa-debugcon example
The <serial> opening tag is paired with the </console> closing tag; that's
a mismatch. The question is then whether to modify the former to
<console>, or the latter to </serial>.
Per section "Relationship between serial ports and consoles", <serial> is
used for emulated (not paravirt) consoles, and it's the type that's
suitable for early debug output (such as from firmware). Thus, change
</console> to </serial>.
Cc: "Daniel P. Berrangé" <berrange@redhat.com> Cc: Andrea Bolognani <abologna@redhat.com> Fixes: 654968381df0256c047d2ecd4542ccc90dc57ad0 Signed-off-by: Laszlo Ersek <lersek@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Jiang Jiacheng [Fri, 24 Feb 2023 09:27:12 +0000 (17:27 +0800)]
qemu: support set parallel migration compression method
Add new compress methods zlib and zstd for parallel migration,
these method should be used with migration option --comp-methods
and will be processed in 'qemuMigrationParamsSetCompression'.
Note that only one compress method could be chosen for parallel
migration and they cann't be used in compress migration.
Jiang Jiacheng [Fri, 24 Feb 2023 09:27:11 +0000 (17:27 +0800)]
virsh: Add migrate options to set parallel compress level
Add migrate options: --compression-zlib-level
--compression-zstd-level
These options are used to set compress level for "zlib"
or "zstd" during parallel migration if the compress method
is specified.
Jiang Jiacheng [Fri, 24 Feb 2023 09:27:10 +0000 (17:27 +0800)]
Add public API for parallel compression method
Add description for VIR_MIGRATE_PARAM_COMPRESSION, it will
be reused in choosing compression method during parallel migration.
Add public API VIR_MIGRATE_PARAM_COMPRESSION_ZLIB_LEVEL,
VIR_MIGRATE_PARAM_COMPRESSION_ZSTD_LEVEL for migration APIs
to support set compress level during parallel migration.
Peter Krempa [Wed, 17 May 2023 12:17:56 +0000 (14:17 +0200)]
docs: xsl: Simplify templating XSL
Wrap the auto-generated pages (API ref and hvsupport.html) in the proper
top level element similarly to what the pages generated from RST have to
remove the extra case when templating our web.
(Best viewed with 'git show -w')
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Peter Krempa [Wed, 17 May 2023 14:48:09 +0000 (16:48 +0200)]
docs: newapi.xsl: Remove support for generating index page
Since we need to generate API docs for multiple input files the index
page is not useful for us and was replaced by a manual one. Drop the XSL
for generating it.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Peter Krempa [Wed, 17 May 2023 14:41:35 +0000 (16:41 +0200)]
docs: html: Add a manually written index page
The auto-generated index contains only references to one run of the
generator but we in total run it 4 times missing the admin, lxc, and
qemu specific apis.
Rewrite it manually so that we can drop the generator for it.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Peter Krempa [Wed, 17 May 2023 13:59:30 +0000 (15:59 +0200)]
hvsupport: Split out common APIs from hypervisor API section
Common APIs such as virConnectOpen/Close and similar which are used by
the non-hypervisor drivers in libvirt are grouped together with
hypervisor drivers, which makes the table very wide.
Split them out into a separate group and clean up the list of hypervisor
drivers.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Peter Krempa [Wed, 17 May 2023 10:55:20 +0000 (12:55 +0200)]
docs: Remove XSLT table of contents generator
The only remaining page was 'hvsupport.html' which is generated by
'scripts/hvsupport.py'. The script already has all the data to generate
the table of contents internally so we can remove the whole complicated
template.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Peter Krempa [Tue, 16 May 2023 13:50:47 +0000 (15:50 +0200)]
css: mobile: Fix responsive design of 'docs' and 'knowledgebase' pages
When the pages were converted to rST it required changes to how the
panels are created. This change was not reproduced in the specific media
override for narrow displays and thus made those pages unusable.
Note that two lines per document are needed as some rst2html5 versions
format a <div class='section'> and others do a <section> element
instead.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Peter Krempa [Tue, 16 May 2023 08:22:39 +0000 (10:22 +0200)]
conf: numa: Allow formatting 'none' values for 'associativity' and 'policy' of cache
The parser makes the values mandatory and also the qemu code implements
actions for those values. The formatter skips them though. Since
format+parse is used to copy the XML at startup a definition with those
values can't be started.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2203709 Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Mon, 15 May 2023 11:55:31 +0000 (13:55 +0200)]
conf: Introduce MTE domain feature
The Memory Tagging Extensions are hardware acceleration present
in some ARM processors that allow memory error detection [1].
Introduce a domain XML knob that turns them on or off.
1: https://www.arm.com/blogs/blueprint/memory-safety-arm-memory-tagging-extension Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
qemu_domin: Account for NVMe disks when calculating memlock limit on hotplug
During hotplug of a NVMe disk we need to adjust the memlock
limit. The computation of the limit is handled by
qemuDomainGetMemLockLimitBytes() which looks at given domain
definition and accounts for various device types (as different
types require different amounts). But during disk hotplug the
disk is not added to domain definition until the very last
moment. Therefore, qemuDomainGetMemLockLimitBytes() has this
@forceVFIO argument which tells it to assume VFIO even if there
are no signs of VFIO in domain definition. And this kind of
works, until the amount needed for NVMe disks changed (in
v9.3.0-rc1~52). What's missing in the commit is making @forceVFIO
behave the same as if there was an NVMe disk present in the
domain definition.
But, we can do even better - just mimic whatever we're doing for
hostdevs. IOW - introduce qemuDomainAdjustMaxMemLockNVMe() that
behaves the same as qemuDomainAdjustMaxMemLockHostdev().
There are subtle differences though:
1) qemuDomainAdjustMaxMemLockHostdev() can afford placing hostdev
right at the end of vm->def->hostdevs, because the array was
already reallocated (at the beginning of
qemuDomainAttachHostPCIDevice()). But
qemuDomainAdjustMaxMemLockNVMe() doesn't have that luxury.
2) qemuDomainAdjustMaxMemLockHostdev() places a
virDomainHostdevDef pointer into domain definition, while
qemuDomainStorageSourceAccessModifyNVMe() (which calls
qemuDomainAdjustMaxMemLock()) sees a virStorageSource pointer
but domain definition contains virDomainDiskDef. But that's
okay, we can create a dummy disk definition and append it into
the domain definition.
After this, qemuDomainAdjustMaxMemLock() can be called with
@forceVFIO = false, as the disk is now part of domain definition
(when computing the new limit).
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2014030#c28 Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Peter Krempa [Mon, 15 May 2023 12:20:16 +0000 (14:20 +0200)]
conf: backup: Fix logic for generating default backup filenames
If the 'disk->store' property is already allocated which happens e.g.
when the disk is described by the backup XML but the optional filename
is not filled in 'virDomainBackupDefAssignStore' would not fill in the
default location.
Fix the logic to do it also if a 'virStorageSource' categorizes as
empty.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
which results in no MOCK_* macros defined in
virmockstathelpers.c, i.e. no stat() mocking, nada. The reason
for this simplification are these musl commits [1][2] which
removed all 64 bit aliases. And that's not what our logic for
deciding what flavor of stat() to mock counted with.
Nevertheless, we do build with Alpine Linux in our CI, so how
come we don't see this problem there? Well, simply because Alpine
Linux maintainers decided to revert the commits [3][4]. But on
distributions that use vanilla musl, this problem can be seen
easily.
projects / libvirt.git / log