]> xenbits.xensource.com Git - libvirt.git/log
libvirt.git
5 years agodocs: convert kbase/locking-sanlock.html.in to RST
Daniel P. Berrangé [Wed, 20 Nov 2019 14:05:50 +0000 (14:05 +0000)]
docs: convert kbase/locking-sanlock.html.in to RST

This is a semi-automated conversion. The first conversion is done using
"pandoc -f html -t rst". The result is then editted manually to apply
the desired heading markup, and fix a few things that pandoc gets wrong.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agodocs: convert kbase/locking-lockd.html.in to RST
Daniel P. Berrangé [Wed, 20 Nov 2019 14:05:50 +0000 (14:05 +0000)]
docs: convert kbase/locking-lockd.html.in to RST

This is a semi-automated conversion. The first conversion is done using
"pandoc -f html -t rst". The result is then editted manually to apply
the desired heading markup, and fix a few things that pandoc gets wrong.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agodocs: convert kbase/locking.html.in to RST
Daniel P. Berrangé [Wed, 20 Nov 2019 14:05:50 +0000 (14:05 +0000)]
docs: convert kbase/locking.html.in to RST

This is a semi-automated conversion. The first conversion is done using
"pandoc -f html -t rst". The result is then editted manually to apply
the desired heading markup, and fix a few things that pandoc gets wrong.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agodocs: convert kbase/secureusage.html.in to RST
Daniel P. Berrangé [Wed, 20 Nov 2019 14:05:50 +0000 (14:05 +0000)]
docs: convert kbase/secureusage.html.in to RST

This is a semi-automated conversion. The first conversion is done using
"pandoc -f html -t rst". The result is then editted manually to apply
the desired heading markup, and fix a few things that pandoc gets wrong.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agodocs: convert kbase/launch_security_sev.html.in to RST
Daniel P. Berrangé [Wed, 20 Nov 2019 14:05:50 +0000 (14:05 +0000)]
docs: convert kbase/launch_security_sev.html.in to RST

This is a semi-automated conversion. The first conversion is done using
"pandoc -f html -t rst". The result is then editted manually to apply
the desired heading markup, and fix a few things that pandoc gets wrong.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agodocs: convert kbase/domainstatecapture.html.in to RST
Daniel P. Berrangé [Wed, 20 Nov 2019 14:05:50 +0000 (14:05 +0000)]
docs: convert kbase/domainstatecapture.html.in to RST

This is a semi-automated conversion. The first conversion is done using
"pandoc -f html -t rst". The result is then editted manually to apply
the desired heading markup, and fix a few things that pandoc gets wrong.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agodocs: add a minimal style guide for writing RST docs
Daniel P. Berrangé [Mon, 18 Nov 2019 18:37:09 +0000 (18:37 +0000)]
docs: add a minimal style guide for writing RST docs

Most importantly we document the required heading markup so that we get
consistency across the docs. Also mention that docs should have a table
of contents if they have headings & are likely longer than one page of
text.

The 3-space indent rule may sound wierd, but that's what python has
recommended and thus what tools like pandoc emit. Rather than try to
reindent things to 4-space, just accept this RST norm.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agodocs: add styling for <tt> element
Daniel P. Berrangé [Wed, 20 Nov 2019 17:44:38 +0000 (17:44 +0000)]
docs: add styling for <tt> element

Although <tt> is deprecated in HTML5, the rst2html command will still
emit it, in preference to <code> tags, so we must style it too.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agodocs: relax CSS context match for pretty tables
Daniel P. Berrangé [Mon, 11 Nov 2019 16:50:01 +0000 (16:50 +0000)]
docs: relax CSS context match for pretty tables

We currently only render pretty tables if they have the "top_table"
class set. All of our tables set this, except for the ACL & migration
doc tables, which should have set it, and the API reference which does
not want it.

Simplify life by rendering all tables in a pretty style and remove the
need for the "top_table" class entirely. A small rule turns off the
pretty style for the API reference where tables are a hack used to
render enums with horizontal alignment.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agodocs: generate permalinks correctly for rst2html output
Daniel P. Berrangé [Mon, 11 Nov 2019 15:53:02 +0000 (15:53 +0000)]
docs: generate permalinks correctly for rst2html output

The rst2html output generates the links for headings in a slightly
different way than we do for docs written in HTML, so we must match
another scenario when generating back links.

rst2html will also use <h1> tags for both the document title and
the first level of section titles, so we must expand the matching
to allow for this too.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agodocs: adapt filling of <head> section for rst2html output
Daniel P. Berrangé [Fri, 18 Oct 2019 14:58:56 +0000 (15:58 +0100)]
docs: adapt filling of <head> section for rst2html output

The HTML from rst2html doesn't have <h1> immediately under the <body>
tag, instead there is at least one <div> in between.

There are also many things added in the <head> section that we don't
want to have copied over, since our templating system already adds
suitable <head> elements.

We only need to copy the <script> to make index.html work.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agodocs: introduce rst2html as a mandatory tool for building docs
Daniel P. Berrangé [Fri, 18 Oct 2019 13:18:36 +0000 (14:18 +0100)]
docs: introduce rst2html as a mandatory tool for building docs

The rst2html tool is provided by python docutils, and as the name
suggests, it converts RST documents into HTML.

Basic rules are added for integrating RST docs into the website
build process.

This enables us to start writing docs on our website in RST format
instead of HTML, without changing the rest of our website templating
system away from XSLT yet.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agodocs: move docs about remote driver URIs into URI docs
Daniel P. Berrangé [Tue, 8 Oct 2019 16:43:38 +0000 (17:43 +0100)]
docs: move docs about remote driver URIs into URI docs

The docs about remote URIs in uri.html are somewhat sparse with the full
docs being in remote.html. Move all the URI content from remote.html
into uri.html so the user only needs to look in one place for URI info.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agodocs: split TLS certificate setup into its own file
Daniel P. Berrangé [Tue, 8 Oct 2019 16:08:24 +0000 (17:08 +0100)]
docs: split TLS certificate setup into its own file

The generation and deployment of x509 certificates for TLS is complex
and verbose and thus deserves its own standalone page.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agotests: Make check-file-access.pl accept files through argv
Michal Privoznik [Wed, 4 Dec 2019 13:58:04 +0000 (14:58 +0100)]
tests: Make check-file-access.pl accept files through argv

The script needs two files to open:

  $(builddir)/test_file_access.txt, and
  $(srcdir)/file_access_whitelist.txt.

However, the script is opening the files from the $CWD which
won't work for a VPATH build. Make the script accept paths to the
files through @ARGV and tune the Makefile.am to pass them.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agotests: rewrite test argv line wrapper in Python
Daniel P. Berrangé [Fri, 30 Aug 2019 12:22:54 +0000 (13:22 +0100)]
tests: rewrite test argv line wrapper in Python

As part of a goal to eliminate Perl from libvirt build tools,
rewrite the test-wrap-argv.pl tool in Python.

This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.

Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agosrc: rewrite remote protocol checker in Python
Daniel P. Berrangé [Fri, 30 Aug 2019 12:22:54 +0000 (13:22 +0100)]
src: rewrite remote protocol checker in Python

As part of a goal to eliminate Perl from libvirt build tools,
rewrite the pdwtags processing script in Python.

The original inline shell and perl code was completely
unintelligible. The new python code is a manual conversion
that attempts todo basically the same thing.

Tested-by: Cole Robinson <crobinso@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agobuild: only support python3 binary
Daniel P. Berrangé [Tue, 3 Dec 2019 16:29:12 +0000 (16:29 +0000)]
build: only support python3 binary

python2 will be end of life by the time of the next
libvirt release. All our supported build targets, including
CentOS7, have a python3 build available.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agorng: Extend the range of driveUnit
Han Han [Wed, 4 Dec 2019 09:35:39 +0000 (17:35 +0800)]
rng: Extend the range of driveUnit

Since the max unit of virtio scsi disk is 16383, update the range of
driveUnit to it.

Signed-off-by: Han Han <hhan@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agorng: Separate virtio serial port and ccid slot from driveUnit
Han Han [Wed, 4 Dec 2019 09:35:38 +0000 (17:35 +0800)]
rng: Separate virtio serial port and ccid slot from driveUnit

Signed-off-by: Han Han <hhan@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agoconf: fix logic error for scsi units
Han Han [Wed, 4 Dec 2019 09:35:37 +0000 (17:35 +0800)]
conf: fix logic error for scsi units

Introduced in c8007fdc5d2, it should use 'greater than max' instead of
'equal or greater than max' for the condition of checking invalid scsi
unit.

Signed-off-by: Han Han <hhan@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agotests: stop static linking to libvirt code in tests
Daniel P. Berrangé [Wed, 27 Nov 2019 17:11:20 +0000 (17:11 +0000)]
tests: stop static linking to libvirt code in tests

If we static link to libvirt_util.la then we can't override functions in
this file by simply implementing them in the test code. Any tests should
dynamic link to the main libvirt.la and ensure symbols are exported.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agoscripts: fix list indexing when printing mis-ordered symbols
Daniel P. Berrangé [Tue, 3 Dec 2019 17:14:22 +0000 (17:14 +0000)]
scripts: fix list indexing when printing mis-ordered symbols

The python array slice syntax expects the first and last indexes,
not the first length and element count.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agodocs: move hyperv_wmi_generator.py to the scripts/ directory
Daniel P. Berrangé [Tue, 8 Oct 2019 10:19:30 +0000 (11:19 +0100)]
docs: move hyperv_wmi_generator.py to the scripts/ directory

Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agodocs: move esx_vi_generator.py to the scripts/ directory
Daniel P. Berrangé [Tue, 8 Oct 2019 10:19:30 +0000 (11:19 +0100)]
docs: move esx_vi_generator.py to the scripts/ directory

Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agodocs: move reformat-news.py to the scripts/ directory
Daniel P. Berrangé [Tue, 8 Oct 2019 10:19:30 +0000 (11:19 +0100)]
docs: move reformat-news.py to the scripts/ directory

Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agodocs: move apibuild.py to the scripts/ directory
Daniel P. Berrangé [Tue, 8 Oct 2019 10:19:30 +0000 (11:19 +0100)]
docs: move apibuild.py to the scripts/ directory

Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agotests: stop linking virt tests to secondary drivers
Daniel P. Berrangé [Tue, 3 Dec 2019 17:29:23 +0000 (17:29 +0000)]
tests: stop linking virt tests to secondary drivers

The hard dependancy between the virt drivers and the network
or storage drivers was removed quite a while back now, so
the tests no longer need to link to these drivers.

Reviewed-by: Laine Stump <laine@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agospec: mingw: add deps for genprotocol.pl usage
Cole Robinson [Tue, 3 Dec 2019 15:50:05 +0000 (10:50 -0500)]
spec: mingw: add deps for genprotocol.pl usage

As of 0985a9597bb03 we no longer distribute generated files. mingw
needs to call genprotocol.pl itself, which means it needs rpcgen
and cpp installed.

Signed-off-by: Cole Robinson <crobinso@redhat.com>
5 years agoqemu: process: Re-process qemu capability lockout in qemuProcessPrepareQEMUCaps
Peter Krempa [Mon, 25 Nov 2019 12:01:13 +0000 (13:01 +0100)]
qemu: process: Re-process qemu capability lockout in qemuProcessPrepareQEMUCaps

We clear some capabilities here so the lockouts need to be
re-evaluated.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoqemu: capabilities: Lock out incremental backup capability without blockdev
Peter Krempa [Mon, 25 Nov 2019 11:59:51 +0000 (12:59 +0100)]
qemu: capabilities: Lock out incremental backup capability without blockdev

Blockdev is required to do incremental backups properly. Add a helper
function for locking out capabilities and export it to allow re-doing
the processing if a different code path modifies capabilities.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoqemu: caps: Don't check capability before clearing it
Peter Krempa [Sun, 24 Nov 2019 08:55:42 +0000 (09:55 +0100)]
qemu: caps: Don't check capability before clearing it

Checking whether a qemu capability set right before clearing it without
any other logic doesn't make sense.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoqemu: process: Move handling of qemu capability overrides
Peter Krempa [Sun, 24 Nov 2019 08:48:30 +0000 (09:48 +0100)]
qemu: process: Move handling of qemu capability overrides

Do all post-processing of capabilities in qemuProcessPrepareQEMUCaps.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoqemu: process: Move clearing of QEMU_CAPS_CHARDEV_FD_PASS to qemuProcessPrepareQEMUCaps
Peter Krempa [Sun, 24 Nov 2019 08:41:49 +0000 (09:41 +0100)]
qemu: process: Move clearing of QEMU_CAPS_CHARDEV_FD_PASS to qemuProcessPrepareQEMUCaps

Move the post-processing of the QEMU_CAPS_CHARDEV_FD_PASS flag to the
new function.

The clearing of the capability is based on the presence of
VIR_QEMU_PROCESS_START_STANDALONE so we must also pass in the process
start flags.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoqemu: process: Move clearing of the BLOCKDEV capability to qemuProcessPrepareQEMUCaps
Peter Krempa [Sun, 24 Nov 2019 08:23:54 +0000 (09:23 +0100)]
qemu: process: Move clearing of the BLOCKDEV capability to qemuProcessPrepareQEMUCaps

Start aggregating all capability post-processing code in one place.

The comment was modified while moving it as it was mentioning floppies
which are no longer clearing the blockdev capability.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemu: Move and rename qemuDomainUpdateQEMUCaps
Peter Krempa [Sun, 24 Nov 2019 08:16:11 +0000 (09:16 +0100)]
qemu: Move and rename qemuDomainUpdateQEMUCaps

The function is now used only in qemu_process.c so move it there and
name it 'qemuProcessPrepareQEMUCaps' which is more appropriate to what
it's doing.

The reworded comment now mentions that it will also post-process the
caps for VM startup.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemu: process: Don't try to redetect missing qemuCaps on reconnect
Peter Krempa [Sun, 24 Nov 2019 07:35:59 +0000 (08:35 +0100)]
qemu: process: Don't try to redetect missing qemuCaps on reconnect

The redetection was originally added in 43c01d3838 as a way to recover
from libvirtd upgrade from the time when we didn't persist the qemu
capabilities in the status XML. Also this the oldest supported qemu by
more than two years.

Even if somebody would have a running VM running at least qemu 1.5 with
such an old libvirt we certainly wouldn't do the right thing by
redetecting the capabilities and then trying to communicate with qemu.

For now it will be the best to just stop considering this scenario any
more and error out for such VM.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemu: process: Make it obvious that virDomainDefPostParse is called with NULL opaque
Peter Krempa [Sat, 23 Nov 2019 08:43:35 +0000 (09:43 +0100)]
qemu: process: Make it obvious that virDomainDefPostParse is called with NULL opaque

Commit c90fb5a828a added explicit use of the private copy of the qemu
capabilities to various places. The change to qemuProcessInit was bogus
though as at the point where we re-initiate the post parse callbacks
priv->qemuCaps is still NULL as we clear it after shutdown of the VM and
don't initiate it until a later point.

Using the value from priv->qemuCaps might mislead readers of the code
into thinking that something useful is being passed at that point so go
with an explicit NULL instead.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemu: Always reset @info in qemuDomainGetJobInfo
Peter Krempa [Fri, 22 Nov 2019 16:39:43 +0000 (17:39 +0100)]
qemu: Always reset @info in qemuDomainGetJobInfo

qemuDomainGetJobInfo didn't always reset the return data in @info.
Thankfully this wouldn't be a problem as the RPC layer does it but we
should do it anyways.

Since we reset the struct we don't have to set the type to
VIR_DOMAIN_JOB_NONE as the value is 0.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoAPI: Introduce VIR_DOMAIN_JOB_SUCCESS field for virDomainGetJobStats
Peter Krempa [Thu, 21 Nov 2019 17:01:06 +0000 (18:01 +0100)]
API: Introduce VIR_DOMAIN_JOB_SUCCESS field for virDomainGetJobStats

The statistics fields are used in two places:

1) virDomainGetJobStats where the job type which ultimately holds
whether the job was successful or not is returned via a different
argument.

2) The virConnectDomainEventJobCompleted event where we report just the
statistics via typed parameters.

Since it might be useful to report the event also for jobs which
completed unsuccessfully and we don't have the means to transport the
state via a different variable with the event let's add a new field
which will hold the success state.

Since this is meant primarily for completed jobs a plain boolean is
sufficient to convey whether the job was successful or not.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agovirsh: domjobinfo: Add switch to print raw fields
Peter Krempa [Mon, 25 Nov 2019 14:06:17 +0000 (15:06 +0100)]
virsh: domjobinfo: Add switch to print raw fields

Introduce --rawstats which prints all statistics fields from the new API
similarly to how the virsh event handler prints them.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agovirsh: domjobinfo: Allow printing stats also for failed and other jobs
Peter Krempa [Thu, 21 Nov 2019 16:44:00 +0000 (17:44 +0100)]
virsh: domjobinfo: Allow printing stats also for failed and other jobs

Introduce the --anystats flag which does not skip the printing of the
stats if the job was unsuccessful.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agovirsh: domjobinfo: Print also job operation for failed jobs
Peter Krempa [Thu, 21 Nov 2019 16:36:18 +0000 (17:36 +0100)]
virsh: domjobinfo: Print also job operation for failed jobs

Printing that a job failed is rather unhelpful. Print at least the
operation which failed.

Achieve this by moving the check whether to print stats later but
replace it with a check which will skip printing of the operation if
there's no job.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoqemu: Implement VIR_DOMAIN_JOB_STATS_KEEP_COMPLETED
Peter Krempa [Mon, 25 Nov 2019 13:19:10 +0000 (14:19 +0100)]
qemu: Implement VIR_DOMAIN_JOB_STATS_KEEP_COMPLETED

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agovirsh: Implement VIR_DOMAIN_JOB_STATS_KEEP_COMPLETED for 'domjobinfo'
Peter Krempa [Fri, 22 Nov 2019 13:26:56 +0000 (14:26 +0100)]
virsh: Implement VIR_DOMAIN_JOB_STATS_KEEP_COMPLETED for 'domjobinfo'

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agoapi: Allow keeping completed domain job stats when reading them
Peter Krempa [Fri, 22 Nov 2019 12:23:31 +0000 (13:23 +0100)]
api: Allow keeping completed domain job stats when reading them

virDomainGetJobStats destroys the completed statistics on the first
read. Give the user possibility to keep them around if they wish so.

Add a flag VIR_DOMAIN_JOB_STATS_KEEP_COMPLETED which will read the stats
without destroying them.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agovirsh: domain: Extract the code converting domain job stats to virDomainJobInfo
Peter Krempa [Fri, 22 Nov 2019 14:06:08 +0000 (15:06 +0100)]
virsh: domain: Extract the code converting domain job stats to virDomainJobInfo

To simplify the stats printer code we convert the new statistics from
the typed parameter list into the old stats structure.

Extract this code since it takes a lot of space.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
5 years agotests: fix name of 32-bit x86 QEMU binary
Daniel P. Berrangé [Fri, 29 Nov 2019 16:33:37 +0000 (16:33 +0000)]
tests: fix name of 32-bit x86 QEMU binary

The 32-bit x86 binary is called qemu-system-i386, not
qemu-system-i686. This mistake across many test XML files was
not noticed because the mistake was also made in testutilsqemu.c
when mocking the capabilities.

Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agoqemu: make 'xz' image compression viable by using -3
Daniel P. Berrangé [Thu, 28 Nov 2019 12:07:04 +0000 (12:07 +0000)]
qemu: make 'xz' image compression viable by using -3

For managed save we can choose between various compression
methods. I randomly tested the 'xz' program on a 8 GB guest
and was surprised to have to wait > 50 minutes for it to
finish compressing, with 'xz' burning 100% cpu for the
entire time. Despite the impressive compression, this is
completely useless in the real world as it is far too long
to wait to save the VM.

The 'xz' binary defaults to '-6' optimization level which
aims for high compression, with moderate memory usage,
at the expense of speed.

This change switches it to use the '-3' optimization level
which is documented as being the one that optimizes speed
at expense of compression. Even with this, it will still
outperform all the other options in terms of compression
level. It is a little less than x4 faster than '-6' which
means it starts to be a viable choice to use 'xz' for
people who really want best compression.

The test results on a 1 GB, fairly freshly booted VM are
as follows

  format | save  | restore  size
  =======+=======+=============
  raw    |   05s |    1s  | 428 MB
  lzop   |   05s |    3s  | 160 MB
  gzip   |   29s |    5s  | 118 MB
  bz2    |   54s |   22s  | 114 MB
  xz     | 4m37s |   13s  |  86 MB
  xz -3  | 1m20s |   12s  |  95 MB

Based on this we can say

 * For moderate compression with no noticable loss in speed

       => use lzop

 * For high compression with moderate loss in speed

       => use gzip

 * For best compression with significant loss in speed

       => use xz

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agomaint: post-post-version-bump version bump to 6.0.0
Ján Tomko [Tue, 3 Dec 2019 11:59:01 +0000 (12:59 +0100)]
maint: post-post-version-bump version bump to 6.0.0

My commit 3bbe10205b79c8803e69c61897a0491d7a3f57c0 forgot
to update the news.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
5 years agospec: Fix printing test-suite.log when make check fails
Jiri Denemark [Tue, 3 Dec 2019 10:49:50 +0000 (11:49 +0100)]
spec: Fix printing test-suite.log when make check fails

Commit v5.7.0-248-g03449e2504 removed "cd tests" without updating the
patch to test-suite.log.

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
5 years agomaint: post-version bump version bump to 6.0.0
Ján Tomko [Tue, 3 Dec 2019 11:42:28 +0000 (12:42 +0100)]
maint: post-version bump version bump to 6.0.0

Signed-off-by: Ján Tomko <jtomko@redhat.com>
5 years agostorage: backend_disk: Implement the deletePool function
Erik Skultety [Thu, 28 Nov 2019 14:21:18 +0000 (15:21 +0100)]
storage: backend_disk: Implement the deletePool function

This is a very simple and straightforward implementation of the opposite
what buildPool does for the disk backend.
The background for this change comes from an existing test case in TCK
which does use the delete method for a pool of type disk, but it
truly could not have ever worked since the implementation simply
wasn't there for the pool of type disk.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agoselinux: Drop needless getfilecon()-s
Michal Privoznik [Thu, 28 Nov 2019 08:37:22 +0000 (09:37 +0100)]
selinux: Drop needless getfilecon()-s

When SELinux support was first introduced the libselinux library
wasn't that advanced and setfilecon_raw() or fsetfilecon_raw()
could fail even when the target context was set. Looking at the
current code [1][2] this is no longer the case. We can drop our
workarounds.

1: https://github.com/SELinuxProject/selinux/blob/master/libselinux/src/setfilecon.c#L10
2: https://github.com/SELinuxProject/selinux/blob/master/libselinux/src/fsetfilecon.c#L10

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agomaint: Post-release version bump to 5.11.0
Jim Fehlig [Tue, 3 Dec 2019 04:20:37 +0000 (21:20 -0700)]
maint: Post-release version bump to 5.11.0

Signed-off-by: Jim Fehlig <jfehlig@suse.com>
5 years agonews: Fix XML validation
Jim Fehlig [Tue, 3 Dec 2019 03:48:24 +0000 (20:48 -0700)]
news: Fix XML validation

The news schema requires two digits for both month and day in the date
attribute. s/2/02/ in the day value of date to fix the following
'make check' failure

2165) Checking ../docs/news.xml against ../news.rng                     ...
libvirt: XML Util error : XML document failed to validate against schema:
Unable to validate doc against /home/jfehlig/virt/upstream/libvirt/build/../docs/schemas/../news.rng
Element release failed to validate attributes

Signed-off-by: Jim Fehlig <jfehlig@suse.com>
5 years agoRelease of libvirt-5.10.0
Daniel Veillard [Mon, 2 Dec 2019 19:15:08 +0000 (20:15 +0100)]
Release of libvirt-5.10.0

* docs/news.xml: updated for the release

Signed-off-by: Daniel Veillard <veillard@redhat.com>
5 years agoqemu: blockcopy: Fix conditions when virStorageSource should be initialized
Peter Krempa [Fri, 29 Nov 2019 13:33:05 +0000 (14:33 +0100)]
qemu: blockcopy: Fix conditions when virStorageSource should be initialized

Commit 4b58fdf280a which enabled block copy also for network
destinations needed to limit when the 'mirror' storage source is
initialized in cases when we e.g. don't have an appropriate backend.

Limiting it just to virStorageFileSupportsCreate is too restrictive as
for example we can't precreate block devices and thus wouldn't
initialize the 'mirror' but since it's a local source we'd try to
examine it. This would fail since it wouldn't be initialized.

Fix it by introducing a more granular check whether certain operations
are supported and fix the check interlocks.

https://bugzilla.redhat.com/show_bug.cgi?id=1778058

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemu: blockcopy: Report error on image format detection failure
Peter Krempa [Fri, 29 Nov 2019 12:25:47 +0000 (13:25 +0100)]
qemu: blockcopy: Report error on image format detection failure

We tolerate image format detection during block copy in very specific
circumstances, but the code didn't error out on failure of the format
detection.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agosrc: fix pkg-config path for API XML files
Daniel P. Berrangé [Mon, 2 Dec 2019 13:18:01 +0000 (13:18 +0000)]
src: fix pkg-config path for API XML files

The API XML files are generated files, so live in the build dir not the
source dir.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agospec: Disable gluster on i686 on RHEL-8
Jiri Denemark [Fri, 29 Nov 2019 13:58:01 +0000 (14:58 +0100)]
spec: Disable gluster on i686 on RHEL-8

Gluster 6.0 is not built on i686 for RHEL-8, which prevents libvirt from
building. Let's just disable gluster there as all we need are client
libraries anyway.

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com>
5 years agoremote_daemon_stream: Hold an extra reference to stream in daemonStreamFilter
Michal Privoznik [Mon, 25 Nov 2019 14:13:17 +0000 (15:13 +0100)]
remote_daemon_stream: Hold an extra reference to stream in daemonStreamFilter

In v5.9.0-273-g8ecab214de I've tried to fix a lock ordering
problem, but introduced a crasher. Problem is that because the
client lock is unlocked (in order to honour lock ordering) the
stream we are currently checking in daemonStreamFilter() might be
freed and thus stream->priv might not even exist when the control
get to virMutexLock() call.

To resolve this, grab an extra reference to the stream and handle
its cleanup should the refcounter reach zero after the deref.
If that's the case and we are the only ones holding a reference
to the stream, we MUST return a positive value to make
virNetServerClientDispatchRead() break its loop where it iterates
over filters. The problem is, if we did not do so, then
"filter = filter->next" line will read from a memory that was
just freed (freeing a stream also unregisters its filter).

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemu: Fix indexes in statistics of iothreads
Peter Krempa [Fri, 29 Nov 2019 08:11:15 +0000 (09:11 +0100)]
qemu: Fix indexes in statistics of iothreads

In commit 2ccb5335dc4 I've refactored how we fill the typed parameters
for domain statistics. The commit introduced a regression in the
formating of stats for IOthreads by using the array index to label the
entries as it's common for all other types of statistics rather than
the iothread IDs used for iothreads.

Since only the design of iothread deviates from the common approach used
in all other statistic types this was not caught.

https://bugzilla.redhat.com/show_bug.cgi?id=1778014

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agolib: Fix documentation for the count field of VIR_DOMAIN_STATS_IOTHREAD
Peter Krempa [Fri, 29 Nov 2019 08:40:47 +0000 (09:40 +0100)]
lib: Fix documentation for the count field of VIR_DOMAIN_STATS_IOTHREAD

The original implementation used QEMU_ADD_COUNT_PARAM which added the
'count' suffix, but 'cnt' was documented. Fix the documentation to
conform with the original implementation.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agonss: Don't fail on empty files
Michal Privoznik [Thu, 28 Nov 2019 13:25:07 +0000 (14:25 +0100)]
nss: Don't fail on empty files

Before we rewrote nss plugin so that it doesn't use libvirt's
internal functions it used virLeaseReadCustomLeaseFile() to parse
.status files. After the rewrite it's using read() + yajl_parse()
+ yajl_complete_parse(). There's one catch though,
virLeaseReadCustomLeaseFile() skipped over empty files.

An empty .status file is created when a network is started. This
is because we configure dnsmasq to use our leasehelper. So the
first thing it does it calls it as follows:

  DNSMASQ_INTERFACE=virbr0 /usr/libexec/libvirt_leaseshelper init

which causes the leasehelper to create empty virbr0.status file.
If there is only one libvirt network then that is no problem -
there are no other .status files to parse anyway. But if there
are two or more networks then the first empty .status file causes
whole parsing process and subsequently the whole name lookup
process to fail.

Fixes: v5.7.0-rc1~343
Reported-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agodocs: don't run xmllint when generating API HTML files
Daniel P. Berrangé [Wed, 27 Nov 2019 14:03:45 +0000 (14:03 +0000)]
docs: don't run xmllint when generating API HTML files

After generating the API HTML files we run xmllint in docs/html/*.html
to validate the correctness. Since

  commit 0aa8536f14b08dfb5188d0bb3aba532680500d38
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   Wed Nov 20 14:49:26 2019 +0000

    docs: generate API reference pages for admin, qemu & lxc libraries

we have many rules generating files into docs/html/. The xmllint
calls for each rule are picking up files which are part-generated by
other parallel build rules resulting in transient errors like:

  GEN      html/index.html
  GEN      html/index-admin.html
  GEN      html/index-qemu.html
  GEN      html/index-lxc.html
  GEN      hvsupport.html.in
html/index-lxc.html:1: parser error : Document is empty

^
make[4]: *** [Makefile:2407: html/index-qemu.html] Error 1

The easiest solution is to move the xmllint rules to the 'make check'
phase of the build.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agotests: improve build time for qemuxml2argvtest
Daniel P. Berrangé [Wed, 27 Nov 2019 16:20:02 +0000 (16:20 +0000)]
tests: improve build time for qemuxml2argvtest

On Fedora 31 with GCC 9.2.1, compiling qemuxml2argvtest takes
about 36 seconds since

  commit 30c6d99209cd36bba8e14795622f658827698522
  Author: Jiri Denemark <jdenemar@redhat.com>
  Date:   Thu Oct 24 17:51:42 2019 +0200

    qemuxml2argvtest: Update host arch for DO_TEST*ARCH* tests

The optimizer is hitting some pathological performance behaviour due to
the high number of branches in the mymain() method.

Pushing the branch tests down into the testCompareXMLToArgv method
brings the compile time down to 3 seconds.

This likely related to this GCC bug:

  https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58479

Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agoqemu: Fix migration without parameters
Jiri Denemark [Wed, 27 Nov 2019 16:21:58 +0000 (17:21 +0100)]
qemu: Fix migration without parameters

The virTypedParamsFilter function doesn't mind params == NULL if nparams
is zero. And there's no need to check for params == NULL && nparams > 0
because this is checked higher in the stack.

In fact all the virCheckNonNull* checks in virTypedParamsFilter are
useless.

https://bugzilla.redhat.com/show_bug.cgi?id=1777094

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
5 years agotravis: explicitly require xcode 10.3 image for macOS
Daniel P. Berrangé [Wed, 27 Nov 2019 13:46:54 +0000 (13:46 +0000)]
travis: explicitly require xcode 10.3 image for macOS

The default macOS image in travis is broken, throwing python
exceptions when trying to install glib. Explicitly ask for the
newer 10.3 image which works correctly.

We now need to also point to the homebrew installed libxml2
rather than the OS distro provided one, since the OS distro
one has a pkg-config file present, but no actual header
files.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agonews: Update for 5.10.0 release
Michal Privoznik [Wed, 27 Nov 2019 10:19:47 +0000 (11:19 +0100)]
news: Update for 5.10.0 release

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemu: blockjob: Finish handling job with broken data
Peter Krempa [Tue, 26 Nov 2019 14:13:57 +0000 (15:13 +0100)]
qemu: blockjob: Finish handling job with broken data

Now that we have a separate job type which will not trigger normal code
paths for terminating job we can remove the ad-hoc handling.

This possibly fixes the issue of a broken job inheriting the disk and
then finishing in which case we'd not detach the backing chain.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoqemu: blockjob: Introduce "broken" block job type
Peter Krempa [Tue, 26 Nov 2019 13:55:05 +0000 (14:55 +0100)]
qemu: blockjob: Introduce "broken" block job type

To better track jobs we couldn't parse let's introduce a new job type
which will clarify semantics internally in few places.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoqemu: blockjob: Separate clearing of per-job data
Peter Krempa [Tue, 26 Nov 2019 12:39:18 +0000 (13:39 +0100)]
qemu: blockjob: Separate clearing of per-job data

We will need to clear per-job type data when we will be marking a
blockjob as broken in the new way. Extract the code for future reuse.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoqemu: blockjob: Don't stop processing the finished job early
Peter Krempa [Tue, 26 Nov 2019 13:29:42 +0000 (14:29 +0100)]
qemu: blockjob: Don't stop processing the finished job early

Both failure to refresh and to dismiss the job are very unlikely but if
they happen there's not much we can do about the blockjob.

The concluded job handlers treat it as if the job failed if we don't
update the state to 'QEMU_BLOCKJOB_STATE_COMPLETED' which is probably
the safest thing to do here.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoqemu: blockjob: Mark job with broken data but tracked by qemu as reconnected
Peter Krempa [Tue, 26 Nov 2019 13:16:37 +0000 (14:16 +0100)]
qemu: blockjob: Mark job with broken data but tracked by qemu as reconnected

Otherwise it would get dropped later on as untracked despite us knowing
about it. Additionally since we cancelled it we must wait to dismiss it
which would not be possible if we unregister it. This also opened a
window for a race condition since the job state change event of the
just-cancelled job might be delivered prior to us unregistering the job
in which case everything would work properly.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoqemu: blockjob: Log blockjobs which are dropped when untracked by qemu
Peter Krempa [Tue, 26 Nov 2019 13:14:54 +0000 (14:14 +0100)]
qemu: blockjob: Log blockjobs which are dropped when untracked by qemu

Since we don't know what happened to the job we can't do much about it
but we can at least log that this happened.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoqemu: blockjob: Fix deadlock when terminating job with invalid data
Peter Krempa [Tue, 26 Nov 2019 12:16:36 +0000 (13:16 +0100)]
qemu: blockjob: Fix deadlock when terminating job with invalid data

We must exit the monitor prior to refusing other work, otherwise the VM
object will become unusable.

This bug was introduced in commit v5.5.0-244-gc412383796 but thankfully
the code path was not excercised without QEMU_CAPS_BLOCKDEV.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoqemu: process: Move block job refresh after async job recovery
Peter Krempa [Tue, 26 Nov 2019 12:04:30 +0000 (13:04 +0100)]
qemu: process: Move block job refresh after async job recovery

Block jobs may be members of async jobs so it makes more sense to
refresh block job state after we do steps for async job recovery.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoqemu: blockjob: Properly propagate cancellation of blockjobs
Peter Krempa [Tue, 26 Nov 2019 08:28:22 +0000 (09:28 +0100)]
qemu: blockjob: Properly propagate cancellation of blockjobs

qemu returns an error message in the job statistics even if the job was
cancelled to emphasize it was not successful. Libvirt didn't properly
transform it into QEMU_BLOCKJOB_STATE_CANCELLED though.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoqemu: monitor: Finish implementation of infrastructure for 'query-jobs'
Peter Krempa [Mon, 25 Nov 2019 20:28:14 +0000 (21:28 +0100)]
qemu: monitor: Finish implementation of infrastructure for 'query-jobs'

Commit ed56851f1bc6f5 didn't wire up fetching of the statistics for the
job which are reported by 'query-jobs'.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoutil: whitelist Oracle ACFS as a shared filesystem
Daniel P. Berrangé [Tue, 26 Nov 2019 16:29:47 +0000 (16:29 +0000)]
util: whitelist Oracle ACFS as a shared filesystem

The magic number is taken from the coreutils stat.c file since
there is no constant for it in normal system headers.

Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agolibxl: fix typo in error message
Daniel P. Berrangé [Tue, 26 Nov 2019 15:59:13 +0000 (15:59 +0000)]
libxl: fix typo in error message

Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agoRevert "qemu: Don't repeat virDomainObjEndAPI in qemuDomainBlockPull"
Peter Krempa [Wed, 27 Nov 2019 12:40:14 +0000 (13:40 +0100)]
Revert "qemu: Don't repeat virDomainObjEndAPI in qemuDomainBlockPull"

This reverts commit 421c9550f5446729b513ee50f5c44e6f6969b5a2

qemuDomainBlockPullCommon calls virDomainObjEndAPI internally so the
original commit made us shed two references of @vm instead of one
getting us into a premature free of @vm.

This is not a straight revert as qemuDomainBlockPull was modified
meanwhile. I've also added a warning comment that @vm is consumed.

https://bugzilla.redhat.com/show_bug.cgi?id=1777230

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoall: don't wait for driver lock during startup
Michal Privoznik [Tue, 26 Nov 2019 14:28:22 +0000 (15:28 +0100)]
all: don't wait for driver lock during startup

There are two daemons that wait for acquiring their pid files:
virtnetworkd and virtstoraged. This is undesirable as the idea
is to quit early if unable to acquire the pid file.

Fixes: v5.6.0-rc1~207.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
5 years agocheck-symfile: Use pythonesque string formatting instead of perl
Peter Krempa [Mon, 25 Nov 2019 15:33:53 +0000 (16:33 +0100)]
check-symfile: Use pythonesque string formatting instead of perl

Commit d30a1ad0443 translated the symbol file checker from perl to
python by doing a literal translation in most cases. Unfortunately one
string formatting operation was not really translated into python
leaving users with non-helpful error:

'Symbol $1 is listed twice'

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
5 years agonet/qemu: move vlan/bandwidth validation out of network driver osstest/frozen/xen-4.13-testing
Laine Stump [Tue, 1 Oct 2019 18:05:58 +0000 (14:05 -0400)]
net/qemu: move vlan/bandwidth validation out of network driver

In the past the network driver was (mistakenly) being called for all
interfaces, not just those of type='network', and so it had a chance
to validate all interface configs after the actual type of the
interface was known.

But since the network driver has been more completely/properly
separated from qemu, the network driver isn't called during the
startup of any interfaces except those with type='network', so this
validation no longer takes place for, e.g. <interface type='bridge'>
(or direct, etc). This in turn meant that a config could erroneously
specify a vlan tag, or bandwidth settings, for a type of interface
that didn't support it, and the domain would start without complaint,
just silently ignoring those settings.

This patch moves those validation checks out of the network driver,
and into virDomainActualNetDefValidate() so they will be done for all
interfaces, not just type='network'.

https://bugzilla.redhat.com/1741121
Signed-off-by: Laine Stump <laine@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoconf: add hypervisor agnostic, domain start-time, validation function for NetDef
Laine Stump [Fri, 18 Oct 2019 19:48:13 +0000 (15:48 -0400)]
conf: add hypervisor agnostic, domain start-time, validation function for NetDef

<interface> devices (virDomainNetDef) are a bit different from other
types of devices in that their actual type may come from a network (in
the form of a port connection), and that doesn't happen until the
domain is started. This means that any validation of an <interface> at
parse time needs to be a bit liberal in what it accepts - when
type='network', you could think that something is/isn't allowed, but
once the domain is started and a port is created by the configured
network, the opposite might be true.

To solve this problem hypervisor drivers need to do an extra
validation step when the domain is being started. I recently (commit
3cff23f7, libvirt 5.7.0) added a function to peform such validation
for all interfaces to the QEMU driver -
qemuDomainValidateActualNetDef() - but while that function is a good
single point to call for the multiple places that need to "start" an
interface (domain startup, device hotplug, device update), it can't be
called by the other hypervisor drivers, since 1) it's in the QEMU
driver, and 2) it contains some checks specific to QEMU. For
validation that applies to network devices on *all* hypervisors, we
need yet another interface validation function that can be called by
any hypervisor driver (not just QEMU) right after its network port has
been created during domain startup or hotplug. This patch adds that
function - virDomainActualNetDefValidate(), in the conf directory,
and calls it in appropriate places in the QEMU, lxc, and libxl
drivers.

This new function is the place to put all network device validation
that 1) is hypervisor agnostic, and 2) can't be done until we know the
"actual type" of an interface.

There is no framework for validation at domain startup as there is for
post-parse validation, but I don't want to create a whole elaborate
system that will only be used by one type of device. For that reason,
I just made a single function that should be called directly from the
hypervisors, when they are initializing interfaces to start a domain,
right after conditionally allocating the network port (and regardless
of whether or not that was actually needed). In the case of the QEMU
driver, qemuDomainValidateActualNetDef() is already called in all the
appropriate places, so we can just call the new function from
there. In the case of the other hypervisors, we search for
virDomainNetAllocateActualDevice() (which is the hypervisor-agnostic
function that calls virNetworkPortCreateXML()), and add the call to our
new function right after that.

The new function itself could be plunked down into many places in the
code, but we already have 3 validation functions for network devices
in 2 different places (not counting any basic validation done in
virDomainNetDefParseXML() itself):

1) post-parse hypervisor-agnostic
   (virDomainNetDefValidate() - domain_conf.c:6145)
2) post-parse hypervisor-specific
   (qemuDomainDeviceDefValidateNetwork() - qemu_domain.c:5498)
3) domain-start hypervisor-specific
   (qemuDomainValidateActualNetDef() - qemu_domain.c:5390)

I placed (3) right next to (2) when I added it, specifically to avoid
spreading validation all over the code. For the same reason, I decided
to put this new function right next to (1) - this way if someone needs
to add validation specific to qemu, they go to one location, and if
they need to add validation applying to everyone, they go to the
other. It looks a bit strange to have a public function in between a
bunch of statics, but I think it's better than the alternative of
further fragmentation. (I'm open to other ideas though, of course.)

Signed-off-by: Laine Stump <laine@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoconf: change args/return values of remaining virDomainNetGetActual*() to const
Laine Stump [Tue, 1 Oct 2019 17:06:24 +0000 (13:06 -0400)]
conf: change args/return values of remaining virDomainNetGetActual*() to const

These all just return a scalar value, so there's no daisy-chained
fallout from changing them, and they can easily be combined in a
single patch.

Signed-off-by: Laine Stump <laine@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoconf: return a const from virDomainNetGetActualVirtPortProfile
Laine Stump [Tue, 1 Oct 2019 17:56:35 +0000 (13:56 -0400)]
conf: return a const from virDomainNetGetActualVirtPortProfile

This also isn't required (due to the vportprofile being stored in the
NetDef as a pointer rather than being directly contained), but it
seemed dishonest to not mark it as const (and thus permit users to
modify its contents)

Signed-off-by: Laine Stump <laine@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoconf: make virDomainNetGetActualBandwidth arg/return value const
Laine Stump [Tue, 1 Oct 2019 16:52:03 +0000 (12:52 -0400)]
conf: make virDomainNetGetActualBandwidth arg/return value const

In this case, the virNetDevBandwidthPtr that is returned is not to a
region within the virDomainNetDef arg, but points elsewhere (the
NetDef has the pointer, not the entire object), so technically it's
not necessary to make the return value a const, but it's a bit
disingenuous to *not* do it.

Signed-off-by: Laine Stump <laine@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoconf: make virDomainNetGetActualVlan arg/return val const
Laine Stump [Tue, 1 Oct 2019 16:25:47 +0000 (12:25 -0400)]
conf: make virDomainNetGetActualVlan arg/return val const

This is needed if we want to call the function when the
virDomainNetDef* we have is a const.

Since virDomainNetGetActualVlan returns a pointer to memory that is
within the virDomainNetDefPtr arg, the returned pointer must also be
made const. This leads to a cascade of other virNetDevVlanPtr's that
must be changed to "const virNetDevVlan *".

Signed-off-by: Laine Stump <laine@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoqemu: add mac address to error messages in qemuDomainValidateActualNetDef
Laine Stump [Fri, 27 Sep 2019 15:47:21 +0000 (11:47 -0400)]
qemu: add mac address to error messages in qemuDomainValidateActualNetDef

This makes it easier to understand which interface's config caused the
error.

Signed-off-by: Laine Stump <laine@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
5 years agoqemu_capabilities: Use proper free function for caps->cpuModels
Michal Privoznik [Mon, 25 Nov 2019 15:35:27 +0000 (16:35 +0100)]
qemu_capabilities: Use proper free function for caps->cpuModels

The cpuModels member of _virQEMUCapsAccel struct is not a
virObject but regular struct with a free function defined:
qemuMonitorCPUDefsFree(). Use that when clearing parent structure
instead of virObjectUnref() to avoid a memleak:

==212322== 57,275 (48 direct, 57,227 indirect) bytes in 3 blocks are definitely lost in loss record 623 of 627
==212322==    at 0x4838B86: calloc (vg_replace_malloc.c:762)
==212322==    by 0x554A158: g_malloc0 (in /usr/lib64/libglib-2.0.so.0.6000.6)
==212322==    by 0x17B14BF5: qemuMonitorCPUDefsNew (qemu_monitor.c:3587)
==212322==    by 0x17B27BA7: qemuMonitorJSONGetCPUDefinitions (qemu_monitor_json.c:5616)
==212322==    by 0x17B14B0B: qemuMonitorGetCPUDefinitions (qemu_monitor.c:3559)
==212322==    by 0x17A6AFBB: virQEMUCapsFetchCPUDefinitions (qemu_capabilities.c:2571)
==212322==    by 0x17A6B2CC: virQEMUCapsProbeQMPCPUDefinitions (qemu_capabilities.c:2629)
==212322==    by 0x17A70C00: virQEMUCapsInitQMPMonitorTCG (qemu_capabilities.c:4769)
==212322==    by 0x17A70DDF: virQEMUCapsInitQMPSingle (qemu_capabilities.c:4820)
==212322==    by 0x17A70E99: virQEMUCapsInitQMP (qemu_capabilities.c:4848)
==212322==    by 0x17A71044: virQEMUCapsNewForBinaryInternal (qemu_capabilities.c:4891)
==212322==    by 0x17A7119C: virQEMUCapsNewData (qemu_capabilities.c:4923)

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
5 years agoqemu: Use host-model CPU on s390 by default
Jiri Denemark [Tue, 12 Nov 2019 14:28:33 +0000 (15:28 +0100)]
qemu: Use host-model CPU on s390 by default

On s390 machines host-passthrough and host-model CPUs result in the same
guest ABI (with QEMU new enough to be able to tell us what "host" CPU is
expanded to, which was implemented around 2.9.0). So instead of using
host-passthrough CPU when there's no CPU specified in a domain XML we
can safely use host-model and benefit from CPU compatibility checks
during migration, snapshot restore and similar operations.

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agocpu_s390: Don't check match attribute for host-model CPUs
Jiri Denemark [Tue, 12 Nov 2019 14:27:45 +0000 (15:27 +0100)]
cpu_s390: Don't check match attribute for host-model CPUs

The match attribute is only relevant for custom mode CPUs. Reporting
failure when match == 'minimum' regardless on CPU mode can cause
unexpected failures. We should only report the error for custom CPUs. In
fact, calling virCPUs390Update on a custom mode CPU should always report
an error as optional features are not supported on s390 either.

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agocpu_conf: Don't format empty model for host-model CPUs
Jiri Denemark [Tue, 12 Nov 2019 14:54:12 +0000 (15:54 +0100)]
cpu_conf: Don't format empty model for host-model CPUs

Most likely for historical reasons our CPU def formatting code is
happily adding useless <model fallback='allow'/> for host-model CPUs. We
can just drop it.

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agocpu_conf: Fix default value for CPU match attribute
Jiri Denemark [Tue, 12 Nov 2019 14:26:23 +0000 (15:26 +0100)]
cpu_conf: Fix default value for CPU match attribute

Commit v0.8.4-66-g95ff6b18ec (9 years ago) changed the default value for
the cpu/@match attribute to 'exact' in a rather complicated way. It did
so only if <model> subelement was present and set -1 otherwise (which is
not expected to ever happen). Thus the following two equivalent XML
elements:

    <cpu mode='host-model'/>

and

    <cpu mode='host-model'>
      <model/>
    </cpu>

would be parsed differently. The former would end up with match == -1
while the latter would have match == 1 ('exact'). This is not a big deal
since the match attribute is ignored for host-model CPUs, but we can
simplify the code and make it a little bit saner anyway.

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
5 years agoqemu: added tests of the new default video type selection algorithm
Pavel Mores [Mon, 25 Nov 2019 10:54:26 +0000 (11:54 +0100)]
qemu: added tests of the new default video type selection algorithm

The test case for x86_64 and neither cirrus nor vga capability is of the
xml2argv type because it actually fails to parse the XML at all [*] which
is something that xml2xml tests don't seem to handle.  xml2argv test fails
to produce a qemu argv for this case which xml2argv tests can handle.

[*] This is a consequence of the decision not to have a fallback if the
obvious choices (cirrus and vga) aren't viable due to missing QEMU caps.

Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Pavel Mores <pmores@redhat.com>
5 years agoqemu: the actual change of default video devide type selection algorithm
Pavel Mores [Mon, 25 Nov 2019 10:54:25 +0000 (11:54 +0100)]
qemu: the actual change of default video devide type selection algorithm

If a graphics device was added to XML that had no video device, libvirt
automatically added a video device which was always of type 'cirrus' on
x86_64, even if the underlying qemu didn't support cirrus.

This patch refines a bit the decision about the type of the video device.
Based on QEMU capabilities, cirrus is still preferred but only added if
QEMU supports it, otherwise VGA is used if supported by QEMU.  There is now
no fallback as libvirt only aspires to generate a basic working config and
leaves anything more specific up to higher-level management tools.

Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Pavel Mores <pmores@redhat.com>
5 years agoqemu: prepare existing test for change of the default video device type
Pavel Mores [Mon, 25 Nov 2019 10:54:24 +0000 (11:54 +0100)]
qemu: prepare existing test for change of the default video device type

The test relied implicitly on default video device being cirrus.  As we're
about to change that the test would start failing.  To avoid this, just make
the test's requirement explicit.

Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Pavel Mores <pmores@redhat.com>