Peter Maydell [Thu, 5 Jul 2018 17:24:28 +0000 (18:24 +0100)]
Merge remote-tracking branch 'remotes/stsquad/tags/pull-code-coverage-and-build-tweaks-050718-3' into staging
Code coverage and other build tweaks
- revert 208ecb3e (and drop filter for mingw, tweak for check-tcg)
- some travis speed-ups
- modernise code coverage support
- docker image cleanups
- clean-up binfmt_misc docker infrastructure
- add debian-powerpc-user-cross image for ppc32 build
# gpg: Signature made Thu 05 Jul 2018 17:00:02 BST
# gpg: using RSA key FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>"
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44
* remotes/stsquad/tags/pull-code-coverage-and-build-tweaks-050718-3:
docker: add linux-user powered cross builder for QEMU
docker: add special rule for deboostrapped images
docker: add special handling for FROM:debian-%-user targets
docker: debian-bootstrap.pre allow customising of variant/url
docker: drop QEMU build-dep from bootstrap
docker: Do not run tests in 'intermediate' images
docker: Clean the MXE base image
docker: ubuntu: Use SDL2
docker: ubuntu: Update the package list before installing new ones
linux-user: add gcov support to preexit_cleanup
linux-user: introduce preexit_cleanup
build-system: add coverage-report target
build-system: add clean-coverage target
travis: add gcovr summary for GCOV build
docker: add gcovr to travis image
.gitignore: add .gcov files
build-system: remove per-test GCOV reporting
travis: test out-of-tree builds
travis: do not waste time cloning unused submodules
Revert "Makefile: Rename TARGET_DIRS to TARGET_LIST"
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Alex Bennée [Fri, 29 Jun 2018 19:41:26 +0000 (20:41 +0100)]
docker: add linux-user powered cross builder for QEMU
We can't use cross compilers in the current Debian stable and Debian
sid is sketchy as hell. So for powerpc fall back to dog-fooding our
own linux-user to do the build.
As we can only build the base image with a suitably configured
source tree we fall back to checking for its existence when we can't
build it from scratch. However this does mean you don't have to keep
a static powerpc-linux-user in your active configuration just to
update the cross build image.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Alex Bennée [Fri, 29 Jun 2018 16:57:57 +0000 (17:57 +0100)]
docker: add special rule for deboostrapped images
We might as well have a custom rule for this. For one thing the
dependencies are different. As the primary dependency for
docker-image-% could never be docker-image-debian-bootstrap we can
drop that test in the main rule as well.
Missing EXECUTABLE, DEB_ARCH and DEB_TYPE are treated as hard faults
now. We also error out if the EXECUTABLE file isn't there. We should
really do this with a dependency on any source rules but currently
subdir-FOO-linux-user isn't enough on a clean build.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Alex Bennée [Fri, 29 Jun 2018 16:46:49 +0000 (17:46 +0100)]
docker: add special handling for FROM:debian-%-user targets
These will have been build with debootstrap so we need to check
against the debian-bootstrap dockerfile. This does mean sticking to
debian-FOO-user as the naming conventions for boot-strapped images.
The actual cross image is built on top.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Alex Bennée [Mon, 2 Jul 2018 13:02:44 +0000 (14:02 +0100)]
docker: debian-bootstrap.pre allow customising of variant/url
We default to the buildd variant as most of our images are for
building. However lets give the user the ability to specify "minbase"
if they want to create a simple base image for experimentation.
Allowing the tweaking of DEB_URL means we can also bootstrap other
Debian based OS's. For example:
make docker-binfmt-image-debian-ubuntu-bionic-arm64 \
DEB_ARCH=arm64 DEB_TYPE=bionic \
DEB_VARIANT=minbase DEB_URL=http://ports.ubuntu.com/ \
EXECUTABLE=./aarch64-linux-user/qemu-aarch64
Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Alex Bennée [Fri, 29 Jun 2018 16:46:02 +0000 (17:46 +0100)]
docker: drop QEMU build-dep from bootstrap
This is best done with any child images that actually need it.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reported-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
docker: ubuntu: Update the package list before installing new ones
Since docker caches the different layers, updating the package
list does not invalidate the previous "apt-get update" layer,
and it is likely "apt-get install" hits an outdated repository.
See https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#apt-get
This fixes:
$ make docker-image-ubuntu V=1
./tests/docker/docker.py build qemu:ubuntu tests/docker/dockerfiles/ubuntu.docker --add-current-user
Sending build context to Docker daemon 3.072kB
[...]
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/m/mesa/libgles2-mesa_17.0.7-0ubuntu0.16.04.2_amd64.deb 404 Not Found
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/m/mesa/libgles2-mesa-dev_17.0.7-0ubuntu0.16.04.2_amd64.deb 404 Not Found
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
The command '/bin/sh -c apt-get -y install $PACKAGES' returned a non-zero code: 100
tests/docker/Makefile.include:40: recipe for target 'docker-image-ubuntu' failed
make: *** [docker-image-ubuntu] Error 1
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Alex Bennée [Fri, 22 Jun 2018 16:23:44 +0000 (17:23 +0100)]
linux-user: add gcov support to preexit_cleanup
As we don't always take the normal exit path when running a guest we
can skip the normal exit destructors where gcov normally dumps it's
info. The GCC manual suggests long running programs use __gcov_dump()
to flush out the coverage state periodically so we use that here.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Alex Bennée [Fri, 22 Jun 2018 16:09:10 +0000 (17:09 +0100)]
linux-user: introduce preexit_cleanup
To avoid repeating ourselves move our preexit clean-up code into a
helper function. I figured the continuing effort to split of the
syscalls made it worthwhile creating a new file for it now.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Alex Bennée [Wed, 20 Jun 2018 13:04:24 +0000 (14:04 +0100)]
build-system: add coverage-report target
This will build a coverage report under the current directory in
reports/coverage. At the users option a report can be generated by
directly invoking something like:
make foo/bar/coverage-report.html
Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Alex Bennée [Wed, 20 Jun 2018 11:34:45 +0000 (12:34 +0100)]
build-system: add clean-coverage target
This can be used to remove any stale coverage data before any
particular test run. This is useful for analysing individual tests.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>---
Alex Bennée [Wed, 20 Jun 2018 11:00:07 +0000 (12:00 +0100)]
travis: add gcovr summary for GCOV build
This gives a more useful summary, sorted by descending % coverage,
after the tests have run. The final numbers will give an idea if our
coverage is getting better or worse.
To keep the width sane we need to post process the file that the old
gcovr tool generates. This is done with a mix of sed, awk and column
in the scripts/coverage-summary.sh script.
As quite a lot of lines don't get covered at all we filter out all the
0% lines. If the file doesn't appear it is not being exercised.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Alex Bennée [Mon, 25 Jun 2018 09:06:35 +0000 (10:06 +0100)]
docker: add gcovr to travis image
Useful for debugging if nothing else as the gcovr on the Travis images
are a little old.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Alex Bennée [Wed, 20 Jun 2018 10:35:47 +0000 (11:35 +0100)]
.gitignore: add .gcov files
These are temporary files generated on gcov runs and shouldn't be
included in the source tree.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Alex Bennée [Wed, 20 Jun 2018 10:28:51 +0000 (11:28 +0100)]
build-system: remove per-test GCOV reporting
I'm not entirely sure who's using this information and certainly in a
CI environment it just washes over as additional noise. Later patches
will provide new reporting options so a user who wants to analyse
individual tests will be able to use that to get the information.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Force one config to build 'out-of-tree' (object files and executables
are created in a tree outside the project source code).
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
travis: do not waste time cloning unused submodules
Builds only require:
- dtc
- keycodemapdb
- capstone
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[AJB: drop wget cache] Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Alex Bennée [Wed, 4 Jul 2018 06:30:11 +0000 (07:30 +0100)]
Revert "Makefile: Rename TARGET_DIRS to TARGET_LIST"
This reverts commit 208ecb3e1acc8d55dab49fdf721a86d513691688. This was
causing problems by making DEF_TARGET_LIST pointless and having to
jump through hoops to build on mingw with a dully enabled config.
This includes a change to fix the per-guest TCG test probe which was
added after 208ecb3 and used TARGET_LIST.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Cc: Paolo Bonzini <pbonzini@redhat.com>
Peter Maydell [Thu, 5 Jul 2018 14:53:04 +0000 (15:53 +0100)]
Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging
Block layer patches:
- qcow2: Use worker threads for compression to improve performance of
'qemu-img convert -W' and compressed backup jobs
- blklogwrites: New filter driver to log write requests to an image in
the dm-log-writes format
- file-posix: Fix image locking during image creation
- crypto: Fix memory leak in error path
- Error out instead of silently truncating node names
# gpg: Signature made Thu 05 Jul 2018 11:24:33 BST
# gpg: using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6
* remotes/kevin/tags/for-upstream:
file-posix: Unlock FD after creation
file-posix: Fix creation locking
block/blklogwrites: Add an option for the update interval of the log superblock
block/blklogwrites: Add an option for appending to an old log
block/blklogwrites: Change log_sector_size from int64_t to uint64_t
block/crypto: Fix memory leak in create error path
block: Don't silently truncate node names
block: Add blklogwrites
block: Move two block permission constants to the relevant enum
qcow2: add compress threads
qcow2: refactor data compression
qemu-img: allow compressed not-in-order writes
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
$ cov-make-library -of scripts/coverity-model.xmldb scripts/coverity-model.c
output file: scripts/coverity-model.xmldb
Compiling scripts/coverity-model.c with command /opt/cov-sa-2018.06/bin/cov-emit --dir /tmp/cov-armbru/930a6fb31e5f464fc1a53354b2deb66b/cov-make-library-emit -w --no_error_recovery --emit_header_functions --no_implicit_decl --preinclude /opt/cov-sa-2018.06/library/decls.h --c scripts/coverity-model.c
"scripts/coverity-model.c", line 110: error #20: identifier "replay_file" is
undefined
if (replay_file) {
^
Emit for file '/work/armbru/qemu/scripts/coverity-model.c' complete.
[ERROR] 1 error detected in the compilation of "scripts/coverity-model.c".
ERROR: cov-emit returned with code 1
Broken in commit 04a0afe5285. Fix by dumbing down.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180626085642.4973-1-armbru@redhat.com> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
* remotes/armbru/tags/pull-monitor-2018-07-03-v2: (32 commits)
qapi: Polish command flags documentation in qapi-code-gen.txt
monitor: Improve some comments
qmp: Clean up capability negotiation after commit 02130314d8c
qobject: Let qobject_from_jsonf() fail instead of abort
qmp: Switch timestamp_put() to qdict_from_jsonf_nofail()
qmp: Add some comments around null responses
qmp: Simplify monitor_qmp_respond()
qmp: Replace get_qmp_greeting() by qmp_greeting()
qmp: Replace monitor_json_emitter{,raw}() by qmp_{queue,send}_response()
qmp: Use QDict * instead of QObject * for response objects
qmp: De-duplicate error response building
qobject: New qdict_from_jsonf_nofail()
monitor: Peel off @mon_global wrapper
monitor: Rename use_io_thr to use_io_thread
qmp: Don't let JSON errors jump the queue
qmp: Don't let malformed in-band commands jump the queue
tests/qmp-test: Demonstrate QMP errors jumping the queue
qmp: Simplify code around monitor_qmp_dispatch_one()
qmp: Always free QMPRequest with qmp_request_free()
qmp: Revert change to handle_qmp_command tracepoint
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Max Reitz [Wed, 4 Jul 2018 14:47:50 +0000 (16:47 +0200)]
file-posix: Fix creation locking
raw_apply_lock_bytes() takes a bit mask of "permissions that are NOT
shared".
Also, make the "perm" and "shared" variables uint64_t, because I do not
particularly like using ~ on signed integers (and other permission masks
are usually uint64_t, too).
Reported-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Ari Sundholm [Wed, 4 Jul 2018 14:59:36 +0000 (17:59 +0300)]
block/blklogwrites: Add an option for the update interval of the log superblock
This is a way to ensure that the log superblock is periodically
updated. Before, this was only done on flush requests, which may
not be enough if the VM exits abnormally, omitting the final flush.
The default interval is 4096 write requests.
Signed-off-by: Ari Sundholm <ari@tuxera.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Kevin Wolf [Wed, 4 Jul 2018 11:28:29 +0000 (13:28 +0200)]
block: Don't silently truncate node names
If the user passes a too long node name string, we silently truncate it
to fit into BlockDriverState.node_name, i.e. to 31 characters. Apart
from surprising the user when the node has a different name than
requested, this also bypasses the check for duplicate names, so that the
same name can be assigned to multiple nodes.
Fix this by just making too long node names an error.
Reported-by: Peter Krempa <pkrempa@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Implements a block device write logging system, similar to Linux kernel
device mapper dm-log-writes. The write operations that are performed
on a block device are logged to a file or another block device. The
write log format is identical to the dm-log-writes format. Currently,
log markers are not supported.
This functionality can be used for crash consistency and fs consistency
testing. By implementing it in qemu, tests utilizing write logs can be
be used to test non-Linux drivers and older kernels.
The driver accepts an optional parameter to set the sector size used
for logging. This makes the driver require all requests to be aligned
to this sector size and also makes offsets and sizes of writes in the
log metadata to be expressed in terms of this value (the log format has
a granularity of one sector for offsets and sizes). This allows
accurate logging of writes to guest block devices that have unusual
sector sizes.
The implementation is based on the blkverify and blkdebug block
drivers.
Signed-off-by: Aapo Vienamo <aapo@tuxera.com> Signed-off-by: Ari Sundholm <ari@tuxera.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Do data compression in separate threads. This significantly improve
performance for qemu-img convert with -W (allow async writes) and -c
(compressed) options.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Make a separate function for compression to be parallelized later.
- use .avail_out field instead of .next_out to calculate size of
compressed data. It looks more natural and it allows to keep dest to
be void pointer
- set avail_out to be at least one byte less than input, to be sure
avoid inefficient compression earlier
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Peter Maydell [Thu, 5 Jul 2018 07:21:25 +0000 (08:21 +0100)]
Merge remote-tracking branch 'remotes/mdroth/tags/qga-pull-2018-07-03-tag' into staging
qemu-ga patch queue for soft-freeze
* add systemd suspend support
* add used/total space stats for guest-get-fsinfo
* fixes for guest-get-fsinfo over PCI bridges
* MSI installer and schema doc fixes
* guard against unbounded allocations in guest-file-read
* add some additional qga test cases
* remotes/mdroth/tags/qga-pull-2018-07-03-tag:
qga: removing bios_supports_mode
qga: systemd hibernate/suspend/hybrid-sleep support
qga: removing switch statements, adding run_process_child
qga: guest_suspend: decoupling pm-utils and sys logic
qga: bios_supports_mode: decoupling pm-utils and sys logic
qga: refactoring qmp_guest_suspend_* functions
qemu-ga: make get-fsinfo work over pci bridges
qga-win: Fixing msi upgrade disallow in WiX file
qga/schema: fix documentation for GuestOSInfo
test-qga: add trivial tests for some commands
qga-win: add driver path usage to GuestFilesystemInfo
qga: add mountpoint usage info to GuestFilesystemInfo
qga: check bytes count read by guest-file-read
qga: unset frozen state if no mount points are frozen
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Bitmap lock/unlock were added to bdrv_enable_dirty_bitmap in 8b1402ce80d, but some places were not updated correspondingly, which
leads to trying to take this lock twice, which is dead-lock. Fix this.
Actually, iotest 199 (about dirty bitmap postcopy migration) is broken
now, and this fixes it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20180625165745.25259-3-vsementsov@virtuozzo.com Signed-off-by: John Snow <jsnow@redhat.com>
Add _locked version of bdrv_enable_dirty_bitmap, to fix dirty bitmap
migration in the following patch.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20180625165745.25259-2-vsementsov@virtuozzo.com Signed-off-by: John Snow <jsnow@redhat.com>
qapi: Polish command flags documentation in qapi-code-gen.txt
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-33-armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-32-armbru@redhat.com>
qmp: Clean up capability negotiation after commit 02130314d8c
qmp_greeting() offers capabilities to the client, and
qmp_qmp_capabilities() accepts or denies capabilities requested by the
client. The two compute the set of available capabilities
independently. Not nice.
Clean this up as follows. Compute available capabilities just once in
monitor_qmp_caps_reset(), and store them in Monitor member
qmp.capab_offered[]. Have qmp_greeting() and qmp_qmp_capabilities()
use that. Both are now oblivious of capability details.
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-31-armbru@redhat.com>
qobject: Let qobject_from_jsonf() fail instead of abort
qobject_from_jsonf() aborts on error, unlike qobject_from_jsonv(),
which returns null. Since all remaining users of qobject_from_jsonf()
cope fine with null, change it to return null.
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-30-armbru@redhat.com>
qmp: Switch timestamp_put() to qdict_from_jsonf_nofail()
There's just one use of qobject_from_jsonf() to parse a JSON object
left: timestamp_put(). Switch it to qdict_from_jsonf_nofail().
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-29-armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-28-armbru@redhat.com>
monitor_qmp_respond() takes both a response object and an error
object. If an error object is non-null, the response object must be
null, and the response is built from the error object.
Of the two callers, one always passes a null response object, and one
a null error object. Move building the response object from the error
object to the latter, and drop the error object parameter.
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-27-armbru@redhat.com>
get_qmp_greeting() returns a QDict * as QObject *. It's caller
converts it right back.
Return QDict * instead. While there, rename to qmp_greeting().
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-26-armbru@redhat.com>
qmp: Replace monitor_json_emitter{,raw}() by qmp_{queue,send}_response()
monitor_json_emitter() and monitor_json_emitter_raw() are
unnecessarily general: they can send arbitrary JSON values, even
though we only ever use them for QMP, which may send only JSON
objects.
Specialize the argument from QObject * to QDict *, and rename to
qmp_queue_response(), qmp_send_response().
All callers but one lose an upcast. The lone exception gains a
downcast; the next commit will get rid of it.
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-25-armbru@redhat.com>
qmp: Use QDict * instead of QObject * for response objects
By using the more specific type, we get fewer downcasts. The
downcasts are safe, but not obviously so, at least not locally.
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-24-armbru@redhat.com>
All callers of qmp_build_error_object() duplicate the code to wrap it
in a response object. Replace it by qmp_error_response() that
captures the duplicated code, including error_free().
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-23-armbru@redhat.com>
Many uses of qobject_from_jsonf() convert JSON objects. Create new
convenience function qdict_from_jsonf_nofail() that includes the
conversion to QDict. The next few commits will put it to use.
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-22-armbru@redhat.com>
Wrapping global variables in a struct without a use for the wrapper
struct buys us nothing but longer lines. Unwrap them.
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-21-armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-20-armbru@redhat.com>
handle_qmp_command() reports JSON syntax errors right away. This is
wrong when OOB is enabled, because the errors can "jump the queue"
then.
The previous commit fixed the same bug for semantic errors, by
delaying the checking until dispatch. We can't delay the checking, so
delay the reporting.
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-19-armbru@redhat.com>
qmp: Don't let malformed in-band commands jump the queue
handle_qmp_command() reports certain errors right away. This is wrong
when OOB is enabled, because the errors can "jump the queue" then, as
the previous commit demonstrates.
To fix, we need to delay errors until dispatch. Do that for semantic
errors, mostly by reverting ill-advised parts of commit cf869d53172
"qmp: support out-of-band (oob) execution". Bonus: doesn't run
qmp_dispatch_check_obj() twice, once in handle_qmp_command(), and
again in do_qmp_dispatch(). That's also due to commit cf869d53172.
The next commit will fix queue jumping for syntax errors.
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-18-armbru@redhat.com>
tests/qmp-test: Demonstrate QMP errors jumping the queue
When OOB is enabled, out-of-band commands are executed right away,
everything else is queued. This lets out-of-band commands "jump the
queue".
However, certain errors are always reported right away, and therefore
can jump the queue even when the erroneous input does not request
out-of-band execution. These errors are pretty unlikely to occur in
production, but it's wrong all the same. Mark FIXME.
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180703085358.13941-17-armbru@redhat.com>
qmp: Simplify code around monitor_qmp_dispatch_one()
Change monitor_qmp_dispatch_one() to take its parameters unwrapped,
move monitor_resume() to the one caller that needs it, rename the
function to monitor_qmp_dispatch().
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-16-armbru@redhat.com>
qmp: Always free QMPRequest with qmp_request_free()
monitor_qmp_dispatch_one() frees a QMPRequest manually, because it
needs to keep a reference to ->id. Premature optimization. Take an
additional reference so we can use qmp_request_free().
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-15-armbru@redhat.com>
qmp: Revert change to handle_qmp_command tracepoint
Commit 71da4667db6 "monitor: separate QMP parser and dispatcher" moved
the handle_qmp_command tracepoint from handle_qmp_command() to
monitor_qmp_dispatch_one(). This delays tracing from enqueue time to
dequeue time. Revert that. Dequeue remains adequately visible via
tracepoint monitor_qmp_cmd_in_band.
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-14-armbru@redhat.com>
qmp: Redo how the client requests out-of-band execution
Commit cf869d53172 "qmp: support out-of-band (oob) execution" added a
general mechanism for command-independent arguments just for an
out-of-band flag:
The "control" key is introduced to store this extra flag. "control"
field is used to store arguments that are shared by all the commands,
rather than command specific arguments. Let "run-oob" be the first.
However, it failed to reject unknown members of "control". For
instance, in QMP command
Instead of fixing this, revert the general "control" mechanism
(because YAGNI), and do it the way I initially proposed, with key
"exec-oob". Simpler code, simpler interface.
Commit cf869d53172 "qmp: support out-of-band (oob) execution"
accidentally made qemu-ga accept and ignore "control". Fix that.
Out-of-band execution in a monitor that doesn't support it now fails
with
{"error": {"class": "GenericError", "desc": "QMP input member 'control' is unexpected"}}
instead of
{"error": {"class": "GenericError", "desc": "Please enable out-of-band first for the session during capabilities negotiation"}}
The old description is suboptimal when out-of-band cannot not be
enabled, or the command doesn't support out-of-band execution.
The new description is a bit unspecific, but it'll do.
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-12-armbru@redhat.com>
tests/test-qga: Demonstrate the guest-agent ignores "control"
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-11-armbru@redhat.com>
qmp qemu-ga: Revert change that accidentally made qemu-ga accept "id"
Commit cf869d53172 "qmp: support out-of-band (oob) execution" changed
how we check "id":
Note that in the patch I exported qmp_dispatch_check_obj() to be
used to check the request earlier, and at the same time allowed
"id" field to be there since actually we always allow that.
The part after "and" is ill-advised: it makes qemu-ga accept and
ignore "id". Revert.
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-10-armbru@redhat.com>
tests/test-qga: Demonstrate the guest-agent ignores "id"
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-9-armbru@redhat.com>
qmp: Make "id" optional again even in "oob" monitors
Commit cf869d53172 "qmp: support out-of-band (oob) execution" made
"id" mandatory for all commands when the client accepted capability
"oob". This is rather onerous when you play with QMP by hand, and
unnecessarily so: only out-of-band commands need an ID for reliable
matching of response to command.
Revert that part of commit cf869d53172 for now, but have documentation
advise on the need to use "id" with out-of-band commands.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180703085358.13941-8-armbru@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
tests/qmp-test: Test in-band command doesn't overtake
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-7-armbru@redhat.com>
tests/qmp-test tests an out-of-band command overtaking a slow in-band
command. To do that, it needs:
1. An in-band command that *reliably* takes long enough to be
overtaken.
2. An out-of-band command to do the overtaking.
3. To avoid delays, a way to make the in-band command complete quickly
after it was overtaken.
To satisfy these needs, commit 469638f9cb3 provides the rather
peculiar oob-capable QMP command x-oob-test:
* With "lock": true, it waits for a global semaphore.
* With "lock": false, it signals the global semaphore.
To satisfy 1., the test runs x-oob-test in-band with "lock": true.
To satisfy 2. and 3., it runs x-oob-test out-of-band with "lock": false.
Note that waiting for a semaphore violates the rules for oob-capable
commands. Running x-oob-test with "lock": true hangs the monitor
until you run x-oob-test with "lock": false on another monitor (which
you might not have set up).
Having an externally visible QMP command that may hang the monitor is
not nice. Let's apply a little more ingenuity to the problem. Idea:
have an existing command block on reading a FIFO special file, unblock
it by opening the FIFO for writing.
where ID2 is a different arbitrary string. Since there's no migration
to pause, the command will fail, but that's fine; instant failure is
still a test of out-of-band responses overtaking in-band commands.
For 3., open FIFO for writing.
Drop QMP command x-oob-test.
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-6-armbru@redhat.com>
[Error checking tweaked]
Events are broadcast to all monitors. If another monitor's client has
a command with the same ID in flight, the event will incorrectly claim
that command was dropped. This must be fixed before out-of-band
execution can graduate from "experimental".
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-5-armbru@redhat.com>
OOB documentation is spread over qmp-spec.txt sections 2.2.1
Capabilities and 2.3 Issuing Commands. The amount of detail is a bit
distracting there. Move the meat of the matter to new section 2.3.1
Out of band execution.
Throw in a few other improvements while there:
* 2.2 Server Greeting: Drop advice to search entire capabilities
array; should be obvious.
* 3. QMP Examples
- 3.1 Server Greeting: Update greeting to the one we expect for the
release. Now shows capability "oob". Update qmp-intro.txt
likewise.
- 3.2 Capabilities negotiation: Show client accepting capability
"oob".
- 3.7 Out-of-band execution: New.
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-4-armbru@redhat.com>
[Whitespace tidied up]
bios_support_mode verifies if the guest has support for a certain
suspend mode but it doesn't inform back which suspend tool
provides it. The caller, guest_suspend, executes all suspend
strategies in order again.
After adding systemd suspend support, bios_support_mode now will
verify for support for systemd, then pmutils, then Linux sys state
file. In a worst case scenario where both systemd and pmutils isn't
supported but Linux sys state is:
- bios_supports_mode will check for systemd, then pmutils, then
Linux sys state. It will tell guest_suspend that there is support,
but it will not tell who provides it;
- guest_suspend will try to execute (and fail) systemd suspend,
then pmutils suspend, to only then use the Linux sys suspend.
The time spent executing systemd and pmutils suspend was wasted
and could be avoided, but only bios_support_mode knew it but
didn't inform it back.
A quicker approach is to nuke bios_supports_mode and control
whether we found support at all with a bool flag inside
guest_suspend. guest_suspend will search for suspend support
and execute it as soon as possible. If the a given suspend
mechanism fails, continue to the next. If no suspend
support is found, the "not supported" message is still being
sent back to the user.
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
qga: systemd hibernate/suspend/hybrid-sleep support
pmutils isn't being supported by newer OSes like Fedora 27
or Mint. This means that the only suspend option QGA offers
for these guests are writing directly into the Linux sys state
file. This also means that QGA also loses the ability to do
hybrid suspend in those guests - this suspend mode is only
available when using pmutils.
Newer guests can use systemd facilities to do all the suspend
types QGA supports. The mapping in comparison with pmutils is:
This is a cleanup of the resulting code after detaching
pmutils and Linux sys state file logic:
- remove the SUSPEND_MODE_* macros and use an enumeration
instead. At the same time, drop the switch statements
at the start of each function and use the enumeration
index to get the right binary/argument;
- create a new function called run_process_child(). This
function uses g_spawn_sync() to execute a shell command,
returning the exit code. This is a common operation in the
pmutils functions and will be used in the systemd implementation
as well, so this function will avoid code repetition.
There are more places inside commands-posix.c where this new
run_process_child function can also be used, but one step
at a time.
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
*check/propagate local_err before setting errp directly Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
qga: guest_suspend: decoupling pm-utils and sys logic
Following the same logic of the previous patch, let's also
decouple the suspend logic from guest_suspend into specialized
functions, one for each strategy we support at this moment.
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
qga: bios_supports_mode: decoupling pm-utils and sys logic
In bios_supports_mode there is a verification to assert if
the chosen suspend mode is supported by the pmutils tools and,
if not, we see if the Linux sys state files supports it.
This verification is done in the same function, one after
the other, and it works for now. But, when adding a new
suspend mechanism that will not necessarily follow the same
return 0 or 1 logic of pmutils, this code will be hard
to deal with.
This patch decouple the two existing logics into their own
functions, pmutils_supports_mode and linux_sys_state_supports_mode,
which in turn are used inside bios_support_mode. The existing
logic is kept but now it's easier to extend it.
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
To be able to add new suspend mechanisms we need to detach
the existing QMP functions from the current implementation
specifics.
At this moment we have functions such as qmp_guest_suspend_ram
calling bios_suspend_mode and guest_suspend passing the
pmutils command and arguments as parameters. This patch
removes this logic from the QMP functions, moving them to
the respective functions that will have to deal with which
binary to use.
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Iterate over the PCI bridges to lookup the PCI device associated with
the block device.
This allows to lookup the driver under the following syspath:
/sys/devices/pci0000:00/0000:00:02.2/0000:03:00.0/virtio2/block/vda/vda3
It also works with an "old-style" Q35 libvirt hierarchy: root complex
-> DMI-PCI bridge -> PCI-PCI bridge -> virtio controller, ex:
/sys/devices/pci0000:00/0000:00:03.0/0000:01:01.0/0000:02:01.0/virtio1/block/vda/vda3
The setup can be reproduced with the following qemu command line
(Thanks Marcel for help):
Issue: When upgrading qemu-ga using the msi from an old version
to a newer one, the upgrade is not allowed by the msi
showing this error message "Another version of this product
is already installed."
Fix: For the upgrade to be allowed by the msi the WiX file must
provide three things:
1. Changing product's Id. (assigning it to "*")
2. Constant product's UpgradeId. (exists)
3. Changing version. (exists)
Before the patch, the commands for !CONFIG_VNC are stubs that fail
like this:
{"error": {"class": "GenericError",
"desc": "The feature 'vnc' is not enabled"}}
Afterwards, they fail like this:
{"error": {"class": "CommandNotFound",
"desc": "The command FOO has not been found"}}
I call that an improvement, because it lets clients distinguish
between command unavailable (class CommandNotFound) and command failed
(class GenericError).
Will return "unknown command: 'info vnc'" when VNC is compiled
out (same as error for spice when --disable-spice)
Occurrences of VNC (case insensitive) in the schema that aren't
covered by this change:
* add_client
Command has other uses, including "socket bases character devices".
These are unconditional as far as I can tell.
* set_password, expire_password
In theory, these commands could be used for managing any service's
password. In practice, they're used for VNC and SPICE services.
They're documented for "remote display session" / "remote display
server".
The service is selected by argument @protocol. The code special-cases
protocol-specific argument checking, then calls a protocol-specific
function to do the work. If it fails, the command fails with "Could
not set password". It does when the service isn't compiled in (it's a
stub then).
We could make these commands conditional on the conjunction of all
services [currently: defined(CONFIG_VNC) || defined(CONFIG_SPICE)],
but I doubt it's worthwhile.
* change
Command has other uses, namely changing media.
This patch inlines a stub; no functional change.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Gerd Hoffmann <kraxel@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180703155648.11933-14-marcandre.lureau@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
Wrap generated code with #if/#endif using an 'ifcontext' on
QAPIGenCSnippet objects.
This makes a conditional event's qapi_event_send_FOO() compile-time
conditional, but its enum QAPIEvent member remains unconditional for
now. A follow up patch "qapi-event: add 'if' condition to implicit
event enum" will improve this.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180703155648.11933-11-marcandre.lureau@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
qapi-introspect: add preprocessor conditions to generated QLit
This commit adds 'ifcond' conditions to top-level QLit objects.
Future work will add them to object and enum type members, i.e. within
QLit objects.
Extend the QLit generator to_qlit() to accept (@obj, @cond) tuples in
addition to just @obj. The tuple causes the QLit generated for
objects for @obj with #if/#endif conditions for @cond.
See generated tests/test-qmp-introspect.c. Example diff after this
patch:
Add helpers to wrap generated code with #if/#endif lines.
A later patch wants to use QAPIGen for generating C snippets rather
than full C files with copyright headers etc. Splice in class
QAPIGenCCode between QAPIGen and QAPIGenC.
Add a 'with' statement context manager that will be used to wrap
generator visitor methods. The manager will check if code was
generated before adding #if/#endif lines on QAPIGenCSnippet
objects. Used in the following patches.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180703155648.11933-7-marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
qapi: leave the ifcond attribute undefined until check()
We commonly initialize attributes to None in .init(), then set their
real value in .check(). Accessing the attribute before .check()
yields None. If we're lucky, the code that accesses the attribute
prematurely chokes on None.
It won't for .ifcond, because None is a legitimate value.
Leave the ifcond attribute undefined until check().
Suggested-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180703155648.11933-4-marcandre.lureau@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
qapi: pass 'if' condition into QAPISchemaEntity objects
Built-in objects remain unconditional. Explicitly defined objects use
the condition specified in the schema. Implicitly defined objects
inherit their condition from their users. For most of them, there is
exactly one user, so the condition to use is obvious. The exception
is wrapped types generated for simple union variants, which can be
shared by any number of simple unions. The tight condition would be
the disjunction of the conditions of these simple unions. For now,
use the wrapped type's condition instead. Much simpler and good
enough for now.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180703155648.11933-3-marcandre.lureau@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
projects / qemu-xen.git / log