Wang King [Thu, 18 May 2017 04:06:42 +0000 (12:06 +0800)]
util: Do not leak @handles in stop netlink event service
Commit e3ba4025 introduced srv->handles and VIR_RESIZE_N to allocate
@handles as necessary, but did not free the handles during when calling
virNetlinkEventServiceStop.
Wang King [Thu, 18 May 2017 04:06:41 +0000 (12:06 +0800)]
util: Deduplicate code in virNetlinkEventServiceStopAll
Commit 15a71e60 introduced the virNetlinkEventServiceStopAll function, and
the code in virNetlinkEventServiceStop is copied to this function, so just
call virNetlinkEventServiceStop instead.
Erik Skultety [Mon, 6 Mar 2017 16:18:48 +0000 (17:18 +0100)]
nodedev: Introduce mdev capability for mediated devices
Start discovering the mediated devices on the host system and format the
attributes for the mediated device into the XML. Compared to the parent
device which reports generic information about the abstract mediated
devices types, a child device only reports the type name it has been
instantiated from and the IOMMU group number, since that's device
specific compared to the rest of the info that can be gathered about
mediated devices at the moment.
This patch introduces both the formatting and parsing routines, updates
nodedev.rng schema, adding a testcase as well.
Erik Skultety [Mon, 6 Mar 2017 16:17:34 +0000 (17:17 +0100)]
nodedev: Introduce the mdev capability to a PCI parent device
The parent device needs to report the generic stuff about the supported
mediated devices types, like device API, available instances, type name,
etc. Therefore this patch introduces a new nested capability element of
type 'mdev_types' with the resulting XML of the following format:
Erik Skultety [Mon, 6 Mar 2017 16:20:00 +0000 (17:20 +0100)]
nodedev: Introduce new mdev_types and mdev nodedev capabilities
The reason for introducing two capabilities, one for the device itself
(cap 'mdev') and one for the parent device listing the available types
('mdev_types'), is that we should be able to do
'virsh nodedev-list --cap' not only for existing mdev devices but also
for devices that support creation of mdev devices, since one day libvirt
might be actually able to create the mdev devices in an automated way
(just like we do for NPIV/vHBA).
Erik Skultety [Wed, 26 Apr 2017 12:47:15 +0000 (14:47 +0200)]
mdev: Pass a uuidstr rather than an mdev object to some util functions
Namely, this patch is about virMediatedDeviceGetIOMMUGroup{Dev,Num}
functions. There's no compelling reason why these functions should take
an object, on the contrary, having to create an object every time one
needs to query the IOMMU group number, discarding the object afterwards,
seems odd.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Commit 8e09663 "pci: recognize/report GEN4 (PCIe 4.0) card 16GT/s Link
speed" introduced another speed into enum, but mistakenly also altered
field width, so one bit of link width was included there.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Michal Privoznik [Wed, 27 Apr 2016 12:21:10 +0000 (14:21 +0200)]
virsh: Implement sparse stream to vol-upload
Similarly to previous commit, implement sparse streams feature
for vol-upload. This is, however, slightly different approach,
because we must implement a function that will tell us whether
we are in a data section or in a hole. But there's no magic
hidden in here.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Tue, 12 Apr 2016 13:35:04 +0000 (15:35 +0200)]
virsh: Implement sparse stream to vol-download
Add a new --sparse switch that does nothing more than
enables the sparse streams feature for this command. Among with
the switch new helper function is introduced: virshStreamSkip().
This is the callback that is called whenever daemon sends us a
hole. In the callback we reflect the hole in underlying file by
seeking as many bytes as told.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
These flags to APIs will tell if caller wants to use sparse
stream for storage transfer. At the same time, it's safe to
enable them in storage driver frontend and rely on our backends
checking the flags. This way we can enable specific flags only on
some specific backends, e.g. enable
VIR_STORAGE_VOL_DOWNLOAD_SPARSE_STREAM for filesystem backend but
not iSCSI backend.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Tue, 12 Apr 2016 13:17:30 +0000 (15:17 +0200)]
gendispatch: Introduce @sparseflag for our calls
Now, not all APIs are going to support sparse streams. To some it
makes no sense at all, e.g. virDomainOpenConsole() or
virDomainOpenChannel(). To others, we will need a special flag to
indicate that client wants to enable sparse streams. Instead of
having to write RPC dispatchers by hand we can just annotate in
our .x files that a certain flag to certain RPC call enables this
feature. For instance:
Michal Privoznik [Wed, 27 Apr 2016 08:51:46 +0000 (10:51 +0200)]
fdstream: Implement sparse stream
Basically, what is needed here is to introduce new message type
for the messages passed between the event loop callbacks and the
worker thread that does all the I/O. The idea is that instead of
a queue of read buffers we will have a queue where "hole of size
X" messages appear. That way the event loop callbacks can just
check the head of the queue and see if the worker thread is in
data or a hole section and how long the section is.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Mon, 11 Apr 2016 09:58:19 +0000 (11:58 +0200)]
daemonStreamHandleRead: Wire up seekable stream
Whenever client is able to receive some data from stream
daemonStreamHandleRead is called. But now the behaviour of this
function needs to be changed a bit. Previously it just read data
from underlying file (of chardev or whatever) and sent those
through the stream to client. This model will not work any longer
because it does not differentiate whether underlying file is in
data or hole section. Therefore, at the beginning of this
function add code that checks this situation and acts
accordingly.
So after the this, when wanting to send some data we always check
whether we are not in a hole and if so, skip it an inform client
about its size.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Fri, 20 May 2016 14:35:13 +0000 (16:35 +0200)]
virNetClientStream: Wire up VIR_NET_STREAM_HOLE
Whenever server sends a client stream packet (either regular with
actual data or stream skip one) it is queued on @st->rx. So the
list is a mixture of both types of stream packets. So now that we
have all the helpers needed we can wire their processing up. But
since virNetClientStreamRecvPacket doesn't support
VIR_STREAM_RECV_STOP_AT_HOLE flag yet, let's turn all received
skips into zeroes repeating requested times.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Sun, 10 Apr 2016 13:48:27 +0000 (15:48 +0200)]
remote_driver: Implement virStreamSendHole
Now that we have RPC wrappers over VIR_NET_STREAM_HOLE we can
start wiring them up. This commit wires up situation when a
client wants to send a hole to daemon.
To keep stream offsets synchronous, upon successful call on the
daemon skip the same hole in local part of the stream.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
This is a function that handles an incoming STREAM_HOLE packet.
Even though it is not wired up yet, it will be soon. At the
beginning do couple of checks whether server plays nicely and
sent us a STREAM_HOLE packed only after we've enabled sparse
streams. Then decodes the message payload to see how big the hole
is and stores it in passed @length argument.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Fri, 12 May 2017 09:26:33 +0000 (11:26 +0200)]
daemon: Implement VIR_NET_STREAM_HOLE handling
Basically, whenever the new type of stream packet arrives to the
daemon call this function that decodes it and calls
virStreamSendHole(). Otherwise a regular data stream packet has
arrived and therefore continue its processing.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Wed, 13 Apr 2016 14:38:09 +0000 (16:38 +0200)]
Teach wireshark plugin about VIR_NET_STREAM_HOLE
Ideally, this would be generated, but to achieve that
corresponding XDR definitions needed to go into a different .x
file. But they belong just to the one that they are right now.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
This is a special type of stream packet, that is bidirectional
and contains information regarding how many bytes each side will
be skipping in the stream.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Mon, 11 Apr 2016 14:50:04 +0000 (16:50 +0200)]
Add new flag to daemonCreateClientStream and virNetClientStreamNew
Add a new argument to daemonCreateClientStream in order to allow for
future expansion to mark that a specific stream can be used to skip
data, such as the case with sparsely populated files. The new flag will
be the eventual decision point between client/server to decide whether
both ends can support and want to use sparse streams.
A new bool 'allowSkip' is added to both _virNetClientStream and
daemonClientStream in order to perform the tracking.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Sat, 21 May 2016 12:25:14 +0000 (14:25 +0200)]
Introduce virStreamInData
This is just an internal API, that calls corresponding function
in stream driver. This function will set @data = 1 if the
underlying file is in data section, or @data = 0 if it is in a
hole. At any rate, @length is set to number of bytes remaining in
the section the file currently is.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Sat, 21 May 2016 13:47:39 +0000 (15:47 +0200)]
Introduce virStreamSparseSendAll
This is just a wrapper over new function that have been just
introduced: virStreamSendHole() . It's very similar to
virStreamSendAll() except it handles sparse streams well.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Sat, 21 May 2016 13:17:51 +0000 (15:17 +0200)]
Introduce virStreamSparseRecvAll
This is just a wrapper over new functions that have been just
introduced: virStreamRecvFlags(), virStreamRecvHole(). It's very
similar to virStreamRecvAll() except it handles sparse streams
well.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Fri, 20 May 2016 13:12:59 +0000 (15:12 +0200)]
Introduce VIR_STREAM_RECV_STOP_AT_HOLE flag
Add a new flag to virStreamRecvFlags in order to handle being able to
stop reading from the stream so that the consumer can generate a "hole"
in stream target. Generation of a hole replaces the need to receive and
handle a sequence of zero bytes for sparse stream targets.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Fri, 20 May 2016 13:03:11 +0000 (15:03 +0200)]
Introduce virStreamRecvHole
This function is basically a counterpart for virStreamSendHole().
If one side of a stream called virStreamSendHole() the other
should call virStreamRecvHole() to get the size of the hole.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Sun, 10 Apr 2016 10:01:20 +0000 (12:01 +0200)]
Introduce virStreamSendHole
This API is used to tell the other side of the stream to skip
some bytes in the stream. This can be used to create a sparse
file on the receiving side of a stream.
It takes @length argument, which says how big the hole is. This
skipping is done from the current point of stream. Since our
streams are not rewindable like regular files, we don't need
@whence argument like seek(2) has.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Thu, 19 May 2016 14:53:35 +0000 (16:53 +0200)]
Introduce virStreamRecvFlags
This patch is adding the virStreamRecvFlags as a variant to the
virStreamRecv function in order to allow for future expansion of
functionality for processing sparse streams using a @flags
argument.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Thu, 16 Jun 2016 08:42:47 +0000 (10:42 +0200)]
util: Introduce virFileInData
This function takes a FD and determines whether the current
position is in data section or in a hole. In addition to that,
it also determines how much bytes are there remaining till the
current section ends.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Thu, 13 Apr 2017 09:49:30 +0000 (11:49 +0200)]
virfdstream: Use messages instead of pipe
One big downside of using the pipe to transfer the data is that
we can really transfer just bare data. No metadata can be carried
through unless some formatted messages are introduced. That would
be quite painful to achieve so let's use a message queue. It's
fairly easy to exchange info between threads now that iohelper is
no longer used.
The reason why we cannot use the FD for plain files directly is
that despite us setting noblock flag on the FD, any
read()/write() blocks regardless (which is a show stopper since
those parts of the code are run from the event loop) and poll()
reports such FD as always readable/writable - even though the
subsequent operation might block.
The pipe is still not gone though. It is used to signal the event
loop that an event occurred (e.g. data is available for reading
in the queue, or vice versa).
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Gordon Messmer [Fri, 12 May 2017 21:27:27 +0000 (14:27 -0700)]
virsh: add --io when attaching disks to guests
virt-install and virt-manager both default to explicitly setting
"io='native'" in the disk "driver" tag. virsh, however, does not and also
does not provide an option to specify that setting at all. As a result,
disks use a different IO mechanism (the default, "threads") when attached
post-setup using virsh. Adding this option allows users to keep disk
performance consistent for disks attached at install, and those attached
afterward.
Peter Krempa [Wed, 17 May 2017 15:16:15 +0000 (17:16 +0200)]
qemu: driver: Allow passing disk target as top image with block commit
Since we allow active layer block commit the users are allowed to commit
the top of the chain (e.g. vda) into the backing image. The API would
not accept that parameter, as it tried to look up the image in the
backing chain.
Add the ability to use the top level image target name explicitly as the
top image of the block commit operation.
Andrea Bolognani [Fri, 12 May 2017 12:38:08 +0000 (14:38 +0200)]
gic: Remove VIR_GIC_VERSION_DEFAULT
The QEMU default is GICv2, and some of the code in libvirt
relies on the exact value. Stop pretending that's not the
case and use GICv2 explicitly where needed.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Pavel Hrdina [Thu, 11 May 2017 12:09:35 +0000 (14:09 +0200)]
qemu: improve detection of UNIX path generated by libvirt
Currently we consider all UNIX paths with specific prefix as generated
by libvirt, but that's a wrong assumption. Let's make the detection
better by actually checking whether the whole path matches one of the
paths that we generate or generated in the past.
The UNIX path isn't stored in config XML since libvirt-1.3.1.
The debian etch distro was end-of-life a long time ago so we no
longer need the ULLONG_MAX hack. In any case gnulib now provides
an equivalent fix by default, and so our definition now triggers
syntax-check rule failure
src/internal.h:# define ULLONG_MAX ULONG_LONG_MAX
maint.mk: define the above via some gnulib .h file
maint.mk:843: recipe for target 'sc_prohibit_always-defined_macros' failed
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Ján Tomko [Fri, 17 Mar 2017 07:26:49 +0000 (08:26 +0100)]
qemu: format kernel_irqchip on the command line
Add kernel_irqchip=split/on to the QEMU command line
and a capability that looks for it in query-command-line-options
output. For the 'split' option, use a version check
since it cannot be reasonably probed.
If a shutdown is expected because it was triggered via libvirt we can
also expect the monitor to close. In those cases do not report an
internal error like:
"internal error: End of file from qemu monitor"
Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
Pavel Hrdina [Thu, 11 May 2017 15:16:13 +0000 (17:16 +0200)]
util: introduce virStringMatch
Simply tries to match the provided regex on a string and returns
the result. Useful if caller don't care about the matched substring
and want to just test if some pattern patches a string.
John Ferlan [Fri, 12 May 2017 10:46:20 +0000 (06:46 -0400)]
conf: Fix resource leak in virCapabilitiesInitCaches
The @type from virFileReadValueString needs to be VIR_FREE each time
through the loop since it's not saved and since cleanup can be reached
prior to decoding it for @kernel_type amd bank->type, the cleanup code
needs to also have a VIR_FREE
Andrea Bolognani [Thu, 11 May 2017 16:28:00 +0000 (18:28 +0200)]
spec: Support maintenance releases on mingw
The regular spec file contains code to deal with the fact
that maintenance releases are uploaded to their own
directory: copy it over to the mingw spec file so that it's
possible to build maintenance releases there as well.
This also switches the source URL from FTP to HTTP for
consistency with the main spec file.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
The Win32 platform can not do link time overrides in the same way
that we can on POSIX / ELF based platforms, so we cannot build
the virfilewrapper.c code reliably. Just stub it out on Win32
so it is a no-op. Tests that use this file are already written
to skip on Win32.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
The test programs depend on virfilewrapper.h as well as the
virfilewrapper.c. Adding the dep ensures that virfilewrapper.h
gets included in the dist tarball.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
GCC complains that inlining virStringTrimOptionalNewline is not
likely on some platforms:
cc1: warnings being treated as errors
../../src/util/virfile.c: In function 'virFileReadValueBitmap':
../../src/util/virstring.h:292: error: inlining failed in call to 'virStringTrimOptionalNewline': call is unlikely and code size would grow [-Winline]
../../src/util/virfile.c:3987: error: called from here [-Winline]
Inlining this function is not going to be a measurable performance
benefit either, since the time required to execute it is going to
be dominated by running of strlen() over the string, not by the
function call overhead.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Cole Robinson [Fri, 5 May 2017 00:08:55 +0000 (20:08 -0400)]
spec: Update version check for maint Source URL
New maint release version numbers of just A.B.C format, not the old
A.B.C.D format. Adjust the check that dynamically changes the Source
URL for maint releases
Signed-off-by: Cole Robinson <crobinso@redhat.com> Acked-by: Andrea Bolognani <abologna@redhat.com>
If __lxstat() and __xstat() functions are not available, build fails with:
CC virfilewrapper.o
virfilewrapper.c:180:5: error: no previous prototype for function '__lxstat' [-Werror,-Wmissing-prototypes]
int __lxstat(int ver, const char *path, struct stat *sb)
^
virfilewrapper.c:208:5: error: no previous prototype for function '__xstat' [-Werror,-Wmissing-prototypes]
int __xstat(int ver, const char *path, struct stat *sb)
Luckily, we already check presence of these functions in configure
using AC_CHECK_FUNCS, so just don't wrap these if they're not available.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Added only in drivers that were already calling
virCapabilitiesInitNUMA(). Instead of refactoring all the callers to
behave the same way in case of error, just follow what the callers are
doing for all the functions.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
We're only adding only info about L3 caches, we can add more
later (just by changing one line), but for now that's more than enough
without overwhelming anyone.
XML snippet of how this should look like (also seen as part of the commit):
util: Remove virsysfs and instead enhance virFileReadValue* functions
It is no longer needed thanks to the great virfilewrapper.c. And this
way we don't have to add a new set of functions for each prefixed
path.
While on that, add two functions that weren't there before, string and
scaled integer reading ones. Also increase the length of the string
being read by one to accompany for the optional newline at the
end (i.e. change INT_STRLEN_BOUND to INT_BUFSIZE_BOUND).
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
This mock (which is actually not mock at all, see later) can redirect
all accesses to a path into another path. There is no need to
create mocks for particular directories, you just create a directory
with all the data a redirect the test there.
In the future, this should also be able to register callbacks for
calls/paths, e.g. when the test is going to write into anything under
"/sys/devices", call function fce(); Then in the open() call we would
add information about the fd into some structure and in write() we
would call fce() with parameters like @path to write to, @data to
be written and pointer to optional return value, so that fce() itself
could stop the call from happening or change its behaviour. But
that's an idea for a latter day.
This is not a mock because it will not be preloaded, but compiled in
the test itself. See future patches for usage.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
So, because mingw is somehow OK with dereferencing a pointer within a
VIR_DEBUG macro, compared to outside of it to which it complained with a
"potential NULL pointer dereference" error (still a false positive), we
can make the code a tiny bit cleaner.
Sighed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Erik Skultety <eskultet@redhat.com>
Andrea Bolognani [Fri, 21 Apr 2017 14:47:02 +0000 (16:47 +0200)]
Add YouCompleteMe support
YouCompleteMe[1] is a vim plugin that implements semantic
code completion using libclang.
For non-trivial projects such as libvirt, the plugin needs
some help figuring out where to find the various header
files: generate its configuration file at configure time
so that the plugin works out of the box.
Andrea Bolognani [Fri, 21 Apr 2017 14:05:19 +0000 (16:05 +0200)]
Add color_coded support
color_coded[1] is a vim plugin that implements semantic
syntax highlighting using libclang.
For non-trivial projects such as libvirt, the plugin needs
some help figuring out where to find the various header
files: generate its configuration file at configure time
so that the plugin works out of the box.
The API docs extractor, ESX code generator and keycodemapdb tools
rely on python. Historically every platform that this present, but
with switch to Python3 by default, we're increasingly seeing
installs without a /usr/bin/python.
This tightens up the check during configure, so it exits immediately
if python is missing, rather than leaving an empty $(PYTHON) make
variable which leads to more obscure errors later.
Also add it as a build dep for Mingw, since Fedora build roots no
longer get python2 by default. This was not previously a major
problem, since both ESX & API generated files were included in
EXTRA_DIST, but the keycodemapdb generated files are not, so we
require python all the time now.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
When adding a nwfilter onto the list in
virNWFilterObjListAssignDef() this array is re-allocated to match
demand for new size. However, it is never freed leading to a
leak:
==26535== 136 bytes in 1 blocks are definitely lost in loss record 1,079 of 1,250
==26535== at 0x4C2E2BE: realloc (vg_replace_malloc.c:785)
==26535== by 0x54BA28E: virReallocN (viralloc.c:245)
==26535== by 0x54BA384: virExpandN (viralloc.c:294)
==26535== by 0x54BA657: virInsertElementsN (viralloc.c:436)
==26535== by 0x55DB011: virNWFilterObjListAssignDef (virnwfilterobj.c:362)
==26535== by 0x55DB530: virNWFilterObjListLoadConfig (virnwfilterobj.c:503)
==26535== by 0x55DB635: virNWFilterObjListLoadAllConfigs (virnwfilterobj.c:539)
==26535== by 0x2AC5A28B: nwfilterStateInitialize (nwfilter_driver.c:250)
==26535== by 0x5621C64: virStateInitialize (libvirt.c:770)
==26535== by 0x124379: daemonRunStateInit (libvirtd.c:881)
==26535== by 0x554AC78: virThreadHelper (virthread.c:206)
==26535== by 0x8F5F493: start_thread (in /lib64/libpthread-2.23.so)
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
projects / libvirt.git / log