Eric Blake [Fri, 24 Jun 2011 15:02:39 +0000 (09:02 -0600)]
docs: document correct flag name
When adding virDomainGetVcpusFlags in commit ea3f5c6, I did
enough rebasing that the doc comments in libvirt.c no longer
matched the final chosen enum names in libvirt.h.
And now we've gone ahead and deprecated the names
VIR_DOMAIN_VCPU_{LIVE,CONFIG}.
Eric Blake [Fri, 24 Jun 2011 04:17:19 +0000 (22:17 -0600)]
build: avoid python 2.4 build failure
I'm not sure when Py_ssize_t was introduced; but Fedora 14 Python 2.7
has it, while RHEL 5 Python 2.4 lacks it. It should be easy enough
to adjust if someone runs into problems.
* python/typewrappers.h (Py_ssize_t): Define for older python.
Eric Blake [Fri, 24 Jun 2011 03:53:08 +0000 (21:53 -0600)]
build: fix NUMA build on RHEL 5
Use NUMA's older nodemask_t (fixed-size map) rather than the newer
'struct bitmask' (variable-size) in order to still compile on RHEL 5,
with its numactl-devel-0.9.8.
* src/qemu/qemu_process.c [HAVE_NUMA]: Prefer back-compat mode.
(qemuProcessInitNumaMemoryPolicy): Use older nodemask_t.
Make sure virnetprotocol.[ch] are in BUILT_SOURCES
To ensure virnetprotocol.[ch] are generated before any other
files, add them to BUILT_SOURCES and MAINTAINERCLEANFILES.
At the same time, move ESX_DRIVER_GENERATED out of DISTCLEAN
and into MAINTAINERCLEANFILES, since they are included in
EXTRA_DIST
* src/Makefile.am: Add virnetprotocol.[ch] to BUILT_SOURCES
The Makefile.am rules for generating RPC protocol had a couple
of bugs
- A instance of remote/rpcgen_fix.pl was not changed
to rpc/genprotocol.pl
- A dep from rpc/virnetmessage.h on the generated
rpc/virnetprotocol.h was missing
- The generated rpc/virnetprotocol.[ch] were not listed
in MAINTAINERCLEANFILES
Fix job type set in qemuMigrationPrepareDirect/PrepareTunnel
The qemuMigrationPrepareDirect/PrepareTunnel methods accidentally
set the domain job to QEMU_JOB_MIGRATION_OUT when it should have
been QEMU_JOB_MIGRATION_IN. This didn't have any ill-effect, but
it is none-the-less wrong.
Avoid high privileges taint warning for QEMU session driver
The code emitting taint warnings was mistakenly thinking
that guests run from the QEMU session driver were tainted
for having high privileges. This is of course nonsense
since the session driver is always unprivileged
* src/qemu/qemu_domain.c: Don't warn for high privileges in
non-privileged QEMU
Allow automatic kill of guests when a connection is closed
If an application is using libvirt + KVM as a piece of its
internal infrastructure to perform a specific task, it can
be desirable to guarentee the VM dies when the virConnectPtr
disconnects from libvirtd. This ensures the app can't leak
any VMs it was using. Adding VIR_DOMAIN_START_AUTOKILL as
a flag when starting guests enables this to be done.
* include/libvirt/libvirt.h.in: All VIR_DOMAIN_START_AUTOKILL
* src/qemu/qemu_driver.c: Support automatic killing of guests
upon connection close
* tools/virsh.c: Add --autokill flag to 'start' and 'create'
commands
At step 2, a QEMU process is lauched in the destination to
accept the incoming migration. Occasionally the process
that is controlling the migration workflow aborts, and fails
to call step 4, Finish. This leaves a QEMU process running
on the target (albeit with paused CPUs). Unfortunately because
step 2 actives a job on the QEMU process, it is unkillable by
normal means.
By registering the VM for autokill against the src virConnectPtr
in step 2, we can ensure that the guest is forcefully killed off
if the connection is closed without step 4 being invoked
* src/qemu/qemu_migration.c: Register autokill in PrepareDirect
and PrepareTunnel. Unregister autokill on successful run
of Finish
* src/qemu/qemu_process.c: Unregister autokill when stopping a
process
Add infrastructure to automatically destroy guests when a connection closes
Sometimes it is useful to be able to automatically destroy a guest when
a connection is closed. For example, kill an incoming migration if
the client managing the migration dies. This introduces a map between
guest 'uuid' strings and virConnectPtr objects. When a connection is
closed, any associated guests are killed off.
* src/qemu/qemu_conf.h: Add autokill hash table to qemu driver
* src/qemu/qemu_process.c, src/qemu/qemu_process.h: Add APIs
for performing autokill of guests associated with a connection
* src/qemu/qemu_driver.c: Initialize autodestroy map
For controlled shutdown we issue a 'system_powerdown' command
to the QEMU monitor. This triggers an ACPI event which (most)
guest OS wire up to a controlled shutdown. There is no equiv
ACPI event to trigger a controlled reboot. This patch attempts
to fake a reboot.
- In qemuDomainObjPrivatePtr we have a bool fakeReboot
flag.
- The virDomainReboot method sets this flag and then
triggers a normal 'system_powerdown'.
- The QEMU process is started with '-no-shutdown'
so that the guest CPUs pause when it powers off the
guest
- When we receive the 'POWEROFF' event from QEMU JSON
monitor if fakeReboot is not set we invoke the
qemuProcessKill command and shutdown continues
normally
- If fakeReboot was set, we spawn a background thread
which issues 'system_reset' to perform a warm reboot
of the guest hardware. Then it issues 'cont' to
start the CPUs again
* src/qemu/qemu_command.c: Add -no-shutdown flag if
we have JSON support
* src/qemu/qemu_domain.h: Add 'fakeReboot' flag to
qemuDomainObjPrivate struct
* src/qemu/qemu_driver.c: Fake reboot using the
system_powerdown command if JSON support is available
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Add
binding for system_reset command
* src/qemu/qemu_process.c: Reset the guest & start CPUs if
fakeReboot is set
To facilitate creation of new clients using XDR RPC services,
pull alot of the remote driver code into a set of reusable
objects.
- virNetClient: Encapsulates a socket connection to a
remote RPC server. Handles all the network I/O for
reading/writing RPC messages. Delegates RPC encoding
and decoding to the registered programs
- virNetClientProgram: Handles processing and dispatch
of RPC messages for a single RPC (program,version).
A program can register to receive async events
from a client
- virNetClientStream: Handles generic I/O stream
integration to RPC layer
Each new client program now merely needs to define the list of
RPC procedures & events it wants and their handlers. It does
not need to deal with any of the network I/O functionality at
all.
Introduce generic RPC module for advertising via MDNS
Allow RPC servers to advertise themselves using MDNS,
via Avahi
* src/rpc/virnetserver.c, src/rpc/virnetserver.h: Allow
registration of MDNS services via avahi
* src/rpc/virnetserverservice.c, src/rpc/virnetserverservice.h: Add
API to fetch the listen port number
* src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Add API to
fetch the local port number
* src/rpc/virnetservermdns.c, src/rpc/virnetservermdns.h: Represent
an MDNS advertisement
To facilitate creation of new daemons providing XDR RPC services,
pull a lot of the libvirtd daemon code into a set of reusable
objects.
* virNetServer: A server contains one or more services which
accept incoming clients. It maintains the list of active
clients. It has a list of RPC programs which can be used
by clients. When clients produce a complete RPC message,
the server passes this onto the corresponding program for
handling, and queues any response back with the client.
* virNetServerClient: Encapsulates a single client connection.
All I/O for the client is handled, reading & writing RPC
messages.
* virNetServerProgram: Handles processing and dispatch of
RPC method calls for a single RPC (program,version).
Multiple programs can be registered with the server.
* virNetServerService: Encapsulates socket(s) listening for
new connections. Each service listens on a single host/port,
but may have multiple sockets if on a dual IPv4/6 host.
Each new daemon now merely has to define the list of RPC procedures
& their handlers. It does not need to deal with any network related
functionality at all.
This extends the basic virNetSocket APIs to allow them to have
a handle to the TLS/SASL session objects, once established.
This ensures that any data reads/writes are automagically
passed through the TLS/SASL encryption layers if required.
* src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Wire up
SASL/TLS encryption
Generic module for handling SASL authentication & encryption
This provides two modules for handling SASL
* virNetSASLContext provides the process-wide state, currently
just a whitelist of usernames on the server and a one time
library init call
* virNetTLSSession provides the per-connection state, ie the
SASL session itself. This also include APIs for providing
data encryption/decryption once the session is established
Generic module for handling TLS encryption and x509 certs
This provides two modules for handling TLS
* virNetTLSContext provides the process-wide state, in particular
all the x509 credentials, DH params and x509 whitelists
* virNetTLSSession provides the per-connection state, ie the
TLS session itself.
The virNetTLSContext provides APIs for validating a TLS session's
x509 credentials. The virNetTLSSession includes APIs for performing
the initial TLS handshake and sending/recving encrypted data
Introduce a generic object for using network sockets
Introduces a simple wrapper around the raw POSIX sockets APIs
and name resolution APIs. Allows for easy creation of client
and server sockets with correct usage of name resolution APIs
for protocol agnostic socket setup.
It can listen for UNIX and TCP stream sockets.
It can connect to UNIX, TCP streams directly, or indirectly
to UNIX sockets via an SSH tunnel or external command
* src/Makefile.am: Add to libvirt-net-rpc.la
* src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Generic
sockets APIs
* tests/Makefile.am: Add socket test
* tests/virnetsockettest.c: New test case
* tests/testutils.c: Avoid overriding LIBVIRT_DEBUG settings
* tests/ssh.c: Dumb helper program for SSH tunnelling tests
Provide a simple object for encoding/decoding RPC messages
This provides a new struct that contains a buffer for the RPC
message header+payload, as well as a decoded copy of the message
header. There is an API for applying a XDR encoding & decoding
of the message headers and payloads. There are also APIs for
maintaining a simple FIFO queue of message instances.
Expected usage scenarios are:
To send a message
msg = virNetMessageNew()
...fill in msg->header fields..
virNetMessageEncodeHeader(msg)
...loook at msg->header fields to determine payload filter
virNetMessageEncodePayload(msg, xdrfilter, data)
...send msg->bufferLength worth of data from buffer
To receive a message
msg = virNetMessageNew()
...read VIR_NET_MESSAGE_LEN_MAX of data into buffer
virNetMessageDecodeLength(msg)
...read msg->bufferLength-msg->bufferOffset of data into buffer
virNetMessageDecodeHeader(msg)
...look at msg->header fields to determine payload filter
virNetMessageDecodePayload(msg, xdrfilter, data)
...run payload processor
* src/Makefile.am: Add to libvirt-net-rpc.la
* src/rpc/virnetmessage.c, src/rpc/virnetmessage.h: Internal
message handling API.
* testutils.c, testutils.h: Helper for printing binary differences
* virnetmessagetest.c: Validate all XDR encoding/decoding
Defines the basics of a generic RPC protocol in XDR
This patch defines the basics of a generic RPC protocol in XDR.
This is wire ABI compatible with the original remote_protocol.x.
It takes everything except for the RPC calls / events from that
protocol
- The basic header virNetMessageHeader (aka remote_message_header)
- The error object virNetMessageError (aka remote_error)
- Two dummy objects virNetMessageDomain & virNetMessageNetwork
sadly needed to keep virNetMessageError ABI compatible with
the old remote_error
The RPC protocol supports method calls, async events and
bidirectional data streams as before
* src/Makefile.am: Add rules for generating RPC code from
protocol & define a new libvirt-net-rpc.la helper library
* src/rpc/virnetprotocol.x: New generic RPC protocol
David S. Wang [Thu, 23 Jun 2011 16:35:30 +0000 (09:35 -0700)]
documenting the 802.1Qbh parameters of a 'direct' interface
This patch adds documentation about the 802.1Qbh related parameters
of the virtualport element for 'direct' interfaces.
Signed-off-by: David S. Wang <dwang2@cisco.com> Signed-off-by: Roopa Prabhu <roprabhu@cisco.com> Signed-off-by: Christian Benvenuti <benve@cisco.com> Signed-off-by: Vasanthy Kolluri <vkolluri@cisco.com>
Eric Blake [Wed, 22 Jun 2011 14:47:11 +0000 (08:47 -0600)]
build: update to latest gnulib
Gnulib has been busy, with 397 commits; it's easier to update now
even without any known libvirt issue to be fixed, rather than
having to analyze an even larger changeset later on.
* .gnulib: Update to latest, for lots of changes.
* bootstrap: Synchronize to upstream.
Stefan Berger [Thu, 23 Jun 2011 15:54:11 +0000 (11:54 -0400)]
build: fix compilation on systems missing libnl and new includes
This patch fixes the compilation of netlink.c and interface.c on those
systems missing either libnl or that have an older linux/if_link.h
include file not supporting macvtap or VF_PORTS.
WITH_MACVTAP is '1' if newer include files were detected, '0' otherwise.
IFLA_PORT_MAX is defined in linux/if_link.h if yet more functionality is
supported.
Cole Robinson [Tue, 21 Jun 2011 00:06:49 +0000 (20:06 -0400)]
python: Generate virStreamFree but don't expose in bindings
Turns out I was right in removing this the first time :) This is
needed in our custom __del__ function, but the C code wasn't
being generated. Add new infrastructure to do what we want
Matthew Booth [Thu, 23 Jun 2011 03:28:29 +0000 (11:28 +0800)]
storage: fix volDelete return when volume still being allocated
volDelete used to return VIR_ERR_INTERNAL_ERROR when attempting to
delete a volume which was still being allocated. It should return
VIR_ERR_OPERATION_INVALID.
* src/storage/storage_driver.c: Fix return of volDelete.
Eric Blake [Wed, 22 Jun 2011 19:51:32 +0000 (13:51 -0600)]
virStorageVol: avoid PATH_MAX-sized array
POSIX allows implementations where PATH_MAX is undefined, leading
to compilation error. Not to mention that even if it is defined,
it is often wasteful in relation to the amount of data being stored.
All clients of vol->key were audited, and found not to care about
whether key is static or dynamic, except for these offenders:
Stefan Berger [Wed, 22 Jun 2011 18:28:57 +0000 (14:28 -0400)]
cleanup: make several interface functions commonly available
In a second cleanup step this patch makes several interface functions from macvtap.c commonly available by moving them into interface.c and prefixing their names with 'iface'. Those functions taking Linux-specific structures as parameters are only visible on Linux.
ifaceRestoreMacAddress returns the return code from the ifaceSetMacAddr call and display an error message if setting the MAC address did not work. The caller is unchanged and still ignores the return code (which is ok).
Stefan Berger [Wed, 22 Jun 2011 18:17:36 +0000 (14:17 -0400)]
cleanup: make nlComm commonly available
In a first cleanup step, make nlComm from macvtap.c commonly available
for other code to use. Since nlComm uses Linux-specific structures as
parameters it's prototype is only visible on Linux.
Eric Blake [Wed, 22 Jun 2011 16:40:24 +0000 (10:40 -0600)]
maint: remove syntax-check exception
We weren't using the @FOO@ notation for a Makefile substitution,
but instead for a sed rule, so using [@]FOO@ instead avoids the
need to exempt this syntax check.
* cfg.mk (_makefile_at_at_check_exceptions): Delete.
* tools/Makefile.am (virt-xml-validate, virt-pki-validate): Avoid
tripping syntax-check.
Reported by Daniel P. Berrange.
Daniel Veillard [Mon, 20 Jun 2011 03:25:34 +0000 (11:25 +0800)]
Fix to python API extractor and API doc generation
This fixes a number of issues most of them raised by Eric Blake on the
generated documentation output:
- parsing of "long long int" and similar
- add parsing of unions within a struct
- remove spurious " * " fron comments on structure fields and enums
- fix concatenation of base type and name in arrays
- extend XSLT to cope with union in structs
* docs/apibuild.py: fix and extend API extraction tool
* docs/newapi.xsl: extend the stylesheets to cope with union in
public structures
Michal Privoznik [Mon, 20 Jun 2011 08:26:47 +0000 (10:26 +0200)]
qemu: domain I/O asynchronous handling
For virtio disks and interfaces, qemu allows users to enable or disable
ioeventfd feature. This means, qemu can execute domain code, while
another thread waits for I/O event. Basically, in some cases it is win,
in some loss. This feature is available via 'ioeventfd' attribute in disk
and interface <driver> element. It accepts 'on' and 'off'. Leaving this
attribute out defaults to hypervisor decision.
Gettext reserves the empty string for internal use, and it must
not be passed through _(). We were violating this for commands
that (for whatever reason) used "" for their description.
set and restore MAC address of a NIC when using PASSTHROUGH mode
The following patch addresses the problem that when a PASSTHROUGH
mode DIRECT NIC connection is made the MAC address of the NIC is
not automatically set and reset to the configured VM MAC and
back again.
The attached patch fixes this problem by setting and resetting the MAC
while remembering the previous setting while the VM is running.
This also works if libvirtd is restarted while the VM is running.
If registering our own event loop implementation written in python,
any handles or timeouts callbacks registered by libvirt C code must
be wrapped in a python function. There is some argument trickery that
makes this all work, by wrapping the user passed opaque value in
a tuple, along with the callback function.
Problem is, the current setup requires the user's event loop to know
about this trickery, rather than just treating the opaque value
as truly opaque.
Fix this in a backwards compatible manner, and adjust the example
python event loop to do things the proper way.
Cole Robinson [Wed, 15 Jun 2011 21:54:30 +0000 (17:54 -0400)]
Promote virEvent*Handle/Timeout to public API
Since we virEventRegisterDefaultImpl is now a public API, callers need
a way to invoke the default registered Handle and Timeout functions. We
already have general functions for these internally, so promote
them to the public API.
Cole Robinson [Wed, 15 Jun 2011 20:54:16 +0000 (16:54 -0400)]
python: Implement virStreamSend/RecvAll helpers
Pure python implementation. The handler callbacks have been altered
a bit compared to the C API: RecvAll doesn't pass length of the data read
since that can be trivially obtained from python string objects, and SendAll
requires the handler to return the string data to send rather than
store the data in a string pointer.
Cole Robinson [Tue, 14 Jun 2011 20:07:43 +0000 (16:07 -0400)]
python: Implement virStreamSend/Recv
The return values for the python version are different that the C version
of virStreamSend: on success we return a string, an error raises an exception,
and if the stream would block we return int(-2). We need to do this
since strings aren't passed by reference in python.
Osier Yang [Tue, 21 Jun 2011 08:20:15 +0000 (16:20 +0800)]
conf: Make full use of goto label
* virDomainDefParse: There is a goto label "no_memory", which
reports OOM error, and then fallthrough label "error". This
patch changes things like following:
The 'char *cur' variable was being assigned from a
'const char *' string, thus discarding constness.
As well as causing a compile warning, it masked a
piece of code which attempts to assign to the
previously const string.
* tools/virsh.c: Fix const-ness of 'cur' variable in vcpupin
Matthias Bolte [Thu, 16 Jun 2011 09:30:23 +0000 (11:30 +0200)]
remote generator: Allow to annotate arrays with typecasts
Removes special case code from the generator and handle additional
methods.
The generated version of remoteDispatchDomainPinVcpu(Flags) has no
length check, but this check was useless anyway as it was applied to
data that was already deserialized from its XDR form.
Taku Izumi [Fri, 10 Jun 2011 06:41:18 +0000 (15:41 +0900)]
vcpupin: add vcpupin resetting feature to qemu driver
Pinning to all physical cpus means resetting, hence it is preferable to
delete vcpupin setting of XML.
This patch changes qemu driver to delete vcpupin setting by invoking
virDomainVcpupinDel API when pinning the specified virtual cpu to
all host physical cpus.
Taku Izumi [Fri, 10 Jun 2011 06:38:55 +0000 (15:38 +0900)]
vcpupin: improve vcpupin definition of virsh vcpupin
When using vcpupin command, we have to speficy comma-separated list as cpulist,
but this is tedious in case the number of phsycal cpus is large.
This patch improves this by introducing special markup "-" and "^" which are
similar to XML schema of "cpuset" attribute.
Osier Yang [Mon, 20 Jun 2011 07:16:16 +0000 (15:16 +0800)]
numatune: Support NUMA memory tuning in qemu driver
Implemented as setting NUMA policy between fork and exec as a hook,
using libnuma. Only support memory tuning on domain process currently.
For the nodemask out of range, will report soft warning instead of
hard error in libvirt layer. (Kernel will be silent as long as one
of set bit in the nodemask is valid on the host. E.g. For a host
has two NUMA nodes, kernel will be silent for nodemask "01010101").
So, soft warning is the only thing libvirt can do, as one might want
to specify the numa policy prior to a node that doesn't exist yet,
however, it may come as hotplug soon.
Wen Congyang [Mon, 20 Jun 2011 02:48:07 +0000 (10:48 +0800)]
build: fix building error when building without libvirtd
When building libvirt without libvirtd, we will receive the following error
message:
make[3]: Entering directory `/home/wency/rpmbuild/BUILD/libvirt-0.9.2/tools'
CC virsh-virsh.o
CC virsh-console.o
GEN virt-xml-validate
GEN virt-pki-validate
CCLD virsh
./src/.libs/libvirt.so: undefined reference to `numa_available'
./src/.libs/libvirt.so: undefined reference to `numa_max_node'
collect2: ld returned 1 exit status
The reason is that: we check numactl only when building qemu driver, and qemu
driver will not be built when bulding without libvirtd. So with_numactl's
value is check and we will not link libnuma.so.
In the other function, we call numa_available() and numa_max_node() only
when HAVE_NUMACTL is 1. We should do the same check in the function nodeGetMemoryStats().
Jamie Strandboge [Mon, 20 Jun 2011 03:53:24 +0000 (11:53 +0800)]
apparmor: implement AppArmorSetFDLabel()
During a savevm operation, libvirt will now use fd migration if qemu
supports it. When the AppArmor driver is enabled, AppArmorSetFDLabel()
is used but since this function simply returns '0', the dynamic AppArmor
profile is not updated and AppArmor blocks access to the save file. This
patch implements AppArmorSetFDLabel() to get the pathname of the file by
resolving the fd symlink in /proc, and then gives that pathname to
reload_profile(), which fixes 'virsh save' when AppArmor is enabled.
projects / libvirt.git / log