Michal Privoznik [Sat, 28 May 2016 11:21:43 +0000 (13:21 +0200)]
docs: Teach apibuild to deal with (1U << 31) too
The apibuild script is a terrifying beast that parses some source
files of ours and produces an XML representation of them. When it
comes to parsing enums we have in some header files, it tries to
be clever and detect a value that an enum member has (or if it is
an alias for a different member). Whilst doing that it has to
deal with values we give to the members in many formats. At some
places we just pass the value in decimal:
The script tries to parse all of these resulting in the following
tokens: "1", "VIR_CONNECT_LIST_DOMAINS_ACTIVE", "1<<31"; Then, the
script tries to turn these into integers using python's eval()
function. This function succeeds on the first and the last
tokens. But, if we were to modify the last example so that it's
of the following form:
the token representing enum's member value will then be "1U<<31".
So our parsing is good. Unfortunately, python is not aware of the
difference between signed and unsigned C types, therefore eval()
fails over this token and the parser falls back thinking it's an
alias to another enum member. Well it's not.
The solution is to transform [0-9]U into [0-9] as for our
purposes here it's the same thing.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Eric Blake [Mon, 4 Jan 2016 20:45:51 +0000 (13:45 -0700)]
maint: update to latest gnulib
Fix a regression in checking for realpath (which caused link
failures regarding duplicate rpl_canonicalize_file_name), and
fix the mingw build regarding unsetenv.
Cole Robinson [Fri, 20 May 2016 19:50:16 +0000 (15:50 -0400)]
spec: Advertise nvram paths of official fedora edk2 builds
Fedora now ships edk2 firmware in its official repos, so adapt
the nvram path list to match. Eventually we can remove the nightly
links as well once some integration kinks have been worked out,
and documentation updated.
Move the macro building into the %build target, which lets us
build up a shell variable and make things a bit more readable
Katerina Koukiou [Fri, 27 May 2016 09:23:20 +0000 (12:23 +0300)]
lxc: Fix virLXCDomainObjBeginJob position in lxcDomainSetMemoryParameters
Adjust the code to perform the virLXCDomainObjBeginJob first
and then the call virDomainLiveConfigHelperMethod.
As Ján Tomko pointed out, in virDomainLiveConfigHelperMethod,
there is a check to see if the domain is active when AFFECT_LIVE is set.
Since virLXCDomainObjBeginJob unlocks the virDomainObjPtr lock,
the domain could possibly be destroyed while we wait for the job
and the check results would no longer be valid.
Eric Blake [Mon, 4 Jan 2016 20:45:51 +0000 (13:45 -0700)]
maint: update to latest gnulib
Pulls in several portability fixes, including the fact that gnulib
now only works on platforms with two's complement signed integers.
Also makes for a smaller delta on the next update (we are waiting
on a license change to unsetenv for the sake of mingw).
* .gnulib: Update to latest.
* bootstrap: Resync from upstream.
* tests/virstringtest.c: Drop use of obsolete probes of integer
properties.
Dawid Zamirski [Thu, 26 May 2016 15:30:11 +0000 (11:30 -0400)]
esx: do not store escaped password in esxVI_Context.
This patch fixes an issue where screenshot API call was failing when
the esx/vcenter password contains special characters such as
apostrophee. The reason for failures was that passwords were escaped
for XML and stored in esxVI_Context which was then passed to raw CURL
API calls where the password must be passed in original form to
authenticate successfully. So this patch addresses this by storing
original passwords in the esxVI_Context struct and escape only for
esxVI_Login call.
Andrea Bolognani [Mon, 23 May 2016 16:00:36 +0000 (18:00 +0200)]
qemu: Fix error message when PCI bridge has index <= bus
Commit ff2126225df0 changed the error message to be more
detailed about the failure at hand; however, while the new
error message claims that "bus must be <= index", the error
message is displayed if "idx <= addr->bus", ie. when bus
is larger than or *equal to* index.
Change the error message to report the correct constraint,
and format it in a way that mirrors the check exactly to
make it clearer to people reading the code. The new error
message reads "index must be larger than bus".
daemon: cleanup state drivers in order reverse to init order
This patch aims to fix observed crash on daemon shutdown. Main thread is in
the process of state drivers cleanup, network driver is cleaned up and
qemu driver is not yet. Meanwhile eof event from qemu process triggers
qemuProcessStop -> networkReleaseActualDevice and crash happens as
network driver is already cleaned up.
Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
Dawid Zamirski [Mon, 23 May 2016 21:22:55 +0000 (17:22 -0400)]
esx: use newer virtualHW version for 5.1+ hosts
This is because there's a known issue where ESX will refuse to attach
drives bigger than 4TB when virtualHW < 9. Therefore, to avoid that
use the higher virtualHW for hosts that support it.
Dawid Zamirski [Mon, 23 May 2016 21:22:53 +0000 (17:22 -0400)]
esx: add pciBridge devices when SCSI is used
When a SCSI controller is present, ESX adds several pciBridge devices
to vmx file. This fixes an error message where it refuses to create VM
due to not enough PCI devices available. This applies only to virtualHW
version >= 7.
Laine Stump [Tue, 10 May 2016 17:14:32 +0000 (13:14 -0400)]
conf: permit auto-assignment of controller indexes
Hand-entering indexes for 20 PCI controllers is not as tedious as
manually determining and entering their PCI addresses, but it's still
annoying, and the algorithm for determining the proper index is
incredibly simple (in all cases except one) - just pick the lowest
unused index.
The one exception is USB2 controllers because multiple controllers in
the same group have the same index. For these we look to see if 1) the
most recently added USB controller is also a USB2 controller, and 2)
the group *that* controller belongs to doesn't yet have a controller
of the exact model we're just now adding - if both are true, the new
controller gets the same index, but in all other cases we just assign
the lowest unused index.
With this patch in place and combined with the automatic PCI address
assignment, we can define a PCIe switch with several ports like this:
Laine Stump [Sun, 8 May 2016 20:28:40 +0000 (16:28 -0400)]
conf: make virDomainControllerFindUnusedIndex() more generally usable
Make virDomainControllerFindUnusedIndex() a global function so that it
can be used outside domain_conf.c (as well as higher up in
domain_conf.c itself)/ Also make its DomainDef arg a const* so that
functions which only have a const* to the domain can use it.
Laine Stump [Sun, 8 May 2016 19:56:23 +0000 (15:56 -0400)]
conf/qemu: make IS_USB2_CONTROLLER globally available
IS_USB2_CONTROLLER() is useful in more places aside from just when
assigning PCI addresses in QEMU, and is checking for enum values that
are all defined in conf/domain_conf.h anyway, so define it there
instead.
Chunyan Liu [Fri, 13 May 2016 06:21:23 +0000 (14:21 +0800)]
libxl: add .domainInterfaceAddresses
Add .domainInterfaceAddresses so that user can have a way to
get domain interface address by 'virsh domifaddr'. Currently
it only supports '--source lease'.
Jim Fehlig [Mon, 23 May 2016 21:56:01 +0000 (15:56 -0600)]
libxl: default to qemu driver for network disks
Xen only supports network-based disks with the qemu (aka qdisk) driver.
Set the driverName to 'qemu' in libxlDomainDeviceDefPostParse() if
not already set. When starting a domain with network-based disks,
ensure the driverName is 'qemu'.
Peter Krempa [Mon, 23 May 2016 14:08:19 +0000 (16:08 +0200)]
qemu: driver: Allow disk update of startupPolicy/snapshot for all disks
The libvirt internal bits can be changed for disks that don't otherwise
support changing media. Remove the switch statement and allow changes of
non-source data for all disks.
Peter Krempa [Mon, 23 May 2016 13:54:11 +0000 (15:54 +0200)]
qemu: driver: Move around code to avoid need to rollback
qemuDomainChangeDiskLive rolled back few changes to the disk definition
if changing of the media failed. This can be avoided by moving some code
around.
John Ferlan [Fri, 20 May 2016 12:42:29 +0000 (08:42 -0400)]
lxc: Fix lxcDomainDestroyFlags endjob processing
Commit id '15ccb0dbf' added job functions for the lxc driver; however,
for shutdown and nonpersistent path, the vm was removed from the domain
object list and the vm pointer cleared before the endjob.
Adjust the code to perform the endjob first and then perform the
ObjListRemove as long as the vm wasn't NULL. This follows more closely
models from qemu and libxl
Where it was set to 0 on domain startup if qemu did not support the
QEMUD_CMD_FLAG_DEVICE capability, to clear the addresses at shutdown,
because QEMU might make up different ones next time.
As of commit f5dd58a6088cfc6e8bd354b693d399807a8ec395
CommitDate: 2012-07-11 11:19:05 +0200
qemu: Extended qemuDomainAssignAddresses to be callable from
everywhere.
this was broken, when the persistentAddrs = 0 assignment was moved
inside qemuDomainAssignPCIAddresses and while it pretends to check
for !QEMU_CAPS_DEVICE, its parent qemuDomainAssignAddresses is only
called if QEMU_CAPS_DEVICE is present.
John Ferlan [Fri, 20 May 2016 11:41:09 +0000 (07:41 -0400)]
qemu: Remove dead code
Since commit id '20a0fa8e' removed the QEMU_CAPS_DEVICE, Coverity notes
that it's no longer possible to have 'addrs' be NULL when checking for
a live domain since qemuDomainPCIAddressSetCreate would have jumped to
cleanup if addrs was NULL.
Andrea Bolognani [Mon, 23 May 2016 09:54:44 +0000 (11:54 +0200)]
conf: nodedev: Set PCI_PHYSICAL_FUNCTION flag more carefully
Instead of setting the flag before parsing the PCI address, set
it afterwards. This ensure we can never end up in a situation
where the flag has been set but pci_dev.physical_function has
not been filled in.
Andrea Bolognani [Wed, 25 May 2016 08:01:58 +0000 (10:01 +0200)]
pci: Fix virPCIGetPhysicalFunction()'s callers
Commit c8b1a83605e4 changed the function, making it
impossible for callers to be able to tell whether a
non-negative return value means "physical function
address found and parsed correctly" or "couldn't find
corresponding physical function".
The important difference between the two being that,
in the latter case, the returned pointer is NULL and
should never, ever be dereferenced.
In order to cope with these changes, the callers
have to be updated.
Peter Krempa [Mon, 23 May 2016 14:32:06 +0000 (16:32 +0200)]
qemu: hotplug: Fix error reported when cdrom tray is locked
Commit 1fad65d49aae364576bd91352a001249510f8d4e used a really big hammer
and overwrote the error message that might be reported by qemu if the
tray is locked. Fix it by reporting the error only if no error is
currently set.
Error after commit mentioned above:
error: internal error: timed out waiting for disk tray status update
New error:
error: internal error: unable to execute QEMU command 'eject': Tray of
device 'drive-ide0-0-0' is not open
Peter Krempa [Thu, 19 May 2016 13:29:02 +0000 (15:29 +0200)]
qemu: process: Fix and improve disk data extraction
Extract information for all disks and update tray state and source only
for removable drives. Additionally store whether a drive is removable
and whether it has a tray.
Joao Martins [Tue, 24 May 2016 10:56:08 +0000 (11:56 +0100)]
xenconfig: xm: check for driver on disk format
When reviewing libxl vif typename series[0] I found a bug
on xen-xm formatter where "virsh domxml-to-native xen-xm file.xml"
can lead to a NULL dereference if the disk driver isn't specified.
Fix this by checking for driver before writing/testing it down.
Laine Stump [Fri, 13 May 2016 17:20:54 +0000 (13:20 -0400)]
lxc: support <interface type='ethernet'>
This is identical to type='bridge', but without the "connect to a
bridge" part, so it can be handled by using the same functions (and
often even the same cases in switch statements), after renaming
virLXCProcessSetupInterfaceBridged() to virLXCProcessInterfaceTap()
and enhancing it to skip bridge-related items when brname == NULL.
To be truly useful, we need to support setting the ip address on the
host side veth as well as guest side veth (already supported for
type='bridge'), as well as setting the peer address for both.
The <script> element (supported by type='ethernet' in qemu) isn't
supported in this patch. An error is logged at domain start time if it
is encountered. This may be changed in a later patch.
Katerina Koukiou [Fri, 20 May 2016 15:17:01 +0000 (18:17 +0300)]
lxc: completely rework reference counting
This patch follows the pattern used in qemu driver regarding
reference counting.
It changes lxcDomObjFromDomain() to ref the domain (using
virDomainObjListFindByUUIDRef()) and adds virDomainObjEndAPI() which
should be the only function in which the return value of
virObjectUnref() is checked. This makes all reference counting
deterministic and makes the code a bit clearer.
The problem is fixed by unlocking interface before calling
virNWFilterInstantiateFilterLate to avoid updateMutex and interface ordering
deadlocks. Otherwise we are going to instantiate the filter while holding
interface lock, which will try to lock updateMutex, and if some other thread
instantiating a filter in parallel is holding updateMutex and is trying to
lock interface, both will deadlock.
Also it is safe to unlock interface before virNWFilterInstantiateFilterLate
because learnIPAddressThread stopped capturing packets and applied necessary
rules on the interface, while instantiating a new filter doesn't require a
locked interface.
Fix the regex for excluding files for this syntax-rule. The rule "include/"
will not work, because we are matching the whole line like this
"^(...|include/|...)$ so we need to use "include/libvirt/libvirt.+". The second
issue is that we are using only one '$' but there should be two of those at the
end. The last small adjustment is to escape dots '.' so it match only dot.
Pavel Hrdina [Sun, 22 May 2016 13:23:08 +0000 (15:23 +0200)]
makefile: fix build on systems where gnutls is not in /usr/include
We need to append GNUTLS_CFLAGS while building utils because virtcrypto
is using it. This fixes build on freebsd where gnutuls is in
/usr/local/include.
Pavel Hrdina [Sun, 22 May 2016 13:20:39 +0000 (15:20 +0200)]
qemuxml2argvtest: skip test that depends on gnutls_cipher_encrypt()
Test disk-drive-network-rbd-auth-AES depends on existence of
gnutls_cipher_encrypt() function which was introduced in gnutls 2.10.0.
On systems without this function we should skip this test.
vz: add error code for case if vm is already stopped
If try to stop VM or container which is already stopped than
Virtuozzo 7 returns code PRL_ERR_INVALID_ACTION_REQUESTED.
Error code PRL_ERR_DISP_VM_IS_NOT_STARTED is used in Virtuozzo 6
Ján Tomko [Fri, 20 May 2016 12:02:55 +0000 (14:02 +0200)]
tests: remove disk-drive-fat test
This test requests a read-only virtual FAT drive on the IDE bus.
Read-only IDE drives are unsupported, but libvirt only displays
the error if it has the QEMU_CAPS_DRIVE_READONLY capability.
Read-write FAT drives are also unsupported.
Laine Stump [Tue, 17 May 2016 18:08:13 +0000 (14:08 -0400)]
qemu: auto-assign addresses when <address type='pci'/> is specified
Rather than only assigning a PCI address when no address is given at
all, also do it when the config says that the address type is 'pci',
but it gives no address (virDeviceInfoPCIAddressWanted()).
There are also several places after parsing but prior to address
assignment where code previously expected that any info with address
type='pci' would have a *valid* PCI address, which isn't always the
case - now we check not only for type='pci', but also for a valid
address (virDeviceInfoPCIAddressPresent()).
The test case added in this patch was directly copied from Cole's patch titled:
Laine Stump [Tue, 17 May 2016 18:06:36 +0000 (14:06 -0400)]
bhyve: auto-assign addresses when <address type='pci'/> is specified
Rather than only assigning a PCI address when no address is given at
all, also do it when the config says that the address type is 'pci',
but it gives no address.
Laine Stump [Tue, 17 May 2016 18:03:00 +0000 (14:03 -0400)]
conf: allow type='pci' addresses with no address attributes specified
Prior to this, <address type='pci'/> wasn't allowed when parsing
(domain+bus+slot+function needed to be a "valid" PCI address, meaning
that at least one of domain/bus/slot had to be non-0), the RNG
required bus to be specified, and if type was set to PCI when
formatting, domain+bus+slot+function would always be output.
This makes all the address attributes optional during parse and RNG
validation, and suppresses domain+bus+slot+function if domain+bus+slot
are all 0 (NB: if d+b+s are all 0, any value for function is
nonsensical as that will never happen in the real world, and after
the next patch we will always assign a real working address to any
empty PCI address before it is ever output to anywhere).
Note that explicitly setting all attributes to 0 is equivalent to
setting none of them, which is okay, since 0000:00:00 is reserved in
any PCI bus setup, and can't be used anyway.
Laine Stump [Wed, 18 May 2016 16:04:42 +0000 (12:04 -0400)]
conf: new functions to check if PCI address is wanted/present
In order to allow <address type='pci'/> with no other attributes to
mean "I want a PCI address, but any PCI address will do" (just as
having no <address> at all usually indicates), we will need to change
several places in the code from a simple "info->type == (or !=)
VIR_DOMAIN_DEVICE_ADDRESS_TYPE_(PCI|NONE)" into something slightly
more complex, this patch adds to new functions that take a
virDomainDeviceInfoPtr and return true/false depending on 1) whether
the current state of the info indicates that we "want" a PCI address
for this device (virDeviceInfoPCIAddressWanted()) and 2) whether this
device already has a valid PCI address
(virDeviceInfoPCIAddressPresent()).
Both of these functions required the simpler check for whether a pci
address is "empty" (i.e. all of its attributes are 0, which can never
happen in a real PCI address, since slot 0 of bus 0 of domain 0 is
always reserved), so that function is also added.
Laine Stump [Tue, 17 May 2016 17:55:02 +0000 (13:55 -0400)]
conf: move virDomainDeviceInfo definition from domain_conf.h to device_conf.h
Also moves all the subordinate structs. This is necessary due to a new
inline function that will be defined in device_conf.h, and also makes
sense, because it is the *device* info that's in the struct. (Actually
a lot more stuff from domain_conf.h could move to this newer file, but
I didn't want to disturb any more than necessary).
Michal Privoznik [Tue, 17 May 2016 14:26:53 +0000 (16:26 +0200)]
virtestmock: Mock stat() properly
There is a lot to explain, but I try to make it as short as
possible. I'd start by pasting some parts of sys/stat.h:
extern int stat (const char *__restrict __file,
struct stat *__restrict __buf) __THROW __nonnull ((1, 2));
extern int __REDIRECT_NTH (stat, (const char *__restrict __file,
struct stat *__restrict __buf), stat64)
__nonnull ((1, 2));
__extern_inline int
__NTH (stat (const char *__path, struct stat *__statbuf))
{
return __xstat (_STAT_VER, __path, __statbuf);
}
Only one of these is effective at once, due to some usage of
the mess we are dealing with in here. So, basically, while
compiling or linking stat() in our code can be transformed into
some other func. Or a dragon.
Now, if you read stat(2) manpage, esp. "C library/kernel
differences" section, you'll learn that glibc uses some tricks
for older applications to work. I haven't gotten around actual
code that does this, but based on my observations, if 'stat'
symbol is found, glibc assumes it's dealing with ancient
application. Unfortunately, it can be just ours stat coming from
our mock. Therefore, calling stat() from a test will end up in
our mock. But since glibc is not exposing the symbol anymore, our
call of real_stat() will SIGSEGV immediately as the pointer to
function is NULL. Therefore, we should expose only those symbols
we know glibc has.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Mon, 16 May 2016 15:15:46 +0000 (17:15 +0200)]
tests: Drop VIR_MOCK_CALL_STAT
It wasn't as great idea as I thought. Thing around stat() are
more complicated than that. Therefore we need to revert 86d1705a8ace349b plus drop use of the macro as introduced in
later patches.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
If they're available and we need to pass secrets to qemu, then use the
qemu domain secret object in order to pass the secrets for RBD volumes
instead of passing the base64 encoded secret on the command line.
The goal is to make AES secrets the default and have no user interaction
required in order to allow using the AES mechanism. If the mechanism
is not available, then fall back to the current plain mechanism using
a base64 encoded secret.
New APIs:
qemu_domain.c:
qemuDomainGetSecretAESAlias:
Generate/return the secret object alias for an AES Secret Info type.
This will be called from qemuDomainSecretAESSetup.
qemuDomainSecretAESSetup: (private)
This API handles the details of the generation of the AES secret
and saves the pieces that need to be passed to qemu in order for
the secret to be decrypted. The encrypted secret based upon the
domain master key, an initialization vector (16 byte random value),
and the stored secret. Finally, the requirement from qemu is the IV
and encrypted secret are to be base64 encoded.
qemu_command.c:
qemuBuildSecretInfoProps: (private)
Generate/return a JSON properties object for the AES secret to
be used by both the command building and eventually the hotplug
code in order to add the secret object. Code was designed so that
in the future perhaps hotplug could use it if it made sense.
qemuBuildObjectSecretCommandLine (private)
Generate and add to the command line the -object secret for the
secret. This will be required for the subsequent RBD reference
to the object.
qemuBuildDiskSecinfoCommandLine (private)
Handle adding the AES secret object.
Adjustments:
qemu_domain.c:
The qemuDomainSecretSetup was altered to call either the AES or Plain
Setup functions based upon whether AES secrets are possible (we have
the encryption API) or not, we have secrets, and of course if the
protocol source is RBD.
qemu_command.c:
Adjust the qemuBuildRBDSecinfoURI API's in order to generate the
specific command options for an AES secret, such as:
where the 'id=' value is the secret object alias generated by
concatenating the disk alias and "-aesKey0". The 'keyid= $masterKey'
is the master key shared with qemu, and the -drive syntax will
reference that alias as the 'password-secret'. For the -drive
syntax, the 'id=myname' is kept to define the username, while the
'key=$base64 encoded secret' is removed.
While according to the syntax described for qemu commit '60390a21'
or as seen in the email archive:
it is possible to pass a plaintext password via a file, the qemu
commit 'ac1d8878' describes the more feature rich 'keyid=' option
based upon the shared masterKey.
Add tests for checking/comparing output.
NB: For hotplug, since the hotplug code doesn't add command line
arguments, passing the encoded secret directly to the monitor
will suffice.
John Ferlan [Thu, 19 May 2016 17:05:36 +0000 (13:05 -0400)]
util: Introduce virCryptoGenerateRandom
Move the logic from qemuDomainGenerateRandomKey into this new
function, altering the comments, variable names, and error messages
to keep things more generic.
NB: Although perhaps more reasonable to add soemthing to virrandom.c.
The virrandom.c was included in the setuid_rpc_client, so I chose
placement in vircrypto.
John Ferlan [Wed, 18 May 2016 15:29:06 +0000 (11:29 -0400)]
util: Introduce encryption APIs
Introduce virCryptoHaveCipher and virCryptoEncryptData to handle
performing encryption.
virCryptoHaveCipher:
Boolean function to determine whether the requested cipher algorithm
is available. It's expected this API will be called prior to
virCryptoEncryptdata. It will return true/false.
virCryptoEncryptData:
Based on the requested cipher type, call the specific encryption
API to encrypt the data.
Currently the only algorithm support is the AES 256 CBC encryption.
John Ferlan [Thu, 12 May 2016 16:15:44 +0000 (12:15 -0400)]
tests: Add mock for virRandomBytes
Create a mock for virRandomBytes to generate a not so random value.
This should be usable by other tests that need a not so random number
to be generated by including the virrandommock at preload.
The "random number" generated is based upon the size of the expected
stream of bytes being returned where each byte in the result gets
the index of the array - hence a 4 byte array returns 0x00010203.
projects / libvirt.git / log