David Scott [Fri, 18 Jun 2010 12:37:34 +0000 (13:37 +0100)]
CA-41839: Moving to RPMs for xapi had the side-effect of renaming the CLI RPM from xe-cli-... to xapi-xe... This patch changes it back for now to minimise unwanted churn.
Rob Hoes [Thu, 17 Jun 2010 16:31:00 +0000 (17:31 +0100)]
Remove bond-slave filter from PIF.scan
This filter uses sysfs to find out whether an interface is a bond slave. This does not work when the openvswitch backend is used. Besides, it does not seem to be necessary (anymore?) anyway.
PIF.scan creates a new PIF for each interface it finds if this interface is physical and there is no PIF with the same MAC. The slave interfaces always have the same MAC as the bond master PIF (bridging backend) or their "real" MACs as stored in their PIFs (openvswitch backend). Therefore, when a bond is in effect, the MACs of all bond slaves have corresponding PIFs in the DB, so PIF.scan won't create new ones.
Rob Hoes [Thu, 17 Jun 2010 16:31:00 +0000 (17:31 +0100)]
CA-40910: Only copy physical PIFs on pool join
On pool join, any bonds or VLANs on the joining host are supposed to be ignored. When the host reboots after the join, it will inherit the bond/VLAN setup from the pool master. Therefore, Bond and VLAN objects are not copied from the joining host to the pool. However, bond and VLAN master PIFs are, with is not good. This patch fixes that.
Rob Hoes [Thu, 17 Jun 2010 16:31:00 +0000 (17:31 +0100)]
Split off bond/VLAN reconstruction code from Dbsync_slave
When a pool slave starts up, it needs synchronise its bonds and VLANs with the pool master. This code is in the Dbsync_slave module, but does not really belong there (DBsync_slave is for synchronising DB fields such as PIF.currently_attached with the state of the physical system). This patch gives the bond/VLAN recreation code its own module.
Jonathan Knowles [Wed, 19 May 2010 10:50:11 +0000 (11:50 +0100)]
[CA-41286] Fixes a memory accounting error in the squeezing code.
When calculating an upper bound for the memory usage of an HVM domain with no balloon driver, the squeezing code would double-count (inappropriately) the shadow memory area.
This caused the squeezing algorithm to underestimate the amount of usable memory of any host running one or more HVM domains without balloon drivers.
This change removes the double-counting error, and also fixes a bug that caused the squeezer to add (inappropriately) the shadow memory size to Xen maxmem.
Signed-off-by: Jonathan Knowles <jonathan.knowles@eu.citrix.com> Acked-by: Dave Scott <dave.scott@eu.citrix.com>
David Scott [Thu, 13 May 2010 13:31:01 +0000 (14:31 +0100)]
Revert c/s 848:dddf116cddb5 - VM.other_config:auto_poweron is inexplicable either way around, so revert back to the old behaviour until we come up with a new plan (probably involving merging this with HA VM restart)
Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Rob Hoes [Thu, 17 Jun 2010 16:31:00 +0000 (17:31 +0100)]
Updated API evolution document
This document is meant to describe how the elements of the XenAPI may evolve over time, and provide compatibility guidelines. This is only a proposal and is still incomplete; we need some discussion about it, especially to establish what sort of compatibility guarantees we can give.
Zheng Li [Tue, 4 May 2010 05:22:08 +0000 (06:22 +0100)]
A few bugfixes and a few minor improvements to current xe cmdline tool implementation
Bugfixes:
* xe command line options doesn't mix well with XE_EXTRA_ARGS varialbe. E.g. setting XE_EXTRA_ARGS to "username=xxxx,password=yyyy" (or any non-nil valid configuration) and calling "xe -s <some server> vm-list" will break xe. Note that this is a common user case in a cluster-like environment where all the machines have the same user/passwd config, where one can conveniently set user/passwd in XE_EXTRA_ARGS for once and connect to different servers by only specifying different "-s" arguments in the cmdline.
* Setting "compat=true" in xe's RC file won't work. E.g. xe vm-clone vm-name=<vm name> new-name=<new vm name> with "compat=true" in ~/.xe won't work (but with "compat=true" in XE_EXTRA_ARGS or in xe cmdline will work).
* Setting a password with comma via XE_EXTRA_ARGS will break the logic. After the fix, it's possible to specify that by using backslash to escape the comma (e.g. password=pass\,word)
* clean up the options handling logic, so that cmdline options, RC file setting and XE_EXTRA_ARGS variable can mix consistently even in some corner cases and follow the natural priority: cmdline option > XE_EXTRA_ARGS > ~/.xe RC > default settings
Improvements:
* change options "-debug" and "-debug-on-fail" to "--debug" and "--debug-on-fail", so that every command line option now follows the common naming convention of -shortcut v.s.--full-name (with the only standard exception of having both "-help" and "--help"). AFAICS, both debug options are (maybe deliberately) not documented in the manual, so changing the names might not be a big issue regarding compatibilities.
* complete the pair relation between command line options and RC/environment variables. There were some missings from either side: e.g. "compat=xxxx" has no "--compat" correspondence and "--debug"("--debug-on-fail") has not "debug=xxxx" in par.
Zheng Li [Fri, 23 Apr 2010 18:30:04 +0000 (19:30 +0100)]
Add missing edition "XD"
"XD" edition seems to be missing here. The more interesting thing is that when we fail because of this bug, the log file will say "unknown edition" as expected, but the cli's error message will say "daemon was not found".
Zheng Li [Fri, 23 Apr 2010 17:54:31 +0000 (18:54 +0100)]
Enable native/bytecode compilation choices, and some Makefile simplification.
* Enable the independent bytecode compilation mode.
Previously this was unfortunately blocked by the hard wired xapi-client.cmxa in the Makefile. Now we should be able to load the bytecode version of xapi client lib in the OCaml interpreter for interactive development with instant feedback. Use $(COMPILE_NATIVE) and $(COMPILE_BYTE) to control these options.
* Change a few "XXX_TARGET = $(if $(equal $(XXX_CONF), yes), $(XXX_OBJ), none)" to "XXX_TARGET = $(if $(XXX_CONF), $(XXX_OBJ))"
The judge conditions are not strictly equivalent in semantics. The former holds only if XXX_CONF equals to "yes", the later condition holds when XXX_CONF is set to any non-negative string (not "no", "false", "0", none etc.). But I think it should be the same regarding XAPI's current Makefiles, and with more simplicity and tolerance. A lot more simplication and optimization can be done to these Makefiles, it's just a matter of time. Moreover XAPI should consider to update to new version of OMake (which should be better, but unfortunately not fully compatible with the old one).
David Scott [Fri, 9 Apr 2010 18:57:43 +0000 (19:57 +0100)]
CA-39291: Work around firewalls which kill idle TCP connections by inserting a small empty block into an export every 5s or so.
The failure happens whenever a disk has a lot of zeroes in it: the TCP connection goes idle while the server is scanning for the next non-zero block. Even setting SO_KEEPALIVE on the stunnel sockets and reducing the window probe interval down to 30s didn't fix it. We wish to keep the ability to have a basic client do an export via HTTP GET so we can't add application-level keepalives to the protocol... we must add them to the export itself.
Note this change is backwards compatible. The receiver code expects:
* a common prefix
* a monotonically increasing chunk number
* the first and last blocks to be the same size and included verbatim (even if all zeroes)
* blocks of zeroes the same size as the first block represented as gaps in the increasing chunk number sequence
Therefore including extra files of length 0 in the stream will be ignored provided they
* share the common prefix and chunk numbering scheme
* are not the first or last blocks
Signed-off-by: David Scott <dave.scott@eu.citrix.com>
David Scott [Fri, 28 May 2010 14:25:56 +0000 (15:25 +0100)]
Move the generation of ocaml/util/version.ml into the Makefile so that it can be done before the main build. This unbreaks the rpm build since 'hg id' will fail when run in a plain non-repo directory of sources.
Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Ewan Mellor [Thu, 29 Apr 2010 09:22:35 +0000 (10:22 +0100)]
CA-37279: Stunnel error from WLB connect "No route to host"
Understand "No route to host" as an expected error message from stunnel, and turn that into API error WLB_UNKNOWN_HOST if we see it when contacting WLB.
This is two patches, one for xen-api-libs.hg, and one for xen-api.hg.
Signed-off-by: Ewan Mellor <ewan.mellor@eu.citrix.com>
Zheng Li [Tue, 20 Apr 2010 18:18:53 +0000 (19:18 +0100)]
Cope with the stunnel zombie process issue.
Some versions of stunnel (old versions, or some new ones bulit with some particular Linux distribution versions) have the zombie process issue when called from xe. When it happens, the main stunnel process won't exit for long time after xe closing its communication channel, seems to be waiting for its children processes which are however staying in "defunct" status. The issue was also reported on the server side when stunnel is called by xapi daemon, so it would be useful to set the Stunnel.disconnect arguments properly there as well. However this demands more work to identify which setting is safe for each occurrence, so I'll leave it for future. Moreover, currently xe doesn't wait for the second stunnel connection process (HTTP Get/Put), this should also be fixed in the future. The question is: does xe really needs to care about the status of stunnel in most cases? If not, why not using the double fork tricks everywhere; if yes, a SSL library might be more appropriate than 3rd party tools such as stunnel.
Ian Campbell [Tue, 20 Apr 2010 16:51:08 +0000 (17:51 +0100)]
The OpenVswitch project is trying to standardise on using OpenVswitch rather than simply vswitch so accept both in network.conf and report the mode as openvswitch.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Yang Hongyang [Tue, 20 Apr 2010 13:53:51 +0000 (14:53 +0100)]
Add strict check to VCPUs-params param-key,according to Xen Cloud Platform Administrator's Guide,there's only 'weight','cap' or 'mask' param-key available.
Zheng Li [Sat, 10 Apr 2010 09:15:21 +0000 (10:15 +0100)]
Fix a bug in the previous -debug-on-fail patch
I didn't realize that xe would initial an auxilary stunnel link in certain opearions, so that some of the global variables we stored for the main link might be overwritten unexpectedly. Now the protection is added. What a typical example about how global mutable variable is bad, but I won't really regret having chosen such a solution because it demanded the least code modification, and hence safer for such complex software in that sense.
David Scott [Mon, 12 Apr 2010 13:44:41 +0000 (14:44 +0100)]
CA-40134: when looking up a domid, always look up the domain by uuid rather than relying on the value in the master's database (which could be out of sync).
In particular if the master is suddenly powercycled then recent domid updates might be lost. This results in VMs which are impossible to shutdown.
Looking up the domain by uuid should be strictly better than using the master's version: both are moving targets (eg over migrate) and so we already rely on the per-VM lock to protect (most) accesses.
The only problem is the code is slightly inefficient: it still has to contact the master to look up the VM's uuid and then has to list all domains on the host to build up a table of uuid -> domid. This can be improved later.
Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Rob Hoes [Fri, 9 Apr 2010 09:56:14 +0000 (10:56 +0100)]
CA-38844: CPUID maskability
Distinguish between not maskable ("no") (CPU does not have Intel FlexMigration or AMB Extended Migration), only base features are maskable ("base"), and base+extended features are maskable ("full") in Host.cpu_info:maskable.
Note: this patch should go in together with the flexmigration patch in xen-api-libs.hg.
David Scott [Wed, 7 Apr 2010 20:09:16 +0000 (21:09 +0100)]
CA-39952: explicitly blank {allowed,current}_operations fields in VM exports. The values stored here are redundant sources of potential import failures on older s/w versions.
The operation enums are often extended rendering them unparsable by older software versions. Although we don't guarantee that a new export can be imported on an old host it nevertheless should almost always work.. (apart from this)
Signed-off-by: David Scott <dave.scott@eu.citrix.com>
David Scott [Wed, 7 Apr 2010 16:43:56 +0000 (17:43 +0100)]
CA-38463: backend vifs now have proper "device" symlinks in /sys so to tell the difference between them and a real "physical" interface, look to see whether they link to devices/xen-backend/...
This prevents PIF.scan from accidentally introducing vifX.Y as PIFs...
Signed-off-by: David Scott <dave.scott@eu.citrix.com>
David Scott [Wed, 7 Apr 2010 16:06:31 +0000 (17:06 +0100)]
CA-39972: try extra hard to kill stunnel in 'service xapissl stop'
Previously we sent one SIGTERM and waited for up to 3 minutes. In a quick test, out of 1000 back-to-back 'service xapissl restart' calls, one took the full 3 minutes, as if the signal was ignored.
Now we send additional SIGTERMS as we go around the loop, one per second. In a quick test, 10000 back-to-back 'service xapissl restart' calls completed without any taking more than a few seconds.
Signed-off-by: David Scott <dave.scott@eu.citrix.com>
David Scott [Wed, 7 Apr 2010 16:06:30 +0000 (17:06 +0100)]
CA-39887: Totally throw away VM RRD updates when domains are paused.
Previously we kept the RRD data but silenced the dirty_memory signal. This would allow the following sequence:
1. domain created
domain = paused; memory = 0
<-- RRD updated with memory = 0
<-- memory not considered 'dirty' because domain is paused
2. domain built
domain = paused; memory = some interesting value
<-- RRD updated with memory = some interesting value
<-- memory not considered 'dirty' because domain is paused
3. domain unpaused
domain = running; memory = some interesting value
<-- RRD updated with memory = some interesting value
<-- memory not considered 'dirty' because memory value has *not* changed in the RRD
Now we ignore the RRD updates when the domain is paused. This means that, when the domain is finally unpaused, the new memory value will always be considered to have changed.
Signed-off-by: David Scott <dave.scott@eu.citrix.com>
David Scott [Wed, 7 Apr 2010 10:46:46 +0000 (11:46 +0100)]
CA-39401: on server boot only start VMs with auto_poweron=true which have their affinity set to the local host.
The auto_poweron=true mechanism made sense when all pools were of size 1. It's a bit odd with multi-host pools.. surely these days you would HA protect VMs you care about?
This change makes the auto_poweron mechanism a bit less inexplicable. At some point we ought to merge this with HA.
Signed-off-by: David Scott <dave.scott@eu.citrix.com>
David Scott [Wed, 7 Apr 2010 10:42:09 +0000 (11:42 +0100)]
CA-34888: the Java bindings tests expect there to be enough memory to install from a Win7 (64 bit) template plus a bit. Bump up the default size of the simulated host in xiu.
Signed-off-by: David Scott <dave.scott@eu.citrix.com>
David Scott [Wed, 7 Apr 2010 10:36:23 +0000 (11:36 +0100)]
CA-39745: When a failure to hotplug a disk is detected, check whether the underlying device was a physical CDROM with an empty drive. In this case throw HOST_CD_DRIVE_EMPTY.
Signed-off-by: David Scott <dave.scott@eu.citrix.com>
David Scott [Wed, 7 Apr 2010 10:35:06 +0000 (11:35 +0100)]
CA-39745: watch /local/domain/%d/error/backend/vbd/%d/%d/error for informative errors such as "2 creating vbd structure". This prevents us timing out after 20 minutes if something goes wrong with a blkback device.
A further patch will be needed to provide some decent error diagnosis.
Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Rob Hoes [Wed, 7 Apr 2010 10:34:09 +0000 (11:34 +0100)]
CA-39461: Fix Network.pool_introduce
The bug in Network.pool_introduce led to problems when adding a host to a pool, when this host has a Network with a name different from any Network on the pool. Networks and PIFs were not properly recreated on the pool in this case.
David Scott [Tue, 6 Apr 2010 09:57:56 +0000 (10:57 +0100)]
CA-39889: hotplug PCI devices into guests in the order specified in the (original) other-config key.
There were two problems:
1. xapi was accidentally reversing the list provided in the other_config key (a side-effect of a fold)
2. when listing the PCI devices present in xenstore, the Device.PCI.list function was passing the device order number back in the place reserved for "PCI bus ID". This caused the devices to be hotplugged in reverse over reboot.
Now the behaviour is:
1. devices are hotplugged in the order they are found in the other_config key
2. the hotplug ordering is stable across start/internal reboot/external reboot
Signed-off-by: David Scott <dave.scott@eu.citrix.com>
[CA-39743] Improvements to the wait_xen_free_mem function.
Improvements include:
* We read the values of free_memory and scrub_memory atomically (rather than separately).
* We exit early if (free_memory < required_memory) and (scrub_memory = 0) as it's unlikely that more will become free.
* We time out more quickly than before (64 seconds rather than 256 seconds).
Signed-off-by: Jonathan Knowles <jonathan.knowles@eu.citrix.com>
Marcus Granado [Thu, 1 Apr 2010 10:53:47 +0000 (11:53 +0100)]
CA-38974: improve parameter reporting during obj.destroy in auditlog
Obj.destroy has the nasty side-effect of removing the object name from the database.
Therefore, for .destroy actions, no obj name is shown in the audit record parameters.
This patch caches the object name before Obj.destroy is called, working around this
side-effect.
Signed-off-by: Marcus Granado <marcus.granado@eu.citrix.com>
David Scott [Thu, 1 Apr 2010 10:52:20 +0000 (11:52 +0100)]
CA-38723: make "xe snapshot-" commands advertise "uuid" parameters (rather than "snapshot-uuid") for consistency with other commands but still secretly accept "snapshot-uuid" to avoid breaking anything.
Signed-off-by: David Scott <dave.scott@eu.citrix.com>
projects / xcp / xen-api.git / log