Dan Smith [Thu, 8 Dec 2016 20:25:37 +0000 (12:25 -0800)]
Cleanup after any failed libvirt spawn
When we go to spawn a libvirt domain, we catch a few types of exceptions
and perform cleanup before failing the operation. For some reason, we
don't do this universally, which means that we leave things like network
devices laying around (from plug_vifs()). If a delete comes later, it
should clean those things up. However, if a subsequent failure prevents
that, and especially if we do a local delete at the API, we'll leak those
interfaces.
As seen in at least one real-world situation, this can cause us to leak
interfaces until we have tens of thousands of them on the system, which
then causes secondary failures.
Since we run the cleanup() routine for certain failures, it certainly
seems appropriate to run it always and not leave residue until a
successful delete is performed.
Sylvain Bauza [Wed, 2 Nov 2016 11:28:02 +0000 (12:28 +0100)]
Extend get_all_by_filters to support resource criteria
Given the scheduler wants to know which RPs can support a set of different
requests, each one having a resource class with an amount, we need to
modify the current ResourceProviderList method for returning a subset.
The proposal for the request parameter is a dictionary of amounts keyed
by the resource class name.
Dan Smith [Fri, 2 Dec 2016 20:12:24 +0000 (12:12 -0800)]
Fix up non-cells-aware context managers in test_db_api
We recently converted all the db_api calls to use pick_context_manager
so that they will work with CellsV2. There were, however, still some
instances in test_db_api that did the old thing for testing low-level
bits. This cleans those up in preparation for cells patches to come.
Since we have to use the pick decorators, this converts several things
to use a decorated closure for that reason. No real functional change
though.
Dan Smith [Thu, 1 Dec 2016 16:13:01 +0000 (08:13 -0800)]
Add SingleCellSimple fixture
This fixture makes it trivial to mock out all the cell listing stuff
for the simple case where you just want to assume a single cell that
is configured as the default database.
Pavel Kholkin [Thu, 1 Dec 2016 15:37:28 +0000 (18:37 +0300)]
[proxy-api] microversion 2.39 deprecates image-metadata proxy API
Almost all proxy APIs were deprecated in microversion 2.36.
But the sub-resource image-metadata of image was forgotten to deprecate.
This patch deprecates the image-metdata API from 2.39.
Dan Smith [Thu, 10 Nov 2016 21:19:32 +0000 (13:19 -0800)]
Make RPCFixture support multiple connections
For testing cells, we will need to track the driver instances that
we give out by url. This normally just works with a conventional
oslo.messaging driver, but the fake driver keeps internal data
structures for simulating its bus. If we end up with clients creating
a new instance of the driver in the rpc switching code, we'll never
be able to send messages to services because we'll always have
private/separate data structures.
So, this makes the fixture wrap the transport creation stuff
and unify references by url. In order to make this work, some
retooling of rpc.init() is done, which makes it more in line with
the recent additions we had for wrapping transport initialization
per connection anyway.
For now, a lot of our tests can't handle the possibility of
multiple RPC connections due to them looking at the global
transport_url configuration. So for the moment, even though this
makes the fixture support multiple independent connections, we
collapse any such attempts down to a single connection to the
default broker.
Note: this requires a fix in oslo.messaging 5.14.0
tests: avoid starting compute service twice in sriov functional test
SRIOV functional tests that starts two or more guest were able to start
the compute service multiple times with the same hostname, which affected the
correctness of the tests.
This patch will make sure that the compute service is started only once.
Closes-Bug: #1647776 Co-Authored-By: Dan Smith <dansmith@redhat.com>
Change-Id: I8556ce068571d8e496e6fba756c1977c1d2c3ca1
tests: generate correct pci addresses for fake pci devices
fakelibvirt library was not generating a correct pci address for
its fake pci devices. PCI slot field would remain constant in all
generated devices.
While this issue would be transparrent for most of the tests,
but test_create_server_with_PF_no_VF would be affected, as it
should lookup VFs by its addresses.
In latest devstack, nova-serialproxy fails to start because it crashed
when it tries to register the cli options.
The issue is that it tried to register an array of options by invoking
conf.register_cli_opt(), when multiple options need to be register
through conf.register_cli_opts().
Matthew Booth [Tue, 22 Nov 2016 12:02:18 +0000 (12:02 +0000)]
libvirt: Mock imagebackend template funcs in ImageBackendFixture
This represents a small change to how we test the arguments passed to
a template function. Most tests which test cache() currently don't
directly test the callback function. Some test the callback function
which was passed to cache(), but this is undesirable as:
* It breaks untestably if you replace it with a wrapper
* You can't test the arguments which were passed to it
To make this easier to test, and because a subsequent change alters
this slightly in ways we want to make obvious, we update
ImageBackendFixture to execute the callback function when cache() is
called. We pre-emptively mock all callback methods so they are not
actually called. Test can assert on these mocks to check that the
intended callback was called, and the arguments used.
Matt Riedemann [Mon, 5 Dec 2016 21:24:05 +0000 (16:24 -0500)]
Handle MarkerNotFound from cell0 database
When listing instances in the cellv2 world we look them up
from three locations:
1. Build requests which exist before the instances are created
in the cell database (after the scheduler picks a host to
build the instance). Currently instances and build requests
are both created before casting to conductor, but that's going
away in Ocata with the support for multiple cellsv2 cells.
2. The cell0 database for instances which failed to get scheduled
to a compute host (and therefore a cell).
3. The actual cell database that the instance lives in. Currently
that's only a single traditional nova database, but could be one
of multiple cellsv2 cells when we add that support in Ocata.
If a marker is passed in when listing instances, if the instance
lives in an actual cell database, we'll get a MarkerNotFound failure
from cell0 because the instance doesn't exist in cell0, but we check
cell0 before we check the cell database. This makes the instance
listing short-circuit and fail with a 400 from the REST API.
This patch simply handles the MarkerNotFound when listing instances
from the cell0 database and ignores it so we can continue onto the
cell database.
melanie witt [Fri, 18 Nov 2016 17:18:24 +0000 (17:18 +0000)]
Add a CellDatabases test fixture
As we progress with the Cells v2 scheduling interaction work, we need
to be able to have switching between multiple databases work in our
functional tests. The existing Database fixture doesn't work in this
case because each connection switch via target_cell results in a new,
empty sqlite database, and main_context_manager is global in the DB
API and always points at the same sqlite database.
This adds a fixture that creates a new sqlite database per cell
database, runs migrations, and keeps track of the databases using
identifiers provided when cell databases are added to the fixture.
It patches get_context_manager, create_context_manager, and target_cell
to return the matching database connection according to identifier,
simulating switching between multiple databases in a single test.
Markus Zoeller [Tue, 6 Dec 2016 10:40:25 +0000 (11:40 +0100)]
libvirt: virtlogd: use virtlogd for char devices
This change makes actual usage of the "logd" sub-element for char devices.
The two REST APIs ``os-getConsoleOutput`` and ``os-getSerialConsole`` can
now be satisfied at the same time. This is valid for any combination of:
* char device element: "console", "serial"
* char device type: "tcp", "pty"
There is also no need to create multiple different device types anymore.
If we have a tcp device, we don't need the pty device anymore. The logging
will be done in the tcp device.
Markus Zoeller [Mon, 7 Nov 2016 09:01:45 +0000 (10:01 +0100)]
libvirt: create consoles in an understandable/extensible way
This change refactors the way the consoles of a libvirt guest get
created. This is basically just a reshuffle of code and an extraction
of methods with the goal to make the plethora of conditionals easier
to understand. Also, future enhancements should be easier this way.
For example, the blueprint libvirt-virtlogd (targeted for Ocata) will
have to be integrated in this console creation flow.
During the implementation I noticed that the host caps are *not*
needed for creation. That was an unnecessary special case for s390x
which didn't make any sense as the guest arch is the important piece.
That's the reason I dropped the "caps" parameter of the method
"_create_consoles". That also made it necessary to adjust the unit
tests.
I also took the chance to rename the "guest" parameter, which represents
the domain *configuration object*, to "guest_cfg". This is (almost) used
in every other place in the libvirt driver.
jichenjc [Fri, 18 Nov 2016 20:51:15 +0000 (04:51 +0800)]
Add more log when delete orphan node
we have following log when delete opphan node
INFO nova.compute.manager Deleting orphan compute node xx
we might need to know why those node are removed so
we need additional log info about the removal.
however, it's not complete about why it's removed and
what's removed unless we dig into database layer
Dan Smith [Tue, 1 Nov 2016 15:54:59 +0000 (08:54 -0700)]
Require cellsv2 setup before migrating to Ocata
We have code going into Ocata that needs to be sure that cell and
host mappings are in place. Since this was required homework in
Newton, we can land a migration to intentionally fail if this was
not completed.
This is, however, a little difficult to require because a first-time
deployment will be initialized schema-wise with none of these records,
which is also sane. So, we look to see if any flavors are defined as
a sentinel to indicate that this is an upgrade of an existing
deployment instead of a first-time event. Not perfect, but since this
is really just a helper for the user, it seems like a reasonable
risk.
Matt Riedemann [Sun, 4 Dec 2016 20:08:04 +0000 (15:08 -0500)]
Handle ComputeHostNotFound when listing hypervisors
Compute node resources must currently be deleted manually
in the database, and as such they can reference service
records which have been deleted via the services delete API.
Because of this when listing hypervisors (compute nodes), we
may get a ComputeHostNotFound error when trying to lookup a
service record for a compute node where the service was
deleted. This causes the API to fail with a 500 since it's not
handled.
This change handles the ComputeHostNotFound when looping over
compute nodes in the hypervisors index and detail methods and
simply ignores them.
Matt Riedemann [Fri, 2 Dec 2016 20:28:01 +0000 (15:28 -0500)]
Fix placement API version history 1.1 title
This needs to use dashes otherwise it doesn't render
properly in the docs. Also adds an informative title
for the 1.1 microversion since that seems to be a pattern
we're going with now in 1.2.
Roman Dobosz [Fri, 2 Dec 2016 09:22:27 +0000 (10:22 +0100)]
placement: Perform build list of standard classes once
To prevent rebuilding list of standard resource classes every time
destroy() method is called on ResourceClass object, move it to class
attribute. Also, removed get_standards call in destroy method of
ResourceClass, and simply access list through the cache object.
Jay Pipes [Mon, 31 Oct 2016 19:33:40 +0000 (15:33 -0400)]
placement: REST API for resource classes
This patch adds support for a REST API for CRUD operations on custom
resource classes:
GET /resource_classes: return all resource classes
POST /resource_classes: create a new custom resource class
PUT /resource_classes/{name}: update name of custom resource class
DELETE /resource_classes/{name}: deletes a custom resource class
GET /resource_classes/{name}: get a single resource class
Balazs Gibizer [Mon, 5 Dec 2016 15:10:08 +0000 (16:10 +0100)]
support polling free notification testing
There are multiple instance action notifications where the only
way to test notification sample is to wait for the notfication
to arrive as there is no indication on the REST API that the
certain action has been finished.
Until now the only way to implement such test was to poll the
fake_notifier.VERSIONED_NOTIFICATIONS list to see if the notification
is arrived.
This patch adds a polling free solution based on threading.Event
- Remove newlines between options
- Place option name on the same line as the declaration
- Use 'Related options', not 'Interdepencies to other options'
- Slightly reword some options to provide a clear summary-description
help text
ghanshyam [Mon, 5 Dec 2016 09:11:25 +0000 (18:11 +0900)]
Merge v21 view builder for ips API
Now v2 and v2.1 code is merged and we do not need
to maintain the different set of view builder.
Previously there were different way of building the
complete response for ips API. v2 used to do with extension
and v2.1 with view builder itself.
Jay Pipes [Mon, 21 Nov 2016 20:21:25 +0000 (15:21 -0500)]
placement: adds ResourceClass.save()
Implement the ability to update the name of a custom resource class. We
raise similar exceptions to ResourceClass.destroy() if the user attempts
to modify the name of a standard resource class.
Some hyperv unit tests was creating a "fake" directory under the nova
root directory. This cause this directory to show up on "git status"
as untracked.
projects / osstest / openstack-nova.git / log