]> xenbits.xensource.com Git - osstest/openstack-nova.git/commit
fix "down" nova-compute service spuriously marked as "up"
authorChris Friesen <chris.friesen@windriver.com>
Fri, 27 Mar 2015 15:23:48 +0000 (09:23 -0600)
committerChris Friesen <chris.friesen@windriver.com>
Wed, 10 Jun 2015 19:18:06 +0000 (13:18 -0600)
commitb9bae02af2168ad64d3b3d28c97c3853cee73272
treee28f205d89441dfbd7c7268bb7628c7cdf577c0c
parente5fe8fd6af2d768cade29f849d5baa5873e3c72b
fix "down" nova-compute service spuriously marked as "up"

Currently we use the auto-updated "updated_at" field to determine
whether a service is "up".  An end-user can cause the "updated_at"
field to be updated by disabling or enabling the service, thus
potentially causing a service that is unavailable to be detected
as "up".  This could result in the scheduler trying to assign
instances to an unavailable compute node, or in the system
mistakenly preventing evacuation of an instance.

The fix is to add a new field to explicitly track the timestamp of
the last time the service sent in a status report and use that if
available when testing whether the service is up.

DocImpact
This commit will cause a behaviour change for the DB servicegroup
driver.  It will mean that enabling/disabling the service will
cause the "updated_at" field to change (as before) but that will
no longer be tied to the "up/down" status of the service. So
"nova service-list" could show the service as "down" even if it
shows a recent "updated_at".  (But this could happen for the other
servicegroup drivers already.)

Closes-Bug: #1420848
Change-Id: Ied7d47363d0489bca3cf2c711217e1a3b7d24a03
nova/db/sqlalchemy/api.py
nova/db/sqlalchemy/migrate_repo/versions/294_add_service_heartbeat.py [new file with mode: 0644]
nova/db/sqlalchemy/models.py
nova/objects/service.py
nova/servicegroup/drivers/db.py
nova/tests/unit/compute/test_resource_tracker.py
nova/tests/unit/db/test_migrations.py
nova/tests/unit/objects/test_objects.py
nova/tests/unit/objects/test_service.py
nova/tests/unit/servicegroup/test_db_servicegroup.py