ia64/xen-unstable

changeset 19527:0e24e9674ded

tools: Always use sane upstream (`native') python paths

Previously, by default we would install our python modules into
/usr/lib/python/xen, for example /usr/lib/python/xen/__init__.py.
Upstream python's standard install location (a) includes the Python
version number and (b) puts things in site-packages by default.

Our best conjecture for the reason for this was an attempt to make the
installs portable between different python versions. However, that
doesn't work because compiled python modules (.pyc), and C python
extensions corresponding to one version of python, are not compatible
across different versions of python.

This is why upstream include the version number.

site-packages is the standard location for locally-installed packages
and is automatically included on the python search path.

In this change, we abandon our own unusual python path setup:

* Invoke setup.py in an entirely standard manner. We pass
PREFIX and DESTDIR using the appropriate options provided by
setup.py for those purposes (adding them to setup.py calls
which were previously lacking them).

* Since the installation locations are now on the standard
python path, we no longer need to add anything to the path
in any of our python utilities. Therefore remove all that
code from every python script. (Many of these scripts
unconditionally added /usr/lib/python and /usr/lib64/python which
is wrong even in the old world.)

* There is no longer any special `Xen python path'. xen-python-path
is no longer needed. It is no longer called by anything in our
tree. However since out-of-tree callers may still invoke it, we
retain it. It now prints a fixed string referring to a directory
which does not to exist; callers (who use it to augment their
python path) will thus add a nonexistent directory to their python
path which is harmless.

* Remove various workarounds including use of setup.py --home
(which is intended for something completely different).

* Remove tests for the XEN_PYTHON_NATIVE_INSTALL build-time
environment variable. The new behaviour is the behaviour which we
should have had if this variable had been set. That is, it is now
as if this variable was always set but also bugs in the resulting
install have been fixed.

This should be a proper fix for the bug addressed by c/s 19515.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
author Keir Fraser <keir.fraser@citrix.com>
date Wed Apr 08 19:13:04 2009 +0100 (2009-04-08)
parents a6003404e95b
children 0ed8616b99d6
files Makefile tools/misc/sxp-pretty tools/misc/xen-bugtool tools/misc/xen-python-path tools/misc/xend tools/misc/xm tools/misc/xsview tools/pygrub/Makefile tools/pygrub/src/pygrub tools/python/Makefile tools/python/scripts/test_hvm_create.py tools/python/scripts/test_vm_create.py tools/python/scripts/xapi.py tools/security/Makefile tools/security/python/xensec_tools/acm_getlabel tools/security/xensec_gen.py tools/sv/index.psp tools/vnet/scripts/vn
line diff
     1.1 --- a/Makefile	Wed Apr 08 19:10:33 2009 +0100
     1.2 +++ b/Makefile	Wed Apr 08 19:13:04 2009 +0100
     1.3 @@ -188,11 +188,7 @@ help:
     1.4  	@echo '  clean-tboot      - clean the tboot module if it exists'
     1.5  	@echo
     1.6  	@echo 'Environment:'
     1.7 -	@echo '  XEN_PYTHON_NATIVE_INSTALL=y'
     1.8 -	@echo '                   - native python install or dist'
     1.9 -	@echo '                     install into prefix/lib/python<VERSION>'
    1.10 -	@echo '                     instead of <PREFIX>/lib/python'
    1.11 -	@echo '                     true if set to non-empty value, false otherwise'
    1.12 +	@echo '  [ this documentation is sadly not complete ]'
    1.13  
    1.14  # Use this target with extreme care!
    1.15  .PHONY: uninstall
     2.1 --- a/tools/misc/sxp-pretty	Wed Apr 08 19:10:33 2009 +0100
     2.2 +++ b/tools/misc/sxp-pretty	Wed Apr 08 19:13:04 2009 +0100
     2.3 @@ -23,14 +23,6 @@ import os.path
     2.4  import pprint
     2.5  import sys
     2.6  
     2.7 -result = commands.getstatusoutput(os.path.join(os.path.dirname(sys.argv[0]),
     2.8 -                                               'xen-python-path'))
     2.9 -if result[0] != 0:
    2.10 -    print >>sys.stderr, result[1]
    2.11 -    sys.exit(1)
    2.12 -
    2.13 -sys.path.append(result[1])
    2.14 -
    2.15  import xen.xend.sxp as sxp
    2.16  
    2.17  def main():
     3.1 --- a/tools/misc/xen-bugtool	Wed Apr 08 19:10:33 2009 +0100
     3.2 +++ b/tools/misc/xen-bugtool	Wed Apr 08 19:13:04 2009 +0100
     3.3 @@ -6,9 +6,6 @@
     3.4  
     3.5  import sys
     3.6  
     3.7 -sys.path.append('/usr/lib/python')
     3.8 -sys.path.append('/usr/lib64/python')
     3.9 -
    3.10  from xen.util import bugtool
    3.11  
    3.12  
     4.1 --- a/tools/misc/xen-python-path	Wed Apr 08 19:10:33 2009 +0100
     4.2 +++ b/tools/misc/xen-python-path	Wed Apr 08 19:13:04 2009 +0100
     4.3 @@ -17,31 +17,8 @@
     4.4  # Copyright (C) 2007 XenSource Inc.
     4.5  #============================================================================
     4.6  
     4.7 -
     4.8 -# Use the auxbin module in Xend to determine the correct Python path.  We
     4.9 -# take the first installed instance of auxbin that we find, and then run it
    4.10 -# to determine the correct path, appending that to sys.path.
    4.11 -
    4.12 -AUXBIN = 'xen/util/auxbin.py'
    4.13 -
    4.14 -import os
    4.15 -import os.path
    4.16 -import sys
    4.17 +# Nowadays we install xen in the standard python site-packages
    4.18 +# directories.  This script is still provided for the benefit of old
    4.19 +# out-of-xen-tree callers.  It is deprecated and will be removed.
    4.20  
    4.21 -usr   = os.path.dirname(os.path.dirname(sys.argv[0]))
    4.22 -list  = [ os.path.join(usr,'lib64') ]
    4.23 -list += [ os.path.join(usr,'lib') ]
    4.24 -list += ['/usr/lib64', '/usr/lib']
    4.25 -
    4.26 -for l in list:
    4.27 -    for p in ['python%s' % sys.version[:3], 'python']:
    4.28 -        for k in ['', 'site-packages/']:
    4.29 -            d = os.path.join(l, p, k)
    4.30 -            if os.path.exists(os.path.join(d, AUXBIN)):
    4.31 -                sys.path.append(d)
    4.32 -                import xen.util.auxbin
    4.33 -                print os.path.join(xen.util.auxbin.libpath(), p)
    4.34 -                sys.exit(0)
    4.35 -
    4.36 -print >>sys.stderr, "Cannot find Xen Python modules."
    4.37 -sys.exit(1)
    4.38 +print '/dev/enoent/xen/python-path'
     5.1 --- a/tools/misc/xend	Wed Apr 08 19:10:33 2009 +0100
     5.2 +++ b/tools/misc/xend	Wed Apr 08 19:13:04 2009 +0100
     5.3 @@ -33,14 +33,6 @@ import signal
     5.4  import time
     5.5  import commands
     5.6  
     5.7 -xpp = os.path.join(os.path.dirname(sys.argv[0]), 'xen-python-path')
     5.8 -if os.path.exists(xpp):
     5.9 -    result = commands.getstatusoutput(xpp)
    5.10 -    if result[0] != 0:
    5.11 -        print >>sys.stderr, result[1]
    5.12 -        sys.exit(1)
    5.13 -    sys.path.append(result[1])
    5.14 -
    5.15  from xen.xend.server import SrvDaemon
    5.16  
    5.17  class CheckError(ValueError):
     6.1 --- a/tools/misc/xm	Wed Apr 08 19:10:33 2009 +0100
     6.2 +++ b/tools/misc/xm	Wed Apr 08 19:13:04 2009 +0100
     6.3 @@ -2,9 +2,6 @@
     6.4  #  -*- mode: python; -*-
     6.5  import sys
     6.6  
     6.7 -# add fallback path for non-native python path installs if needed
     6.8 -sys.path.append('/usr/lib/python')
     6.9 -sys.path.append('/usr/lib64/python')
    6.10  from xen.xm import main
    6.11  
    6.12  main.main(sys.argv)
     7.1 --- a/tools/misc/xsview	Wed Apr 08 19:10:33 2009 +0100
     7.2 +++ b/tools/misc/xsview	Wed Apr 08 19:13:04 2009 +0100
     7.3 @@ -2,8 +2,6 @@
     7.4  
     7.5  import sys
     7.6  
     7.7 -sys.path.append('/usr/lib/python')
     7.8 -sys.path.append('/usr/lib64/python')
     7.9  from xen.xsview import main
    7.10  
    7.11  main.main(sys.argv)
     8.1 --- a/tools/pygrub/Makefile	Wed Apr 08 19:10:33 2009 +0100
     8.2 +++ b/tools/pygrub/Makefile	Wed Apr 08 19:13:04 2009 +0100
     8.3 @@ -9,15 +9,10 @@ build:
     8.4  	CC="$(CC)" CFLAGS="$(CFLAGS)" python setup.py build
     8.5  
     8.6  .PHONY: install
     8.7 -ifndef XEN_PYTHON_NATIVE_INSTALL
     8.8  install: all
     8.9 -	CC="$(CC)" CFLAGS="$(CFLAGS)" python setup.py install --root="$(DESTDIR)" --home="$(PREFIX)" --prefix="" --force
    8.10 +	CC="$(CC)" CFLAGS="$(CFLAGS)" python setup.py install \
    8.11 +		--prefix="$(PREFIX)" --root="$(DESTDIR)" --force
    8.12  	$(INSTALL_DIR) $(DESTDIR)/var/run/xend/boot
    8.13 -else
    8.14 -install: all
    8.15 -	CC="$(CC)" CFLAGS="$(CFLAGS)" python setup.py install --root="$(DESTDIR)"
    8.16 -	$(INSTALL_DIR) $(DESTDIR)/var/run/xend/boot
    8.17 -endif
    8.18  
    8.19  .PHONY: clean
    8.20  clean:
     9.1 --- a/tools/pygrub/src/pygrub	Wed Apr 08 19:10:33 2009 +0100
     9.2 +++ b/tools/pygrub/src/pygrub	Wed Apr 08 19:13:04 2009 +0100
     9.3 @@ -21,8 +21,6 @@ import platform
     9.4  import curses, _curses, curses.wrapper, curses.textpad, curses.ascii
     9.5  import getopt
     9.6  
     9.7 -sys.path = [ '/usr/lib/python', '/usr/lib64/python' ] + sys.path
     9.8 -
     9.9  import fsimage
    9.10  import grub.GrubConf
    9.11  import grub.LiloConf
    10.1 --- a/tools/python/Makefile	Wed Apr 08 19:10:33 2009 +0100
    10.2 +++ b/tools/python/Makefile	Wed Apr 08 19:13:04 2009 +0100
    10.3 @@ -54,13 +54,9 @@ refresh-po: $(POTFILE)
    10.4  	$(MSGFMT) -c -o $@ $<
    10.5  
    10.6  .PHONY: install
    10.7 -ifndef XEN_PYTHON_NATIVE_INSTALL
    10.8  install: install-messages install-dtd
    10.9 -	CC="$(CC)" CFLAGS="$(CFLAGS)" python setup.py install --root="$(DESTDIR)" --home="$(PREFIX)" --prefix="" --force
   10.10 -else
   10.11 -install: install-messages install-dtd
   10.12 -	CC="$(CC)" CFLAGS="$(CFLAGS)" python setup.py install --root="$(DESTDIR)" --force
   10.13 -endif
   10.14 +	CC="$(CC)" CFLAGS="$(CFLAGS)" python setup.py install \
   10.15 +		--prefix="$(PREFIX)" --root="$(DESTDIR)" --force
   10.16  
   10.17  install-dtd: all
   10.18  	$(INSTALL_DIR) $(DESTDIR)$(DOCDIR)
    11.1 --- a/tools/python/scripts/test_hvm_create.py	Wed Apr 08 19:10:33 2009 +0100
    11.2 +++ b/tools/python/scripts/test_hvm_create.py	Wed Apr 08 19:13:04 2009 +0100
    11.3 @@ -74,7 +74,6 @@ console_cfg = {
    11.4  
    11.5  import sys
    11.6  import time
    11.7 -sys.path.append('/usr/lib/python')
    11.8  
    11.9  from xapi import connect, execute
   11.10  
    12.1 --- a/tools/python/scripts/test_vm_create.py	Wed Apr 08 19:10:33 2009 +0100
    12.2 +++ b/tools/python/scripts/test_vm_create.py	Wed Apr 08 19:13:04 2009 +0100
    12.3 @@ -93,7 +93,6 @@ console_cfg = {
    12.4  
    12.5  import sys
    12.6  import time
    12.7 -sys.path.append('/usr/lib/python')
    12.8  
    12.9  from xapi import connect, execute
   12.10  
    13.1 --- a/tools/python/scripts/xapi.py	Wed Apr 08 19:10:33 2009 +0100
    13.2 +++ b/tools/python/scripts/xapi.py	Wed Apr 08 19:13:04 2009 +0100
    13.3 @@ -20,7 +20,6 @@ import sys
    13.4  import time
    13.5  import re
    13.6  import os
    13.7 -sys.path.append('/usr/lib/python')
    13.8  
    13.9  from xen.util.xmlrpclib2 import ServerProxy
   13.10  from optparse import *
    14.1 --- a/tools/security/Makefile	Wed Apr 08 19:10:33 2009 +0100
    14.2 +++ b/tools/security/Makefile	Wed Apr 08 19:13:04 2009 +0100
    14.3 @@ -40,9 +40,6 @@ ifeq ($(ACM_SECURITY),y)
    14.4  all: build
    14.5  
    14.6  .PHONY: install
    14.7 -ifndef XEN_PYTHON_NATIVE_INSTALL
    14.8 -install: LIBPATH=$(shell PYTHONPATH=../python/xen/util python -c "import auxbin; print auxbin.libpath()")
    14.9 -endif
   14.10  install: all $(ACM_CONFIG_FILE)
   14.11  	$(INSTALL_DIR) $(DESTDIR)$(SBINDIR)
   14.12  	$(INSTALL_PROG) $(ACM_INST_TOOLS) $(DESTDIR)$(SBINDIR)
   14.13 @@ -63,11 +60,8 @@ install: all $(ACM_CONFIG_FILE)
   14.14  	$(INSTALL_DATA) $(ACM_INST_HTML) $(DESTDIR)$(ACM_SECGEN_HTMLDIR)
   14.15  	$(INSTALL_DIR) $(DESTDIR)$(ACM_SECGEN_CGIDIR)
   14.16  	$(INSTALL_PROG) $(ACM_INST_CGI) $(DESTDIR)$(ACM_SECGEN_CGIDIR)
   14.17 -ifndef XEN_PYTHON_NATIVE_INSTALL
   14.18 -	python python/setup.py install --install-lib="$(DESTDIR)$(LIBPATH)/python"
   14.19 -else
   14.20 -	python python/setup.py install --root="$(DESTDIR)"
   14.21 -endif
   14.22 +	python python/setup.py install \
   14.23 +		--prefix="$(PREFIX)" --root="$(DESTDIR)" --force
   14.24  else
   14.25  .PHONY: all
   14.26  all:
    15.1 --- a/tools/security/python/xensec_tools/acm_getlabel	Wed Apr 08 19:10:33 2009 +0100
    15.2 +++ b/tools/security/python/xensec_tools/acm_getlabel	Wed Apr 08 19:13:04 2009 +0100
    15.3 @@ -4,10 +4,6 @@ import sys
    15.4  import traceback
    15.5  import getopt
    15.6  
    15.7 -# add fallback path for non-native python path installs if needed
    15.8 -sys.path.insert(-1, '/usr/lib/python')
    15.9 -sys.path.insert(-1, '/usr/lib64/python')
   15.10 -
   15.11  from xen.util.security import ACMError, err, get_ssid
   15.12  
   15.13  # getopt.gnu_getopt is better, but only exists in Python 2.3+.  Use
    16.1 --- a/tools/security/xensec_gen.py	Wed Apr 08 19:10:33 2009 +0100
    16.2 +++ b/tools/security/xensec_gen.py	Wed Apr 08 19:13:04 2009 +0100
    16.3 @@ -17,10 +17,6 @@
    16.4  
    16.5  import sys
    16.6  
    16.7 -# Add fallback path for non-native python path installs if needed
    16.8 -sys.path.append( '/usr/lib/python' )
    16.9 -sys.path.append( '/usr/lib64/python' )
   16.10 -
   16.11  from xen.xensec_gen import main
   16.12  
   16.13  main.main( )
    17.1 --- a/tools/sv/index.psp	Wed Apr 08 19:10:33 2009 +0100
    17.2 +++ b/tools/sv/index.psp	Wed Apr 08 19:13:04 2009 +0100
    17.3 @@ -1,6 +1,5 @@
    17.4  <%
    17.5  import sys
    17.6 -sys.path.append( "/usr/lib/python" )
    17.7  
    17.8  debug = True and False
    17.9  
    18.1 --- a/tools/vnet/scripts/vn	Wed Apr 08 19:10:33 2009 +0100
    18.2 +++ b/tools/vnet/scripts/vn	Wed Apr 08 19:13:04 2009 +0100
    18.3 @@ -27,9 +27,6 @@ import socket
    18.4  import sys
    18.5  from getopt import getopt, GetoptError
    18.6  
    18.7 -sys.path.append('/usr/lib/python')
    18.8 -sys.path.append('/usr/lib64/python')
    18.9 -
   18.10  from xen.xend import sxp
   18.11  from xen.xend.PrettyPrint import prettyprint
   18.12