ia64/xen-unstable

changeset 8198:a9c35cd7e559

This patch adds a new domain config option, 'cpus' which is a list of
CPUs a domains' vcpus can use. The older 'cpu' config option is
prepended to the list of cpus to use and will keep the behavior of
pinning VCPU0.

The cpus option supports ranges and negation, so:

cpus = "0-3,5,^1" produces -> [0,2,3,5]

The list is circular, so in a domain with the following config:

vcpus = 4
cpus = "0,3,7" # Use any of 0, 3, 7 for this domain.

would see vcpus 0-3 pinned to cpus 0,3,7,0 respectively.

Also, the pin operation is moved before the memory reservation as vcpu
to cpu mapping will be helpful for future NUMA work when trying to
allocate pages close to the physical cpus being used.

An update to the display of cpumap was needed to normalize the cpumap
values to the range of possible cpus.

I've also included some text for the xmdomain.cfg(5) man page.

Signed-off-by: Ryan Harper <ryanh@us.ibm.com>
author emellor@leeni.uk.xensource.com
date Fri Dec 02 15:52:47 2005 +0000 (2005-12-02)
parents 30eb074a560a
children b21be984945a b77ceb2481b3
files docs/man/xmdomain.cfg.pod.5 tools/examples/xmexample.vmx tools/examples/xmexample.vti tools/examples/xmexample1 tools/examples/xmexample2 tools/examples/xmexample3 tools/python/xen/xend/XendDomainInfo.py tools/python/xen/xm/create.py tools/python/xen/xm/main.py
line diff
     1.1 --- a/docs/man/xmdomain.cfg.pod.5	Fri Dec 02 15:48:55 2005 +0000
     1.2 +++ b/docs/man/xmdomain.cfg.pod.5	Fri Dec 02 15:52:47 2005 +0000
     1.3 @@ -156,6 +156,16 @@ Specifies which CPU the domain should be
     1.4  the first cpu, 1 the second, and so on.  This defaults to -1, which
     1.5  means Xen is free to pick which CPU to start on.
     1.6  
     1.7 +=item B<cpus>
     1.8 +
     1.9 +Specifies a list of CPUs on which the domains' VCPUs are allowed to
    1.10 +execute upon.  The syntax supports ranges (0-3), and negation, ^1.
    1.11 +For instance:
    1.12 +
    1.13 +    cpus = "0-3,5,^1"
    1.14 +
    1.15 +Will result in CPUs 0, 2, 3, 5 being available for use by the domain.
    1.16 +
    1.17  =item B<extra>
    1.18  
    1.19  Extra information to append to the end of the kernel parameter line.
     2.1 --- a/tools/examples/xmexample.vmx	Fri Dec 02 15:48:55 2005 +0000
     2.2 +++ b/tools/examples/xmexample.vmx	Fri Dec 02 15:52:47 2005 +0000
     2.3 @@ -30,8 +30,10 @@ name = "ExampleVMXDomain"
     2.4  # the number of cpus guest platform has, default=1
     2.5  vcpus=1
     2.6  
     2.7 -# Which CPU to start domain on? 
     2.8 -#cpu = -1   # leave to Xen to pick
     2.9 +# List of which CPUS this domain is allowed to use, default Xen picks
    2.10 +#cpus = ""         # leave to Xen to pick
    2.11 +#cpus = "0"        # all vcpus run on CPU0
    2.12 +#cpus = "0-3,5,^1" # run on cpus 0,2,3,5
    2.13  
    2.14  # Optionally define mac and/or bridge for the network interfaces.
    2.15  # Random MACs are assigned if not given.
     3.1 --- a/tools/examples/xmexample.vti	Fri Dec 02 15:48:55 2005 +0000
     3.2 +++ b/tools/examples/xmexample.vti	Fri Dec 02 15:52:47 2005 +0000
     3.3 @@ -23,8 +23,10 @@ memory = 256
     3.4  # A name for your domain. All domains must have different names.
     3.5  name = "ExampleVMXDomain"
     3.6  
     3.7 -# Which CPU to start domain on? 
     3.8 -#cpu = -1   # leave to Xen to pick
     3.9 +# List of which CPUS this domain is allowed to use, default Xen picks
    3.10 +#cpus = ""         # leave to Xen to pick
    3.11 +#cpus = "0"        # all vcpus run on CPU0
    3.12 +#cpus = "0-3,5,^1" # run on cpus 0,2,3,5
    3.13  
    3.14  # Disable vif for now
    3.15  nics=0
     4.1 --- a/tools/examples/xmexample1	Fri Dec 02 15:48:55 2005 +0000
     4.2 +++ b/tools/examples/xmexample1	Fri Dec 02 15:52:47 2005 +0000
     4.3 @@ -22,8 +22,10 @@ memory = 64
     4.4  # A name for your domain. All domains must have different names.
     4.5  name = "ExampleDomain"
     4.6  
     4.7 -# Which CPU to start domain on? 
     4.8 -#cpu = -1   # leave to Xen to pick
     4.9 +# List of which CPUS this domain is allowed to use, default Xen picks
    4.10 +#cpus = ""         # leave to Xen to pick
    4.11 +#cpus = "0"        # all vcpus run on CPU0
    4.12 +#cpus = "0-3,5,^1" # run on cpus 0,2,3,5
    4.13  
    4.14  # Number of Virtual CPUS to use, default is 1
    4.15  #vcpus = 1
     5.1 --- a/tools/examples/xmexample2	Fri Dec 02 15:48:55 2005 +0000
     5.2 +++ b/tools/examples/xmexample2	Fri Dec 02 15:52:47 2005 +0000
     5.3 @@ -51,9 +51,11 @@ memory = 64
     5.4  # so we use the vmid to create a name.
     5.5  name = "VM%d" % vmid
     5.6  
     5.7 -# Which CPU to start domain on? 
     5.8 -#cpu = -1   # leave to Xen to pick
     5.9 -cpu = vmid  # set based on vmid (mod number of CPUs)
    5.10 +# List of which CPUS this domain is allowed to use, default Xen picks
    5.11 +#cpus = ""         # leave to Xen to pick
    5.12 +#cpus = "0"        # all vcpus run on CPU0
    5.13 +#cpus = "0-3,5,^1" # run on cpus 0,2,3,5
    5.14 +#cpus = "%s" % vmid # set based on vmid (mod number of CPUs)
    5.15  
    5.16  # Number of Virtual CPUS to use, default is 1
    5.17  #vcpus = 1
     6.1 --- a/tools/examples/xmexample3	Fri Dec 02 15:48:55 2005 +0000
     6.2 +++ b/tools/examples/xmexample3	Fri Dec 02 15:52:47 2005 +0000
     6.3 @@ -51,9 +51,11 @@ memory = 64
     6.4  # so we use the vmid to create a name.
     6.5  name = "VM%d" % vmid
     6.6  
     6.7 -# Which CPU to start domain on? 
     6.8 -#cpu = -1   # leave to Xen to pick
     6.9 -cpu = vmid  # set based on vmid (mod number of CPUs)
    6.10 +# List of which CPUS this domain is allowed to use, default Xen picks
    6.11 +#cpus = ""         # leave to Xen to pick
    6.12 +#cpus = "0"        # all vcpus run on CPU0
    6.13 +#cpus = "0-3,5,^1" # run on cpus 0,2,3,5
    6.14 +cpus = "%s" % vmid # set based on vmid (mod number of CPUs)
    6.15  
    6.16  #----------------------------------------------------------------------------
    6.17  # Define network interfaces.
     7.1 --- a/tools/python/xen/xend/XendDomainInfo.py	Fri Dec 02 15:48:55 2005 +0000
     7.2 +++ b/tools/python/xen/xend/XendDomainInfo.py	Fri Dec 02 15:52:47 2005 +0000
     7.3 @@ -286,6 +286,7 @@ def parseConfig(config):
     7.4          result[e[0]] = get_cfg(e[0], e[1])
     7.5  
     7.6      result['cpu']       = get_cfg('cpu',       int)
     7.7 +    result['cpus']      = get_cfg('cpus',      str)
     7.8      result['image']     = get_cfg('image')
     7.9  
    7.10      try:
    7.11 @@ -299,6 +300,43 @@ def parseConfig(config):
    7.12              'Invalid configuration setting: vcpus = %s: %s' %
    7.13              (sxp.child_value(result['image'], 'vcpus', 1), str(exn)))
    7.14  
    7.15 +    try:
    7.16 +        # support legacy config files with 'cpu' parameter
    7.17 +        # NB: prepending to list to support previous behavior
    7.18 +        #     where 'cpu' parameter pinned VCPU0.
    7.19 +        if result['cpu']:
    7.20 +           if result['cpus']:
    7.21 +               result['cpus'] = "%s,%s" % (str(result['cpu']), result['cpus'])
    7.22 +           else:
    7.23 +               result['cpus'] = str(result['cpu'])
    7.24 +
    7.25 +        # convert 'cpus' string to list of ints
    7.26 +        # 'cpus' supports a list of ranges (0-3), seperated by
    7.27 +        # commas, and negation, (^1).  
    7.28 +        # Precedence is settled by  order of the string:
    7.29 +        #     "0-3,^1"   -> [0,2,3]
    7.30 +        #     "0-3,^1,1" -> [0,1,2,3]
    7.31 +        if result['cpus']:
    7.32 +            cpus = []
    7.33 +            for c in result['cpus'].split(','):
    7.34 +                if c.find('-') != -1:             
    7.35 +                    (x,y) = c.split('-')
    7.36 +                    for i in range(int(x),int(y)+1):
    7.37 +                        cpus.append(int(i))
    7.38 +                else:
    7.39 +                    # remove this element from the list 
    7.40 +                    if c[0] == '^':
    7.41 +                        cpus = [x for x in cpus if x != int(c[1])]
    7.42 +                    else:
    7.43 +                        cpus.append(int(c))
    7.44 +
    7.45 +            result['cpus'] = cpus
    7.46 +        
    7.47 +    except ValueError, exn:
    7.48 +        raise VmError(
    7.49 +            'Invalid configuration setting: cpus = %s: %s' %
    7.50 +            (result['cpus'], exn))
    7.51 +
    7.52      result['backend'] = []
    7.53      for c in sxp.children(config, 'backend'):
    7.54          result['backend'].append(sxp.name(sxp.child0(c)))
    7.55 @@ -486,6 +524,7 @@ class XendDomainInfo:
    7.56              defaultInfo('on_reboot',    lambda: "restart")
    7.57              defaultInfo('on_crash',     lambda: "restart")
    7.58              defaultInfo('cpu',          lambda: None)
    7.59 +            defaultInfo('cpus',         lambda: [])
    7.60              defaultInfo('cpu_weight',   lambda: 1.0)
    7.61  
    7.62              # some domains don't have a config file (e.g. dom0 )
    7.63 @@ -1130,9 +1169,15 @@ class XendDomainInfo:
    7.64              xc.domain_setmaxmem(self.domid, m)
    7.65              xc.domain_memory_increase_reservation(self.domid, m, 0, 0)
    7.66  
    7.67 -            cpu = self.info['cpu']
    7.68 -            if cpu is not None and cpu != -1:
    7.69 -                xc.domain_pincpu(self.domid, 0, 1 << cpu)
    7.70 +            # repin domain vcpus if a restricted cpus list is provided
    7.71 +            # this is done prior to memory allocation to aide in memory
    7.72 +            # distribution for NUMA systems.
    7.73 +            cpus = self.info['cpus']
    7.74 +            if cpus is not None and len(cpus) > 0:
    7.75 +                for v in range(0, self.info['max_vcpu_id']+1):
    7.76 +                    # pincpu takes a list of ints
    7.77 +                    cpu = [ int( cpus[v % len(cpus)] ) ]
    7.78 +                    xc.domain_pincpu(self.domid, v, cpu)
    7.79  
    7.80              self.createChannels()
    7.81  
     8.1 --- a/tools/python/xen/xm/create.py	Fri Dec 02 15:48:55 2005 +0000
     8.2 +++ b/tools/python/xen/xm/create.py	Fri Dec 02 15:52:47 2005 +0000
     8.3 @@ -154,7 +154,11 @@ gopts.var('maxmem', val='MEMORY',
     8.4  
     8.5  gopts.var('cpu', val='CPU',
     8.6            fn=set_int, default=None,
     8.7 -          use="CPU to run the domain on.")
     8.8 +          use="CPU to run the VCPU0 on.")
     8.9 +
    8.10 +gopts.var('cpus', val='CPUS',
    8.11 +          fn=set_int, default=None,
    8.12 +          use="CPUS to run the domain on.")
    8.13  
    8.14  gopts.var('lapic', val='LAPIC',
    8.15            fn=set_int, default=0,
    8.16 @@ -572,6 +576,8 @@ def make_config(vals):
    8.17      
    8.18      if vals.cpu is not None:
    8.19          config.append(['cpu', vals.cpu])
    8.20 +    if vals.cpus is not None:
    8.21 +        config.append(['cpus', vals.cpus])
    8.22      if vals.cpu_weight is not None:
    8.23          config.append(['cpu_weight', vals.cpu_weight])
    8.24      if vals.blkif:
     9.1 --- a/tools/python/xen/xm/main.py	Fri Dec 02 15:48:55 2005 +0000
     9.2 +++ b/tools/python/xen/xm/main.py	Fri Dec 02 15:52:47 2005 +0000
     9.3 @@ -459,7 +459,9 @@ def xm_vcpu_list(args):
     9.4              for x in server.xend_node()[1:]:
     9.5                  if len(x) > 1 and x[0] == 'nr_cpus':
     9.6                      nr_cpus = int(x[1])
     9.7 -                    cpumap = filter(lambda x: x < nr_cpus, cpumap)
     9.8 +                    # normalize cpumap by modulus nr_cpus, and drop duplicates
     9.9 +                    cpumap = dict.fromkeys(
    9.10 +                                map(lambda x: x % nr_cpus, cpumap)).keys()
    9.11                      if len(cpumap) == nr_cpus:
    9.12                          return "any cpu"
    9.13                      break