xen/arm: update the docs about heterogeneous computing

Introduce a new document about big.LITTLE and update the documentation
of hmp-unsafe.

Also update the warning messages to point users to the docs.

Signed-off-by: Stefano Stabellini <sstabellini@kernel.org>
Acked-by: Julien Grall <julien.grall@arm.com>
CC: jbeulich@suse.com
CC: konrad.wilk@oracle.com
CC: tim@xen.org
CC: wei.liu2@citrix.com
CC: andrew.cooper3@citrix.com
CC: George.Dunlap@eu.citrix.com
CC: ian.jackson@eu.citrix.com

diff --git a/docs/misc/arm/big.LITTLE.txt b/docs/misc/arm/big.LITTLE.txt
new file mode 100644 (file)
index 0000000..b6ea1c9
--- /dev/null
+++ b/docs/misc/arm/big.LITTLE.txt
@@ -0,0 +1,46 @@
+big.LITTLE is a form of heterogeneous computing that comes with two
+types of general purpose cpu cores: big cores, more powerful and with a
+higher power consumption rate, and LITTLE cores, less powerful and
+cheaper to run. For example, Cortex A53 and Cortex A57 cpus. Typically,
+big cores are only recommended for burst activity, especially in
+battery powered environments. Please note that Xen doesn't not use any
+board specific power management techniques at the moment, it only uses
+WFI. It is recommended to check the vendor's big.LITTLE and power
+management documentation before using it in a Xen environment.
+
+
+big and LITTLE cores are fully compatible in terms of instruction sets,
+but can differ in many subtle ways. For example, their cacheline sizes
+might differ. For this reason, vcpu migration between big and LITTLE
+cores can lead to data corruptions.
+
+Today, the Xen scheduler does not have support for big.LITTLE,
+therefore, it might unknowingly move any vcpus between big and LITTLE
+cores, potentially leading to breakages. To avoid this kind of issues,
+at boot time Xen disables all cpus that differ from the boot cpu.
+
+
+Expert users can enable all big.LITTLE cores by passing hmp-unsafe=true
+to the Xen command line [1]. Given the lack of big.LITTLE support in the
+scheduler, it is only safe if the cpu affinity of all domains is
+manually specified, so that the scheduler is not allowed to switch a
+vcpu from big to LITTLE or vice versa.
+
+In the case of dom0, dom0_vcpus_pin needs to be added to the Xen command
+line options [1]. For DomUs, the `cpus' option should be added to all VM
+config files [2].
+
+For example, if the first 4 cpus are big and the last 4 are LITTLE, the
+following options run all domain vcpus on either big or LITTLE cores
+(not both):
+
+  cpus = "0-3"
+  cpus = "4-7"
+
+The following option runs one domain vcpu as big and one as LITTLE:
+
+  cpus = ["0-3", "4-7"]
+
+
+[1] docs/misc/xen-command-line.markdown
+[2] docs/man/xl.cfg.pod.5

diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index f224bbd8326d84e5695db35613620e376ffc1113..b353352adfa83f70105e9752d5a11c45b1c6f8df 100644 (file)
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -1020,7 +1020,12 @@ Control Xens use of the APEI Hardware Error Source Table, should one be found.
 
 Say yes at your own risk if you want to enable heterogenous computing
 (such as big.LITTLE). This may result to an unstable and insecure
-platform. When the option is disabled (default), CPUs that are not
+platform, unless you manually specify the cpu affinity of all domains so
+that all vcpus are scheduled on the same class of pcpus (big or LITTLE
+but not both). vcpu migration between big cores and LITTLE cores is not
+supported. See docs/misc/arm/big.LITTLE.txt for more information.
+
+When the hmp-unsafe option is disabled (default), CPUs that are not
 identical to the boot CPU will be parked and not used by Xen.
 
 ### hpetbroadcast

diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
index 122c0b554b0d6485b28d791f8a8a636981852200..04efb335a77ab7896ec5d78a5a64c15521279ac8 100644 (file)
--- a/xen/arch/arm/smpboot.c
+++ b/xen/arch/arm/smpboot.c
@@ -266,7 +266,8 @@ void __init smp_init_cpus(void)
 
     if ( opt_hmp_unsafe )
         warning_add("WARNING: HMP COMPUTING HAS BEEN ENABLED.\n"
-                    "It has implications on the security and stability of the system.\n");
+                    "It has implications on the security and stability of the system,\n"
+                    "unless the cpu affinity of all domains is specified.\n");
 }
 
 int __init
@@ -308,13 +309,15 @@ void start_secondary(unsigned long boot_phys_offset,
     /*
      * Currently Xen assumes the platform has only one kind of CPUs.
      * This assumption does not hold on big.LITTLE platform and may
-     * result to instability and insecure platform. Better to park them
-     * for now.
+     * result to instability and insecure platform (unless cpu affinity
+     * is manually specified for all domains). Better to park them for
+     * now.
      */
     if ( !opt_hmp_unsafe &&
          current_cpu_data.midr.bits != boot_cpu_data.midr.bits )
     {
-        printk(XENLOG_ERR "CPU%u MIDR (0x%x) does not match boot CPU MIDR (0x%x).\n",
+        printk(XENLOG_ERR "CPU%u MIDR (0x%x) does not match boot CPU MIDR (0x%x),\n"
+               "disable cpu (see big.LITTLE.txt under docs/).\n",
                smp_processor_id(), current_cpu_data.midr.bits,
                boot_cpu_data.midr.bits);
         stop_cpu();