]> xenbits.xensource.com Git - people/royger/xen.git/log
people/royger/xen.git
3 years agox86/emul: Update emulation stubs to be CET-IBT compatible
Andrew Cooper [Mon, 1 Nov 2021 10:09:59 +0000 (10:09 +0000)]
x86/emul: Update emulation stubs to be CET-IBT compatible

All indirect branches need to land on an endbr64 instruction.

For stub_selftests(), use endbr64 unconditionally for simplicity.  For ioport
and instruction emulation, add endbr64 conditionally.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
3 years agox86: Introduce helpers/checks for endbr64 instructions
Andrew Cooper [Fri, 26 Nov 2021 15:34:08 +0000 (15:34 +0000)]
x86: Introduce helpers/checks for endbr64 instructions

... to prevent the optimiser creating unsafe code.  See the code comment for
full details.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/traps: Rework write_stub_trampoline() to not hardcode the jmp
Andrew Cooper [Mon, 1 Nov 2021 12:36:33 +0000 (12:36 +0000)]
x86/traps: Rework write_stub_trampoline() to not hardcode the jmp

For CET-IBT, we will need to optionally insert an endbr64 instruction at the
start of the stub.  Don't hardcode the jmp displacement assuming that it
starts at byte 24 of the stub.

Also add extra comments describing what is going on.  The mix of %rax and %rsp
is far from trivial to follow.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/alternatives: Clear CR4.CET when clearing CR0.WP
Andrew Cooper [Mon, 1 Nov 2021 10:17:59 +0000 (10:17 +0000)]
x86/alternatives: Clear CR4.CET when clearing CR0.WP

This allows us to have CET active much earlier in boot.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/setup: Read CR4 earlier in __start_xen()
Andrew Cooper [Mon, 1 Nov 2021 10:19:57 +0000 (10:19 +0000)]
x86/setup: Read CR4 earlier in __start_xen()

This is necessary for read_cr4() to function correctly.  Move the EFER caching
at the same time.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
3 years agox86: Build check for embedded endbr64 instructions
Marek Marczykowski-Górecki [Fri, 26 Nov 2021 15:34:08 +0000 (15:34 +0000)]
x86: Build check for embedded endbr64 instructions

An interesting corner case occurs when the byte sequence making up endbr64
ends up on a non-instruction boundary.  Such embedded instructions mark legal
indirect branch targets as far as the CPU is concerned, which aren't legal as
far as the logic is concerned.

When CET-IBT is active, check for embedded byte sequences.  Example failures
look like:

  check-endbr.sh xen-syms Fail: Found 2 embedded endbr64 instructions
  0xffff82d040325677: test_endbr64 at /local/xen.git/xen/arch/x86/x86_64/entry.S:28
  0xffff82d040352da6: init_done at /local/xen.git/xen/arch/x86/setup.c:675

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
3 years agox86: Use control flow typechecking where possible
Andrew Cooper [Thu, 11 Nov 2021 13:09:19 +0000 (13:09 +0000)]
x86: Use control flow typechecking where possible

Now all indirect branch targets have been annotated, turn on typechecking to
catch issues in the future.

This extension isn't in a released version of GCC yet, so provide a container
to use with the extension included, and add it to CI.  RANDCONFIG is necessary
because some stubs for compiled-out subsystems are used as function pointer
targets.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/bugframe: CFI hardening
Andrew Cooper [Tue, 2 Nov 2021 20:58:59 +0000 (20:58 +0000)]
x86/bugframe: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

run_in_exception_handler() managed to escape typechecking, as the compiler
can't see where function pointer gets called.  After adding some ad-hoc
typechecking, it turns out that dump_execution_state() alone differs in
const-ness from the other users of run_in_exception_handler().

Introduce a new show_execution_state_nonconst() to make the typechecking
happy.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/stack: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 17:04:02 +0000 (18:04 +0100)]
x86/stack: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

The function typecheck in switch_stack_and_jump() is incompatible with control
flow typechecking.  It's ok for reset_stack_and_jump_ind(), but for
reset_stack_and_jump(), it would force us to endbr64 the targets which are
branched to directly.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/misc: CFI hardening
Andrew Cooper [Thu, 28 Oct 2021 11:31:20 +0000 (12:31 +0100)]
x86/misc: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/time: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 16:40:17 +0000 (17:40 +0100)]
x86/time: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/pt: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 18:47:47 +0000 (19:47 +0100)]
x86/pt: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/dpci: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 14:47:20 +0000 (15:47 +0100)]
x86/dpci: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/psr: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 12:48:47 +0000 (13:48 +0100)]
x86/psr: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/apei: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 18:37:20 +0000 (19:37 +0100)]
x86/apei: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/irq: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 17:45:59 +0000 (18:45 +0100)]
x86/irq: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/p2m: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 14:47:59 +0000 (15:47 +0100)]
x86/p2m: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/hap: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 14:44:02 +0000 (15:44 +0100)]
x86/hap: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/shadow: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 14:41:15 +0000 (15:41 +0100)]
x86/shadow: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/logdirty: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 16:35:26 +0000 (17:35 +0100)]
x86/logdirty: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/guest: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 18:23:09 +0000 (19:23 +0100)]
x86/guest: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/cpu: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 18:11:52 +0000 (19:11 +0100)]
x86/cpu: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/pmu: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 18:07:04 +0000 (19:07 +0100)]
x86/pmu: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/mce: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 10:15:03 +0000 (11:15 +0100)]
x86/mce: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/hvmsave: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 15:30:57 +0000 (16:30 +0100)]
x86/hvmsave: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/quirks: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 19:43:29 +0000 (20:43 +0100)]
x86/quirks: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/idle: CFI hardening
Andrew Cooper [Thu, 28 Oct 2021 12:32:34 +0000 (13:32 +0100)]
x86/idle: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/mtrr: CFI hardening
Andrew Cooper [Thu, 28 Oct 2021 12:48:54 +0000 (13:48 +0100)]
x86/mtrr: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/nmi: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 13:05:07 +0000 (14:05 +0100)]
x86/nmi: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/apic: CFI hardening
Andrew Cooper [Thu, 28 Oct 2021 13:05:05 +0000 (14:05 +0100)]
x86/apic: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/power: CFI hardening
Andrew Cooper [Thu, 28 Oct 2021 11:00:06 +0000 (12:00 +0100)]
x86/power: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

cpufreq_governor_dbs() has no external callers so make it static.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/ucode: CFI hardening
Andrew Cooper [Thu, 28 Oct 2021 10:35:25 +0000 (11:35 +0100)]
x86/ucode: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/emul: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 16:28:04 +0000 (17:28 +0100)]
x86/emul: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

pv_emul_is_mem_write() is only used in a single file.  Move it out of its
header file, so it doesn't risk being duplicated in multiple translation
units.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/hvm: CFI hardening for device emulation
Andrew Cooper [Fri, 29 Oct 2021 17:40:17 +0000 (18:40 +0100)]
x86/hvm: CFI hardening for device emulation

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/hvm: CFI hardening for hvm_funcs
Andrew Cooper [Fri, 29 Oct 2021 19:15:24 +0000 (20:15 +0100)]
x86/hvm: CFI hardening for hvm_funcs

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

In svm.c, make a few rearrangements.  svm_update_guest_cr() has no external
callers so can become static, but needs moving along with svm_fpu_enter() to
avoid a forward declaration.  Move svm_fpu_leave() too, to match.  Also move
svm_update_guest_efer() to drop its forward declaration.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86: CFI hardening for request_irq()
Andrew Cooper [Fri, 29 Oct 2021 13:08:31 +0000 (14:08 +0100)]
x86: CFI hardening for request_irq()

... and friends; alloc_direct_apic_vector() and set_direct_apic_vector().

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agoxen/misc: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 13:02:13 +0000 (14:02 +0100)]
xen/misc: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agoxen/console: CFI hardening
Andrew Cooper [Thu, 28 Oct 2021 13:06:28 +0000 (14:06 +0100)]
xen/console: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agoxen/video: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 18:41:13 +0000 (19:41 +0100)]
xen/video: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agoxen/iommu: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 17:30:29 +0000 (18:30 +0100)]
xen/iommu: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

AMD's parse_ppr_log_entry() has no external callers, so becomes static.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agoxen/decompress: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 19:57:23 +0000 (20:57 +0100)]
xen/decompress: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agoxen/vpci: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 10:57:06 +0000 (11:57 +0100)]
xen/vpci: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agoxen/keyhandler: CFI hardening
Andrew Cooper [Thu, 28 Oct 2021 10:18:45 +0000 (11:18 +0100)]
xen/keyhandler: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Tweak {IRQ_,}KEYHANDLER() to use a named initialiser instead of requiring a
pointer cast to compile in the IRQ case.

Reposition iommu_dump_page_tables() to avoid a forward declaration.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agoxen/tasklet: CFI hardening
Andrew Cooper [Thu, 28 Oct 2021 09:58:37 +0000 (10:58 +0100)]
xen/tasklet: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

The function pointer cast in hvm_vcpu_initialise() is undefined behaviour.

While it happens to function correctly, it is not compatible with control flow
typechecking, so introduce a new hvm_assert_evtchn_irq_tasklet() to handle the
parameter type conversion in a legal way.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agoxen/hypfs: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 14:42:42 +0000 (15:42 +0100)]
xen/hypfs: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Juergen Gross <jgross@suse.com>
3 years agoxen/evtchn: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 13:35:51 +0000 (14:35 +0100)]
xen/evtchn: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agoxen/sched: CFI hardening
Andrew Cooper [Thu, 28 Oct 2021 09:56:53 +0000 (10:56 +0100)]
xen/sched: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Dario Faggioli <dfaggioli@suse.com>
3 years agoxsm: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 20:26:04 +0000 (21:26 +0100)]
xsm: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>
3 years agoxsm/flask/ss: CFI hardening
Andrew Cooper [Fri, 29 Oct 2021 14:32:08 +0000 (15:32 +0100)]
xsm/flask/ss: CFI hardening

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>
3 years agoxen: CFI hardening for open_softirq()
Andrew Cooper [Fri, 29 Oct 2021 09:58:21 +0000 (10:58 +0100)]
xen: CFI hardening for open_softirq()

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agoxen: CFI hardening for IPIs
Andrew Cooper [Thu, 28 Oct 2021 09:56:53 +0000 (10:56 +0100)]
xen: CFI hardening for IPIs

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agoxen: CFI hardening for call_rcu()
Andrew Cooper [Thu, 28 Oct 2021 09:28:35 +0000 (10:28 +0100)]
xen: CFI hardening for call_rcu()

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agoxen: CFI hardening for init_timer()
Andrew Cooper [Thu, 28 Oct 2021 09:42:25 +0000 (10:42 +0100)]
xen: CFI hardening for init_timer()

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agoxen: CFI hardening for continue_hypercall_on_cpu()
Andrew Cooper [Thu, 28 Oct 2021 09:56:49 +0000 (10:56 +0100)]
xen: CFI hardening for continue_hypercall_on_cpu()

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agoxen: CFI hardening for acpi_table_parse()
Andrew Cooper [Thu, 28 Oct 2021 10:30:00 +0000 (11:30 +0100)]
xen: CFI hardening for acpi_table_parse()

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agoxen: CFI hardening for notifier callbacks
Andrew Cooper [Fri, 29 Oct 2021 09:11:55 +0000 (10:11 +0100)]
xen: CFI hardening for notifier callbacks

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agoxen: CFI hardening for __initcall()
Andrew Cooper [Thu, 28 Oct 2021 09:24:13 +0000 (10:24 +0100)]
xen: CFI hardening for __initcall()

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agoxen: CFI hardening for custom_param()
Andrew Cooper [Wed, 27 Oct 2021 13:21:51 +0000 (14:21 +0100)]
xen: CFI hardening for custom_param()

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

The "watchdog_timeout" and "cpu_type" handlers were missing __init.

The "numa", "acpi", "irq_vector_map" and "flask" handlers can skip forward
declarations by altering the custom_param() position.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agoxen: CFI hardening for x86 hypercalls
Andrew Cooper [Thu, 21 Oct 2021 17:38:50 +0000 (18:38 +0100)]
xen: CFI hardening for x86 hypercalls

Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.

Use cf_check to annotate function pointer targets for the toolchain.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86: Introduce support for CET-IBT
Andrew Cooper [Thu, 21 Oct 2021 17:38:50 +0000 (18:38 +0100)]
x86: Introduce support for CET-IBT

CET Indirect Branch Tracking is a hardware feature designed to provide
forward-edge control flow integrity, protecting against jump/call oriented
programming.

IBT requires the placement of endbr{32,64} instructions at the target of every
indirect call/jmp, and every entrypoint.

However, the default -fcf-protection=branch places an endbr{32,64} on every
function which far more than necessary, and reduces the quantity of protection
afforded.  Therefore, we use manual placement using the cf_check attribute.

It is necessary to check for both compiler and assembler support, as the
notrack prefix can be emitted in certain cases.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/kexec: Annotate embedded data with ELF metadata
Andrew Cooper [Wed, 16 Feb 2022 18:24:43 +0000 (18:24 +0000)]
x86/kexec: Annotate embedded data with ELF metadata

Scanning for embedded endbranch instructions involves parsing the .text
disassembly.  Data in the kexec trampoline has no ELF metadata, so objdump
treats it as instructions and tries to disassemble.  Convert:

  ffff82d040396108 <compatibility_mode_far>:
  ffff82d040396108:       00 00                   add    %al,(%rax)
  ffff82d04039610a:       00 00                   add    %al,(%rax)
  ffff82d04039610c:       10 00                   adc    %al,(%rax)

  ffff82d04039610e <compat_mode_gdt_desc>:
  ffff82d04039610e:       17                      (bad)
          ...

  ffff82d040396118 <compat_mode_gdt>:
          ...
  ffff82d040396120:       ff                      (bad)
  ffff82d040396121:       ff 00                   incl   (%rax)
  ffff82d040396123:       00 00                   add    %al,(%rax)
  ffff82d040396125:       93                      xchg   %eax,%ebx
  ffff82d040396126:       cf                      iret
  ffff82d040396127:       00 ff                   add    %bh,%bh
  ffff82d040396129:       ff 00                   incl   (%rax)
  ffff82d04039612b:       00 00                   add    %al,(%rax)
  ffff82d04039612d:       9b                      fwait
  ffff82d04039612e:       cf                      iret
          ...

  ffff82d040396130 <compat_mode_idt>:
          ...

  ffff82d0403961b6 <kexec_reloc_size>:
  ffff82d0403961b6:       b6 01                   mov    $0x1,%dh
          ...

to:

  ffff82d040396108 <compatibility_mode_far>:
  ffff82d040396108:       00 00 00 00 10 00                               ......

  ffff82d04039610e <compat_mode_gdt_desc>:
  ffff82d04039610e:       17 00 00 00 00 00 00 00 00 00                   ..........

  ffff82d040396118 <compat_mode_gdt>:
          ...
  ffff82d040396120:       ff ff 00 00 00 93 cf 00 ff ff 00 00 00 9b cf 00 ................

  ffff82d040396130 <compat_mode_idt>:
  ffff82d040396130:       00 00 00 00 00 00                               ......

  ffff82d040396136 <reloc_stack>:
          ...

Most data just gains type and size metadata.

The reloc_stack label is the wrong end of the data block to have a size, so
move it to the lowest address and introduce .Lreloc_stack_base as a
replacement.  Also, fix the fact that it is misaligned by 2 bytes.

While kexec_reloc_size could gain metadata, it's use in the linker
assertion (while correct) is deeply confusing to follow.  Drop it entirely,
using a linker symbol instead to denote the end of the trampoline.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
3 years agox86: Don't use the hypercall table for calling compat hypercalls
Juergen Gross [Mon, 1 Nov 2021 15:20:10 +0000 (16:20 +0100)]
x86: Don't use the hypercall table for calling compat hypercalls

Today the *_op_compat hypercalls call the modern handler functions by
using the entries from the hypercall table. This is resulting in a
not needed indirect function call which can be avoided by using the
correct handler function directly. This is basically a revert of
commit 1252e282311734 ("x86/pv: Export pv_hypercall_table[] rather
than working around it in several ways"), which reasoning no longer
applies, as shim no longer modifies the hypercall table.

The hypercall table can now be made static as there is no external
reference to it any longer.

Commit 834cb8761051f7 ("x86/PV32: fix physdev_op_compat handling")
can be reverted, too, as using the direct call of the correct handler
is already handled fine without that patch.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
3 years agox86/pv-shim: Don't modify the hypercall table
Juergen Gross [Mon, 1 Nov 2021 15:20:09 +0000 (16:20 +0100)]
x86/pv-shim: Don't modify the hypercall table

When running as pv-shim the hypercall is modified today in order to
replace the functions for __HYPERVISOR_event_channel_op and
__HYPERVISOR_grant_table_op hypercalls.

Change this to call the related functions from the normal handlers
instead when running as shim. The performance implications are not
really relevant, as a normal production hypervisor will not be
configured to support shim mode, so the related calls will be dropped
due to optimisation of the compiler.

Note that for the CONFIG_PV_SHIM_EXCLUSIVE case there is a dummy
wrapper do_grant_table_op() needed, as in this case grant_table.c
isn't being built.

Signed-off-by: Juergen Gross <jgross@suse.com>
Split out of series.  To compile in isolation, the compat_platform_op()
prototype needs correcting, and header files need rearranging to avoid the
compat_platform_op_t/multicall_entry_compat_t guest handles being declared
multiple times.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
3 years agoxen/xsm: Move {do,compat}_flask_op() declarations into a header
Andrew Cooper [Fri, 29 Oct 2021 22:05:25 +0000 (23:05 +0100)]
xen/xsm: Move {do,compat}_flask_op() declarations into a header

Declaring sideways like this is unsafe, because the compiler can't check that
the implementation in flask_op.c still has the same type.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>
3 years agoxen/sort: Switch to an extern inline implementation
Andrew Cooper [Sat, 30 Oct 2021 21:13:55 +0000 (22:13 +0100)]
xen/sort: Switch to an extern inline implementation

There are exactly 3 callers of sort() in the hypervisor.  Callbacks in a tight
loop like this are problematic for performance, especially with Spectre v2
protections, which is why extern inline is used commonly by libraries.

Both ARM callers pass in NULL for the swap function, and while this might seem
like an attractive option at first, it causes generic_swap() to be used, which
forced a byte-wise copy.  Provide real swap functions so the compiler can
optimise properly, which is very important for ARM downstreams where
milliseconds until the system is up matters.

This is also important for Control Flow Integrity schemes (e.g. x86 CET-IBT,
ARM BTI), because tagged function(s) performing an arbitrary length swap of
two arbitrary pointers is a very valuable gadget for an attacker.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien Grall <jgrall@amazon.com>
3 years agocirrus: update FreeBSD to 12.3
Roger Pau Monne [Tue, 22 Feb 2022 13:56:29 +0000 (14:56 +0100)]
cirrus: update FreeBSD to 12.3

Switch from using a FreeBSD 12.2 to a 12.3 image.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
3 years agox86/Kconfig: introduce option to select retpoline usage
Roger Pau Monne [Fri, 18 Feb 2022 14:34:16 +0000 (15:34 +0100)]
x86/Kconfig: introduce option to select retpoline usage

Add a new Kconfig option under the "Speculative hardening" section
that allows selecting whether to enable retpoline. This depends on the
underlying compiler having retpoline support.

Requested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
3 years agox86/clang: add retpoline support
Roger Pau Monne [Fri, 18 Feb 2022 14:34:15 +0000 (15:34 +0100)]
x86/clang: add retpoline support

Detect whether the compiler supports clang retpoline option and enable
by default if available, just like it's done for gcc.

Note clang already disables jump tables when retpoline is enabled, so
there's no need to also pass the fno-jump-tables parameter. Also clang
already passes the return address in a register always on amd64, so
there's no need for any equivalent mindirect-branch-register
parameter.

Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
3 years agox86/retpoline: split retpoline compiler support into separate option
Roger Pau Monne [Fri, 18 Feb 2022 14:34:14 +0000 (15:34 +0100)]
x86/retpoline: split retpoline compiler support into separate option

Keep the previous option as a way to signal generic retpoline support
regardless of the underlying compiler, while introducing a new
CC_HAS_INDIRECT_THUNK that signals whether the underlying compiler
supports retpoline.

No functional change intended.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
3 years agoCI: Coverity tweaks
Andrew Cooper [Sat, 19 Feb 2022 16:28:08 +0000 (16:28 +0000)]
CI: Coverity tweaks

 * Use workflow_dispatch to allow manual creation of the job.
 * Use parallel builds; the workers have two vCPUs.  Also, use the build-*
   targets rather than the ones which expand to dist-*.
 * Shrink the dependency list further.  build-essential covers make and gcc,
   while bridge-utils and iproute2 are runtime dependencies not build
   dependencies.  Alter bzip2 to libbz2-dev.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
3 years agoxen: Rename asprintf() to xasprintf()
Andrew Cooper [Sat, 19 Feb 2022 17:15:27 +0000 (17:15 +0000)]
xen: Rename asprintf() to xasprintf()

Coverity reports that there is a memory leak in
ioreq_server_alloc_rangesets().  This would be true if Xen's implementation of
asprintf() had glibc's return semantics, but it doesn't.

Rename to xasprintf() to reduce confusion for Coverity and other developers.

While at it, fix style issues.  Rearrange ioreq_server_alloc_rangesets() to
use a tabulated switch statement, and not to have a trailing space in the
rangeset name for an unknown range type.

Coverity-ID: 1472735
Coverity-ID: 1500265
Fixes: 780e918a2e54 ("add an implentation of asprintf() for xen")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
3 years agotests/resource: Initialise gnttab before xenforeignmemory_map_resource()
Andrew Cooper [Sat, 19 Feb 2022 16:59:01 +0000 (16:59 +0000)]
tests/resource: Initialise gnttab before xenforeignmemory_map_resource()

It's the 'addr' input to mmap(), and currently consuming stack rubble.

Coverity-ID: 1500115
Fixes: c7a7f14b9299 ("tests/resource: Extend to check that the grant frames are mapped correctly")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
3 years agoCI/Coverity: Do not build QEMU, SeaBIOS or OVMF
Roger Pau Monne [Fri, 18 Feb 2022 12:00:42 +0000 (13:00 +0100)]
CI/Coverity: Do not build QEMU, SeaBIOS or OVMF

Such external projects should have their own Coverity runs, and
there's not much point in also making them part of our scan (apart
from greatly increasing the amount of code scanned).

Trim the dependencies now that QEMU is not built.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
3 years agoCI: add github workflow to run Coverity scans
Roger Pau Monne [Fri, 18 Feb 2022 12:00:41 +0000 (13:00 +0100)]
CI: add github workflow to run Coverity scans

Add a workflow that performs a build like it's done by osstest
Coverity flight and uploads the result to Coverity for analysis. The
build process is exactly the same as the one currently used in
osstest, and it's also run at the same time (bi-weekly).

This has one big benefit over using osstest: we no longer have to care
about keeping the Coverity tools up to date in osstest.

Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
3 years agoCI: Clean up alpine containers
Andrew Cooper [Thu, 17 Feb 2022 21:16:35 +0000 (21:16 +0000)]
CI: Clean up alpine containers

 * `apk --no-cache` is the preferred way of setting up containers, and it does
   shrink the image by a few MB.
 * Neither container needs curl-dev or automake.
 * Flex and bison are needed for Xen, so move to the Xen block.

No practical change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
3 years agoCI: Add gnu grep to alpine containers
Andrew Cooper [Tue, 15 Feb 2022 20:49:10 +0000 (20:49 +0000)]
CI: Add gnu grep to alpine containers

A forthcoming change is going to want more support than busybox's grep can
provide.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
3 years agox86: replace a few do_div() uses
Jan Beulich [Fri, 18 Feb 2022 13:47:25 +0000 (14:47 +0100)]
x86: replace a few do_div() uses

When the macro's "return value" is not used, the macro use can be
replaced by a simply division, avoiding some obfuscation.

According to my observations, no change to generated code.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
3 years agovideo/vesa: adjust (not just) command line option handling
Jan Beulich [Fri, 18 Feb 2022 13:46:27 +0000 (14:46 +0100)]
video/vesa: adjust (not just) command line option handling

Document the remaining option. Add section annotation to the variable
holding the parsed value as well as a few adjacent ones. Adjust the
types of font_height and vga_compat.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
3 years agovideo/vesa: drop "vesa-remap" command line option
Jan Beulich [Fri, 18 Feb 2022 13:45:45 +0000 (14:45 +0100)]
video/vesa: drop "vesa-remap" command line option

If we get mode dimensions wrong, having the remapping size controllable
via command line option isn't going to help much. Drop the option.

While adjusting this also
- add __initdata to the variable,
- use ROUNDUP() instead of open-coding it.

Requested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
3 years agovideo/vesa: drop "vesa-mtrr" command line option
Jan Beulich [Fri, 18 Feb 2022 13:45:14 +0000 (14:45 +0100)]
video/vesa: drop "vesa-mtrr" command line option

Now that we use ioremap_wc() for mapping the frame buffer, there's no
need for this option anymore. As noted in the change introducing the
use of ioremap_wc(), mtrr_add() didn't work in certain cases anyway.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
3 years agovideo/vesa: unmap frame buffer when relinquishing console
Jan Beulich [Fri, 18 Feb 2022 13:44:32 +0000 (14:44 +0100)]
video/vesa: unmap frame buffer when relinquishing console

There's no point in keeping the VA space occupied when no further output
will occur.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
3 years agox86: move .text.kexec
Jan Beulich [Fri, 18 Feb 2022 13:43:58 +0000 (14:43 +0100)]
x86: move .text.kexec

The source file requests page alignment - avoid a padding hole by
placing it right after .text.entry. On average this yields a .text size
reduction of 2k.

Requested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
3 years agox86: introduce ioremap_wc()
Jan Beulich [Fri, 18 Feb 2022 13:42:39 +0000 (14:42 +0100)]
x86: introduce ioremap_wc()

In order for a to-be-introduced ERMS form of memcpy() to not regress
boot performance on certain systems when video output is active, we
first need to arrange for avoiding further dependency on firmware
setting up MTRRs in a way we can actually further modify. On many
systems, due to the continuously growing amounts of installed memory,
MTRRs get configured with at least one huge WB range, and with MMIO
ranges below 4Gb then forced to UC via overlapping MTRRs. mtrr_add(), as
it is today, can't deal with such a setup. Hence on such systems we
presently leave the frame buffer mapped UC, leading to significantly
reduced performance when using REP STOSB / REP MOVSB.

On post-PentiumII hardware (i.e. any that's capable of running 64-bit
code), an effective memory type of WC can be achieved without MTRRs, by
simply referencing the respective PAT entry from the PTEs. While this
will leave the switch to ERMS forms of memset() and memcpy() with
largely unchanged performance, the change here on its own improves
performance on affected systems quite significantly: Measuring just the
individual affected memcpy() invocations yielded a speedup by a factor
of over 250 on my initial (Skylake) test system. memset() isn't getting
improved by as much there, but still by a factor of about 20.

While adding {__,}PAGE_HYPERVISOR_WC, also add {__,}PAGE_HYPERVISOR_WT
to, at the very least, make clear what PTE flags this memory type uses.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
3 years agoIOMMU/PCI: propagate get_device_group_id() failure
Jan Beulich [Fri, 18 Feb 2022 13:19:42 +0000 (14:19 +0100)]
IOMMU/PCI: propagate get_device_group_id() failure

The VT-d hook can indicate an error, which shouldn't be ignored. Convert
the hook's return value to a proper error code, and let that bubble up.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
3 years agoVT-d: replace flush_all_cache()
Jan Beulich [Fri, 18 Feb 2022 13:18:51 +0000 (14:18 +0100)]
VT-d: replace flush_all_cache()

Let's use infrastructure we have available instead of an open-coded
wbinvd() invocation.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
3 years agoVT-d / x86: re-arrange cache syncing
Jan Beulich [Fri, 18 Feb 2022 13:18:01 +0000 (14:18 +0100)]
VT-d / x86: re-arrange cache syncing

The actual function should always have lived in core x86 code; move it
there, replacing get_cache_line_size() by readily available (except very
early during boot; see the code comment) data. Also rename the function.

Drop the respective IOMMU hook, (re)introducing a respective boolean
instead. Replace a true and an almost open-coding instance of
iommu_sync_cache().

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
3 years agox86/cpuid: add CPUID flag for Extended Destination ID support
Roger Pau Monné [Fri, 18 Feb 2022 08:17:47 +0000 (09:17 +0100)]
x86/cpuid: add CPUID flag for Extended Destination ID support

Introduce the CPUID flag to be used in order to signal the support for
using an extended destination ID in IO-APIC RTEs and MSI address
fields. Such format expands the maximum target APIC ID from 255 to
32768 without requiring the usage of interrupt remapping.

The design document describing the feature can be found at:

http://david.woodhou.se/15-bit-msi.pdf

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
3 years agotools/libxl: don't allow IOMMU usage with PoD
Roger Pau Monné [Fri, 18 Feb 2022 08:03:08 +0000 (09:03 +0100)]
tools/libxl: don't allow IOMMU usage with PoD

Prevent libxl from creating guests that attempts to use PoD together
with an IOMMU, even if no devices are actually assigned.

While the hypervisor could support using PoD together with an IOMMU as
long as no devices are assigned, such usage seems doubtful. There's no
guarantee the guest has PoD no longer be active, and thus a later
assignment of a PCI device to such domain could fail.

Preventing the usage of PoD together with an IOMMU at guest creation
avoids having to add checks for active PoD entries in the device
assignment paths.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
3 years agotools/xenstore: add error indicator to ring page
Juergen Gross [Fri, 18 Feb 2022 08:02:48 +0000 (09:02 +0100)]
tools/xenstore: add error indicator to ring page

In case Xenstore is detecting a malicious ring page modification (e.g.
an invalid producer or consumer index set by a guest) it will ignore
the connection of that guest in future.

Add a new error field to the ring page indicating that case. Add a new
feature bit in order to signal the presence of that error field.

Move the ignore_connection() function to xenstored_domain.c in order
to be able to access the ring page for setting the error indicator.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
3 years agox86/console: process softirqs between warning prints
Roger Pau Monné [Fri, 18 Feb 2022 08:02:16 +0000 (09:02 +0100)]
x86/console: process softirqs between warning prints

Process softirqs while printing end of boot warnings. Each warning can
be several lines long, and on slow consoles printing multiple ones
without processing softirqs can result in the watchdog triggering:

(XEN) [   22.277806] ***************************************************
(XEN) [   22.417802] WARNING: CONSOLE OUTPUT IS SYNCHRONOUS
(XEN) [   22.556029] This option is intended to aid debugging of Xen by ensuring
(XEN) [   22.696802] that all output is synchronously delivered on the serial line.
(XEN) [   22.838024] However it can introduce SIGNIFICANT latencies and affect
(XEN) [   22.978710] timekeeping. It is NOT recommended for production use!
(XEN) [   23.119066] ***************************************************
(XEN) [   23.258865] Booted on L1TF-vulnerable hardware with SMT/Hyperthreading
(XEN) [   23.399560] enabled.  Please assess your configuration and choose an
(XEN) [   23.539925] explicit 'smt=<bool>' setting.  See XSA-273.
(XEN) [   23.678860] ***************************************************
(XEN) [   23.818492] Booted on MLPDS/MFBDS-vulnerable hardware with SMT/Hyperthreading
(XEN) [   23.959811] enabled.  Mitigations will not be fully effective.  Please
(XEN) [   24.100396] choose an explicit smt=<bool> setting.  See XSA-297.
(XEN) [   24.240254] *************************************************(XEN) [   24.247302] Watchdog timer detects that CPU0 is stuck!
(XEN) [   24.386785] ----[ Xen-4.17-unstable  x86_64  debug=y  Tainted:   C    ]----
(XEN) [   24.527874] CPU:    0
(XEN) [   24.662422] RIP:    e008:[<ffff82d04025b84a>] drivers/char/ns16550.c#ns16550_tx_ready+0x3a/0x90

Fixes: ee3fd57acd ('xen: add warning infrastructure')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agorwlock: remove unneeded subtraction
Roger Pau Monné [Fri, 18 Feb 2022 08:01:27 +0000 (09:01 +0100)]
rwlock: remove unneeded subtraction

There's no need to subtract _QR_BIAS from the lock value for storing
in the local cnts variable in the read lock slow path: the users of
the value in cnts only care about the writer-related bits and use a
mask to get the value.

Note that further setting of cnts in rspin_until_writer_unlock already
do not subtract _QR_BIAS.

Originally _QR_BIAS was subtracted from the result of
atomic_add_return_acquire in order to prevent GCC from emitting an
unneeded ADD instruction. This being in the lock slow path such
optimizations don't seem likely to make any relevant performance
difference. Also modern GCC and CLANG versions will already avoid
emitting the ADD instruction.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
3 years agox86/Intel: re-indent family 6 switch() in intel_log_freq()
Jan Beulich [Fri, 18 Feb 2022 08:00:10 +0000 (09:00 +0100)]
x86/Intel: re-indent family 6 switch() in intel_log_freq()

This was left at its previous indentation by e6e3cf191d37 ("x86/Intel:
also display CPU freq for family 0xf") to ease review. Remove the now
unnecessary level of indentation.

No functional change.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
3 years agobuild: fix enforce unique symbols for recent clang version
Anthony PERARD [Fri, 18 Feb 2022 07:59:03 +0000 (08:59 +0100)]
build: fix enforce unique symbols for recent clang version

clang 6.0 and newer behave like gcc in regards for the FILE symbol, so
only the filename rather than the full path to the source file.

clang 3.8.1-24 (in our debian:stretch container) and 3.5.0-10
(in our debian:jessie container) do store the full path to the source
file in the FILE symbol.

Also we have commit 81ecb38b83 ("build: provide option to disambiguate
symbol names") which were using clang 5, and LLVM's commit f5040b9685a7
[1] ("Make .file directive to have basename only") which is part of
"llvmorg-6.0.0" tag but not "release/5.x" branch. Both suggest that
clang change of behavior happened with clang 6.0.

This means that we also need to check clang version to figure out
which command we need to use to redefine symbol.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
[1] https://github.com/llvm/llvm-project/commit/f5040b9685a760e584c576e9185295e54635d51e

3 years agobuild: rework cloc recipe
Anthony PERARD [Fri, 18 Feb 2022 07:58:52 +0000 (08:58 +0100)]
build: rework cloc recipe

We are going to make other modifications to the cloc recipe, so this
patch prepare make those modification easier.

We replace the Makefile meta programming by just a shell script which
should be easier to read and is actually faster to execute.

Instead of looking for files in "$(BASEDIR)", we use "." which is give
the same result overall. We also avoid the need for a temporary file
as cloc can read the list of files from stdin.

No change intended to the output of `cloc`.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
3 years agobuild: prepare to always invoke $(MAKE) from xen/, use $(obj)
Anthony PERARD [Fri, 18 Feb 2022 07:58:01 +0000 (08:58 +0100)]
build: prepare to always invoke $(MAKE) from xen/, use $(obj)

In a future patch, when building a subdirectory, we will set
"obj=$subdir" rather than change directory.

Before that, we add "$(obj)" and "$(src)" in as many places as
possible where we will need to know which subdirectory is been built.
"$(obj)" is for files been generated during the build, and "$(src)" is
for files present in the source tree.

For now, we set both to "." in Rules.mk and Makefile.clean.

A few places don't tolerate the addition of "./", this is because make
remove the leading "./" in targets and dependencies in rules, so these
will be change later.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Julien Grall <jgrall@amazon.com>
Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com> # XSM
3 years agobuild: set ALL_OBJS in main Makefile; move prelink.o to main Makefile
Anthony PERARD [Fri, 18 Feb 2022 07:57:03 +0000 (08:57 +0100)]
build: set ALL_OBJS in main Makefile; move prelink.o to main Makefile

This is to avoid arch/$arch/Makefile having to recurse into parents
directories.

This avoid duplication of the logic to build prelink.o between arches.

In order to do that, we cut the $(TARGET) target in the main Makefile in
two, there is a "prepare" phase/target runned before starting to build
"prelink.o" which will prepare "include/" among other things, then all
the $(ALL_OBJS) will be generated in order to build "prelink.o" and
finally $(TARGET) will be generated by calling into "arch/*/" to make
$(TARGET).

Now we don't need to prefix $(ALL_OBJS) with $(BASEDIR) as it is now
only used from the main Makefile. Other changes is to use "$<" instead
of spelling "prelink.o" in the target "$(TARGET)" in both
arch/*/Makefile.

Beside "prelink.o" been at a different location, no other functional
change intended.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Julien Grall <jgrall@amazon.com>
3 years agoxen/docs: Document how to do passthrough without IOMMU
Stefano Stabellini [Mon, 14 Feb 2022 03:19:56 +0000 (03:19 +0000)]
xen/docs: Document how to do passthrough without IOMMU

This commit creates a new doc to document how to do passthrough without IOMMU.

Signed-off-by: Stefano Stabellini <sstabellini@kernel.org>
Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Acked-by: Julien Grall <jgrall@amazon.com>
Tested-by: Stefano Stabellini <sstabellini@kernel.org>
3 years agoxen/arm: if direct-map domain use native UART address and IRQ number for vPL011
Stefano Stabellini [Mon, 14 Feb 2022 03:19:55 +0000 (03:19 +0000)]
xen/arm: if direct-map domain use native UART address and IRQ number for vPL011

We always use a fix address to map the vPL011 to domains. The address
could be a problem for direct-map domains.

So, for domains that are directly mapped, reuse the address of the
physical UART on the platform to avoid potential clashes.

Do the same for the virtual IRQ number: instead of always using
GUEST_VPL011_SPI, try to reuse the physical SPI number if possible.

Signed-off-by: Stefano Stabellini <sstabellini@kernel.org>
Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
Tested-by: Stefano Stabellini <sstabellini@kernel.org>
3 years agoxen/arm: if direct-map domain use native addresses for GICv3
Stefano Stabellini [Mon, 14 Feb 2022 03:19:54 +0000 (03:19 +0000)]
xen/arm: if direct-map domain use native addresses for GICv3

Today we use native addresses to map the GICv3 for Dom0 and fixed
addresses for DomUs.

This patch changes the behavior so that native addresses are used for
all domain which is using the host memory layout

Considering that DOM0 may not always be directly mapped in the future,
this patch introduces a new helper "domain_use_host_layout()" that
wraps both two check "is_domain_direct_mapped(d) || is_hardware_domain(d)"
for more flexible usage.

Signed-off-by: Stefano Stabellini <sstabellini@kernel.org>
Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
Tested-by: Stefano Stabellini <sstabellini@kernel.org>
3 years agoxen/arm: gate make_gicv3_domU_node with CONFIG_GICV3
Penny Zheng [Mon, 14 Feb 2022 03:19:53 +0000 (03:19 +0000)]
xen/arm: gate make_gicv3_domU_node with CONFIG_GICV3

This commit gates function make_gicv3_domU_node with CONFIG_GICV3.

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Tested-by: Stefano Stabellini <sstabellini@kernel.org>