Jan Beulich [Thu, 24 Feb 2022 10:17:26 +0000 (11:17 +0100)]
SVM: sync VM-exit perf counters with known VM-exit reasons
This has gone out of sync over time, resulting in NPF and XSETBV exits
incrementing the same counter. Introduce a simplistic mechanism to
hopefully keep things in better sync going forward.
Signed-off-by: Jan Beulich <jbeulich@suse.com> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Andrew Cooper [Fri, 21 Jan 2022 03:47:05 +0000 (03:47 +0000)]
x86/hvm: Drop get_shadow_gs_base() hook and use hvm_get_reg()
This is a trivial accessor for an MSR, so use hvm_get_reg() rather than a
dedicated hook. In arch_get_info_guest(), rework the logic to read GS_SHADOW
only once.
get_hvm_registers() is called on current, meaning that diagnostics print a
stale GS_SHADOW from the previous vcpu context switch. Adjust both
implementations to obtain the correct value.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Jan Beulich <jbeulich@suse.com> Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Andrew Cooper [Mon, 21 Feb 2022 17:09:15 +0000 (17:09 +0000)]
x86/IOMMU: Use altcall, and __initconst_cf_clobber
Most IOMMU hooks are already altcall for performance reasons. Convert the
rest of them so we can harden all the hooks in Control Flow Integrity
configurations. This necessitates the use of iommu_{v,}call() in debug builds
too. Switch to using an ASSERT() as all forms should resolve to &iommu_ops.
Move the root iommu_ops from __read_mostly to __ro_after_init now that the
latter exists.
Since c/s 3330013e6739 ("VT-d / x86: re-arrange cache syncing"), vtd_ops is
not modified and doesn't need a forward declaration, so we can use
__initconst_cf_clobber for both VT-d and AMD.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Andrew Cooper [Tue, 30 Nov 2021 21:31:55 +0000 (21:31 +0000)]
x86/vpmu: Harden indirect branches
As all function pointer calls are resolved to direct calls on boot, clobber
the endbr64 instructions too to make life harder for an attacker which has
managed to hijack a function pointer.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Andrew Cooper [Sun, 7 Nov 2021 11:35:50 +0000 (11:35 +0000)]
x86/ucode: Use altcall, and __initconst_cf_clobber
Microcode loading is not a fastpath, but there are control flow integrity
hardening benefits from using altcall, because it allows us to clobber the
endbr64 instructions on all function pointer targets.
Convert the existing microcode_ops pointer into an __ro_after_init structure,
and move {amd,intel}_ucode_ops into __initconst_cf_clobber.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Andrew Cooper [Mon, 14 Feb 2022 12:12:13 +0000 (12:12 +0000)]
x86/hvm: Use __initdata_cf_clobber for hvm_funcs
Now that all calls through hvm_funcs are fully altcall'd, harden all the svm
and vmx function pointer targets. This drops 106 endbr64 instructions.
Clobbering does come with a theoretical risk. The non-pointer fields of
{svm,vmx}_function_table can in theory happen to form a bit pattern matching a
pointer into .text at a legal endbr64 instruction, but this is expected to be
implausible for anything liable to pass code review.
While at it, move hvm_funcs into __ro_after_init now that this exists.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Jan Beulich <jbeulich@suse.com>
of 1655 on an everything-enabled build of Xen, which is ~12%.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Jan Beulich <jbeulich@suse.com> Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Andrew Cooper [Thu, 4 Nov 2021 19:36:23 +0000 (19:36 +0000)]
x86/altcall: Optimise away endbr64 instruction where possible
With altcall, we convert indirect branches into direct ones. With that
complete, none of the potential targets need an endbr64 instruction.
Furthermore, removing the endbr64 instructions is a security defence-in-depth
improvement, because it limits the options available to an attacker who has
managed to hijack a function pointer.
Introduce new .init.{ro,}data.cf_clobber sections. Have _apply_alternatives()
walk over this, looking for any pointers into .text, and clobber an endbr64
instruction if found. This is some minor structure (ab)use but it works
alarmingly well.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Andrew Cooper [Fri, 26 Nov 2021 15:42:48 +0000 (15:42 +0000)]
x86/altcall: Check and optimise altcall targets
When converting indirect to direct calls, there is no need to execute endbr64
instructions. Detect and optimise this case, leaving a warning in the case
that no endbr64 was found, as it likely indicates a build error.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Andrew Cooper [Mon, 1 Nov 2021 15:17:20 +0000 (15:17 +0000)]
x86: Enable CET Indirect Branch Tracking
With all the pieces now in place, turn CET-IBT on when available.
MSR_S_CET, like SMEP/SMAP, controls Ring1 meaning that ENDBR_EN can't be
enabled for Xen independently of PV32 kernels. As we already disable PV32 for
CET-SS, extend this to all CET, adjusting the documentation/comments as
appropriate.
Introduce a cet=no-ibt command line option to allow the admin to disable IBT
even when everything else is configured correctly.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Andrew Cooper [Mon, 1 Nov 2021 21:54:26 +0000 (21:54 +0000)]
x86/EFI: Disable CET-IBT around Runtime Services calls
UEFI Runtime services, at the time of writing, aren't CET-IBT compatible.
Work is ongoing to address this. In the meantime, unconditionally disable IBT.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Andrew Cooper [Mon, 1 Nov 2021 16:13:29 +0000 (16:13 +0000)]
x86/setup: Rework MSR_S_CET handling for CET-IBT
CET-SS and CET-IBT can be independently controlled, so the configuration of
MSR_S_CET can't be constant any more.
Introduce xen_msr_s_cet_value(), mostly because I don't fancy
writing/maintaining that logic in assembly. Use this in the 3 paths which
alter MSR_S_CET when both features are potentially active.
To active CET-IBT, we only need CR4.CET and MSR_S_CET.ENDBR_EN. This is
common with the CET-SS setup, so reorder the operations to set up CR4 and
MSR_S_CET for any nonzero result from xen_msr_s_cet_value(), and set up
MSR_PL0_SSP and SSP if SHSTK_EN was also set.
Adjust the crash path to disable CET-IBT too.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Andrew Cooper [Mon, 1 Nov 2021 17:08:24 +0000 (17:08 +0000)]
x86/entry: Make IDT entrypoints CET-IBT compatible
Each IDT vector needs to land on an endbr64 instruction. This is especially
important for the #CP handler, which will recurse indefinitely if the endbr64
is missing, eventually escalating to #DF if guard pages are active.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Andrew Cooper [Mon, 1 Nov 2021 09:51:16 +0000 (09:51 +0000)]
x86/entry: Make syscall/sysenter entrypoints CET-IBT compatible
Each of MSR_{L,C}STAR and MSR_SYSENTER_EIP need to land on an endbr64
instruction. For sysenter, this is easy.
Unfortunately for syscall, the stubs are already 29 byte long with a limit of
32. endbr64 is 4 bytes. Luckily, there is a 1 byte instruction which can
move from the stubs into the main handlers.
Move the push %rax out of the stub and into {l,c}star_entry(), allowing room
for the endbr64 instruction when appropriate. Update the comment describing
the entry state.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Andrew Cooper [Mon, 1 Nov 2021 12:36:33 +0000 (12:36 +0000)]
x86/traps: Rework write_stub_trampoline() to not hardcode the jmp
For CET-IBT, we will need to optionally insert an endbr64 instruction at the
start of the stub. Don't hardcode the jmp displacement assuming that it
starts at byte 24 of the stub.
Also add extra comments describing what is going on. The mix of %rax and %rsp
is far from trivial to follow.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Jan Beulich <jbeulich@suse.com>
x86: Build check for embedded endbr64 instructions
An interesting corner case occurs when the byte sequence making up endbr64
ends up on a non-instruction boundary. Such embedded instructions mark legal
indirect branch targets as far as the CPU is concerned, which aren't legal as
far as the logic is concerned.
When CET-IBT is active, check for embedded byte sequences. Example failures
look like:
check-endbr.sh xen-syms Fail: Found 2 embedded endbr64 instructions
0xffff82d040325677: test_endbr64 at /local/xen.git/xen/arch/x86/x86_64/entry.S:28
0xffff82d040352da6: init_done at /local/xen.git/xen/arch/x86/setup.c:675
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Andrew Cooper [Thu, 11 Nov 2021 13:09:19 +0000 (13:09 +0000)]
x86: Use control flow typechecking where possible
Now all indirect branch targets have been annotated, turn on typechecking to
catch issues in the future.
This extension isn't in a released version of GCC yet, so provide a container
to use with the extension included, and add it to CI. RANDCONFIG is necessary
because some stubs for compiled-out subsystems are used as function pointer
targets.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Acked-by: Jan Beulich <jbeulich@suse.com>
Andrew Cooper [Tue, 2 Nov 2021 20:58:59 +0000 (20:58 +0000)]
x86/bugframe: CFI hardening
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
run_in_exception_handler() managed to escape typechecking, as the compiler
can't see where function pointer gets called. After adding some ad-hoc
typechecking, it turns out that dump_execution_state() alone differs in
const-ness from the other users of run_in_exception_handler().
Introduce a new show_execution_state_nonconst() to make the typechecking
happy.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Acked-by: Jan Beulich <jbeulich@suse.com>
Andrew Cooper [Fri, 29 Oct 2021 17:04:02 +0000 (18:04 +0100)]
x86/stack: CFI hardening
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
The function typecheck in switch_stack_and_jump() is incompatible with control
flow typechecking. It's ok for reset_stack_and_jump_ind(), but for
reset_stack_and_jump(), it would force us to endbr64 the targets which are
branched to directly.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Acked-by: Jan Beulich <jbeulich@suse.com>
Andrew Cooper [Fri, 29 Oct 2021 16:28:04 +0000 (17:28 +0100)]
x86/emul: CFI hardening
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
pv_emul_is_mem_write() is only used in a single file. Move it out of its
header file, so it doesn't risk being duplicated in multiple translation
units.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Acked-by: Jan Beulich <jbeulich@suse.com>
Andrew Cooper [Fri, 29 Oct 2021 19:15:24 +0000 (20:15 +0100)]
x86/hvm: CFI hardening for hvm_funcs
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
In svm.c, make a few rearrangements. svm_update_guest_cr() has no external
callers so can become static, but needs moving along with svm_fpu_enter() to
avoid a forward declaration. Move svm_fpu_leave() too, to match. Also move
svm_update_guest_efer() to drop its forward declaration.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Acked-by: Jan Beulich <jbeulich@suse.com>
Andrew Cooper [Thu, 28 Oct 2021 09:58:37 +0000 (10:58 +0100)]
xen/tasklet: CFI hardening
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
The function pointer cast in hvm_vcpu_initialise() is undefined behaviour.
While it happens to function correctly, it is not compatible with control flow
typechecking, so introduce a new hvm_assert_evtchn_irq_tasklet() to handle the
parameter type conversion in a legal way.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Acked-by: Jan Beulich <jbeulich@suse.com>
Andrew Cooper [Thu, 21 Oct 2021 17:38:50 +0000 (18:38 +0100)]
x86: Introduce support for CET-IBT
CET Indirect Branch Tracking is a hardware feature designed to provide
forward-edge control flow integrity, protecting against jump/call oriented
programming.
IBT requires the placement of endbr{32,64} instructions at the target of every
indirect call/jmp, and every entrypoint.
However, the default -fcf-protection=branch places an endbr{32,64} on every
function which far more than necessary, and reduces the quantity of protection
afforded. Therefore, we use manual placement using the cf_check attribute.
It is necessary to check for both compiler and assembler support, as the
notrack prefix can be emitted in certain cases.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Acked-by: Jan Beulich <jbeulich@suse.com>
Andrew Cooper [Wed, 16 Feb 2022 18:24:43 +0000 (18:24 +0000)]
x86/kexec: Annotate embedded data with ELF metadata
Scanning for embedded endbranch instructions involves parsing the .text
disassembly. Data in the kexec trampoline has no ELF metadata, so objdump
treats it as instructions and tries to disassemble. Convert:
The reloc_stack label is the wrong end of the data block to have a size, so
move it to the lowest address and introduce .Lreloc_stack_base as a
replacement. Also, fix the fact that it is misaligned by 2 bytes.
While kexec_reloc_size could gain metadata, it's use in the linker
assertion (while correct) is deeply confusing to follow. Drop it entirely,
using a linker symbol instead to denote the end of the trampoline.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Juergen Gross [Mon, 1 Nov 2021 15:20:10 +0000 (16:20 +0100)]
x86: Don't use the hypercall table for calling compat hypercalls
Today the *_op_compat hypercalls call the modern handler functions by
using the entries from the hypercall table. This is resulting in a
not needed indirect function call which can be avoided by using the
correct handler function directly. This is basically a revert of
commit 1252e282311734 ("x86/pv: Export pv_hypercall_table[] rather
than working around it in several ways"), which reasoning no longer
applies, as shim no longer modifies the hypercall table.
The hypercall table can now be made static as there is no external
reference to it any longer.
Commit 834cb8761051f7 ("x86/PV32: fix physdev_op_compat handling")
can be reverted, too, as using the direct call of the correct handler
is already handled fine without that patch.
Signed-off-by: Juergen Gross <jgross@suse.com> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Juergen Gross [Mon, 1 Nov 2021 15:20:09 +0000 (16:20 +0100)]
x86/pv-shim: Don't modify the hypercall table
When running as pv-shim the hypercall is modified today in order to
replace the functions for __HYPERVISOR_event_channel_op and
__HYPERVISOR_grant_table_op hypercalls.
Change this to call the related functions from the normal handlers
instead when running as shim. The performance implications are not
really relevant, as a normal production hypervisor will not be
configured to support shim mode, so the related calls will be dropped
due to optimisation of the compiler.
Note that for the CONFIG_PV_SHIM_EXCLUSIVE case there is a dummy
wrapper do_grant_table_op() needed, as in this case grant_table.c
isn't being built.
Signed-off-by: Juergen Gross <jgross@suse.com>
Split out of series. To compile in isolation, the compat_platform_op()
prototype needs correcting, and header files need rearranging to avoid the
compat_platform_op_t/multicall_entry_compat_t guest handles being declared
multiple times.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Andrew Cooper [Sat, 30 Oct 2021 21:13:55 +0000 (22:13 +0100)]
xen/sort: Switch to an extern inline implementation
There are exactly 3 callers of sort() in the hypervisor. Callbacks in a tight
loop like this are problematic for performance, especially with Spectre v2
protections, which is why extern inline is used commonly by libraries.
Both ARM callers pass in NULL for the swap function, and while this might seem
like an attractive option at first, it causes generic_swap() to be used, which
forced a byte-wise copy. Provide real swap functions so the compiler can
optimise properly, which is very important for ARM downstreams where
milliseconds until the system is up matters.
This is also important for Control Flow Integrity schemes (e.g. x86 CET-IBT,
ARM BTI), because tagged function(s) performing an arbitrary length swap of
two arbitrary pointers is a very valuable gadget for an attacker.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Jan Beulich <jbeulich@suse.com> Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com> Acked-by: Julien Grall <jgrall@amazon.com>
Roger Pau Monne [Fri, 18 Feb 2022 14:34:16 +0000 (15:34 +0100)]
x86/Kconfig: introduce option to select retpoline usage
Add a new Kconfig option under the "Speculative hardening" section
that allows selecting whether to enable retpoline. This depends on the
underlying compiler having retpoline support.
Requested-by: Andrew Cooper <andrew.cooper3@citrix.com> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Roger Pau Monne [Fri, 18 Feb 2022 14:34:15 +0000 (15:34 +0100)]
x86/clang: add retpoline support
Detect whether the compiler supports clang retpoline option and enable
by default if available, just like it's done for gcc.
Note clang already disables jump tables when retpoline is enabled, so
there's no need to also pass the fno-jump-tables parameter. Also clang
already passes the return address in a register always on amd64, so
there's no need for any equivalent mindirect-branch-register
parameter.
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Roger Pau Monne [Fri, 18 Feb 2022 14:34:14 +0000 (15:34 +0100)]
x86/retpoline: split retpoline compiler support into separate option
Keep the previous option as a way to signal generic retpoline support
regardless of the underlying compiler, while introducing a new
CC_HAS_INDIRECT_THUNK that signals whether the underlying compiler
supports retpoline.
No functional change intended.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Andrew Cooper [Sat, 19 Feb 2022 16:28:08 +0000 (16:28 +0000)]
CI: Coverity tweaks
* Use workflow_dispatch to allow manual creation of the job.
* Use parallel builds; the workers have two vCPUs. Also, use the build-*
targets rather than the ones which expand to dist-*.
* Shrink the dependency list further. build-essential covers make and gcc,
while bridge-utils and iproute2 are runtime dependencies not build
dependencies. Alter bzip2 to libbz2-dev.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Andrew Cooper [Sat, 19 Feb 2022 17:15:27 +0000 (17:15 +0000)]
xen: Rename asprintf() to xasprintf()
Coverity reports that there is a memory leak in
ioreq_server_alloc_rangesets(). This would be true if Xen's implementation of
asprintf() had glibc's return semantics, but it doesn't.
Rename to xasprintf() to reduce confusion for Coverity and other developers.
While at it, fix style issues. Rearrange ioreq_server_alloc_rangesets() to
use a tabulated switch statement, and not to have a trailing space in the
rangeset name for an unknown range type.
Coverity-ID: 1472735
Coverity-ID: 1500265 Fixes: 780e918a2e54 ("add an implentation of asprintf() for xen") Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Andrew Cooper [Sat, 19 Feb 2022 16:59:01 +0000 (16:59 +0000)]
tests/resource: Initialise gnttab before xenforeignmemory_map_resource()
It's the 'addr' input to mmap(), and currently consuming stack rubble.
Coverity-ID: 1500115 Fixes: c7a7f14b9299 ("tests/resource: Extend to check that the grant frames are mapped correctly") Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Roger Pau Monne [Fri, 18 Feb 2022 12:00:42 +0000 (13:00 +0100)]
CI/Coverity: Do not build QEMU, SeaBIOS or OVMF
Such external projects should have their own Coverity runs, and
there's not much point in also making them part of our scan (apart
from greatly increasing the amount of code scanned).
Trim the dependencies now that QEMU is not built.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Roger Pau Monne [Fri, 18 Feb 2022 12:00:41 +0000 (13:00 +0100)]
CI: add github workflow to run Coverity scans
Add a workflow that performs a build like it's done by osstest
Coverity flight and uploads the result to Coverity for analysis. The
build process is exactly the same as the one currently used in
osstest, and it's also run at the same time (bi-weekly).
This has one big benefit over using osstest: we no longer have to care
about keeping the Coverity tools up to date in osstest.
Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Andrew Cooper [Thu, 17 Feb 2022 21:16:35 +0000 (21:16 +0000)]
CI: Clean up alpine containers
* `apk --no-cache` is the preferred way of setting up containers, and it does
shrink the image by a few MB.
* Neither container needs curl-dev or automake.
* Flex and bison are needed for Xen, so move to the Xen block.
No practical change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
Jan Beulich [Fri, 18 Feb 2022 13:46:27 +0000 (14:46 +0100)]
video/vesa: adjust (not just) command line option handling
Document the remaining option. Add section annotation to the variable
holding the parsed value as well as a few adjacent ones. Adjust the
types of font_height and vga_compat.
Signed-off-by: Jan Beulich <jbeulich@suse.com> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Jan Beulich [Fri, 18 Feb 2022 13:45:45 +0000 (14:45 +0100)]
video/vesa: drop "vesa-remap" command line option
If we get mode dimensions wrong, having the remapping size controllable
via command line option isn't going to help much. Drop the option.
While adjusting this also
- add __initdata to the variable,
- use ROUNDUP() instead of open-coding it.
Requested-by: Andrew Cooper <andrew.cooper3@citrix.com> Signed-off-by: Jan Beulich <jbeulich@suse.com> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Jan Beulich [Fri, 18 Feb 2022 13:45:14 +0000 (14:45 +0100)]
video/vesa: drop "vesa-mtrr" command line option
Now that we use ioremap_wc() for mapping the frame buffer, there's no
need for this option anymore. As noted in the change introducing the
use of ioremap_wc(), mtrr_add() didn't work in certain cases anyway.
Signed-off-by: Jan Beulich <jbeulich@suse.com> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Jan Beulich [Fri, 18 Feb 2022 13:43:58 +0000 (14:43 +0100)]
x86: move .text.kexec
The source file requests page alignment - avoid a padding hole by
placing it right after .text.entry. On average this yields a .text size
reduction of 2k.
Requested-by: Andrew Cooper <andrew.cooper3@citrix.com> Signed-off-by: Jan Beulich <jbeulich@suse.com> Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Jan Beulich [Fri, 18 Feb 2022 13:42:39 +0000 (14:42 +0100)]
x86: introduce ioremap_wc()
In order for a to-be-introduced ERMS form of memcpy() to not regress
boot performance on certain systems when video output is active, we
first need to arrange for avoiding further dependency on firmware
setting up MTRRs in a way we can actually further modify. On many
systems, due to the continuously growing amounts of installed memory,
MTRRs get configured with at least one huge WB range, and with MMIO
ranges below 4Gb then forced to UC via overlapping MTRRs. mtrr_add(), as
it is today, can't deal with such a setup. Hence on such systems we
presently leave the frame buffer mapped UC, leading to significantly
reduced performance when using REP STOSB / REP MOVSB.
On post-PentiumII hardware (i.e. any that's capable of running 64-bit
code), an effective memory type of WC can be achieved without MTRRs, by
simply referencing the respective PAT entry from the PTEs. While this
will leave the switch to ERMS forms of memset() and memcpy() with
largely unchanged performance, the change here on its own improves
performance on affected systems quite significantly: Measuring just the
individual affected memcpy() invocations yielded a speedup by a factor
of over 250 on my initial (Skylake) test system. memset() isn't getting
improved by as much there, but still by a factor of about 20.
While adding {__,}PAGE_HYPERVISOR_WC, also add {__,}PAGE_HYPERVISOR_WT
to, at the very least, make clear what PTE flags this memory type uses.
Signed-off-by: Jan Beulich <jbeulich@suse.com> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
projects / people / royger / xen.git / log