* This prevents inclusion of drivers/staging/greybus/tools/Android.mk
which will conflict in case we have more than 1 kernel tree in AOSP
source dir.
Change-Id: I335bca7b6d6463b1ffc673ab5367603347516e13
commit 228a26b912287934789023b4132ba76065d9491c upstream.
Future CPUs may implement a clearbhb instruction that is sufficient
to mitigate SpectreBHB. CPUs that implement this instruction, but
not CSV2.3 must be affected by Spectre-BHB.
Add support to use this instruction as the BHB mitigation on CPUs
that support it. The instruction is in the hint space so it will
be treated by a NOP as older CPUs.
Reviewed-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
[ modified for stable: Use a KVM vector template instead of alternatives ]
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Git-commit: ed5dec3fae
Git-repo: https://android.googlesource.com/kernel/common/
Change-Id: I78e5a1888e44ef45294ed35226c4cce43f249126
Signed-off-by: Kishor Krishna Bhat <quic_kishkris@quicinc.com>
commit a5905d6af492ee6a4a2205f0d550b3f931b03d03 upstream.
KVM allows the guest to discover whether the ARCH_WORKAROUND SMCCC are
implemented and to preserve that state during migration through its
firmware register interface.
Add the necessary boiler plate for SMCCC_ARCH_WORKAROUND_3.
Reviewed-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
[ kvm code moved to virt/kvm/arm, removed fw regs ABI. Added 32bit stub ]
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Git-commit: 5f051d32b0
Git-repo: https://android.googlesource.com/kernel/common/
Change-Id: Id7ed70b94bd3a3f0575e13691c4c211bb23c1ec3
Signed-off-by: Kishor Krishna Bhat <quic_kishkris@quicinc.com>
commit 558c303c9734af5a813739cd284879227f7297d2 upstream.
Speculation attacks against some high-performance processors can
make use of branch history to influence future speculation.
When taking an exception from user-space, a sequence of branches
or a firmware call overwrites or invalidates the branch history.
The sequence of branches is added to the vectors, and should appear
before the first indirect branch. For systems using KPTI the sequence
is added to the kpti trampoline where it has a free register as the exit
from the trampoline is via a 'ret'. For systems not using KPTI, the same
register tricks are used to free up a register in the vectors.
For the firmware call, arch-workaround-3 clobbers 4 registers, so
there is no choice but to save them to the EL1 stack. This only happens
for entry from EL0 so if we take an exception due to the stack access,
it will not become re-entrant.
For KVM, the existing branch-predictor-hardening vectors are used.
When a spectre version of these vectors is in use, the firmware call
is sufficient to mitigate against Spectre-BHB. For the non-spectre
versions the sequence of branches is added to the indirect vector.
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: <stable@kernel.org> # <v5.17.x 72bb9dcb6c33c arm64: Add Cortex-X2 CPU part definition
Cc: <stable@kernel.org> # <v5.16.x 2d0d656700d67 arm64: Add Neoverse-N2, Cortex-A710 CPU part definition
Cc: <stable@kernel.org> # <v5.10.x 8a6b88e66233f arm64: Add part number for Arm Cortex-A77
[ modified for stable, moved code to cpu_errata.c removed bitmap of
mitigations, use kvm template infrastructure ]
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Git-commit: c20d551744
Git-repo: https://android.googlesource.com/kernel/common/
Change-Id: Ib5f1191b3e81b68edbcd2a0ba4f45e0c1041ad05
Signed-off-by: Kishor Krishna Bhat <quic_kishkris@quicinc.com>
KVM writes the Spectre-v2 mitigation template at the beginning of each
vector when a CPU requires a specific sequence to run.
Because the template is copied, it can not be modified by the alternatives
at runtime. As the KVM template code is intertwined with the bp-hardening
callbacks all templates must have a bp-hardening callback.
Add templates for calling ARCH_WORKAROUND_3 and one for each value of K
in the brancy-loop. Identify these sequences by a new parameter
template_start, and add a copy of install_bp_hardening_cb() that is able to
install them.
Change-Id: I3dbb2d1e06663cbf39b1d212547ac64d569fd72e
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Git-commit: a68912a3ae
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Kishor Krishna Bhat <quic_kishkris@quicinc.com>
commit dee435be76f4117410bbd90573a881fd33488f37 upstream.
Speculation effect against some high-performance processors can
make use of branch history to influence future speculation as part of
a spectre-v2. This is not mitigated by CSV2 meaning CPUs that
previously reported 'Not affected' are now moderately mitigated by CSV2.
Update the value in /sys/devices/system/cpu/vulnerabilities/spectre_v2
to show the state of the BHB mitigation.
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
[ code move to cpu_errata.c for backport ]
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Git-commit: 7b012f6597
Git-repo: https://android.googlesource.com/kernel/common/
Change-Id: I7dc0fc8e5d198c1d9d7b3d8227a93b43d746c97a
Signed-off-by: Kishor Krishna Bhat <quic_kishkris@quicinc.com>
commit bd09128d16fac3c34b80bd6a29088ac632e8ce09 upstream.
The Spectre-BHB workaround adds a firmware call to the vectors. This
is needed on some CPUs but not others. To avoid the unaffected CPU in
a big or little pair from making the firmware call create per cpu vectors.
The per-cpu vectors only apply when returning from EL0.
Systems using KPTI can use the canonical 'full-fat' vectors directly at
EL1, the trampoline exit code will switch to this_cpu_vector on exit to
EL0. Systems not using KPTI should always use this_cpu_vector.
this_cpu_vector will point at a vector in tramp_vecs or
__bp_harden_el1_vectors, depending on whether KPTI is in use.
Change-Id: I9983edd36ee880f3f951f4933b6fa4fca5a91a5b
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Git-commit: 5b5ca2608f
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Kishor Krishna Bhat <quic_kishkris@quicinc.com>
commit b28a8eebe81c186fdb1a0078263b30576c8e1f42 upstream.
The trampoline code needs to use the address of symbols in the wider
kernel, e.g., vectors. PC-relative addressing wouldn't work as the
trampoline code doesn't run at the address the linker expected.
tramp_ventry uses a literal pool, unless CONFIG_RANDOMIZE_BASE is
set in which case it uses the data page as a literal pool because
the data page can be unmapped when running in user-space, which is
required for CPUs vulnerable to meltdown.
Pull this logic out as a macro instead of adding a third copy
of it.
Change-Id: Ic2c62933b6a49c15f572c40e430a83bfa50dedae
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Git-commit: e18876b523
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Kishor Krishna Bhat <quic_kishkris@quicinc.com>
commit ba2689234be92024e5635d30fe744f4853ad97db upstream.
Some CPUs affected by Spectre-BHB need a sequence of branches, or a
firmware call to be run before any indirect branch. This needs to go
in the vectors. No CPU needs both.
While this can be patched in it would run on all CPUs as there is a
single set of vectors. If the only one part of a big or little combination is
affected the unaffected CPUs have to run the mitigation too.
Create extra vectors that include the sequence. Subsequent patches will
allow affected CPUs to select this set of vectors. Later patches will
modify the loop count to match what the CPU requires.
Change-Id: I83274b2d90e65c1c5bf3b11fb236c08731b83456
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Git-commit: 91429ed04e
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Kishor Krishna Bhat <quic_kishkris@quicinc.com>
commit aff65393fa1401e034656e349abd655cfe272de0 upstream.
kpti is an optional feature for systems not using kpti a set of
vectors for the spectre-bhb mitigations is needed.
Add another set of vectors, __bp_harden_el1_vectors that will be
used if a mitigation is needed and kpti is not in use.
The EL1 ventries are repeated verbatim as there is no additional
work needed for entry from EL1.
Change-Id: Ib7ad5dfccb9dd86e75796ae8b6303024c9019848
Reviewed-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Git-commit: 901c0a20aa
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Kishor Krishna Bhat <quic_kishkris@quicinc.com>
commit a9c406e6462ff14956d690de7bbe5131a5677dc9 upstream.
Adding a second set of vectors to .entry.tramp.text will make it
larger than a single 4K page.
Allow the trampoline text to occupy up to three pages by adding two
more fixmap slots. So previous changes to tramp_valias allowed it to reach
beyond a single page.
Change-Id: Iaecfa089c3c7c2edaffe257a841c3dbb0a15e1a4
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Git-commit: 22fdfcf1c2
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Kishor Krishna Bhat <quic_kishkris@quicinc.com>
commit c47e4d04ba0f1ea17353d85d45f611277507e07a upstream.
Spectre-BHB needs to add sequences to the vectors. Having one global
set of vectors is a problem for big/little systems where the sequence
is costly on cpus that are not vulnerable.
Making the vectors per-cpu in the style of KVM's bh_harden_hyp_vecs
requires the vectors to be generated by macros.
Make the kpti remapping of the kernel optional so the macros can be
used without kpti.
Change-Id: I4dc98f5af303700bd3ae02eb80e17772f52fde36
Reviewed-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Git-commit: 9e056623df
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Kishor Krishna Bhat <quic_kishkris@quicinc.com>
commit 13d7a08352a83ef2252aeb464a5e08dfc06b5dfd upstream.
The macros for building the kpti trampoline are all behind
CONFIG_UNMAP_KERNEL_AT_EL0 and in a region that outputs to the
.entry.tramp.text section.
Move the macros out so they can be used to generate other kinds of
trampoline. Only the symbols that need to be guarded by
CONFIG_UNMAP_KERNEL_AT_EL0 and appear in the .entry.tramp.text section.
Change-Id: I126e6fc751f5ed64ed9916ea21026203c1a403d1
Reviewed-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Git-commit: f689fa53bb
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Kishor Krishna Bhat <quic_kishkris@quicinc.com>
commit ed50da7764535f1e24432ded289974f2bf2b0c5a upstream.
The tramp_ventry macro uses tramp_vectors as the address of the vectors
when calculating which ventry in the 'full fat' vectors to branch to.
While there is a one set of tramp_vectors this will be true.
Adding multiple sets of vectors will break this assumption.
Move the generation of the vectors to a macro and pass the start
of the vectors as an argument to tramp_ventry.
Change-Id: Ia9607bea454f9822be63c8be7bfbf55d3dabedd0
Reviewed-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Git-commit: af484e69b5
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Kishor Krishna Bhat <quic_kishkris@quicinc.com>
commit 6c5bf79b69f911560fbf82214c0971af6e58e682 upstream.
Systems using kpti enter and exit the kernel through a trampoline mapping
that is always mapped even when the kernel is not. tramp_valias is a macro
to find the address of a symbol in the trampoline mapping.
Adding the extra sets of vectors will expand the size of the entry.tramp.text
section to beyond 4K. tramp_valias will be unable to generate addresses
for symbols beyond 4K as it uses the 12 bit immediate of the add
instruction.
As there are now two registers available when tramp_alias is called,
use the extra register to avoid the 4K limit of the 12 bit immediate.
Change-Id: I225b66c724b2d167d92d190715f885b465beb388
Reviewed-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Git-commit: ebcdd80d00
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Kishor Krishna Bhat <quic_kishkris@quicinc.com>
commit c091fb6ae059cda563b2a4d93fdbc548ef34e1d6 upstream.
The trampoline code has a data page that holds the address of the vectors,
which is unmapped when running in user-space. This ensures that with
CONFIG_RANDOMIZE_BASE the randomised address of the kernel cannot be
discovered until after the kernel has been mapped.
If the trampoline text page is extended to include multiple sets of
vectors, it will be larger than a single page making it tricky to
find the data page without knowing the size of the trampoline text
pages, which will vary with PAGE_SIZE.
Move the data page to appear before the text page. This allows the
data page to be found without knowing the size of the trampoline text
pages. 'tramp_vectors' is used to refer to the beginning of the
.entry.tramp.text section,so do that explicitly.
Change-Id: I0034331c077fbd3e49b4934d84ec009b0797bf34
Reviewed-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Git-commit: 266b1ef136
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Kishor Krishna Bhat <quic_kishkris@quicinc.com>
commit 03aff3a77a58b5b52a77e00537a42090ad57b80b upstream.
Kpti stashes x30 in far_el1 while it uses x30 for all its work.
Making the vectors a per cpu data structure will require a second
register.
Allow tramp_exit two registers before it unmaps the kernel by
leaving x30 on the stack and stashing x29 in far_el1.
Change-Id: I324402ec90a8401780ac88d6ed91eaa50fb1911a
Reviewed-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Git-commit: 51acb81130
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Kishor Krishna Bhat <quic_kishkris@quicinc.com>
commit d739da1694a0eaef0358a42b76904b611539b77b upstream.
Subsequent patches will add additional sets of vectors that use
the same tricks as the kpti vectors to reach the full-fat vectors.
The full-fat vectors contain some cleanup for kpti that is patched
in by alternatives when kpti is in use. Once there are additional
vectors, the cleanup will be needed in more cases.
But on big or little systems, the cleanup would be harmful if no
trampoline vector were in use. Instead of forcing CPUs that don't
need a trampoline vector to use one make the trampoline cleanup
optional.
Entry at the top of the vectors will skip the cleanup. The trampoline
vectors can then skip the first instruction, triggering the cleanup
to run.
Change-Id: I1db4a6bc057f191cd596e6cd68d50f6a26a4eddf
Reviewed-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Git-commit: 87eccd56c5
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Kishor Krishna Bhat <quic_kishkris@quicinc.com>
In the default design, every time we change the brightness
in torch node it gets reflected only when the enable bit
in the switch node is toggled from 0 to 1. Add new DT property
to optionally allow brightness change to reflect in realtime,
for as long as the switch node enable bit remains set.
Change-Id: I0bc339ba2488177fff144b2771e9a8db425ced84
Signed-off-by: Kavya Nunna <quic_knunna@quicinc.com>
