Changes in 4.19.320
platform/chrome: cros_ec_debugfs: fix wrong EC message version
hfsplus: fix to avoid false alarm of circular locking
x86/of: Return consistent error type from x86_of_pci_irq_enable()
x86/pci/intel_mid_pci: Fix PCIBIOS_* return code handling
x86/pci/xen: Fix PCIBIOS_* return code handling
x86/platform/iosf_mbi: Convert PCIBIOS_* return codes to errnos
hwmon: (adt7475) Fix default duty on fan is disabled
pwm: stm32: Always do lazy disabling
hwmon: (max6697) Fix underflow when writing limit attributes
hwmon: Introduce SENSOR_DEVICE_ATTR_{RO, RW, WO} and variants
hwmon: (max6697) Auto-convert to use SENSOR_DEVICE_ATTR_{RO, RW, WO}
hwmon: (max6697) Fix swapped temp{1,8} critical alarms
arm64: dts: rockchip: Increase VOP clk rate on RK3328
m68k: atari: Fix TT bootup freeze / unexpected (SCU) interrupt messages
x86/xen: Convert comma to semicolon
m68k: cmpxchg: Fix return value for default case in __arch_xchg()
wifi: brcmsmac: LCN PHY code is used for BCM4313 2G-only device
net/smc: Allow SMC-D 1MB DMB allocations
net/smc: set rmb's SG_MAX_SINGLE_ALLOC limitation only when CONFIG_ARCH_NO_SG_CHAIN is defined
selftests/bpf: Check length of recv in test_sockmap
wifi: cfg80211: fix typo in cfg80211_calculate_bitrate_he()
wifi: cfg80211: handle 2x996 RU allocation in cfg80211_calculate_bitrate_he()
net: fec: Refactor: #define magic constants
net: fec: Fix FEC_ECR_EN1588 being cleared on link-down
ipvs: Avoid unnecessary calls to skb_is_gso_sctp
perf: Fix perf_aux_size() for greater-than 32-bit size
perf: Prevent passing zero nr_pages to rb_alloc_aux()
bna: adjust 'name' buf size of bna_tcb and bna_ccb structures
selftests: forwarding: devlink_lib: Wait for udev events after reloading
media: imon: Fix race getting ictx->lock
saa7134: Unchecked i2c_transfer function result fixed
media: uvcvideo: Allow entity-defined get_info and get_cur
media: uvcvideo: Override default flags
media: renesas: vsp1: Fix _irqsave and _irq mix
media: renesas: vsp1: Store RPF partition configuration per RPF instance
leds: trigger: Unregister sysfs attributes before calling deactivate()
perf report: Fix condition in sort__sym_cmp()
drm/etnaviv: fix DMA direction handling for cached RW buffers
mfd: omap-usb-tll: Use struct_size to allocate tll
ext4: avoid writing unitialized memory to disk in EA inodes
sparc64: Fix incorrect function signature and add prototype for prom_cif_init
PCI: Equalize hotplug memory and io for occupied and empty slots
PCI: Fix resource double counting on remove & rescan
RDMA/mlx4: Fix truncated output warning in mad.c
RDMA/mlx4: Fix truncated output warning in alias_GUID.c
RDMA/rxe: Don't set BTH_ACK_MASK for UC or UD QPs
mtd: make mtd_test.c a separate module
Input: elan_i2c - do not leave interrupt disabled on suspend failure
MIPS: Octeron: remove source file executable bit
powerpc/xmon: Fix disassembly CPU feature checks
macintosh/therm_windtunnel: fix module unload.
bnxt_re: Fix imm_data endianness
ice: Rework flex descriptor programming
netfilter: ctnetlink: use helper function to calculate expect ID
pinctrl: core: fix possible memory leak when pinctrl_enable() fails
pinctrl: single: fix possible memory leak when pinctrl_enable() fails
pinctrl: ti: ti-iodelay: Drop if block with always false condition
pinctrl: ti: ti-iodelay: fix possible memory leak when pinctrl_enable() fails
pinctrl: freescale: mxs: Fix refcount of child
fs/nilfs2: remove some unused macros to tame gcc
nilfs2: avoid undefined behavior in nilfs_cnt32_ge macro
tick/broadcast: Make takeover of broadcast hrtimer reliable
net: netconsole: Disable target before netpoll cleanup
af_packet: Handle outgoing VLAN packets without hardware offloading
ipv6: take care of scope when choosing the src addr
char: tpm: Fix possible memory leak in tpm_bios_measurements_open()
media: venus: fix use after free in vdec_close
hfs: fix to initialize fields of hfs_inode_info after hfs_alloc_inode()
drm/gma500: fix null pointer dereference in cdv_intel_lvds_get_modes
drm/gma500: fix null pointer dereference in psb_intel_lvds_get_modes
m68k: amiga: Turn off Warp1260 interrupts during boot
ext4: check dot and dotdot of dx_root before making dir indexed
ext4: make sure the first directory block is not a hole
wifi: mwifiex: Fix interface type change
leds: ss4200: Convert PCIBIOS_* return codes to errnos
tools/memory-model: Fix bug in lock.cat
hwrng: amd - Convert PCIBIOS_* return codes to errnos
PCI: hv: Return zero, not garbage, when reading PCI_INTERRUPT_PIN
binder: fix hang of unregistered readers
scsi: qla2xxx: Return ENOBUFS if sg_cnt is more than one for ELS cmds
f2fs: fix to don't dirty inode for readonly filesystem
clk: davinci: da8xx-cfgchip: Initialize clk_init_data before use
ubi: eba: properly rollback inside self_check_eba
decompress_bunzip2: fix rare decompression failure
kobject_uevent: Fix OOB access within zap_modalias_env()
rtc: cmos: Fix return value of nvmem callbacks
scsi: qla2xxx: During vport delete send async logout explicitly
scsi: qla2xxx: validate nvme_local_port correctly
perf/x86/intel/pt: Fix topa_entry base length
watchdog/perf: properly initialize the turbo mode timestamp and rearm counter
platform: mips: cpu_hwmon: Disable driver on unsupported hardware
RDMA/iwcm: Fix a use-after-free related to destroying CM IDs
selftests/sigaltstack: Fix ppc64 GCC build
nilfs2: handle inconsistent state in nilfs_btnode_create_block()
kdb: Fix bound check compiler warning
kdb: address -Wformat-security warnings
kdb: Use the passed prompt in kdb_position_cursor()
jfs: Fix array-index-out-of-bounds in diFree
dma: fix call order in dmam_free_coherent
MIPS: SMP-CPS: Fix address for GCR_ACCESS register for CM3 and later
net: ip_rt_get_source() - use new style struct initializer instead of memset
ipv4: Fix incorrect source address in Record Route option
net: bonding: correctly annotate RCU in bond_should_notify_peers()
tipc: Return non-zero value from tipc_udp_addr2str() on error
mISDN: Fix a use after free in hfcmulti_tx()
mm: avoid overflows in dirty throttling logic
PCI: rockchip: Make 'ep-gpios' DT property optional
PCI: rockchip: Use GPIOD_OUT_LOW flag while requesting ep_gpio
parport: parport_pc: Mark expected switch fall-through
parport: Convert printk(KERN_<LEVEL> to pr_<level>(
parport: Standardize use of printmode
dev/parport: fix the array out-of-bounds risk
driver core: Cast to (void *) with __force for __percpu pointer
devres: Fix memory leakage caused by driver API devm_free_percpu()
perf/x86/intel/pt: Export pt_cap_get()
perf/x86/intel/pt: Use helpers to obtain ToPA entry size
perf/x86/intel/pt: Use pointer arithmetics instead in ToPA entry calculation
perf/x86/intel/pt: Split ToPA metadata and page layout
perf/x86/intel/pt: Fix a topa_entry base address calculation
remoteproc: imx_rproc: ignore mapping vdev regions
remoteproc: imx_rproc: Fix ignoring mapping vdev regions
remoteproc: imx_rproc: Skip over memory region when node value is NULL
drm/vmwgfx: Fix overlay when using Screen Targets
net/iucv: fix use after free in iucv_sock_close()
ipv6: fix ndisc_is_useropt() handling for PIO
protect the fetch of ->fd[fd] in do_dup2() from mispredictions
ALSA: usb-audio: Correct surround channels in UAC1 channel map
net: usb: sr9700: fix uninitialized variable use in sr_mdio_read
irqchip/mbigen: Fix mbigen node address layout
x86/mm: Fix pti_clone_pgtable() alignment assumption
net: usb: qmi_wwan: fix memory leak for not ip packets
net: linkwatch: use system_unbound_wq
Bluetooth: l2cap: always unlock channel in l2cap_conless_channel()
net: fec: Stop PPS on driver remove
md/raid5: avoid BUG_ON() while continue reshape after reassembling
clocksource/drivers/sh_cmt: Address race condition for clock events
PCI: Add Edimax Vendor ID to pci_ids.h
udf: prevent integer overflow in udf_bitmap_free_blocks()
wifi: nl80211: don't give key data to userspace
btrfs: fix bitmap leak when loading free space cache on duplicate entry
media: uvcvideo: Ignore empty TS packets
media: uvcvideo: Fix the bandwdith quirk on USB 3.x
jbd2: avoid memleak in jbd2_journal_write_metadata_buffer
s390/sclp: Prevent release of buffer in I/O
SUNRPC: Fix a race to wake a sync task
ext4: fix wrong unit use in ext4_mb_find_by_goal
arm64: Add support for SB barrier and patch in over DSB; ISB sequences
arm64: cpufeature: Force HWCAP to be based on the sysreg visible to user-space
arm64: Add Neoverse-V2 part
arm64: cputype: Add Cortex-X4 definitions
arm64: cputype: Add Neoverse-V3 definitions
arm64: errata: Add workaround for Arm errata 3194386 and 3312417
arm64: cputype: Add Cortex-X3 definitions
arm64: cputype: Add Cortex-A720 definitions
arm64: cputype: Add Cortex-X925 definitions
arm64: errata: Unify speculative SSBS errata logic
arm64: errata: Expand speculative SSBS workaround
arm64: cputype: Add Cortex-X1C definitions
arm64: cputype: Add Cortex-A725 definitions
arm64: errata: Expand speculative SSBS workaround (again)
i2c: smbus: Don't filter out duplicate alerts
i2c: smbus: Improve handling of stuck alerts
i2c: smbus: Send alert notifications to all devices if source not found
bpf: kprobe: remove unused declaring of bpf_kprobe_override
spi: lpspi: Replace all "master" with "controller"
spi: lpspi: Add slave mode support
spi: lpspi: Let watermark change with send data length
spi: lpspi: Add i.MX8 boards support for lpspi
spi: lpspi: add the error info of transfer speed setting
spi: fsl-lpspi: remove unneeded array
spi: spi-fsl-lpspi: Fix scldiv calculation
ALSA: line6: Fix racy access to midibuf
usb: vhci-hcd: Do not drop references before new references are gained
USB: serial: debug: do not echo input by default
usb: gadget: core: Check for unset descriptor
scsi: ufs: core: Fix hba->last_dme_cmd_tstamp timestamp updating logic
tick/broadcast: Move per CPU pointer access into the atomic section
ntp: Clamp maxerror and esterror to operating range
driver core: Fix uevent_show() vs driver detach race
ntp: Safeguard against time_constant overflow
serial: core: check uartclk for zero to avoid divide by zero
power: supply: axp288_charger: Fix constant_charge_voltage writes
power: supply: axp288_charger: Round constant_charge_voltage writes down
tracing: Fix overflow in get_free_elt()
x86/mtrr: Check if fixed MTRRs exist before saving them
drm/bridge: analogix_dp: properly handle zero sized AUX transactions
drm/mgag200: Set DDC timeout in milliseconds
kbuild: Fix '-S -c' in x86 stack protector scripts
netfilter: nf_tables: set element extended ACK reporting support
netfilter: nf_tables: use timestamp to check for set element timeout
netfilter: nf_tables: prefer nft_chain_validate
arm64: cpufeature: Fix the visibility of compat hwcaps
media: uvcvideo: Use entity get_cur in uvc_ctrl_set
drm/i915/gem: Fix Virtual Memory mapping boundaries calculation
exec: Fix ToCToU between perm check and set-uid/gid usage
nvme/pci: Add APST quirk for Lenovo N60z laptop
Linux 4.19.320
Change-Id: I12efa55c04d97f29d34f1a49511948735871b2bd
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
[ Upstream commit 41e71dbb0e0a0fe214545fe64af031303a08524c ]
Guenter reported dodgy crashes on an i386-nosmp build using GCC-11
that had the form of endless traps until entry stack exhaust and then
#DF from the stack guard.
It turned out that pti_clone_pgtable() had alignment assumptions on
the start address, notably it hard assumes start is PMD aligned. This
is true on x86_64, but very much not true on i386.
These assumptions can cause the end condition to malfunction, leading
to a 'short' clone. Guess what happens when the user mapping has a
short copy of the entry text?
Use the correct increment form for addr to avoid alignment
assumptions.
Fixes: 16a3fe634f ("x86/mm/pti: Clone kernel-image on PTE level for 32 bit")
Reported-by: Guenter Roeck <linux@roeck-us.net>
Tested-by: Guenter Roeck <linux@roeck-us.net>
Suggested-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20240731163105.GG33588@noisy.programming.kicks-ass.net
Signed-off-by: Sasha Levin <sashal@kernel.org>
Allow CFI enabled entry code to make indirect calls by also mapping
CFI jump tables, and add a check to ensure the jump table section is
not empty.
Bug: 145297900
Change-Id: I1204c50a139ba62234f3bb4699c50921a831162b
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
[ Upstream commit 825d0b73cd7526b0bb186798583fae810091cbac ]
pti_clone_pmds() assumes that the supplied address is either:
- properly PUD/PMD aligned
or
- the address is actually mapped which means that independently
of the mapping level (PUD/PMD/PTE) the next higher mapping
exists.
If that's not the case the unaligned address can be incremented by PUD or
PMD size incorrectly. All callers supply mapped and/or aligned addresses,
but for the sake of robustness it's better to handle that case properly and
to emit a warning.
[ tglx: Rewrote changelog and added WARN_ON_ONCE() ]
Signed-off-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/alpine.DEB.2.21.1908282352470.1938@nanos.tec.linutronix.de
Signed-off-by: Sasha Levin <sashal@kernel.org>
[ Upstream commit 990784b57731192b7d90c8d4049e6318d81e887d ]
When PTI is disabled at boot time either because the CPU is not affected or
PTI has been disabled on the command line, the boot code still calls into
pti_finalize() which then unconditionally invokes:
pti_clone_entry_text()
pti_clone_kernel_text()
pti_clone_kernel_text() was called unconditionally before the 32bit support
was added and 32bit added the call to pti_clone_entry_text().
The call has no side effects as cloning the page tables into the available
second one, which was allocated for PTI does not create damage. But it does
not make sense either and in case that this functionality would be extended
later this might actually lead to hard to diagnose issues.
Neither function should be called when PTI is runtime disabled. Make the
invocation conditional.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Acked-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Song Liu <songliubraving@fb.com>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20190828143124.063353972@linutronix.de
Signed-off-by: Sasha Levin <sashal@kernel.org>
Fix the section mismatch warning in arch/x86/mm/pti.c:
WARNING: vmlinux.o(.text+0x6972a): Section mismatch in reference from the function pti_clone_pgtable() to the function .init.text:pti_user_pagetable_walk_pte()
The function pti_clone_pgtable() references
the function __init pti_user_pagetable_walk_pte().
This is often because pti_clone_pgtable lacks a __init
annotation or the annotation of pti_user_pagetable_walk_pte is wrong.
FATAL: modpost: Section mismatches detected.
Fixes: 85900ea515 ("x86/pti: Map the vsyscall page if needed")
Reported-by: kbuild test robot <lkp@intel.com>
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andy Lutomirski <luto@kernel.org>
Link: https://lkml.kernel.org/r/43a6d6a3-d69d-5eda-da09-0b1c88215a2a@infradead.org
Merge L1 Terminal Fault fixes from Thomas Gleixner:
"L1TF, aka L1 Terminal Fault, is yet another speculative hardware
engineering trainwreck. It's a hardware vulnerability which allows
unprivileged speculative access to data which is available in the
Level 1 Data Cache when the page table entry controlling the virtual
address, which is used for the access, has the Present bit cleared or
other reserved bits set.
If an instruction accesses a virtual address for which the relevant
page table entry (PTE) has the Present bit cleared or other reserved
bits set, then speculative execution ignores the invalid PTE and loads
the referenced data if it is present in the Level 1 Data Cache, as if
the page referenced by the address bits in the PTE was still present
and accessible.
While this is a purely speculative mechanism and the instruction will
raise a page fault when it is retired eventually, the pure act of
loading the data and making it available to other speculative
instructions opens up the opportunity for side channel attacks to
unprivileged malicious code, similar to the Meltdown attack.
While Meltdown breaks the user space to kernel space protection, L1TF
allows to attack any physical memory address in the system and the
attack works across all protection domains. It allows an attack of SGX
and also works from inside virtual machines because the speculation
bypasses the extended page table (EPT) protection mechanism.
The assoicated CVEs are: CVE-2018-3615, CVE-2018-3620, CVE-2018-3646
The mitigations provided by this pull request include:
- Host side protection by inverting the upper address bits of a non
present page table entry so the entry points to uncacheable memory.
- Hypervisor protection by flushing L1 Data Cache on VMENTER.
- SMT (HyperThreading) control knobs, which allow to 'turn off' SMT
by offlining the sibling CPU threads. The knobs are available on
the kernel command line and at runtime via sysfs
- Control knobs for the hypervisor mitigation, related to L1D flush
and SMT control. The knobs are available on the kernel command line
and at runtime via sysfs
- Extensive documentation about L1TF including various degrees of
mitigations.
Thanks to all people who have contributed to this in various ways -
patches, review, testing, backporting - and the fruitful, sometimes
heated, but at the end constructive discussions.
There is work in progress to provide other forms of mitigations, which
might be less horrible performance wise for a particular kind of
workloads, but this is not yet ready for consumption due to their
complexity and limitations"
* 'l1tf-final' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (75 commits)
x86/microcode: Allow late microcode loading with SMT disabled
tools headers: Synchronise x86 cpufeatures.h for L1TF additions
x86/mm/kmmio: Make the tracer robust against L1TF
x86/mm/pat: Make set_memory_np() L1TF safe
x86/speculation/l1tf: Make pmd/pud_mknotpresent() invert
x86/speculation/l1tf: Invert all not present mappings
cpu/hotplug: Fix SMT supported evaluation
KVM: VMX: Tell the nested hypervisor to skip L1D flush on vmentry
x86/speculation: Use ARCH_CAPABILITIES to skip L1D flush on vmentry
x86/speculation: Simplify sysfs report of VMX L1TF vulnerability
Documentation/l1tf: Remove Yonah processors from not vulnerable list
x86/KVM/VMX: Don't set l1tf_flush_l1d from vmx_handle_external_intr()
x86/irq: Let interrupt handlers set kvm_cpu_l1tf_flush_l1d
x86: Don't include linux/irq.h from asm/hardirq.h
x86/KVM/VMX: Introduce per-host-cpu analogue of l1tf_flush_l1d
x86/irq: Demote irq_cpustat_t::__softirq_pending to u16
x86/KVM/VMX: Move the l1tf_flush_l1d test to vmx_l1d_flush()
x86/KVM/VMX: Replace 'vmx_l1d_flush_always' with 'vmx_l1d_flush_cond'
x86/KVM/VMX: Don't set l1tf_flush_l1d to true from vmx_l1d_flush()
cpu/hotplug: detect SMT disabled by BIOS
...
The kernel image starts out with the Global bit set across the entire
kernel image. The bit is cleared with set_memory_nonglobal() in the
configurations with PCIDs where the performance benefits of the Global bit
are not needed.
However, this is fragile. It means that we are stuck opting *out* of the
less-secure (Global bit set) configuration, which seems backwards. Let's
start more secure (Global bit clear) and then let things opt back in if
they want performance, or are truly mapping common data between kernel and
userspace.
This fixes a bug. Before this patch, there are areas that are unmapped
from the user page tables (like like everything above 0xffffffff82600000 in
the example below). These have the hallmark of being a wrong Global area:
they are not identical in the 'current_kernel' and 'current_user' page
table dumps. They are also read-write, which means they're much more
likely to contain secrets.
Before this patch:
current_kernel:---[ High Kernel Mapping ]---
current_kernel-0xffffffff80000000-0xffffffff81000000 16M pmd
current_kernel-0xffffffff81000000-0xffffffff81e00000 14M ro PSE GLB x pmd
current_kernel-0xffffffff81e00000-0xffffffff81e11000 68K ro GLB x pte
current_kernel-0xffffffff81e11000-0xffffffff82000000 1980K RW GLB NX pte
current_kernel-0xffffffff82000000-0xffffffff82600000 6M ro PSE GLB NX pmd
current_kernel-0xffffffff82600000-0xffffffff82c00000 6M RW PSE GLB NX pmd
current_kernel-0xffffffff82c00000-0xffffffff82e00000 2M RW GLB NX pte
current_kernel-0xffffffff82e00000-0xffffffff83200000 4M RW PSE GLB NX pmd
current_kernel-0xffffffff83200000-0xffffffffa0000000 462M pmd
current_user:---[ High Kernel Mapping ]---
current_user-0xffffffff80000000-0xffffffff81000000 16M pmd
current_user-0xffffffff81000000-0xffffffff81e00000 14M ro PSE GLB x pmd
current_user-0xffffffff81e00000-0xffffffff81e11000 68K ro GLB x pte
current_user-0xffffffff81e11000-0xffffffff82000000 1980K RW GLB NX pte
current_user-0xffffffff82000000-0xffffffff82600000 6M ro PSE GLB NX pmd
current_user-0xffffffff82600000-0xffffffffa0000000 474M pmd
After this patch:
current_kernel:---[ High Kernel Mapping ]---
current_kernel-0xffffffff80000000-0xffffffff81000000 16M pmd
current_kernel-0xffffffff81000000-0xffffffff81e00000 14M ro PSE GLB x pmd
current_kernel-0xffffffff81e00000-0xffffffff81e11000 68K ro GLB x pte
current_kernel-0xffffffff81e11000-0xffffffff82000000 1980K RW NX pte
current_kernel-0xffffffff82000000-0xffffffff82600000 6M ro PSE GLB NX pmd
current_kernel-0xffffffff82600000-0xffffffff82c00000 6M RW PSE NX pmd
current_kernel-0xffffffff82c00000-0xffffffff82e00000 2M RW NX pte
current_kernel-0xffffffff82e00000-0xffffffff83200000 4M RW PSE NX pmd
current_kernel-0xffffffff83200000-0xffffffffa0000000 462M pmd
current_user:---[ High Kernel Mapping ]---
current_user-0xffffffff80000000-0xffffffff81000000 16M pmd
current_user-0xffffffff81000000-0xffffffff81e00000 14M ro PSE GLB x pmd
current_user-0xffffffff81e00000-0xffffffff81e11000 68K ro GLB x pte
current_user-0xffffffff81e11000-0xffffffff82000000 1980K RW NX pte
current_user-0xffffffff82000000-0xffffffff82600000 6M ro PSE GLB NX pmd
current_user-0xffffffff82600000-0xffffffffa0000000 474M pmd
Fixes: 0f561fce4d ("x86/pti: Enable global pages for shared areas")
Reported-by: Hugh Dickins <hughd@google.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: keescook@google.com
Cc: aarcange@redhat.com
Cc: jgross@suse.com
Cc: jpoimboe@redhat.com
Cc: gregkh@linuxfoundation.org
Cc: peterz@infradead.org
Cc: torvalds@linux-foundation.org
Cc: bp@alien8.de
Cc: luto@kernel.org
Cc: ak@linux.intel.com
Cc: Kees Cook <keescook@google.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Juergen Gross <jgross@suse.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Andi Kleen <ak@linux.intel.com>
Link: https://lkml.kernel.org/r/20180802225825.A100C071@viggo.jf.intel.com
The next patch in this series will have to make the definition of
irq_cpustat_t available to entering_irq().
Inclusion of asm/hardirq.h into asm/apic.h would cause circular header
dependencies like
asm/smp.h
asm/apic.h
asm/hardirq.h
linux/irq.h
linux/topology.h
linux/smp.h
asm/smp.h
or
linux/gfp.h
linux/mmzone.h
asm/mmzone.h
asm/mmzone_64.h
asm/smp.h
asm/apic.h
asm/hardirq.h
linux/irq.h
linux/irqdesc.h
linux/kobject.h
linux/sysfs.h
linux/kernfs.h
linux/idr.h
linux/gfp.h
and others.
This causes compilation errors because of the header guards becoming
effective in the second inclusion: symbols/macros that had been defined
before wouldn't be available to intermediate headers in the #include chain
anymore.
A possible workaround would be to move the definition of irq_cpustat_t
into its own header and include that from both, asm/hardirq.h and
asm/apic.h.
However, this wouldn't solve the real problem, namely asm/harirq.h
unnecessarily pulling in all the linux/irq.h cruft: nothing in
asm/hardirq.h itself requires it. Also, note that there are some other
archs, like e.g. arm64, which don't have that #include in their
asm/hardirq.h.
Remove the linux/irq.h #include from x86' asm/hardirq.h.
Fix resulting compilation errors by adding appropriate #includes to *.c
files as needed.
Note that some of these *.c files could be cleaned up a bit wrt. to their
set of #includes, but that should better be done from separate patches, if
at all.
Signed-off-by: Nicolai Stange <nstange@suse.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Global pages are bad for hardening because they potentially let an
exploit read the kernel image via a Meltdown-style attack which
makes it easier to find gadgets.
But, global pages are good for performance because they reduce TLB
misses when making user/kernel transitions, especially when PCIDs
are not available, such as on older hardware, or where a hypervisor
has disabled them for some reason.
This patch implements a basic, sane policy: If you have PCIDs, you
only map a minimal amount of kernel text global. If you do not have
PCIDs, you map all kernel text global.
This policy effectively makes PCIDs something that not only adds
performance but a little bit of hardening as well.
I ran a simple "lseek" microbenchmark[1] to test the benefit on
a modern Atom microserver. Most of the benefit comes from applying
the series before this patch ("entry only"), but there is still a
signifiant benefit from this patch.
No Global Lines (baseline ): 6077741 lseeks/sec
88 Global Lines (entry only): 7528609 lseeks/sec (+23.9%)
94 Global Lines (this patch): 8433111 lseeks/sec (+38.8%)
[1.] https://github.com/antonblanchard/will-it-scale/blob/master/tests/lseek1.c
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Hugh Dickins <hughd@google.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Juergen Gross <jgross@suse.com>
Cc: Kees Cook <keescook@google.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Nadav Amit <namit@vmware.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-mm@kvack.org
Link: http://lkml.kernel.org/r/20180406205518.E3D989EB@viggo.jf.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Summary:
In current kernels, with PTI enabled, no pages are marked Global. This
potentially increases TLB misses. But, the mechanism by which the Global
bit is set and cleared is rather haphazard. This patch makes the process
more explicit. In the end, it leaves us with Global entries in the page
tables for the areas truly shared by userspace and kernel and increases
TLB hit rates.
The place this patch really shines in on systems without PCIDs. In this
case, we are using an lseek microbenchmark[1] to see how a reasonably
non-trivial syscall behaves. Higher is better:
No Global pages (baseline): 6077741 lseeks/sec
88 Global Pages (this set): 7528609 lseeks/sec (+23.9%)
On a modern Skylake desktop with PCIDs, the benefits are tangible, but not
huge for a kernel compile (lower is better):
No Global pages (baseline): 186.951 seconds time elapsed ( +- 0.35% )
28 Global pages (this set): 185.756 seconds time elapsed ( +- 0.09% )
-1.195 seconds (-0.64%)
I also re-checked everything using the lseek1 test[1]:
No Global pages (baseline): 15783951 lseeks/sec
28 Global pages (this set): 16054688 lseeks/sec
+270737 lseeks/sec (+1.71%)
The effect is more visible, but still modest.
Details:
The kernel page tables are inherited from head_64.S which rudely marks
them as _PAGE_GLOBAL. For PTI, we have been relying on the grace of
$DEITY and some insane behavior in pageattr.c to clear _PAGE_GLOBAL.
This patch tries to do better.
First, stop filtering out "unsupported" bits from being cleared in the
pageattr code. It's fine to filter out *setting* these bits but it
is insane to keep us from clearing them.
Then, *explicitly* go clear _PAGE_GLOBAL from the kernel identity map.
Do not rely on pageattr to do it magically.
After this patch, we can see that "GLB" shows up in each copy of the
page tables, that we have the same number of global entries in each
and that they are the *same* entries.
/sys/kernel/debug/page_tables/current_kernel:11
/sys/kernel/debug/page_tables/current_user:11
/sys/kernel/debug/page_tables/kernel:11
9caae8ad6a1fb53aca2407ec037f612d current_kernel.GLB
9caae8ad6a1fb53aca2407ec037f612d current_user.GLB
9caae8ad6a1fb53aca2407ec037f612d kernel.GLB
A quick visual audit also shows that all the entries make sense.
0xfffffe0000000000 is the cpu_entry_area and 0xffffffff81c00000
is the entry/exit text:
0xfffffe0000000000-0xfffffe0000002000 8K ro GLB NX pte
0xfffffe0000002000-0xfffffe0000003000 4K RW GLB NX pte
0xfffffe0000003000-0xfffffe0000006000 12K ro GLB NX pte
0xfffffe0000006000-0xfffffe0000007000 4K ro GLB x pte
0xfffffe0000007000-0xfffffe000000d000 24K RW GLB NX pte
0xfffffe000002d000-0xfffffe000002e000 4K ro GLB NX pte
0xfffffe000002e000-0xfffffe000002f000 4K RW GLB NX pte
0xfffffe000002f000-0xfffffe0000032000 12K ro GLB NX pte
0xfffffe0000032000-0xfffffe0000033000 4K ro GLB x pte
0xfffffe0000033000-0xfffffe0000039000 24K RW GLB NX pte
0xffffffff81c00000-0xffffffff81e00000 2M ro PSE GLB x pmd
[1.] https://github.com/antonblanchard/will-it-scale/blob/master/tests/lseek1.c
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Hugh Dickins <hughd@google.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Juergen Gross <jgross@suse.com>
Cc: Kees Cook <keescook@google.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Nadav Amit <namit@vmware.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-mm@kvack.org
Link: http://lkml.kernel.org/r/20180406205517.C80FBE05@viggo.jf.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
The following code contains dead logic:
162 if (pgd_none(*pgd)) {
163 unsigned long new_p4d_page = __get_free_page(gfp);
164 if (!new_p4d_page)
165 return NULL;
166
167 if (pgd_none(*pgd)) {
168 set_pgd(pgd, __pgd(_KERNPG_TABLE | __pa(new_p4d_page)));
169 new_p4d_page = 0;
170 }
171 if (new_p4d_page)
172 free_page(new_p4d_page);
173 }
There can't be any difference between two pgd_none(*pgd) at L162 and L167,
so it's always false at L171.
Dave Hansen explained:
Yes, the double-test was part of an optimization where we attempted to
avoid using a global spinlock in the fork() path. We would check for
unallocated mid-level page tables without the lock. The lock was only
taken when we needed to *make* an entry to avoid collisions.
Now that it is all single-threaded, there is no chance of a collision,
no need for a lock, and no need for the re-check.
As all these functions are only called during init, mark them __init as
well.
Fixes: 03f4424f34 ("x86/mm/pti: Add functions to clone kernel PMDs")
Signed-off-by: Jike Song <albcamus@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Alan Cox <gnomes@lxorguk.ukuu.org.uk>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Tim Chen <tim.c.chen@linux.intel.com>
Cc: Jiri Koshina <jikos@kernel.org>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Kees Cook <keescook@google.com>
Cc: Andi Lutomirski <luto@amacapital.net>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Greg KH <gregkh@linux-foundation.org>
Cc: David Woodhouse <dwmw@amazon.co.uk>
Cc: Paul Turner <pjt@google.com>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/20180108160341.3461-1-albcamus@gmail.com
Meelis reported that his K8 Athlon64 emits MCE warnings when PTI is
enabled:
[Hardware Error]: Error Addr: 0x0000ffff81e000e0
[Hardware Error]: MC1 Error: L1 TLB multimatch.
[Hardware Error]: cache level: L1, tx: INSN
The address is in the entry area, which is mapped into kernel _AND_ user
space. That's special because we switch CR3 while we are executing
there.
User mapping:
0xffffffff81e00000-0xffffffff82000000 2M ro PSE GLB x pmd
Kernel mapping:
0xffffffff81000000-0xffffffff82000000 16M ro PSE x pmd
So the K8 is complaining that the TLB entries differ. They differ in the
GLB bit.
Drop the GLB bit when installing the user shared mapping.
Fixes: 6dc72c3cbc ("x86/mm/pti: Share entry text PMD")
Reported-by: Meelis Roos <mroos@linux.ee>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Meelis Roos <mroos@linux.ee>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/alpine.DEB.2.20.1801031407180.1957@nanos
