* 'upstream-f2fs-stable-linux-4.19.y' of https://android.googlesource.com/kernel/common: (560 commits)
f2fs: reset wait_ms to default if any of the victims have been selected
f2fs: fix some format WARNING in debug.c and sysfs.c
f2fs: don't call f2fs_issue_discard_timeout() when discard_cmd_cnt is 0 in f2fs_put_super()
f2fs: fix iostat parameter for discard
f2fs: Fix spelling mistake in label: free_bio_enrty_cache -> free_bio_entry_cache
f2fs: avoid build warnining in extent_cache
f2fs: add block_age-based extent cache
f2fs: allocate the extent_cache by default
f2fs: refactor extent_cache to support for read and more
f2fs: remove unnecessary __init_extent_tree
f2fs: move internal functions into extent_cache.c
f2fs: specify extent cache for read explicitly
f2fs: introduce f2fs_is_readonly() for readability
f2fs: remove F2FS_SET_FEATURE() and F2FS_CLEAR_FEATURE() macro
f2fs: do some cleanup for f2fs module init
MAINTAINERS: Add f2fs bug tracker link
f2fs: remove the unused flush argument to change_curseg
f2fs: open code allocate_segment_by_default
f2fs: remove struct segment_allocation default_salloc_ops
f2fs: introduce discard_urgent_util sysfs node
...
Conflicts:
fs/crypto/hooks.c
fs/ext4/ioctl.c
fs/ext4/namei.c
fs/f2fs/checkpoint.c
fs/f2fs/data.c
fs/f2fs/dir.c
fs/f2fs/f2fs.h
fs/f2fs/file.c
fs/f2fs/gc.c
fs/f2fs/hash.c
fs/f2fs/inline.c
fs/f2fs/namei.c
fs/f2fs/node.c
fs/f2fs/node.h
fs/f2fs/recovery.c
fs/f2fs/segment.c
fs/f2fs/segment.h
fs/f2fs/super.c
fs/f2fs/sysfs.c
fs/f2fs/xattr.c
fs/libfs.c
fs/ubifs/dir.c
fs/unicode/utf8-core.c
fs/verity/enable.c
fs/verity/signature.c
include/linux/fs.h
include/linux/fscrypt.h
include/uapi/linux/fsverity.h
Change-Id: I555b2ac03d0bc864b8993a006994c68c0f4f8c41
Changes in 4.19.271
pNFS/filelayout: Fix coalescing test for single DS
net/ethtool/ioctl: return -EOPNOTSUPP if we have no phy stats
RDMA/srp: Move large values to a new enum for gcc13
f2fs: let's avoid panic if extent_tree is not created
Add exception protection processing for vd in axi_chan_handle_err function
nilfs2: fix general protection fault in nilfs_btree_insert()
xhci-pci: set the dma max_seg_size
usb: xhci: Check endpoint is valid before dereferencing it
xhci: Fix null pointer dereference when host dies
xhci: Add a flag to disable USB3 lpm on a xhci root port level.
prlimit: do_prlimit needs to have a speculation check
USB: serial: option: add Quectel EM05-G (GR) modem
USB: serial: option: add Quectel EM05-G (CS) modem
USB: serial: option: add Quectel EM05-G (RS) modem
USB: serial: option: add Quectel EC200U modem
USB: serial: option: add Quectel EM05CN (SG) modem
USB: serial: option: add Quectel EM05CN modem
USB: misc: iowarrior: fix up header size for USB_DEVICE_ID_CODEMERCS_IOW100
usb: core: hub: disable autosuspend for TI TUSB8041
comedi: adv_pci1760: Fix PWM instruction handling
mmc: sunxi-mmc: Fix clock refcount imbalance during unbind
cifs: do not include page data when checking signature
USB: serial: cp210x: add SCALANCE LPE-9000 device id
usb: host: ehci-fsl: Fix module alias
usb: typec: altmodes/displayport: Add pin assignment helper
usb: typec: altmodes/displayport: Fix pin assignment calculation
usb: gadget: g_webcam: Send color matching descriptor per frame
usb: gadget: f_ncm: fix potential NULL ptr deref in ncm_bitrate()
usb-storage: apply IGNORE_UAS only for HIKSEMI MD202 on RTL9210
serial: pch_uart: Pass correct sg to dma_unmap_sg()
serial: atmel: fix incorrect baudrate setup
gsmi: fix null-deref in gsmi_get_variable
Revert "ext4: fix delayed allocation bug in ext4_clu_mapped for bigalloc + inline"
Revert "ext4: fix reserved cluster accounting at delayed write time"
Revert "ext4: add new pending reservation mechanism"
Revert "ext4: generalize extents status tree search functions"
x86/fpu: Use _Alignof to avoid undefined behavior in TYPE_ALIGN
Linux 4.19.271
Change-Id: I4671da1d3451f065227129f08352c71aea37c854
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
This patch introduces a runtime hot/cold data separation method
for f2fs, in order to improve the accuracy for data temperature
classification, reduce the garbage collection overhead after
long-term data updates.
Enhanced hot/cold data separation can record data block update
frequency as "age" of the extent per inode, and take use of the age
info to indicate better temperature type for data block allocation:
- It records total data blocks allocated since mount;
- When file extent has been updated, it calculate the count of data
blocks allocated since last update as the age of the extent;
- Before the data block allocated, it searches for the age info and
chooses the suitable segment for allocation.
Test and result:
- Prepare: create about 30000 files
* 3% for cold files (with cold file extension like .apk, from 3M to 10M)
* 50% for warm files (with random file extension like .FcDxq, from 1K
to 4M)
* 47% for hot files (with hot file extension like .db, from 1K to 256K)
- create(5%)/random update(90%)/delete(5%) the files
* total write amount is about 70G
* fsync will be called for .db files, and buffered write will be used
for other files
The storage of test device is large enough(128G) so that it will not
switch to SSR mode during the test.
Benefit: dirty segment count increment reduce about 14%
- before: Dirty +21110
- after: Dirty +18286
Signed-off-by: qixiaoyu1 <qixiaoyu1@xiaomi.com>
Signed-off-by: xiongping1 <xiongping1@xiaomi.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Changes in 4.19.262
Makefile.extrawarn: Move -Wcast-function-type-strict to W=1
docs: update mediator information in CoC docs
ARM: fix function graph tracer and unwinder dependencies
fs: fix UAF/GPF bug in nilfs_mdt_destroy
firmware: arm_scmi: Add SCMI PM driver remove routine
dmaengine: xilinx_dma: cleanup for fetching xlnx,num-fstores property
dmaengine: xilinx_dma: Report error in case of dma_set_mask_and_coherent API failure
ARM: dts: fix Moxa SDIO 'compatible', remove 'sdhci' misnomer
scsi: qedf: Fix a UAF bug in __qedf_probe()
net/ieee802154: fix uninit value bug in dgram_sendmsg
um: Cleanup syscall_handler_t cast in syscalls_32.h
um: Cleanup compiler warning in arch/x86/um/tls_32.c
usb: mon: make mmapped memory read only
USB: serial: ftdi_sio: fix 300 bps rate for SIO
mmc: core: Replace with already defined values for readability
mmc: core: Terminate infinite loop in SD-UHS voltage switch
rpmsg: qcom: glink: replace strncpy() with strscpy_pad()
nilfs2: fix NULL pointer dereference at nilfs_bmap_lookup_at_level()
nilfs2: fix leak of nilfs_root in case of writer thread creation failure
nilfs2: replace WARN_ONs by nilfs_error for checkpoint acquisition failure
ceph: don't truncate file in atomic_open
random: clamp credited irq bits to maximum mixed
ALSA: hda: Fix position reporting on Poulsbo
scsi: stex: Properly zero out the passthrough command structure
USB: serial: qcserial: add new usb-id for Dell branded EM7455
random: restore O_NONBLOCK support
random: avoid reading two cache lines on irq randomness
random: use expired timer rather than wq for mixing fast pool
wifi: mac80211_hwsim: avoid mac80211 warning on bad rate
Input: xpad - add supported devices as contributed on github
Input: xpad - fix wireless 360 controller breaking after suspend
ALSA: oss: Fix potential deadlock at unregistration
ALSA: rawmidi: Drop register_mutex in snd_rawmidi_free()
ALSA: usb-audio: Fix potential memory leaks
ALSA: usb-audio: Fix NULL dererence at error path
ALSA: hda/realtek: remove ALC289_FIXUP_DUAL_SPK for Dell 5530
mtd: rawnand: atmel: Unmap streaming DMA mappings
iio: dac: ad5593r: Fix i2c read protocol requirements
usb: add quirks for Lenovo OneLink+ Dock
can: kvaser_usb: Fix use of uninitialized completion
can: kvaser_usb_leaf: Fix overread with an invalid command
can: kvaser_usb_leaf: Fix TX queue out of sync after restart
can: kvaser_usb_leaf: Fix CAN state after restart
fs: dlm: fix race between test_bit() and queue_work()
fs: dlm: handle -EBUSY first in lock arg validation
HID: multitouch: Add memory barriers
quota: Check next/prev free block number after reading from quota file
regulator: qcom_rpm: Fix circular deferral regression
Revert "fs: check FMODE_LSEEK to control internal pipe splicing"
parisc: fbdev/stifb: Align graphics memory size to 4MB
riscv: Allow PROT_WRITE-only mmap()
UM: cpuinfo: Fix a warning for CONFIG_CPUMASK_OFFSTACK
PCI: Sanitise firmware BAR assignments behind a PCI-PCI bridge
fbdev: smscufx: Fix use-after-free in ufx_ops_open()
btrfs: fix race between quota enable and quota rescan ioctl
riscv: fix build with binutils 2.38
nilfs2: fix use-after-free bug of struct nilfs_root
ext4: avoid crash when inline data creation follows DIO write
ext4: fix null-ptr-deref in ext4_write_info
ext4: make ext4_lazyinit_thread freezable
ext4: place buffer head allocation before handle start
livepatch: fix race between fork and KLP transition
ftrace: Properly unset FTRACE_HASH_FL_MOD
ring-buffer: Allow splice to read previous partially read pages
ring-buffer: Check pending waiters when doing wake ups as well
ring-buffer: Fix race between reset page and reading page
KVM: x86/emulator: Fix handing of POP SS to correctly set interruptibility
KVM: nVMX: Unconditionally purge queued/injected events on nested "exit"
selinux: use "grep -E" instead of "egrep"
sh: machvec: Use char[] for section boundaries
wifi: ath10k: add peer map clean up for peer delete in ath10k_sta_state()
wifi: mac80211: allow bw change during channel switch in mesh
bpftool: Fix a wrong type cast in btf_dumper_int
spi: mt7621: Fix an error message in mt7621_spi_probe()
wifi: rtl8xxxu: tighten bounds checking in rtl8xxxu_read_efuse()
spi: qup: add missing clk_disable_unprepare on error in spi_qup_resume()
spi: qup: add missing clk_disable_unprepare on error in spi_qup_pm_resume_runtime()
wifi: rtl8xxxu: Fix skb misuse in TX queue selection
bpf: btf: fix truncated last_member_type_id in btf_struct_resolve
wifi: rtl8xxxu: gen2: Fix mistake in path B IQ calibration
net: fs_enet: Fix wrong check in do_pd_setup
bpf: Ensure correct locking around vulnerable function find_vpid()
spi/omap100k:Fix PM disable depth imbalance in omap1_spi100k_probe
netfilter: nft_fib: Fix for rpath check with VRF devices
spi: s3c64xx: Fix large transfers with DMA
vhost/vsock: Use kvmalloc/kvfree for larger packets.
mISDN: fix use-after-free bugs in l1oip timer handlers
sctp: handle the error returned from sctp_auth_asoc_init_active_key
tcp: fix tcp_cwnd_validate() to not forget is_cwnd_limited
net: rds: don't hold sock lock when cancelling work from rds_tcp_reset_callbacks()
bnx2x: fix potential memory leak in bnx2x_tpa_stop()
once: add DO_ONCE_SLOW() for sleepable contexts
net: mvpp2: fix mvpp2 debugfs leak
drm: bridge: adv7511: fix CEC power down control register offset
drm/mipi-dsi: Detach devices when removing the host
platform/chrome: fix double-free in chromeos_laptop_prepare()
platform/x86: msi-laptop: Fix old-ec check for backlight registering
platform/x86: msi-laptop: Fix resource cleanup
drm/bridge: megachips: Fix a null pointer dereference bug
mmc: au1xmmc: Fix an error handling path in au1xmmc_probe()
ASoC: eureka-tlv320: Hold reference returned from of_find_xxx API
drm/msm/dpu: index dpu_kms->hw_vbif using vbif_idx
ALSA: dmaengine: increment buffer pointer atomically
mmc: wmt-sdmmc: Fix an error handling path in wmt_mci_probe()
ASoC: wm8997: Fix PM disable depth imbalance in wm8997_probe
ASoC: wm5110: Fix PM disable depth imbalance in wm5110_probe
ASoC: wm5102: Fix PM disable depth imbalance in wm5102_probe
memory: of: Fix refcount leak bug in of_get_ddr_timings()
soc: qcom: smsm: Fix refcount leak bugs in qcom_smsm_probe()
soc: qcom: smem_state: Add refcounting for the 'state->of_node'
ARM: dts: turris-omnia: Fix mpp26 pin name and comment
ARM: dts: kirkwood: lsxl: fix serial line
ARM: dts: kirkwood: lsxl: remove first ethernet port
ARM: dts: exynos: correct s5k6a3 reset polarity on Midas family
ARM: Drop CMDLINE_* dependency on ATAGS
ARM: dts: exynos: fix polarity of VBUS GPIO of Origen
iio: adc: at91-sama5d2_adc: fix AT91_SAMA5D2_MR_TRACKTIM_MAX
iio: adc: at91-sama5d2_adc: check return status for pressure and touch
iio: inkern: only release the device node when done with it
iio: ABI: Fix wrong format of differential capacitance channel ABI.
clk: oxnas: Hold reference returned by of_get_parent()
clk: berlin: Add of_node_put() for of_get_parent()
clk: tegra: Fix refcount leak in tegra210_clock_init
clk: tegra: Fix refcount leak in tegra114_clock_init
clk: tegra20: Fix refcount leak in tegra20_clock_init
HSI: omap_ssi: Fix refcount leak in ssi_probe
HSI: omap_ssi_port: Fix dma_map_sg error check
media: exynos4-is: fimc-is: Add of_node_put() when breaking out of loop
tty: xilinx_uartps: Fix the ignore_status
media: xilinx: vipp: Fix refcount leak in xvip_graph_dma_init
RDMA/rxe: Fix "kernel NULL pointer dereference" error
RDMA/rxe: Fix the error caused by qp->sk
dyndbg: fix module.dyndbg handling
dyndbg: let query-modname override actual module name
mtd: devices: docg3: check the return value of devm_ioremap() in the probe
ata: fix ata_id_sense_reporting_enabled() and ata_id_has_sense_reporting()
ata: fix ata_id_has_devslp()
ata: fix ata_id_has_ncq_autosense()
ata: fix ata_id_has_dipm()
md/raid5: Ensure stripe_fill happens on non-read IO with journal
xhci: Don't show warning for reinit on known broken suspend
usb: gadget: function: fix dangling pnp_string in f_printer.c
drivers: serial: jsm: fix some leaks in probe
phy: qualcomm: call clk_disable_unprepare in the error handling
staging: vt6655: fix some erroneous memory clean-up loops
firmware: google: Test spinlock on panic path to avoid lockups
serial: 8250: Fix restoring termios speed after suspend
fsi: core: Check error number after calling ida_simple_get
mfd: intel_soc_pmic: Fix an error handling path in intel_soc_pmic_i2c_probe()
mfd: fsl-imx25: Fix an error handling path in mx25_tsadc_setup_irq()
mfd: lp8788: Fix an error handling path in lp8788_probe()
mfd: lp8788: Fix an error handling path in lp8788_irq_init() and lp8788_irq_init()
mfd: sm501: Add check for platform_driver_register()
dmaengine: ioat: stop mod_timer from resurrecting deleted timer in __cleanup()
spmi: pmic-arb: correct duplicate APID to PPID mapping logic
clk: bcm2835: fix bcm2835_clock_rate_from_divisor declaration
clk: ti: dra7-atl: Fix reference leak in of_dra7_atl_clk_probe
mailbox: bcm-ferxrm-mailbox: Fix error check for dma_map_sg
powerpc/math_emu/efp: Include module.h
powerpc/sysdev/fsl_msi: Add missing of_node_put()
powerpc/pci_dn: Add missing of_node_put()
powerpc/powernv: add missing of_node_put() in opal_export_attrs()
x86/hyperv: Fix 'struct hv_enlightened_vmcs' definition
powerpc/64s: Fix GENERIC_CPU build flags for PPC970 / G5
powerpc: Fix SPE Power ISA properties for e500v1 platforms
iommu/omap: Fix buffer overflow in debugfs
iommu/iova: Fix module config properly
crypto: cavium - prevent integer overflow loading firmware
f2fs: fix race condition on setting FI_NO_EXTENT flag
ACPI: video: Add Toshiba Satellite/Portege Z830 quirk
MIPS: BCM47XX: Cast memcmp() of function to (void *)
powercap: intel_rapl: fix UBSAN shift-out-of-bounds issue
thermal: intel_powerclamp: Use get_cpu() instead of smp_processor_id() to avoid crash
NFSD: Return nfserr_serverfault if splice_ok but buf->pages have data
wifi: brcmfmac: fix invalid address access when enabling SCAN log level
openvswitch: Fix double reporting of drops in dropwatch
openvswitch: Fix overreporting of drops in dropwatch
tcp: annotate data-race around tcp_md5sig_pool_populated
wifi: ath9k: avoid uninit memory read in ath9k_htc_rx_msg()
xfrm: Update ipcomp_scratches with NULL when freed
wifi: brcmfmac: fix use-after-free bug in brcmf_netdev_start_xmit()
Bluetooth: L2CAP: initialize delayed works at l2cap_chan_create()
Bluetooth: hci_sysfs: Fix attempting to call device_add multiple times
can: bcm: check the result of can_send() in bcm_can_tx()
wifi: rt2x00: don't run Rt5592 IQ calibration on MT7620
wifi: rt2x00: set correct TX_SW_CFG1 MAC register for MT7620
wifi: rt2x00: set SoC wmac clock register
wifi: rt2x00: correctly set BBP register 86 for MT7620
net: If sock is dead don't access sock's sk_wq in sk_stream_wait_memory
Bluetooth: L2CAP: Fix user-after-free
r8152: Rate limit overflow messages
drm: Use size_t type for len variable in drm_copy_field()
drm: Prevent drm_copy_field() to attempt copying a NULL pointer
drm/amd/display: fix overflow on MIN_I64 definition
drm/vc4: vec: Fix timings for VEC modes
drm: panel-orientation-quirks: Add quirk for Anbernic Win600
platform/x86: msi-laptop: Change DMI match / alias strings to fix module autoloading
drm/amdgpu: fix initial connector audio value
ARM: dts: imx7d-sdb: config the max pressure for tsc2046
ARM: dts: imx6q: add missing properties for sram
ARM: dts: imx6dl: add missing properties for sram
ARM: dts: imx6qp: add missing properties for sram
ARM: dts: imx6sl: add missing properties for sram
ARM: dts: imx6sll: add missing properties for sram
ARM: dts: imx6sx: add missing properties for sram
media: cx88: Fix a null-ptr-deref bug in buffer_prepare()
scsi: 3w-9xxx: Avoid disabling device if failing to enable it
nbd: Fix hung when signal interrupts nbd_start_device_ioctl()
power: supply: adp5061: fix out-of-bounds read in adp5061_get_chg_type()
staging: vt6655: fix potential memory leak
ata: libahci_platform: Sanity check the DT child nodes number
HID: roccat: Fix use-after-free in roccat_read()
md/raid5: Wait for MD_SB_CHANGE_PENDING in raid5d
usb: host: xhci: Fix potential memory leak in xhci_alloc_stream_info()
usb: musb: Fix musb_gadget.c rxstate overflow bug
Revert "usb: storage: Add quirk for Samsung Fit flash"
nvme: copy firmware_rev on each init
usb: idmouse: fix an uninit-value in idmouse_open
clk: bcm2835: Make peripheral PLLC critical
perf intel-pt: Fix segfault in intel_pt_print_info() with uClibc
net: ieee802154: return -EINVAL for unknown addr type
net/ieee802154: don't warn zero-sized raw_sendmsg()
ext4: continue to expand file system when the target size doesn't reach
md: Replace snprintf with scnprintf
efi: libstub: drop pointless get_memory_map() call
inet: fully convert sk->sk_rx_dst to RCU rules
thermal: intel_powerclamp: Use first online CPU as control_cpu
gcov: support GCC 12.1 and newer compilers
Linux 4.19.262
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
Change-Id: If70223b939e3710c4fbc4f7cc522f07d4b4ffd45
[ Upstream commit 07725adc55c0a414c10acb5c8c86cea34b95ddef ]
The following scenarios exist.
process A: process B:
->f2fs_drop_extent_tree ->f2fs_update_extent_cache_range
->f2fs_update_extent_tree_range
->write_lock
->set_inode_flag
->is_inode_flag_set
->__free_extent_tree // Shouldn't
// have been
// cleaned up
// here
->write_lock
In this case, the "FI_NO_EXTENT" flag is set between
f2fs_update_extent_tree_range and is_inode_flag_set
by other process. it leads to clearing the whole exten
tree which should not have happened. And we fix it by
move the setting it to the range of write_lock.
Fixes:5f281fab9b9a3 ("f2fs: disable extent_cache for fcollapse/finsert inodes")
Signed-off-by: Zhang Qilong <zhangqilong3@huawei.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
ERROR: code indent should use tabs where possible
ERROR: spaces required around that ':'
ERROR: incorrect tab
Found serveral code type errors when review the code and fix it.
There is no function change.
Signed-off-by: Zhang Qilong <zhangqilong3@huawei.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
The trace_f2fs_update_extent_tree_range could not record compressed
block length in the cluster of compress file and we just add it.
Signed-off-by: Zhang Qilong <zhangqilong3@huawei.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
The following scenarios exist.
process A: process B:
->f2fs_drop_extent_tree ->f2fs_update_extent_cache_range
->f2fs_update_extent_tree_range
->write_lock
->set_inode_flag
->is_inode_flag_set
->__free_extent_tree // Shouldn't
// have been
// cleaned up
// here
->write_lock
In this case, the "FI_NO_EXTENT" flag is set between
f2fs_update_extent_tree_range and is_inode_flag_set
by other process. it leads to clearing the whole exten
tree which should not have happened. And we fix it by
move the setting it to the range of write_lock.
Fixes:5f281fab9b9a3 ("f2fs: disable extent_cache for fcollapse/finsert inodes")
Signed-off-by: Zhang Qilong <zhangqilong3@huawei.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch supports to inject fault into f2fs_kmem_cache_alloc().
Usage:
a) echo 32768 > /sys/fs/f2fs/<dev>/inject_type or
b) mount -o fault_type=32768 <dev> <mountpoint>
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Compressed inode may suffer read performance issue due to it can not
use extent cache, so I propose to add this unaligned extent support
to improve it.
Currently, it only works in readonly format f2fs image.
Unaligned extent: in one compressed cluster, physical block number
will be less than logical block number, so we add an extra physical
block length in extent info in order to indicate such extent status.
The idea is if one whole cluster blocks are contiguous physically,
once its mapping info was readed at first time, we will cache an
unaligned (or aligned) extent info entry in extent cache, it expects
that the mapping info will be hitted when rereading cluster.
Merge policy:
- Aligned extents can be merged.
- Aligned extent and unaligned extent can not be merged.
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
then, we can add specified entry into rb-tree with 64-bits segment time
as key.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Filesystem including f2fs should support stable page for special
device like software raid, however there is one missing path that
page could be updated while it is writeback state as below, fix
this.
- gc_node_segment
- f2fs_move_node_page
- __write_node_page
- set_page_writeback
- do_read_inode
- f2fs_init_extent_tree
- __f2fs_init_extent_tree
i_ext->len = 0;
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
- Add and use f2fs_<level> macros
- Convert f2fs_msg to f2fs_printk
- Remove level from f2fs_printk and embed the level in the format
- Coalesce formats and align multi-line arguments
- Remove unnecessary duplicate extern f2fs_msg f2fs.h
Signed-off-by: Joe Perches <joe@perches.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
As Dan Carpenter as below:
The patch df634f444ee9: "f2fs: use rb_*_cached friends" from Oct 4,
2018, leads to the following static checker warning:
fs/f2fs/extent_cache.c:606 f2fs_update_extent_tree_range()
error: uninitialized symbol 'leftmost'.
And also Eric Biggers, and Kyungtae Kim reported, there is an UBSAN
warning described as below:
We report a bug in linux-4.20.2: "UBSAN: Undefined behaviour in
fs/f2fs/extent_cache.c"
kernel config: https://kt0755.github.io/etc/config_v4.20_stable
repro: https://kt0755.github.io/etc/repro.4a3e7.c (f2fs is mounted on
/mnt/f2fs/)
This arose in f2fs_update_extent_tree_range (fs/f2fs/extent_cache.c:605).
It seems that, for some reason, its last argument became "24"
although that was supposed to be bool type.
=========================================
UBSAN: Undefined behaviour in fs/f2fs/extent_cache.c:605:4
load of value 24 is not a valid value for type '_Bool'
CPU: 0 PID: 6774 Comm: syz-executor5 Not tainted 4.20.2 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0xb1/0x118 lib/dump_stack.c:113
ubsan_epilogue+0x12/0x94 lib/ubsan.c:159
__ubsan_handle_load_invalid_value+0x17a/0x1be lib/ubsan.c:457
f2fs_update_extent_tree_range+0x1d4a/0x1d50 fs/f2fs/extent_cache.c:605
f2fs_update_extent_cache+0x2b6/0x350 fs/f2fs/extent_cache.c:804
f2fs_update_data_blkaddr+0x61/0x70 fs/f2fs/data.c:656
f2fs_outplace_write_data+0x1d6/0x4b0 fs/f2fs/segment.c:3140
f2fs_convert_inline_page+0x86d/0x2060 fs/f2fs/inline.c:163
f2fs_convert_inline_inode+0x6b5/0xad0 fs/f2fs/inline.c:208
f2fs_preallocate_blocks+0x78b/0xb00 fs/f2fs/data.c:982
f2fs_file_write_iter+0x31b/0xf40 fs/f2fs/file.c:3062
call_write_iter include/linux/fs.h:1857 [inline]
new_sync_write fs/read_write.c:474 [inline]
__vfs_write+0x538/0x6e0 fs/read_write.c:487
vfs_write+0x1b3/0x520 fs/read_write.c:549
ksys_write+0xde/0x1c0 fs/read_write.c:598
__do_sys_write fs/read_write.c:610 [inline]
__se_sys_write fs/read_write.c:607 [inline]
__x64_sys_write+0x7e/0xc0 fs/read_write.c:607
do_syscall_64+0xbe/0x4f0 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x4497b9
Code: e8 8c 9f 02 00 48 83 c4 18 c3 0f 1f 80 00 00 00 00 48 89 f8 48
89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d
01 f0 ff ff 0f 83 9b 6b fc ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007f1ea15edc68 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 00007f1ea15ee6cc RCX: 00000000004497b9
RDX: 0000000000001000 RSI: 0000000020000140 RDI: 0000000000000013
RBP: 000000000071bea0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff
R13: 000000000000bb50 R14: 00000000006f4bf0 R15: 00007f1ea15ee700
=========================================
As I checked, this uninitialized variable won't cause extent cache
corruption, but in order to avoid such kind of warning of both UBSAN
and smatch, fix to initialize related variable.
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Reported-by: Eric Biggers <ebiggers@google.com>
Reported-by: Kyungtae Kim <kt0755@gmail.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
As rbtree supports caching leftmost node natively, update f2fs codes
to use rb_*_cached helpers to speed up leftmost node visiting.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Remove the verbose license text from f2fs files and replace them with
SPDX tags. This does not change the license of any of the code.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
In the call trace below, we might sleep in function dput().
So in order to avoid sleeping under spin_lock, we remove f2fs_mark_inode_dirty_sync
from __try_update_largest_extent && __drop_largest_extent.
BUG: sleeping function called from invalid context at fs/dcache.c:796
Call trace:
dump_backtrace+0x0/0x3f4
show_stack+0x24/0x30
dump_stack+0xe0/0x138
___might_sleep+0x2a8/0x2c8
__might_sleep+0x78/0x10c
dput+0x7c/0x750
block_dump___mark_inode_dirty+0x120/0x17c
__mark_inode_dirty+0x344/0x11f0
f2fs_mark_inode_dirty_sync+0x40/0x50
__insert_extent_tree+0x2e0/0x2f4
f2fs_update_extent_tree_range+0xcf4/0xde8
f2fs_update_extent_cache+0x114/0x12c
f2fs_update_data_blkaddr+0x40/0x50
write_data_page+0x150/0x314
do_write_data_page+0x648/0x2318
__write_data_page+0xdb4/0x1640
f2fs_write_cache_pages+0x768/0xafc
__f2fs_write_data_pages+0x590/0x1218
f2fs_write_data_pages+0x64/0x74
do_writepages+0x74/0xe4
__writeback_single_inode+0xdc/0x15f0
writeback_sb_inodes+0x574/0xc98
__writeback_inodes_wb+0x190/0x204
wb_writeback+0x730/0xf14
wb_check_old_data_flush+0x1bc/0x1c8
wb_workfn+0x554/0xf74
process_one_work+0x440/0x118c
worker_thread+0xac/0x974
kthread+0x1a0/0x1c8
ret_from_fork+0x10/0x1c
Signed-off-by: Zhikang Zhang <zhangzhikang1@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
As Ted reported:
"Hi, I was looking at f2fs's sources recently, and I noticed that there
is a very large number of non-static symbols which don't have a f2fs
prefix. There's well over a hundred (see attached below).
As one example, in fs/f2fs/dir.c there is:
unsigned char get_de_type(struct f2fs_dir_entry *de)
This function is clearly only useful for f2fs, but it has a generic
name. This means that if any other file system tries to have the same
symbol name, there will be a symbol conflict and the kernel would not
successfully build. It also means that when someone is looking f2fs
sources, it's not at all obvious whether a function such as
read_data_page(), invalidate_blocks(), is a generic kernel function
found in the fs, mm, or block layers, or a f2fs specific function.
You might want to fix this at some point. Hopefully Kent's bcachefs
isn't similarly using genericly named functions, since that might
cause conflicts with f2fs's functions --- but just as this would be a
problem that we would rightly insist that Kent fix, this is something
that we should have rightly insisted that f2fs should have fixed
before it was integrated into the mainline kernel.
acquire_orphan_inode
add_ino_entry
add_orphan_inode
allocate_data_block
allocate_new_segments
alloc_nid
alloc_nid_done
alloc_nid_failed
available_free_memory
...."
This patch adds "f2fs_" prefix for all non-static symbols in order to:
a) avoid conflict with other kernel generic symbols;
b) to indicate the function is f2fs specific one instead of generic
one;
Reported-by: Theodore Ts'o <tytso@mit.edu>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Pointer p is initialized with a value that is never read and is later
re-assigned a new value, hence the initialization is redundant and can
be removed.
Cleans up clang warning:
fs/f2fs/extent_cache.c:463:19: warning: Value stored to 'p' during
its initialization is never read
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If noextent_cache mount option is on, we will never initialize extent tree
in inode, but still we're going to access it in f2fs_drop_extent_tree,
result in kernel panic as below:
BUG: unable to handle kernel NULL pointer dereference at 0000000000000038
IP: _raw_write_lock+0xc/0x30
Call Trace:
? f2fs_drop_extent_tree+0x41/0x70 [f2fs]
f2fs_fallocate+0x5a0/0xdd0 [f2fs]
? common_file_perm+0x47/0xc0
? apparmor_file_permission+0x1a/0x20
vfs_fallocate+0x15b/0x290
SyS_fallocate+0x44/0x70
do_syscall_64+0x6e/0x160
entry_SYSCALL64_slow_path+0x25/0x25
This patch fixes to check extent cache status before using in
f2fs_drop_extent_tree.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Introduce __check_rb_tree_consistence to check consistence of rb-tree
based discard cache in runtime.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Introduce rb-tree based discard cache infrastructure to speed up lookup and
merge operation of discard entry.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
[Jaegeuk Kim: initialize dc to avoid build warning]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
rb-tree lookup/update functions are deeply coupled into extent cache
codes, it's very hard to reuse these basic functions, this patch
extracts common rb-tree operation infrastructure for latter reusing.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Since commit ee6d182f2a ("f2fs: remove syncing inode page in all the
cases") delayed inode element updating from inode cache to node page
cache, so once largest cached extent is updated, we can make inode dirty
immediately instead of checking and updating it in the end of extent
cache update.
The above commit didn't clean up unneeded codes in extent_cache.c, let's
finish the job in this patch.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch replace rw semaphore extent_tree_lock with mutex lock
for no read cases with this lock.
Signed-off-by: Yunlei He <heyunlei@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This is to avoid no free segment bug during checkpoint caused by a number of
dirty inodes.
The case was reported by Chao like this.
1. mount with lazytime option
2. fill 4k file until disk is full
3. sync filesystem
4. read all files in the image
5. umount
In this case, we actually don't need to flush dirty inode to inode page during
checkpoint.
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch reduces to call them across the whole tree.
- sync_inode_page()
- update_inode_page()
- update_inode()
- f2fs_write_inode()
Instead, checkpoint will flush all the dirty inode metadata before syncing
node pages.
Note that, this is doable, since we call mark_inode_dirty_sync() for all
inode's field change which needs to update on-disk inode as well.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch calls mark_inode_dirty_sync() for the following on-disk inode
changes.
-> largest
-> ctime/mtime/atime
-> i_current_depth
-> i_xattr_nid
-> i_pino
-> i_advise
-> i_flags
-> i_mode
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Add a new help f2fs_update_data_blkaddr to clean up redundant codes.
Signed-off-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
There are redundant pointer conversion in following call stack:
- at position a, inode was been converted to f2fs_file_info.
- at position b, f2fs_file_info was been converted to inode again.
- truncate_blocks(inode,..)
- fi = F2FS_I(inode) ---a
- ADDRS_PER_PAGE(node_page, fi)
- addrs_per_inode(fi)
- inode = &fi->vfs_inode ---b
- f2fs_has_inline_xattr(inode)
- fi = F2FS_I(inode)
- is_inode_flag_set(fi,..)
In order to avoid unneeded conversion, alter ADDRS_PER_PAGE and
addrs_per_inode to acept parameter with type of inode pointer.
Signed-off-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
On the worst case, we need to scan the whole radix tree and every rb-tree to
free the victimed extent_nodes when shrinking.
Pengyang initially introduced a victim_list to record the victimed extent_nodes,
and free these extent_nodes by just scanning a list.
Later, Chao Yu enhances the original patch to improve memory footprint by
removing victim list.
The policy of lru list shrinking becomes:
1) lock lru list's lock
2) trylock extent tree's lock
3) remove extent node from lru list
4) unlock lru list's lock
5) do shrink
6) repeat 1) to 5)
Signed-off-by: Hou Pengyang <houpengyang@huawei.com>
Signed-off-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
