Path: csiph.com!feeder.erje.net!1.eu.feeder.erje.net!news.unit0.net!news.mixmin.net!aioe.org!bofh.it!news.nic.it!robomod From: Enrico Zini Newsgroups: linux.debian.bugs.dist,linux.debian.kernel Subject: Bug#899027: Calling sendfile(2) on sparse files on tmpfs allocates space for them Date: Fri, 18 May 2018 17:30:01 +0200 Message-ID: X-Mailbox-Line: From debian-bugs-dist-request@lists.debian.org Fri May 18 15:21:18 2018 Old-Return-Path: X-Spam-Flag: NO X-Spam-Score: -4.2 Reply-To: Enrico Zini , 899027@bugs.debian.org Resent-To: debian-bugs-dist@lists.debian.org Resent-Cc: Debian Kernel Team X-Debian-Pr-Message: report 899027 X-Debian-Pr-Package: src:linux X-Debian-Pr-Source: linux X-Spam-Bayes: score:0.0000 Tokens: new, 49; hammy, 150; neutral, 848; spammy, 0. spammytokens: hammytokens:0.000-+--H*M:reportbug, 0.000-+--H*MI:reportbug, 0.000-+--H*x:reportbug, 0.000-+--H*UA:reportbug, 0.000-+--i915 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Mailer: reportbug 7.1.10 X-Debian-Message: from BTS X-Mailing-List: archive/latest/1461375 List-ID: List-URL: Approved: robomod@news.nic.it Lines: 659 Organization: linux.* mail to news gateway Sender: robomod@news.nic.it X-Original-Date: Fri, 18 May 2018 17:19:04 +0200 X-Original-Message-ID: <152665674499.28247.1056582105639615817.reportbug@ploma.enricozini.org> Xref: csiph.com linux.debian.bugs.dist:898435 linux.debian.kernel:61048 Package: src:linux Version: 4.16.5-1 Severity: normal Hello, the code below, if run on a normal file system prints: After creation: 0/1048576 After read: 0/1048576 but if run on a tmpfs it prints: After creation: 0/1048576 After read: 1048576/1048576 This unexpected allocation happens only when using sendfile(2), not while reading the file with read(2). I observed this behaviour on a range of kernels, from Centos7's 3.10.0 to 4.16 currently running in my machine. This is the code to reproduce it: #include #include #include #include #include #include int main() { int fd = open("test", O_WRONLY | O_CREAT, 0644); if (fd == -1) { perror("cannot open for writing"); return 1; } if (ftruncate(fd, 1024*1024) == -1) { perror("cannot ftruncate"); return 1; } close(fd); struct stat st; if (stat("test", &st) == -1) { perror("cannot stat after creation"); return 1; } fprintf(stdout, "After creation: %d/%d\n", st.st_blocks * 512, st.st_size); fd = open("test", O_RDONLY); if (fd == -1) { perror("cannot open for reading"); return 1; } int out = open("/dev/null", O_WRONLY); if (out == -1) { perror("cannot open /dev/null"); return 1; } off_t offset = 0; ssize_t res = sendfile(out, fd, &offset, st.st_size); if (res == -1) { perror("cannot sendfile"); return 1; } if (res != st.st_size) fprintf(stderr, "warning: partial sendfile\n"); if (stat("test", &st) == -1) { perror("cannot stat after read"); return 1; } fprintf(stdout, "After read: %d/%d\n", st.st_blocks * 512, st.st_size); close(out); close(fd); unlink("test"); return 0; } Thanks, Enrico -- Package-specific info: ** Version: Linux version 4.16.0-1-amd64 (debian-kernel@lists.debian.org) (gcc version 7.3.0 (Debian 7.3.0-17)) #1 SMP Debian 4.16.5-1 (2018-04-29) ** Command line: BOOT_IMAGE=/vmlinuz-4.16.0-1-amd64 root=/dev/mapper/ploma--vg-root ro quiet ** Tainted: O (4096) * Out-of-tree module has been loaded. ** Kernel log: Unable to read kernel log; any relevant messages should be attached ** Model information sys_vendor: HP product_name: HP EliteBook x360 1020 G2 product_version: chassis_vendor: HP chassis_version: bios_vendor: HP bios_version: P90 Ver. 01.01 board_vendor: HP board_name: 8300 board_version: KBC Version 56.3A ** Loaded modules: fuse ctr ccm sd_mod sg cdc_ether usbnet r8152 mii uas usb_storage scsi_mod pci_stub vboxpci(O) vboxnetadp(O) vboxnetflt(O) vboxdrv(O) rfcomm bnep wacom usbhid hid_multitouch arc4 hid_generic binfmt_misc nls_ascii nls_cp437 vfat fat iwlmvm mac80211 snd_hda_codec_hdmi snd_hda_codec_generic snd_soc_skl wmi_bmof hp_wmi snd_soc_skl_ipc snd_hda_ext_core iwlwifi snd_soc_sst_dsp snd_soc_sst_ipc snd_soc_acpi snd_soc_core snd_compress cfg80211 intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp btusb kvm_intel btrtl btbcm btintel kvm bluetooth irqbypass intel_cstate intel_uncore intel_rapl_perf drbg ansi_cprng uvcvideo ecdh_generic efi_pstore videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 rfkill videobuf2_common joydev evdev snd_hda_intel videodev media serio_raw snd_hda_codec efivars pcspkr snd_hda_core snd_hwdep snd_pcm i915 snd_timer snd iTCO_wdt tpm_crb iTCO_vendor_support soundcore drm_kms_helper shpchp idma64 drm intel_lpss_pci processor_thermal_device intel_lpss i2c_algo_bit intel_pch_thermal intel_soc_dts_iosf tpm_tis battery wmi soc_button_array int3403_thermal intel_vbtn int340x_thermal_zone tpm_tis_core intel_hid tpm hp_wireless rng_core int3400_thermal sparse_keymap acpi_thermal_rel video acpi_pad ac button parport_pc ppdev lp parport efivarfs ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 fscrypto ecb btrfs zstd_decompress zstd_compress xxhash algif_skcipher af_alg dm_crypt dm_mod raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c crc32c_generic raid1 raid0 multipath linear md_mod crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel pcbc aesni_intel aes_x86_64 crypto_simd glue_helper cryptd nvme xhci_pci psmouse xhci_hcd nvme_core i2c_i801 intel_ish_ipc usbcore usb_common intel_ishtp i2c_hid thermal hid ** PCI devices: 00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v6/7th Gen Core Processor Host Bridge/DRAM Registers [8086:5904] (rev 02) Subsystem: Hewlett-Packard Company Xeon E3-1200 v6/7th Gen Core Processor Host Bridge/DRAM Registers [103c:8300] Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- SERR- 00:02.0 VGA compatible controller [0300]: Intel Corporation HD Graphics 620 [8086:5916] (rev 02) (prog-if 00 [VGA controller]) Subsystem: Hewlett-Packard Company HD Graphics 620 [103c:8300] Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- Kernel driver in use: i915 Kernel modules: i915 00:04.0 Signal processing controller [1180]: Intel Corporation Skylake Processor Thermal Subsystem [8086:1903] (rev 02) Subsystem: Hewlett-Packard Company Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Thermal Subsystem [103c:8300] Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- SERR- Kernel driver in use: proc_thermal Kernel modules: processor_thermal_device 00:13.0 Non-VGA unclassified device [0000]: Intel Corporation Sunrise Point-LP Integrated Sensor Hub [8086:9d35] (rev 21) Subsystem: Hewlett-Packard Company Sunrise Point-LP Integrated Sensor Hub [103c:8300] Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- Kernel modules: intel_ish_ipc 00:14.0 USB controller [0c03]: Intel Corporation Sunrise Point-LP USB 3.0 xHCI Controller [8086:9d2f] (rev 21) (prog-if 30 [XHCI]) Subsystem: Hewlett-Packard Company Sunrise Point-LP USB 3.0 xHCI Controller [103c:8300] Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- SERR- Kernel driver in use: xhci_hcd Kernel modules: xhci_pci 00:14.2 Signal processing controller [1180]: Intel Corporation Sunrise Point-LP Thermal subsystem [8086:9d31] (rev 21) Subsystem: Hewlett-Packard Company Sunrise Point-LP Thermal subsystem [103c:8300] Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- Kernel driver in use: intel_pch_thermal Kernel modules: intel_pch_thermal 00:15.0 Signal processing controller [1180]: Intel Corporation Sunrise Point-LP Serial IO I2C Controller #0 [8086:9d60] (rev 21) Subsystem: Hewlett-Packard Company Sunrise Point-LP Serial IO I2C Controller [103c:8300] Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- Kernel driver in use: intel-lpss Kernel modules: intel_lpss_pci 00:15.1 Signal processing controller [1180]: Intel Corporation Sunrise Point-LP Serial IO I2C Controller #1 [8086:9d61] (rev 21) Subsystem: Hewlett-Packard Company Sunrise Point-LP Serial IO I2C Controller [103c:8300] Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- Kernel driver in use: intel-lpss Kernel modules: intel_lpss_pci 00:15.2 Signal processing controller [1180]: Intel Corporation Sunrise Point-LP Serial IO I2C Controller #2 [8086:9d62] (rev 21) Subsystem: Hewlett-Packard Company Sunrise Point-LP Serial IO I2C Controller [103c:8300] Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- Kernel driver in use: intel-lpss Kernel modules: intel_lpss_pci 00:1c.0 PCI bridge [0604]: Intel Corporation Sunrise Point-LP PCI Express Root Port [8086:9d10] (rev f1) (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- TAbort- Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: Kernel driver in use: pcieport Kernel modules: shpchp 00:1c.4 PCI bridge [0604]: Intel Corporation Sunrise Point-LP PCI Express Root Port #5 [8086:9d14] (rev f1) (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- TAbort- Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: Kernel driver in use: pcieport Kernel modules: shpchp 00:1d.0 PCI bridge [0604]: Intel Corporation Sunrise Point-LP PCI Express Root Port #9 [8086:9d18] (rev f1) (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- TAbort- Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: Kernel driver in use: pcieport Kernel modules: shpchp 00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:9d4e] (rev 21) Subsystem: Hewlett-Packard Company Device [103c:8300] Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- TAbort- SERR- TAbort- SERR- Kernel driver in use: snd_hda_intel Kernel modules: snd_hda_intel, snd_soc_skl 00:1f.4 SMBus [0c05]: Intel Corporation Sunrise Point-LP SMBus [8086:9d23] (rev 21) Subsystem: Hewlett-Packard Company Sunrise Point-LP SMBus [103c:8300] Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- SERR- TAbort- SERR- TAbort- Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: Kernel driver in use: pcieport Kernel modules: shpchp 02:00.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02) (prog-if 00 [Normal decode]) Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- TAbort- Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: Kernel driver in use: pcieport Kernel modules: shpchp 02:01.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02) (prog-if 00 [Normal decode]) Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- TAbort- Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: Kernel driver in use: pcieport Kernel modules: shpchp 02:02.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02) (prog-if 00 [Normal decode]) Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- TAbort- Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: Kernel driver in use: pcieport Kernel modules: shpchp 02:04.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02) (prog-if 00 [Normal decode]) Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- TAbort- Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: Kernel driver in use: pcieport Kernel modules: shpchp 37:00.0 USB controller [0c03]: Intel Corporation JHL6540 Thunderbolt 3 USB Controller (C step) [Alpine Ridge 4C 2016] [8086:15d4] (rev 02) (prog-if 30 [XHCI]) Subsystem: Device [2222:1111] Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- Kernel driver in use: xhci_hcd Kernel modules: xhci_pci 6c:00.0 Network controller [0280]: Intel Corporation Wireless 8265 / 8275 [8086:24fd] (rev 78) Subsystem: Intel Corporation Wireless 8265 / 8275 [8086:8010] Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- Kernel driver in use: iwlwifi Kernel modules: iwlwifi 6d:00.0 Non-Volatile memory controller [0108]: Toshiba America Info Systems XG4 NVMe SSD Controller [1179:0115] (rev 01) (prog-if 02 [NVM Express]) Subsystem: Toshiba America Info Systems XG4 NVMe SSD Controller [1179:0001] Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- Kernel driver in use: nvme Kernel modules: nvme ** USB devices: Bus 004 Device 004: ID 05e3:0749 Genesys Logic, Inc. Bus 004 Device 006: ID 0bda:8153 Realtek Semiconductor Corp. Bus 004 Device 003: ID 0bda:0411 Realtek Semiconductor Corp. Bus 004 Device 002: ID 2109:0817 VIA Labs, Inc. Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 001 Device 004: ID 8087:0a2b Intel Corp. Bus 001 Device 003: ID 05c8:0808 Cheng Uei Precision Industry Co., Ltd (Foxlink) Bus 001 Device 002: ID 138a:0092 Validity Sensors, Inc. Bus 001 Device 079: ID 04a9:190f Canon, Inc. Bus 001 Device 081: ID 04d9:2013 Holtek Semiconductor, Inc. Keyboard [Das Keyboard] Bus 001 Device 082: ID 046d:c069 Logitech, Inc. M-U0007 [Corded Mouse M500] Bus 001 Device 080: ID 05e3:0608 Genesys Logic, Inc. Hub Bus 001 Device 078: ID 05e3:0610 Genesys Logic, Inc. 4-port hub Bus 001 Device 077: ID 0bda:5411 Realtek Semiconductor Corp. Bus 001 Device 076: ID 2109:0102 VIA Labs, Inc. Bus 001 Device 075: ID 2109:2817 VIA Labs, Inc. Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub -- System Information: Debian Release: buster/sid APT prefers testing APT policy: (500, 'testing') Architecture: amd64 (x86_64) Kernel: Linux 4.16.0-1-amd64 (SMP w/4 CPU cores) Locale: LANG=en_IE.UTF-8, LC_CTYPE=en_IE.UTF-8 (charmap=UTF-8), LANGUAGE=en_IE.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages linux-image-4.16.0-1-amd64 depends on: ii initramfs-tools [linux-initramfs-tool] 0.130 ii kmod 25-1 ii linux-base 4.5 Versions of packages linux-image-4.16.0-1-amd64 recommends: ii apparmor 2.12-4 ii firmware-linux-free 3.4 ii irqbalance 1.3.0-0.1+b1 Versions of packages linux-image-4.16.0-1-amd64 suggests: pn debian-kernel-handbook ii extlinux 3:6.03+dfsg1-2 ii grub-efi-amd64 2.02+dfsg1-4 pn linux-doc-4.16 Versions of packages linux-image-4.16.0-1-amd64 is related to: pn firmware-amd-graphics pn firmware-atheros pn firmware-bnx2 pn firmware-bnx2x pn firmware-brcm80211 pn firmware-cavium pn firmware-intel-sound pn firmware-intelwimax pn firmware-ipw2x00 pn firmware-ivtv ii firmware-iwlwifi 20170823-1 pn firmware-libertas pn firmware-linux-nonfree pn firmware-misc-nonfree pn firmware-myricom pn firmware-netxen pn firmware-qlogic pn firmware-realtek pn firmware-samsung pn firmware-siano pn firmware-ti-connectivity pn xen-hypervisor -- no debconf information