Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #208610

Re: Speed Problem Copying Files

From Lothar Schilling <ls@proasyl.de>
Newsgroups linux.debian.user
Subject Re: Speed Problem Copying Files
Date 2019-05-13 10:40 +0200
Message-ID <xXeF3-yg-7@gated-at.bofh.it> (permalink)
References <xVN4f-3Rc-15@gated-at.bofh.it> <xW6zT-7v1-1@gated-at.bofh.it> <xW9xL-M5-3@gated-at.bofh.it>
Organization linux.* mail to news gateway

Show all headers | View raw

Am 10.05.2019 um 10:51 schrieb Lothar Schilling:
> Am 10.05.2019 um 07:48 schrieb David Christensen:
>> On 5/9/19 1:49 AM, Lothar Schilling wrote:
>>> Hi everybody,
>>>
>>> for years I have used CentOS for our server landscape. Now I decided to
>>> give Debian a try. I just set up a Stretch 9.8 system supposed to become
>>> our main backup server. So I set up a backup job wih rsync. But the
>>> going is really very very slooooow. Trying to figure out what's
>>> happening:
>>>
>>>    * iperf -c [host] => bandwith almost 1000 Mbit, that's fine.
>>>    * dd if=/dev/zero of=/daten/testfile bs=1G count=10 oflag=direct =>
>>>      10737418240 Bytes (11 GB, 10 GiB) kopiert, 40,4992 s, 265 MB/s, so
>>>      that's fine as well
>>>
>>> But whenever I try to rsync, cp or scp - whether copy files on the local
>>> hard disk only or over the network - speed goes down to 500 kB/s.
>>>
>>> Filesystem is ext4.
>>>
>>> I don't have any clue about what's going on. Any kind of help would be
>>> appreciated, thank you!
>>>
>>> Lothar
>>
>> On 5/9/19 2:46 AM, Lothar Schilling wrote:> Am 09.05.2019 um 11:14
>> schrieb Jonas Smedegaard:
>>>> Is _only_ transfer speed affected?
>>>>
>>>> I am no expert in this, but imagine that if you rsync massive amounts
>>>> involving hardlinks then memory becomes a problem too.
>>>>
>>>> Perhaps run atop to monitor bottlenecks live
>>>>
>>>>
>>>>    - Jonas
>>>>
>>> It is most definitely not a memory or cpu problem. It's a HP Proliant
>>> with 32 GB RAM and a Intel Xeon CPU 2.40GHz with 4 cores. Also the
>>> problem stays the same if I just copy one large file.
>>
>> On 5/9/19 3:26 AM, Lothar Schilling wrote:> Am 09.05.2019 um 11:51
>> schrieb Kevin DAGNEAUX:
>>>> Check if it's related to the disk speed (hdparm and/or iotop).
>>>>
>>>> Kevin
>>>>
>>> hdparm -tT /dev/sda
>>> /dev/sda:
>>>    Timing cached reads:   13348 MB in  2.00 seconds = 6683.42 MB/sec
>>>    Timing buffered disk reads: 1014 MB in  3.00 seconds = 337.72 MB/sec
>>>
>>> iotop -o (for rsync and cp)
>>> Total DISK READ :       0.00 B/s | Total DISK WRITE :     476.15 K/s
>>> Actual DISK READ:       0.00 B/s | Actual DISK WRITE:     487.86 K/s
>>>     TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND
>>> 19531 be/4 root        0.00 B/s  476.15 K/s  0.00 % 99.24 % rsync
>>> --info=progress2 /daten/testfile /daten/testfile2
>>>
>>> iotop -o (for dd)
>>> Total DISK READ :       0.00 B/s | Total DISK WRITE :     297.68 M/s
>>> Actual DISK READ:       0.00 B/s | Actual DISK WRITE:     297.68 M/s
>>>     TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND
>>> 19557 be/4 root        0.00 B/s  297.68 M/s  0.00 % 99.99 % dd
>>> if=/dev/zero of=/daten/testfile bs=1G count=10 oflag=direct
>>
>> On 5/9/19 5:43 AM, Lothar Schilling wrote:> Am 09.05.2019 um 13:27
>> schrieb Martin:
>>>> [..]
>>>>> hdparm -tT /dev/sda
>>>>> /dev/sda:
>>>>>    Timing cached reads:   13348 MB in  2.00 seconds = 6683.42 MB/sec
>>>>>    Timing buffered disk reads: 1014 MB in  3.00 seconds = 337.72
>> MB/sec
>>>>> iotop -o (for rsync and cp)
>>>>> Total DISK READ :       0.00 B/s | Total DISK WRITE :     476.15 K/s
>>>>> Actual DISK READ:       0.00 B/s | Actual DISK WRITE:     487.86 K/s
>>>>>     TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>   
>> COMMAND
>>>>> 19531 be/4 root        0.00 B/s  476.15 K/s  0.00 % 99.24 % rsync
>>>>> --info=progress2 /daten/testfile /daten/testfile2
>>>>>
>>>>> iotop -o (for dd)
>>>>> Total DISK READ :       0.00 B/s | Total DISK WRITE :     297.68 M/s
>>>>> Actual DISK READ:       0.00 B/s | Actual DISK WRITE:     297.68 M/s
>>>>>     TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>   
>> COMMAND
>>>>> 19557 be/4 root        0.00 B/s  297.68 M/s  0.00 % 99.99 % dd
>>>>> if=/dev/zero of=/daten/testfile bs=1G count=10 oflag=direct
>>>> Show us the 'dd if=/daten/testfile bs=1G oflag=direct
>> of=/dev/null', please.
>>>> If this is as slow as this ~480k/s above, check your disk's health
>> status. Like with smartmontools or some disk-utility software.
>>>> Martin
>>>>
>>> Fast enough...
>>>
>>> dd if=/daten/testfile bs=1G oflag=direct of=/daten/testfile2
>>> 10+0 Datensätze ein
>>> 10+0 Datensätze aus
>>> 10737418240 Bytes (11 GB, 10 GiB) kopiert, 72,7297 s, 148 MB/s
>>>
>>> dd if=/daten/testfile of=/dev/null
>>> 20971520+0 Datensätze ein
>>> 20971520+0 Datensätze aus
>>> 10737418240 Bytes (11 GB, 10 GiB) kopiert, 36,6887 s, 293 MB/s
>> Please reply with your *complete* console session (you can remove all
>> of the above in your reply).
>>
>> David
>>

# cat /etc/debian_version
9.9

# uname -a
Linux [my.server.com] 4.9.0-9-686-pae #1 SMP Debian 4.9.168-1
(2019-04-12) i686 GNU/Linux

# lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0   3,7T  0 disk
├─sda1   8:1    0   476M  0 part /boot
├─sda2   8:2    0  93,1G  0 part /
├─sda3   8:3    0  18,6G  0 part [SWAP]
└─sda4   8:4    0 931,3G  0 part /daten

# mount | egrep 'sd[a-z]'
/dev/sda2 on / type ext4 (rw,relatime,errors=remount-ro,data=ordered)
/dev/sda4 on /daten type ext4 (rw,relatime,data=ordered)
/dev/sda1 on /boot type ext4 (rw,relatime,data=ordered)

# df | egrep 'sd[a-z]'
/dev/sda2       95596964  2241952  88455840    3% /
/dev/sda4      960185376 41249236 870091648    5% /daten
/dev/sda1         463826    57063    378296   14% /boot

# time dd if=/dev/urandom of=test bs=1M count=100 conv=fsync
100+0 Datensätze ein
100+0 Datensätze aus
104857600 Bytes (105 MB, 100 MiB) kopiert, 192,778 s, 544 kB/s
real    3m12,781s
user    0m0,000s
sys     0m1,480s

# echo 3 > /proc/sys/vm/drop_caches && time dd if=test of=/dev/null
204800+0 Datensätze ein
204800+0 Datensätze aus
104857600 Bytes (105 MB, 100 MiB) kopiert, 0,600354 s, 175 MB/s
real    0m0,675s
user    0m0,048s
sys     0m0,340s

# echo 3 > /proc/sys/vm/drop_caches && f=test &&
uh=root@[my.other.server.com] && p=/tmp/test && ssh $uh rm -f $p/$g &&
rsync -a --stats $f $uh:$p
Number of files: 1 (reg: 1)
Number of created files: 1 (reg: 1)
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 104,857,600 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.021 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 104,883,277
Total bytes received: 34
sent 104,883,277 bytes  received 34 bytes  13,984,441.47 bytes/sec
total size is 104,857,600  speedup is 1.00

# echo 3 > /proc/sys/vm/drop_caches && f=test && p=/tmp && rsync -a
--stats $f $p
Number of files: 1 (reg: 1)
Number of created files: 1 (reg: 1)
Number of deleted files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 104,857,600 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.021 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 104,883,283
Total bytes received: 35
sent 104,883,283 bytes  received 35 bytes  629,929.84 bytes/sec
total size is 104,857,600  speedup is 1.00

Back to linux.debian.user | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Speed Problem Copying Files Lothar Schilling <ls@proasyl.de> - 2019-05-09 11:00 +0200
  Re: Speed Problem Copying Files Jonas Smedegaard <jonas@jones.dk> - 2019-05-09 11:20 +0200
    Re: Speed Problem Copying Files Lothar Schilling <ls@proasyl.de> - 2019-05-09 11:50 +0200
      Re: Speed Problem Copying Files Kevin DAGNEAUX <kevin.dagneaux@fiitelcom.fr> - 2019-05-09 12:00 +0200
        Re: Speed Problem Copying Files Lothar Schilling <ls@proasyl.de> - 2019-05-09 12:30 +0200
          Re: Speed Problem Copying Files Martin <keine-eile@online.de> - 2019-05-09 13:30 +0200
            Re: Speed Problem Copying Files Lothar Schilling <ls@proasyl.de> - 2019-05-09 14:50 +0200
              Re: Speed Problem Copying Files Martin <keine-eile@online.de> - 2019-05-09 16:00 +0200
              Re: Speed Problem Copying Files "Thomas Schmitt" <scdbackup@gmx.net> - 2019-05-09 17:20 +0200
      Re: Speed Problem Copying Files Jonas Smedegaard <jonas@jones.dk> - 2019-05-09 12:30 +0200
        Re: Speed Problem Copying Files Jonas Smedegaard <jonas@jones.dk> - 2019-05-09 13:00 +0200
          Re: Speed Problem Copying Files Lothar Schilling <ls@proasyl.de> - 2019-05-09 13:20 +0200
            Re: Speed Problem Copying Files Keith Christian <keith1christian@gmail.com> - 2019-05-09 13:30 +0200
              Re: Speed Problem Copying Files Lothar Schilling <ls@proasyl.de> - 2019-05-09 14:50 +0200
  Re: Speed Problem Copying Files Gene Heskett <gheskett@shentel.net> - 2019-05-09 16:20 +0200
    Re: Speed Problem Copying Files "Jeremy Nicoll" <jn.ml.dbn.25@letterboxes.org> - 2019-05-09 16:30 +0200
      Re: Speed Problem Copying Files Dan Ritter <dsr@randomstring.org> - 2019-05-09 17:00 +0200
      Re: Speed Problem Copying Files Gene Heskett <gheskett@shentel.net> - 2019-05-09 17:00 +0200
  Re: Speed Problem Copying Files David Christensen <dpchrist@holgerdanske.com> - 2019-05-10 07:50 +0200
    Re: Speed Problem Copying Files Lothar Schilling <ls@proasyl.de> - 2019-05-10 11:00 +0200
      Re: Speed Problem Copying Files Lothar Schilling <ls@proasyl.de> - 2019-05-13 10:40 +0200
        Re: Speed Problem Copying Files Lothar Schilling <ls@proasyl.de> - 2019-05-13 11:00 +0200
          Re: Speed Problem Copying Files Tixy <tixy@yxit.co.uk> - 2019-05-13 11:20 +0200
          Re: Speed Problem Copying Files Curt <curty@free.fr> - 2019-05-13 11:40 +0200
          Re: Speed Problem Copying Files Michael Stone <mstone@debian.org> - 2019-05-14 13:40 +0200
        Re: Speed Problem Copying Files Tixy <tixy@yxit.co.uk> - 2019-05-13 11:00 +0200
        Re: Speed Problem Copying Files David Christensen <dpchrist@holgerdanske.com> - 2019-05-14 02:30 +0200
          Re: Speed Problem Copying Files Lothar Schilling <ls@proasyl.de> - 2019-05-14 09:40 +0200
            Re: Speed Problem Copying Files David Christensen <dpchrist@holgerdanske.com> - 2019-05-14 18:30 +0200
    Re: Speed Problem Copying Files Michael Stone <mstone@debian.org> - 2019-05-14 13:30 +0200
      Re: Speed Problem Copying Files David Christensen <dpchrist@holgerdanske.com> - 2019-05-14 21:20 +0200
        Re: Speed Problem Copying Files Michael Stone <mstone@debian.org> - 2019-05-14 22:10 +0200

csiph-web