Page MenuHomePhabricator

SFTP transfers handled by kio_sftp are much slower than those handled by scp
Closed, ResolvedPublic


KDE people needs complete async sftp API to make download on dolphin faster.

Description of problem:
On a 100Mbps network, I can transfer files with an average throughput of 11.1 MB/s with sftp get command in Konsole. Transferring the exact files using Dolphin and kio_sftp gives an average throughput of only 1.2 MB/s.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
0.a. Setup a gigabit network with at least two machines and a multiGB file to download on one side.
0.b. Set up ssh access on machine 2

  1. On machine1 open Dolphin
  2. Click on the "address" bar. The bar becomes editable
  3. enter "sftp://<user>@<ip_of_machine2>:/home/<user>"
  4. Enter password when prompted for one
  5. After directory loads, "copy"(Ctrl-C) a large local, and "paste" (Ctrl-V) it in the Dolphin window. This starts the transfer.
  6. Use the "Network Management" widget to observe transfer speed
  7. Try to copy the same or a similar file with scp: $ scp large.file <user>@<ip_of_machine2>:/home/<user>/

8 Observe the transfer speed either by watching scp output, or using the "Network Management" widget.

Actual results:
kio_sftp: about 1.2 MiB/s average throughput
scp: about 11.1 MiB/s average throughput

Expected results:
Same throughput using both scp and kio_sftp, and same cpu usage.

Additional info:
I transferred one contiguous 65 GiB file. Speeds remain constant in both escenarios. Dolphin scenario almost 2 days downloading some 215GB.

I read someplace in a similar problem solved by increassing some buffer size for samba kio slave.

I copied this bug description from replacing statistics with my own data.

People from kde asks me to send it here.

Event Timeline

epertinez triaged this task as High priority.Apr 16 2018, 2:56 PM
epertinez created this task.

Anyone listening?

asn added a subscriber: asn.Sep 4 2018, 10:26 PM

Yes, this should be better when used with TCP_NODELAY. There will be an update to kio_sftp soon. We are also working on improvements for sftp.

asn added a project: Restricted Project.Sep 4 2018, 10:27 PM
Jakuje added a subscriber: Jakuje.Jun 26 2019, 11:06 AM

The current version should have improved memory handling. Can you retest whether the speed is better now?

Sorry for the long time to reply. I didn't have an environment with a new enough KDE to test it.

I just uploaded and downloaded 1.5GB file from my computer to another one in a closed network. Plasma 5.17.3, qt 5.12.4. Sftp through konsole gave me times of 73.5MB/s to upload, 110.7MB/s to download, so I guess upload time bottle neck was disk writing.

Dolphin times were 35.7MB/s to upload, 101.7MB/s to download.

Buffers and network status appart, I understand it has been solved. Thank you very much!!!

Jakuje closed this task as Resolved.Nov 25 2019, 1:24 PM
Jakuje claimed this task.

Thank you for testing the latest change. The throughput looks good now. Closing the issue.