USB transfer to external NTFS so slow

I am using Fedora 39, with Nautilus running under Sway WM.

When I transfer files TO my 7200rpm external HDD in a Simplecom USB 3 case, file transfer is so slow as to be unusable with Qbittorrent, and transferring any files over 1GB is impractically slow. Unmounting also takes so long I don’t wait and just turn the thing off (over ten minutes after 5GB transfer). 5 GB took 12 minutes to transfer - three times slower than to a USB 2 thumb drive. However file transfer FROM USB HDD to PC is only a couple of minutes.

Will I get any better speeds using Ext4? Or any other recommendations?

NTFS is not native to Linux and it’s normal that writes are slower than reads (maybe it has recently been merged to Linux kernel, but performance isn’t good. )

Yes, probably much faster, and also safer because the filesystem is mature and part of the kernel, no need to rely on the NTFS driver.
If you can, switch over to ext4, xfs or btrfs.

1 Like

Thank you Flo, I will put it to the test when I can.

Check the USB bus version and speed for your external HDD with lsusv -v. Some hubs set speed for all devices to that of the slowest device. Try unplugging devices you aren’t using, and avoid connecting slow devices to USB3 ports.

1 Like

:100:

Also, an older Intel system would route USB traffic over the DMI which is VERY slow if you have several devices plugged up to the machine.

This again can be an issue with the BUS, Intel systems are terrible at this and some Arm External devices also can’t handle speeds higher than USB2.

Do not do this :exclamation: : as you can corrupt the HDD writing. Yes, when you unmount your device there are writes/reads done to the device by the Filesystem/OS. If you do this enough you can corrupt files and the device ( I know from experience. )

1 thing to note also is the device and drives in your Simplecom USB 3 case HDD’s have buffers depending how large those buffers are can determine how fast you can transfer files over to it.

So say you have an older HDD like a WD Blue 500GB Desktop Hard Disk Drive - 7200 RPM SATA 6 Gb/s 16MB , This HDD will never exceed around 80mb/s due to it’s cache being so low and be slower to unmount in certain scenarios as well. But a Western Digital 1TB WD Blue Mobile Hard Drive HDD - 5400 RPM, SATA 6 Gb/s, 128 MB cache , can meet your USB3 bandwidth in certai scenarios because it’s a better drive.

I currently own 36 USB HDD’s I have accumulated over the years for cold storage and migrations. New installs and the like. i own several USB enclosures and can talk to using FileSystems like XFS for large data (videos databases etc. ) or EXT4 (more general use). They are cheap and come in handy for many situations. NTFS is not a good choice here for a Linux workflow IMO. I prefer XFS.

1 Like

As @hamrheadcorvette said do not do this.

As background to understand why; the way linux kernel works is that when you copy a file it first copies the data into buffers in the kernel. Over time these buffers are written out to the disk.

So you see the copy command complete quickly when you have lots of memory in your computer (common these days).

The reason that the unmount is slow is that lots of that data is still in memory and not written to the disk yet. If you care about that data you will need to wait.

If you are curious you can see the size of the backlog by using this command (I think its the Writeback field that matters):

grep Writeback /proc/meminfo

Update:
The problem ‘fixed itself’. After updating to kernal 6.5.12-300.fc39 my transfer speeds are back to around 200MB/s. I can transfer large files (40GB) again with no buffering, no wait time to unmount.

I;m going to update kernal again to latest 6.7.4-200.fc39 and we’ll see what happens :slight_smile: