Please make a inxi -Fzx and post the output here. So we do see the whole picture and can better give you some recommendations.
It looks like that Network Speed really depends on a good working driver. I had similar issues with speed on my computer. Till I changed the adapter. Fortunately I had two build in adapters with two different brands.
Since I chosen the integrated one from Intel with a more famous driver I do have much more peace with networking.