Slow SSH response time

Ask Fedora
1

Hello there,

I have a strange problem with openssh on Fedora 33.

Whenever I use SSH to connect to my debian remote server I get slow response time. It’s not unbearable but I have a slight delay that doesn’t happen with any of my other machines (Windows, WSL, termux on android…). I suppose the problem doesn’t come from the server side as I’ve disabled dns and a bunch of other stuff that are supposed to speed up my connection.

I have tried multiple terminal on fedora but the problem is still present and my google searches are unsuccessful. So you’re my last hope.

Thanks you.

1 Like
2

Hi @phogred , welcome to the forum.

The standard thing to do first is to get as much information as you can, so run your ssh command with the -vvv flags. It’ll spew out a lot of debug information. If you go through that, you should know exactly what’s happening, and hopefully get some idea on what’s causing the delay.

6 Likes
4

Thank you for answering. I don’t see anything strange. Here’s the output:

OpenSSH_8.4p1, OpenSSL 1.1.1i FIPS  8 Dec 2020
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
debug2: checking match for 'final all' host phromaj.b0tnet.com originally phromaj.b0tnet.com
debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: not matched 'final'
debug2: match not found
debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]
debug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]
debug1: configuration requests final Match pass
debug1: re-parsing configuration
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
debug2: checking match for 'final all' host
debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: matched 'final'
debug2: match found
debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]
debug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/home/lucas/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/home/lucas/.ssh/known_hosts2'
debug2: resolving  port 22
debug2: ssh_connect_direct
debug1: Connecting to 
debug1: Connection established.
debug1: identity file /home/lucas/.ssh/id_rsa type 0
debug1: identity file /home/lucas/.ssh/id_rsa-cert type -1
debug1: identity file /home/lucas/.ssh/id_dsa type -1
debug1: identity file /home/lucas/.ssh/id_dsa-cert type -1
debug1: identity file /home/lucas/.ssh/id_ecdsa type -1
debug1: identity file /home/lucas/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/lucas/.ssh/id_ecdsa_sk type -1
debug1: identity file /home/lucas/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /home/lucas/.ssh/id_ed25519 type -1
debug1: identity file /home/lucas/.ssh/id_ed25519-cert type -1
debug1: identity file /home/lucas/.ssh/id_ed25519_sk type -1
debug1: identity file /home/lucas/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /home/lucas/.ssh/id_xmss type -1
debug1: identity file /home/lucas/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.9p1 Debian-10+deb10u2
debug1: match: OpenSSH_7.9p1 Debian-10+deb10u2 pat OpenSSH* compat 0x04000000
debug2: fd 4 setting O_NONBLOCK
debug1: Authenticating to phromaj.b0tnet.com:22 as 'lucas'
debug3: hostkeys_foreach: reading file "/home/lucas/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /home/lucas/.ssh/known_hosts:2
debug3: load_hostkeys: loaded 1 keys from phromaj.b0tnet.com
debug3: order_hostkeyalgs: have matching best-preference key type ecdsa-sha2-nistp256-cert-v01@openssh.com, using HostkeyAlgorithms verbatim
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com,ssh-ed25519,sk-ssh-ed25519@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes128-gcm@openssh.com,aes128-ctr
debug2: ciphers stoc: aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes128-gcm@openssh.com,aes128-ctr
debug2: MACs ctos: hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512
debug2: MACs stoc: hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: aes256-gcm@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: aes256-gcm@openssh.com MAC: <implicit> compression: none
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:
debug3: hostkeys_foreach: reading file "/home/lucas/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /home/lucas/.ssh/known_hosts:2
debug3: load_hostkeys: loaded 1 keys from phromaj.b0tnet.com
debug3: hostkeys_foreach: reading file "/home/lucas/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /home/lucas/.ssh/known_hosts:2
debug3: load_hostkeys: loaded 1 keys from 82.124.142.0
debug1: Host 'phromaj.b0tnet.com' is known and matches the ECDSA host key.
debug1: Found key in /home/lucas/.ssh/known_hosts:2
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey out after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey in after 4294967296 blocks
debug1: Will attempt key: /home/lucas/.ssh/id_rsa
debug1: Will attempt key: /home/lucas/.ssh/id_dsa 
debug1: Will attempt key: /home/lucas/.ssh/id_ecdsa 
debug1: Will attempt key: /home/lucas/.ssh/id_ecdsa_sk 
debug1: Will attempt key: /home/lucas/.ssh/id_ed25519 
debug1: Will attempt key: /home/lucas/.ssh/id_ed25519_sk 
debug1: Will attempt key: /home/lucas/.ssh/id_xmss 
debug2: pubkey_prepare: done
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password
debug3: start over, passed a different list publickey,password
debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /home/lucas/.ssh/id_rsa RSA SHA256:
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 60
debug1: Server accepts key: /home/lucas/.ssh/id_rsa RSA SHA256:
debug3: sign_and_send_pubkey: RSA SHA256:
debug3: sign_and_send_pubkey: signing using rsa-sha2-256
SHA256:
debug3: send packet: type 50
debug3: receive packet: type 52
debug1: Authentication succeeded (publickey).
Authenticated to .
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug3: send packet: type 90
debug1: Requesting no-more-sessions@openssh.com
debug3: send packet: type 80
debug1: Entering interactive session.
debug1: pledge: network
debug3: receive packet: type 80
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug3: receive packet: type 4
debug1: Remote: /home/lucas/.ssh/authorized_keys:1: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
debug3: receive packet: type 4
debug1: Remote: /home/lucas/.ssh/authorized_keys:1: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
debug3: receive packet: type 91
debug2: channel_input_open_confirmation: channel 0: callback start
debug2: fd 4 setting TCP_NODELAY
debug3: ssh_packet_set_tos: set IP_TOS 0x48
debug2: client_session2_setup: id 0
debug2: channel 0: request pty-req confirm 1
debug3: send packet: type 98
debug1: Sending environment.
debug3: Ignored env COLORTERM
debug3: Ignored env DBUS_SESSION_BUS_ADDRESS
debug3: Ignored env DESKTOP_SESSION
debug3: Ignored env DISPLAY
debug3: Ignored env EDITOR
debug3: Ignored env GDMSESSION
debug3: Ignored env GTK_MODULES
debug3: Ignored env HOME
debug3: Ignored env IMSETTINGS_INTEGRATE_DESKTOP
debug3: Ignored env IMSETTINGS_MODULE
debug1: Sending env LANG = fr_FR.utf8
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug3: Ignored env LOGNAME
debug3: Ignored env LS_COLORS
debug3: Ignored env PATH
debug3: Ignored env PWD
debug3: Ignored env QT_IM_MODULE
debug3: Ignored env SESSION_MANAGER
debug3: Ignored env SHELL
debug3: Ignored env SHLVL
debug3: Ignored env SSH_AGENT_PID
debug3: Ignored env SSH_AUTH_SOCK
debug3: Ignored env TERM
debug3: Ignored env USER
debug3: Ignored env VTE_VERSION
debug3: Ignored env WINDOWID
debug3: Ignored env XAUTHORITY
debug3: Ignored env XDG_CONFIG_DIRS
debug3: Ignored env XDG_CURRENT_DESKTOP
debug3: Ignored env XDG_DATA_DIRS
debug3: Ignored env XDG_GREETER_DATA_DIR
debug3: Ignored env XDG_MENU_PREFIX
debug3: Ignored env XDG_RUNTIME_DIR
debug3: Ignored env XDG_SEAT
debug3: Ignored env XDG_SEAT_PATH
debug3: Ignored env XDG_SESSION_CLASS
debug3: Ignored env XDG_SESSION_DESKTOP
debug3: Ignored env XDG_SESSION_ID
debug3: Ignored env XDG_SESSION_PATH
debug3: Ignored env XDG_SESSION_TYPE
debug3: Ignored env XDG_VTNR
debug1: Sending env XMODIFIERS = @im=ibus
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug2: channel 0: request shell confirm 1
debug3: send packet: type 98
debug2: channel_input_open_confirmation: channel 0: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: PTY allocation request accepted on channel 0
debug2: channel 0: rcvd adjust 2097152
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: shell request accepted on channel 0
1 Like
5

Nothing obviously wrong, I agree. If you are still looking for this, perhaps a way to get a bit more information would be to put timestamps on the output, and see where you get a jump. That might be a clue to what it is that takes time. Something like this should work I believe:

unbuffer ssh -vvv … | ts

(unbuffer comes with the package expect and ts with the package moreutils, in case you don’t have them installed already.)

2 Likes
6

It was getting stuck for me on
Feb 03 13:41:37 debug2: x11_get_proto: /usr/bin/xauth list :0 2>/dev/null
Feb 03 13:41:57 Warning: No xauth data; using fake authentication data for X11 forwarding.

1 Like
7

commenting the line ForwardX11 yes in the ~/.ssh/config file speeds up the connection again.

I tried a clean F33 install with and without ForwardX11 yes, but the issue was not seen in either case.

2 Likes
8

Sounds strange. Can you reproduce the slow xauth command outside ssh?

9 
$ time /usr/bin/xauth list :0 2>/dev/null
pc/unix:0  MIT-MAGIC-COOKIE-1  097e4b166013c8370e75194c0914161b
#ffff##:0  MIT-MAGIC-COOKIE-1  097e4b166013c8370e75194c0914161b

real    0m0.003s
user    0m0.001s
sys     0m0.002s

I just restored the ForwardX11 yes and retested.
(un)fortunately the issue has now been resolved. No idea how.