Page MenuHomePhabricator

ServerAliveInterval & ClientAliveInterval not present in libssh version 0.9.0
Open, WishlistPublic

Description

I am working on a callhome functionality provided by libnetconf2 , this uses libssh for connection .
This callhome functionality is basically initiating connection from server to the client.
The connection gets established but after every 10 mins the connection gets disconnected . While checking the logs I found that ssh is terminating the connection.

Log says:
:Session 1: SSH channel poll error (Received SSH_MSG_DISCONNECT: 2:User session has timed out idling after 600000 ms.)

After searching solution through internet I found that "ServerAliveInterval" should be set at sshd_config so that server should not be disconnected till the specified amount of time .
But this option is not available in libssh , which I verified by going through the code of libssh

Following options are only available :

enum ssh_bind_config_opcode_e {
32 /* Known but not allowed in Match block */
33 BIND_CFG_NOT_ALLOWED_IN_MATCH = -4,
34 /* Unknown opcode */
35 BIND_CFG_UNKNOWN = -3,
36 /* Known and not applicable to libssh */
37 BIND_CFG_NA = -2,
38 /* Known but not supported by current libssh version */
39 BIND_CFG_UNSUPPORTED = -1,
40 BIND_CFG_INCLUDE,
41 BIND_CFG_HOSTKEY,
42 BIND_CFG_LISTENADDRESS,
43 BIND_CFG_PORT,
44 BIND_CFG_LOGLEVEL,
45 BIND_CFG_CIPHERS,
46 BIND_CFG_MACS,
47 BIND_CFG_KEXALGORITHMS,
48 BIND_CFG_MATCH,
49 BIND_CFG_PUBKEY_ACCEPTED_KEY_TYPES,
50 BIND_CFG_HOSTKEY_ALGORITHMS,
51
52 BIND_CFG_MAX /* Keep this one last in the list */
53 }

Also after enabling logs, results as:

Unknown option: Protocol, line: 3
Unknown option: Banner, line: 4
Unknown option: RhostsRSAAuthentication, line: 5
Unknown option: HostbasedAuthentication, line: 6
Unknown option: PasswordAuthentication, line: 7
Unknown option: PermitEmptyPasswords, line: 8
Unknown option: ChallengeResponseAuthentication, line: 9
Unknown option: AllowTcpForwarding, line: 10
Unknown option: PidFile, line: 12
Unknown option: TCPKeepAlive, line: 13
Unknown option: ClientAliveCountMax, line: 14
Unknown option: ClientAliveInterval, line: 15
Unknown option: MaxStartups, line: 16
Unknown option: ForceCommand, line: 46
Count 46
Poll callback on socket 12 (POLLOUT ), out buffer 22
Received POLLOUT in connecting state
ssh_handle_key_exchange: current state : 2
Poll callback on socket 12 (POLLOUT ), out buffer 22
Enabling POLLOUT for socket
ssh_handle_key_exchange: current state : 2
Poll callback on socket 12 (POLLIN POLLOUT ), out buffer 0
Received banner: SSH-2.0-OpenSSH_5.3
SSH client banner: SSH-2.0-OpenSSH_5.3
Analyzing banner: SSH-2.0-OpenSSH_5.3

So can you pl tell me how to enable keepalive in server side so that it doesnot gets disconnected after every 10 mins

Event Timeline

Jakuje added a subscriber: Jakuje.Jan 23 2020, 10:43 AM

This is not implemented inside of libssh now, but it can be simply implemented by your application that will send some data in your defined time intervals, for example using ssh_send_ignore().

In OpenSSH, this is implemented using SIGALRM, which is not safe to use from inside of library so we would have to make the poll functions even more complicated to handle this.

One other possibility that could work would be TCPKeepAlive configuration option (from OpenSSH), which could handle this on TCP level (but might not work on all the networks configurations). So still, the first advice holds.

Thanks a lot @Jakuje .
I tried as per your recommendation and it works for me.

Jakuje triaged this task as Wishlist priority.Jan 27 2020, 11:50 AM
Vishalearnz added a comment.EditedJan 28 2020, 8:25 AM

Hi @Jakuje ,

Just one query .
If we are using ssh_send_ignore . Do we get response from the client for the same ?
(Because as I understand correctly .
The ssh_send_ignore returns SSH_OK if sent successfully else error. It doesnot states
if client has received it and responded.)

If we aren't getting a response, then is there an api other than ssh_send_ignore which sends the message and gets back response from the client as success of fail , if it has received the message
.

*Actually I want to implement something like heartbeat message referring to RFC 8071 for netconf callhome
This states that :
If a persistent connection is desired, the NETCONF/RESTCONF server, as the connection initiator, SHOULD actively test the aliveness of the connection using a keep-alive mechanism. For SSH-based connections, per Section 4 of [RFC4254], the server SHOULD
send an SSH_MSG_GLOBAL_REQUEST message with a purposely nonexistent "request name" value(e.g. keepalive@ietf.org) and the "want reply" value set to '1'.
*

So can you pl check and tell me what will be best suitable to this

Regards,
Vishal

Please, check the RFC 4253 describing this message: https://tools.ietf.org/html/rfc4253#section-11.2

The message is just sent and no answer is expected from the remote. But if the TCP connection would be broken, it would probably fail. Requesting an reply is certainly more reliable.

If the expectation from your protocol is to get an answer, you need to use technique as described above (global request). Indeed, I was wrong in the initial comment and indeed, openssh is using global request with keepalive@openssh.com name.

You should be able to use ssh_send_keepalive() from server.h. It should do what you need.

Vishalearnz added a comment.EditedJan 29 2020, 9:49 AM

Hi @Jakuje

I tried using ssh_send_keepalive() but it doesnot serve the purpose . Here I was monitoring the return value of the function (ssh_send_keepalive) .
I first started client and server . In my scenario server shall initiate the connection which it did and this keep alive function was set to send every 30 sec . for which the ssh_send_keepalive() was returning SSH_OK.
But when I killed the client . It was expected that the ssh_send_keepalive() should send back SSH_ERROR but instead it kept on sending back SSH_OK . This means this api is only sending the message but not monitoring the response.
I also checked in libssh code the above function always sends SSH_OK irrespective of any condition , it internally calls ssh_global_request() which is not an exposed API , so I cant use it to check.

Also if I check using ssh_get_status() for monitoring then on session kill it sends back value as 12(on error) whereas 0(on success).

Can you pl tell me if inside ssh_send_keepalive() some callback gets set which gets the reply. If yes then I need to check the callback.

Thanks & Regards,
Vishal

Hi,
the ssh_send_keepalive() does really only the sending. But the return value is ignored since 59ada799. But if the sending failed, the session state should be modified to reflect this if I am right. The response is handled as any other message in ssh_handle_packets() if I am right. The response is anyway SSH_MSG_REQUEST_FAILURE.

This method should be called in the main server loop, which should detect the disconnect anyway if the send failed. For example using ssh_is_connected() check in the main loop should give you an assertion that connection is still alive.

Hi @Jakuje ,

ssh_handle_packets() is not an exposed api . I am unable to use it , also this macro(SSH_MSG_REQUEST_FAILURE) I cant find in the codebase of libssh . So what should I use ?

Just to reiterate my problem statement - I need a way to send signals every 30 sec and get back a response for the same.
So as per what you said steps are as follows:

  1. ssh_send_keepalive() which will send unsolicited keep alive messages.
  2. Define a mainloop, which keeps on chencking the status of ssh_is_connected()
  3. ssh_handle_packets() should be used as a callback in the mainloop , each time ssh_is_connected() returns false.

Am I right?

Thanks & Regards,
Vishal

Hi @Jakuje ,
ssh_handle_packets() is not an exposed api .

Right. But it is called from inside of ssh_global_request() through ssh_handle_packets_termination(), so I revert the previous as no explicit action should be needed from the calling program.

I am unable to use it , also this macro(SSH_MSG_REQUEST_FAILURE) I cant find in the codebase of libssh . So what should I use ?

For historical reasons (cough ... SSH1 ...), it is SSH2_MSG_REQUEST_FAILURE in libssh. But this should not be needed for the calling application to care for this. It is internal of the ssh protocol.

Just to reiterate my problem statement - I need a way to send signals every 30 sec and get back a response for the same.
So as per what you said steps are as follows:

  1. ssh_send_keepalive() which will send unsolicited keep alive messages.
  2. Define a mainloop, which keeps on chencking the status of ssh_is_connected()
  3. ssh_handle_packets() should be used as a callback in the mainloop , each time ssh_is_connected() returns false.

That is my understanding. Without the last point. Also if you already have some channel-handling loop or polling one, you handle this in there I think.

Hi @Jakuje .

In my scenario I need to handle the response i.e if for a keepalive request I donot get a correct response . I need to terminate the session .
But going by what you wrote above " no explicit action should be needed from the calling program" , I am understanding that ssh_send_keepalive() internally calls the following sequence ssh_global_request()-> ssh_handle_packets_termination()->ssh_handle_packets() .
But I need to capture the response which ssh_send_keepalive() doesnot provide me any means to get it.
Also all the other api's like
ssh_global_request(), ssh_handle_packets_termination(), ssh_handle_packets() are not exposed to external world so I cant use it inside my code.

I understand that ssh_handle_packets() will be getting the response by what you said above . But how do I retrieve it.

So Can you pl let me know , how can I capture the response.

Thanks & Regards,
Vishal

I am sorry for a delay.

It looks like there is no way to capture the response at this moment. But still, I am not sure why you need that particular answer and why checking ssh_is_connected() is not enough after calling the ssh_send_keepalive().

Vishalearnz added a comment.EditedFeb 13 2020, 10:34 AM

Thanks Jakuje , Although I am able to use the above two api's to get the status of the connection.
But even after that there is a hard reset which causes the session to be terminated after 30 mins , even though keep alive is being sent.

But I feel there should have been an api' which tracks the response sent by the client like some callback or something , so whenever there is a disconnect the callback gets triggered.

Nevertheless , the api works fine for my needs as of now, but can I increase the idle timeout from 30 mins to more ?
Also I can see that instead of ssh_send_keepalive(), if I use ssh_send_ignore() at fixed intervals , the connection lasts for 60 mins.

Thank you for confirmation that this combination works. But lets clarify what you do -- you are sending every X minutes the ignore or keepalive messages to keep the channel open, but even though you are getting disconnects after 30 minutes. I do not think this is anything in libssh. What are you running in the channels? Port forwarding? Some long-running commands transmitting or not transmitting data? Shells? How does this disconnect look like? Could it be the default value of $TMOUT in bash? Or something on the network layer terminating long-running connections?