Added ssh heartbeat to keep the connection alive

[aylei]

Test1: the new ssh config will have include the following SSH config (need CLIENT UPGRADE)

  ServerAliveInterval 30
  ServerAliveCountMax 3

Test2: manually remove the SSH alive config, the server-side heartbeat still works and does not affect ssh connection (patched the client to output raw msg to a log file):

1756889060.8253598Recevied message: b'\xe0\xdf\xael\xad\xab\x1eBm\x8d\x99\xe8\n3y\xfc\xf8&\xf2g\xd3\x00Y\xb6\xef\x0f\t\xe8\x1dI\xcayLA\xdf\x8e\xa7=<\x833I\xedc\xa2$\xf8\xe8\x94\x96N\xa3\xdeb\x82+\xd5\x90\x1f\xad\x9a\xa7\xc0 v\x83\xc8\xc1'
1756889091.140574Recevied message: b'\x90^U"\xe1\xc6\x10\xa5\xef\xcb\x10\x95db\xfe\xd1\n\xc1a\xfc\xcd\xedL\xd1\x8a\xfbk\x88\x10lPET\xa7J\x1d3\\\x87\x1fGG\xd6\xa3\x992\xef\xe9\xfa9\xe2\xdd_}GG\xf6Y\xa3\xe6s\xa3\x17\xa8\xf3\xa1EY'
1756889121.465916Recevied message: b'\x18\xed[Ny\xc7\xac\x05\xa7\xf4P\x1bi\xde\xbao\xe4\xe7\xd7\xe1\x108?\x91^\x02\xf0h\x9c[0B\xb0\x05\x13yd\x87\x15\xa7\x9e\x92\xcdlQ\\9O\x89vy2\x01\xf6\xa1\xe3^\xf6\x06\xdb|/\x04A\xc1\xd2\x05&'
1756889151.7792552Recevied message: b'\x98\x9e\x1b\x9a\xb2\xf1\xc3\x1e>\x9a\x06\xa9!\xde^\xfc\x84\xb39\x1e\xabevS^\x8f\xe3<a\x86\x8en\x900\x852\x8e\xf9E}\xd1\x17\xb1\xcc\xf5\xa6&7^D\x16\xf4\xe1u\xde^7M\x0c?\xecl\x13o\x82\xab\x8f;'
1756889182.092423Recevied message: b'wn\xec\xbc\x8fy9!\x1c\xcc\x07\xa0o\x95\x15\x81+]\x8a\x1f:\xb0\x8d"\xb1\xb2\x9a\x1b\xe5o\xfa\x82\x0b\xb7\x82\xa8+a\xe9\xda\xb9S\x95j\xcfY%[\x82\x87\x0b\xe1b\x91\xa9\xd0\xcdI\xf4\xdc:\x10\xc4\x1e\x91S\xc7\x8b'
1756889212.410708Recevied message: b'Y\xcd\x0b%\xa4\x1d\xae\xc1\x13\x84\xc5\xb9\x00v\x82\xbc\xf1\xd9,\nu\x1e6\x9b:r\xb1X\xa1\xa8i\x94\xbd\x897K\xb0j\xd5\xfe\xff\xc6\x9c)\xfc\xb6\xeb\r(N\x07c>Z\xec\xa6\xda\xd7$\xbdF\xe8a\x0c\xae`\xf6\xa2'

Test3: open an idle ssh connection, can keep alive.

Tested (run the relevant ones):

• Code formatting: install pre-commit (auto-check on commit) or bash format.sh
• Any manual or new tests for this PR (please specify below)
• All smoke tests: /smoke-test (CI) or pytest tests/test_smoke.py (local)
• Relevant individual tests: /smoke-test -k test_name (CI) or pytest tests/test_smoke.py::test_name (local)
• Backward compatibility: /quicktest-core (CI) or pytest tests/smoke_tests/test_backward_compat.py (local)

[aylei]

/smoke-test

[Michaelvll]

Did we reproduced the disconnection issue on master?

[aylei]

Did we reproduced the disconnection issue on master?

Yes, I think there is another reason cause the disconnection, closing this and will investigate more