Connection to remote host is lost when using Eikon Messenger Bot API (WebSocket)

Cemil Yigit

Hello,

I need your help. I am using the python demo script for the Eikon Messenger API to listen in a chatroom. Unfortunately, the connection to the remote host is lost after a certain time every time. Is it the WebSocket service or threading in Python or am I getting kicked out by Refinitiv?

Thanks.

wasin.w

All,

After investigating this disconnection issue for a while, @Cemil Yigit found that the proxy needs some kind of heartbeat messages or activities every certain minute to keep a WebSocket connection active.

The workaround is sending a ping-pong message with the WebSocket connection as follows:

def on_ping(_, message):
    print("Got a ping! A pong reply has already been automatically sent.")
def on_pong(_, message):
    print("Got a pong! No need to respond")

web_socket_app = websocket.WebSocketApp(
        ws_url,
        on_message=on_message,
        on_error=on_error,
        on_close=on_close,
        on_ping=on_ping,
        on_pong=on_pong,
        subprotocols=['messenger-json'])
wst = threading.Thread(target=web_socket_app.run_forever, kwargs={"sslopt": {"check_hostname": False}, "ping_interval": 60, "ping_timeout": 10, "ping_payload": "2"})

The code also needs to handle reconnection logic (as the example application aims for demonstrating the API workflow only, so it is not cover all use cases).

After ping-pong and reconnection logic are applied to the application, the application works fine.

wasin.w

Hello @Cemil Yigit

Does the WebSocket connection disconnect every 5 minutes? If so, I highly recommend you re-test with the latest version (June 2022 - master branch) of the https://github.com/Refinitiv-API-Samples/Example.MessengerChatBot.Python example.

Please also share the log file too (please remove your credentials before sharing!).

Cemil Yigit

Yes, it seems, that after 5 Minutes it disconnects. I'll check your link.

Cemil Yigit

Hello @wasin.w

Thanks for your post. I downloaded the chatbot_demo_ws.py file from git and tried it. Unfortunately, it now lost the connection after 2 minutes. Here ist some output:

Connecting to WebSocket wss://api.collab.refinitiv.com/services/nt/api/messenger/v1/stream ...

2022-09-06 11:10:02: DEBUG:websocket :Connecting proxy...

2022-09-06 11:10:02: INFO:root :Receive: onopen event. WebSocket Connection is established

2022-09-06 11:10:02: INFO:root :Sent: {.......}

2022-09-06 11:12:40: ERROR:root :Error: Connection to remote host was lost.

2022-09-06 11:12:40: ERROR:websocket :error from callback <function on_close at 0x00000273DA7E5DC0>: on_close() takes 1 positional argument but 3 were given

2022-09-06 11:12:40: ERROR:root :Error: on_close() takes 1 positional argument but 3 were given

Refresh Token

2022-09-06 11:14:02: DEBUG:urllib3.connectionpool :Starting new HTTPS connection (1): api.refinitiv.com:443

2022-09-06 11:14:03: DEBUG:urllib3.connectionpool :https://api.refinitiv.com:443 "POST /auth/oauth2/v1/token HTTP/1.1" 200 None

2022-09-06 11:14:03: ERROR:root :send_ws_connect_request exception: Connection is already closed.

2022-09-06 11:14:03: INFO:root :Sent: {.......}

Cemil Yigit

A log file was not created.

Cemil Yigit

It's funny, when changing 60 to 180 in the while loop, it seems that is works.

wasin.w

Hello @Cemil Yigit

Sorry for my mistake, the demo application does not generate a log file. Can you save entire log messages (please removed your credentials) and share a text file in this post?

I am trying to replicate the issue with the chatbot_demo_ws.py example in my environment (Python 3.9, libraries versions are the same as in the ws-requirements.txt file).

I did not encounter any disconnection yet (Note: I am connecting with my public internet, not the office environment network).

According to the following logs, are you connecting to the Messenger Bot WebSocket server through any firewall or proxy?

2022-09-06 11:10:02: DEBUG:websocket :Connecting proxy...
2022-09-06 11:10:02: INFO:root :Receive: onopen event. WebSocket Connection is established
2022-09-06 11:10:02: INFO:root :Sent: {.......}
2022-09-06 11:12:40: ERROR:root :Error: Connection to remote host was lost.

If so, does your network have any policy that cuts a long connection like WebSocket?

Cemil Yigit

Hello @wasin.w

Thanks for your support so far.

The hint with the proxy is a good one. I will check it and let you know if it's on my side. In the meantime I adjusted

time.sleep(int(expire_time) - 240)

For the moment, it works fine with 240. If it's not working, I'll let you know and push my code here.

Best, Cemil

Cemil Yigit

Hello @wasin.w

Unfortunately, the connection to the server was lost after roughly 5 hours. The relevant part of my code looks like this:

if __name__ == "__main__":

    # logging settings
    log_level = logging.DEBUG
    logging.basicConfig(format="%(asctime)s - %(levelname)s - %(name)s - %(message)s:", level=log_level, datefmt="%Y-%m-%d %H:%M:%S")
    logging.getLogger("urllib3").setLevel(logging.CRITICAL)
    logging.getLogger("websocket").setLevel(logging.CRITICAL)

    # rdp authentication token
    rdp_token = RDPTokenManagement(bot_username, bot_password, app_key, 30)
    access_token, refresh_token, expire_time, logged_in = authen_rdp(rdp_token)
    if not access_token:
        sys.exit(1)

    # join chatrooms
    for chatroom in config["chatrooms"].keys():
        join_room = join_chatroom(access_token, chatroom)

    # connect via a websocket connection
    print(f"connecting to {ws_url}")
    web_socket_app = websocket.WebSocketApp(ws_url, on_message=on_message, on_error=on_error, on_close=on_close, subprotocols=["messenger-json"])
    web_socket_app.on_open = on_open

    # infinite event loop
    wst = threading.Thread(target=web_socket_app.run_forever, kwargs={"sslopt": {"check_hostname": False}})
    wst.start()

    try:

        while True:

            if int(expire_time) > 245:
                expire_time = "300"
                time.sleep(int(expire_time) - 245)
            else:
                sys.exit(1)

            if not web_socket_app.keep_running:
                print(f"{datetime.datetime.now()} new connection established")
                send_ws_connect_request(access_token)

            print(f"{datetime.datetime.now()} token refreshed")
            access_token, refresh_token, expire_time, logged_in = authen_rdp(rdp_token)

            if not access_token:
                sys.exit(1)

            if logged_in:
                send_ws_keepalive(access_token)

    except KeyboardInterrupt:

        web_socket_app.close()

So, if you have an idea, please let me know. Thanks.

The part with

if not web_socket_app.keep_running

was added just now.