"Connection is already closed" error when reconnecting to the websocket in python

Hi,

I'm writing a websocket multithread program in python to get real-time data from Refinitiv.

One thread for sending request, receiving data and put them in a queue.

The other thread is to handle the data in the queue.(put them in my database).

Since sometimes the connection will disconnect for whatever reason. I wrote some reconnection logic in the on_close() function. But somehow once it's successfully reconnected, it said "Connection is already closed". And the program will not proceed to login and request data as shown.

Find more posts tagged with

python

disconnection

websockets

#technology

Accepted answers

nick.zincone

Hi @tobywong

The purpose of the try...except logic is to help determine if there is an unintentional side effect when working in different threads and accessing common resources from multiple threads. What I'm trying to do is help provide some ways to better understand what may be causing your disconnects. It is also good practice to catch unintended issues and gracefully exit and possibly log to help understand/debug.

Another possibility as to why the application disconnects may be a result of an untimely "pong" message sent to the WebSocket server. As you know, the WebSocket server depends on a "keep-alive" heartbeat message (ping/pong) for the connection to stay alive. If the server does not receive a "pong" heartbeat in a timely manner, then it will disconnect. It's clear you are doing this already otherwise you would be disconnecting after a short period of time. However, if you are relying on a "ping" message in order to send a "pong", that could be an issue. For example, depending on how many open subscriptions you have, there is quite a bit of traffic that occurs at market open. Because of this, you may not be able to process the "ping" message in a timely manner thus the server may be unhappy it hasn't heard from you (because you are busy) and disconnects. I would suggest the best way to deal with this is to NOT rely on the "ping", but rather use a timer that fires regularly so you can fire off a "pong" message in a consistent, timely manner. You can find the actual "PingTimeout" value within the login response and maybe set your time to 90% of this value.

Regarding the on_close(), I'm suggesting you define a timer within the on_close() logic. For example, you can define a 15 second timer, then return from your on_close(). This will allow your WebSocket thread to properly complete the close event. Then in about 15 seconds, your timer will fire and you can attempt to reconnect.

For example:

def on_close():
    print("WebSocket closed")

    # Schedule a reconnect after a delay (e.g., 15 seconds)
    delay_seconds = 15
    reconnect_timer = threading.Timer(delay_seconds, reconnect)
    reconnect_timer.start()

All comments

Jirapongse

@tobywong

Thank you for reaching out to us.

You may need to enable trace in the websocket and check the error in the on_error callback method, as mentioned in this discusson on StackOverflow.

nick.zincone

Hi @tobywong

If you are continuously disconnecting, I would try to narrow down why that may be happening. For example, I would suggest you simplify your application to test the health of your connection with 1 thread that simply sends/receives data. If you find success here, then slowly introduce your additional logic - for example, are you locking access to your shared resource (queue)? Perhaps place a try...exception around the access logic of your queue?

With regards to your reconnect logic, I wouldn't recommend you reconnect within your on_close(). The websocket thread may not be in a state to deal with this. Instead, you may want to set a short timer and allow the Websocket::on_close() workflow to complete. Then a few seconds later reconnect.

tobywong

Hi @nick.zincone

It disconnects about once every 1-2 days. I've put a try...except around the access logic. But what do I do if I really does 'catch' an exception?

Do you mean put a timer inside on_close() so that it wait a certain amount of time before trying to reconnect? Since once it jumps to on_close() there is no way to call other functions/do the reconnection other than doing it inside on_close() no?

nick.zincone

Hi @tobywong

For example:

def on_close():
    print("WebSocket closed")

    # Schedule a reconnect after a delay (e.g., 15 seconds)
    delay_seconds = 15
    reconnect_timer = threading.Timer(delay_seconds, reconnect)
    reconnect_timer.start()

tobywong

Hi @nick.zincone

I understand the purpose of try/except now.

Will the argument ping_interval and ping_timeout in web_socket_app.run_forever() do the trick?

Previously I use the default values. Now I've set them to 20 and 15 respectively.

And I also respond with a 'pong' message in on_message() when a 'ping' is received.

I've already made the program or the websocket thread wait 30s in on_close() before reconnecting though.

nick.zincone

Hi @tobywong

I assume you are referring to the market_price_ping.py example.

That example I believe just checks the health using a testing loop - that is not my suggestion. For example, look at how the market_price.py keeps the connection active by responding to 'ping' messages. While this is a general approach that would work in most cases, I'm suggesting above to try an alternative approach (I explain why). The timer I'm suggesting is the type of timer I provided in the code example I provided in my answer above.

In general, I'm suggesting one ongoing timer for your 'pong' messages and a 2nd timer in the event of a disconnect. Once you successfully reconnect, you can disable the 2nd timer. Hope this makes sense.

tobywong

Hi @nick.zincone
Is a ping timeout considered an error(jump to on_error()) or a close event(jump to on_close()) or both? Since if it's an error, I will have to call on_close() in the on_error() function which has the reconnection logic in it. If it's both than it'll keep reconnecting because I try to reconnect in on_close() and on_error() also calls on_close(). It will raise the connection is already open error and jump to on_error() which will call on_close() and reconnect again. It will keep reconnecting until it exceed the limit and then I get banned.

EXPLORE OUR SITES