Asynio library errors when running rdp.StreamingPrices simple test

michaelearl0

asynio library error once every several minutes(~4 mins)

Using Python v3.9.1. Latest RDP libraries via python -m pip install --upgrade refinitiv.dataplatform

The message is like this: (with DEBUG logging enabled)

2021-01-13 01:13:52,214 - websocket - ERROR - error from callback <bound method StreamConnection._ws_message of <OMMStreamConnection(WebSocket 0 - pricing, started daemon 140170804320000)>>: Non-thread-safe operation invoked on an event loop other than the current one
File "/home/originai/trader/originai/signal/refinitiv/refinitiv-env/lib/python3.6/site-packages/websocket/_app.py", line 344, in _callback
callback(*args)
File "/home/originai/trader/originai/signal/refinitiv/refinitiv-env/lib/python3.6/site-packages/refinitiv/dataplatform/delivery/stream/stream_connection.py", line 838, in _ws_message
self._on_messages(messages)
File "/home/originai/trader/originai/signal/refinitiv/refinitiv-env/lib/python3.6/site-packages/refinitiv/dataplatform/delivery/stream/omm_stream_connection.py", line 239, in _on_messages
self._process_response_message(message)

File "/home/originai/trader/originai/signal/refinitiv/refinitiv-env/lib/python3.6/site-packages/refinitiv/dataplatform/delivery/stream/omm_stream_connection.py", line 341, in _process_response_message
self._on_receive_login_message(result)
File "/home/originai/trader/originai/signal/refinitiv/refinitiv-env/lib/python3.6/site-packages/refinitiv/dataplatform/delivery/stream/stream_connection.py", line 671, in _on_receive_login_message
self._login_response_future.set_result(result)
File "/usr/lib/python3.6/asyncio/futures.py", line 298, in set_result
self._schedule_callbacks()
File "/usr/lib/python3.6/asyncio/futures.py", line 214, in _schedule_callbacks
self._loop.call_soon(callback, self)
File "/usr/lib/python3.6/asyncio/base_events.py", line 593, in call_soon
self._check_thread()
File "/usr/lib/python3.6/asyncio/base_events.py", line 632, in _check_thread
"Non-thread-safe operation invoked on an event loop other "
2021-01-13 01:14:07,449 - Session session.platform - Thread 140170804320000 | WebSocket 0 - pricing

umer.nalla

Hi @Cheng-Yi

Thanks for the pip list, I have passed it onto the RDP Python dev team to examine.

In the meantime, if you are able to do so (ideally in a virtualenv), you could try out the version I am using on my PC v1.0.0a7 - available on PyPI - refinitiv-dataplatform · PyPI

umer.nalla

Hi @michaelearl0

Are you able to post the code here or even a smaller snippet which recreates the issue?

Easiest would be if you can attach it as a .txt file.

Cheng-Yi

Hi Umer,

I attached the code asyncio_test_20210113.txt

Please change the login credential to give it a try.

Thanks for taking a look!

umer.nalla

Hi @michaelearl0 / @Cheng-Yi

When I ran your snippet it crashed almost immediately after getting the first Refresh Messages.

However, if I replace the last line of code with something like:

while (True):
    asyncio.get_event_loop().run_until_complete(asyncio.sleep(120))

Then It ran fine for over 10 minutes without a problem.

Please let me know if the above works for you also.

I am not a Python expert so cannot explain why the above is ok and your alternative is not. I will refer this to the RDP Python library development team and hopefully, they will respond with an explanation.

Cheng-Yi

https://community.developers.refinitiv.com/discussion/comment/71773#Comment_71773

Thank you Umer, I will use your method and keep it running for a day and report back.

umer.nalla

https://community.developers.refinitiv.com/discussion/comment/71774#Comment_71774

Note that I have been advised there is a bug in rdp library version 1.0.0a7.post1 with refreshing the access token every 4-5 minutes. I am still using version 1.0.0a7 and perhaps that is why it is working for me.

If the above-suggested code change fails for you, please run the 'pip list' command in a Python console and attach the output as .txt file in your reply.

You can confirm your RDP library version with:

import refinitiv.dataplatform as rdp
rdp.__version__

Cheng-Yi

https://community.developers.refinitiv.com/discussion/comment/71816#Comment_71816

Hi Umer,

I am using 1.0.0a5 version.

pip_list.txt

The while loop method can not avoid the issue.

It still happened once every two~three minutes in my testing.

Thanks!

Cheng-Yi

https://community.developers.refinitiv.com/discussion/comment/71836#Comment_71836

Hi Umer,

I can confirm that after using v1.0.0a7, and python 3.9, your workaround can avoid the issue.

However, if I use .run_forever() the issue still happens, so I think there is still something wrong in the library. But I am fine with the workaround.

BTW, I found that v1.0.0a7 uses asyncio differently (e.g. asyncio.create_task) such that I can not keep using python3.6. I have to update to python 3.9, and it created some pain. I am afraid that other users might get hit by the same issue.

Thanks!