question

Upvotes
Accepted
4 4 7 11

Unable to get data via Refintiv Data Library (refinitiv.data) Python

Hi, it looks like I'm unable to retrieve any data via Refinitiv using the new Refinitiv Data Library.

Sample Code:

import os
os.environ["RD_LIB_CONFIG_PATH"] = "../../Configuration"
import refinitiv.data as rd
rd.open_session()

Code is fine up to here, (desktop) session seems to be successfully opened:

1672970677020.png

However, attempting to get the data as demonstrated in the examples does not seem to work:

rd.get_data(['LSEG.L', 'VOD.L'])

1672970755215.png

Would appreciate any help, thanks.

refinitiv-dataplatform-eikon#technology#productrefinitiv-data-platformdata
1672970677020.png (8.6 KiB)
1672970835284.png (22.0 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Hello @wesley.ng

Thank you for your participation in the forum. Is the reply below satisfactory in resolving your query?


If so please can you click the 'Accept' text next to the appropriate reply? This will guide all community members who have a similar question.

Thanks,


AHS

Please be informed that a reply has been verified as correct in answering the question, and has been marked as such.

Thanks,


AHS

Upvotes
Accepted
79.2k 251 52 74

@wesley.ng

You can enable trace in the Python WebSocket, as shown below.

websocket.enableTrace(True)
   
print("Connecting to WebSocket " + url + " ...")
web_socket_app = websocket.WebSocketApp(url, header=headers,
                                        on_message=on_message,
                                        on_error=on_error,
                                        on_close=on_close,
                                        subprotocols=['tr_json2'])

The output looks like this:

1673323525378.png

Otherwise, you can test the WebSocket connection with the Curl command.

curl --include --no-buffer --header "Connection: Upgrade" --header "Upgrade: websocket" --header "Host: localhost:9060" --header "Origin: http://localhost:9060" --header "Sec-WebSocket-Key: 4FlPV2h/xF20YXwlt7zBiw==" --header "Sec-WebSocket-Version: 13" --header "x-tr-applicationid: <app key>" --header "Sec-WebSocket-Protocol: tr_json2" -v http://localhost:9060/api/rdp/streaming/pricing/v1/WebSocket

The output looks like this:

1673323626716.png

Please make sure that 9060 is the correct port by accessing this URL (http://localhost:9060/api/status) via a web browser.




1673323525378.png (45.4 KiB)
1673323626716.png (65.4 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Hi Jirapongse, the port is correct.

Output 1 with default proxy settings:

----------------------- 
--- response header --- 
HTTP/1.1 200 Connection established 

----------------------- 
--- request header --- 
GET /api/rdp/streaming/pricing/v1/WebSocket HTTP/1.1 
Upgrade: websocket 
Host: localhost:9060 
Origin: http://localhost:9060 
Sec-WebSocket-Key: E6cohDP/n2dJqMYLNwX++w== 
Sec-WebSocket-Version: 13 
Connection: Upgrade 
Sec-WebSocket-Protocol: tr_json2 
User-Agent: Python 
x-tr-applicationid: <id> 

----------------------- 
--- response header --- 
HTTP/1.1 301 Moved Permanently Server: BlueCoat-Security-Appliance 
Location:http://10.190.169.68 
Connection: Close 
-----------------------
Connecting to WebSocket ws://localhost:9060/api/rdp/streaming/pricing/v1/WebSocket ...
scheme http is invalid
WebSocket Closed

Is this a firewall issue?

@wesley.ng

Yes, it is possible.

I found that BlueCoat-Security-Appliance relates to the secure web gateway application.

Hi Jirapongse,


On removing the proxies, I get the following returned:

--- request header ---
Connecting to WebSocket ws://localhost:9060/api/rdp/streaming/pricing/v1/WebSocket ...
GET /api/rdp/streaming/pricing/v1/WebSocket HTTP/1.1 
Upgrade: websocket Host: localhost:9060 Origin: http://localhost:9060 
Sec-WebSocket-Key: Nb8WlSx9JHid4fxn8LM4iQ== 
Sec-WebSocket-Version: 13 
Connection: Upgrade 
Sec-WebSocket-Protocol: tr_json2 
User-Agent: Python 
x-tr-applicationid: <key> 

----------------------- 
--- response header --- 
HTTP/1.1 101 Switching Protocols 
Upgrade: websocket 
Connection: Upgrade 
Sec-WebSocket-Accept: x2LKwD6zgl3oUV+6z+WhBHBwxCw= 
Sec-WebSocket-Protocol: tr_json2 
-----------------------
WebSocket successfully connected!


Thanks.

EDIT: It works now, I just don't have the credentials at the moment and am in touch with the account manager. Thanks!

Upvotes
79.2k 251 52 74

@wesley.ng

Thanks for reaching out to us.

I can run it properly.

[stream_connection] - [_run_websocket_listener] - [OMMSTREAMING_PRICING_0.0] connect
    num_attempt : 0
    url         : ws://localhost:9060/api/rdp/streaming/pricing/v1/WebSocket
    headers     : ['User-Agent: Python', 'x-tr-applicationid: <application id>', 'Authorization: Bearer <token>']
    cookies     : None
    transport   : websocket
    subprotocols: ['tr_json2']
[2023-01-06 12:07:19,216] - [sessions.desktop.workspace.0] - [DEBUG] - [112488 - ThreadOMMSTREAMING_PRICING_0.0] - [stream_connection] - [_on_ws_open] - [OMMSTREAMING_PRICING_0.0] on_ws_open

1672983470882.png

Please share the full Refinitiv Data log when the problem occurred. I would like to verify the version of the RD library and dependencies.

You can also run the following code to test the connection.

import websocket
import threading
import time

url = "ws://localhost:9060/api/rdp/streaming/pricing/v1/WebSocket"
application_id = "x-tr-applicationid: <application id>"
token = "Authorization: Bearer <token>"

headers = ['User-Agent: Python',application_id,token]
 
def on_message(ws, message):
    print("on message")
    
def on_error(ws, error):
    """ Called when websocket error has occurred """
    print(error)


def on_close(ws, close_status_code, close_msg):
    """ Called when websocket is closed """
   
    print("WebSocket Closed")


def on_open(ws):
    """ Called when handshake is complete and websocket is open, send login """

    print("WebSocket successfully connected!")
    

# Start websocket handshake
   
print("Connecting to WebSocket " + url + " ...")
web_socket_app = websocket.WebSocketApp(url, header=headers,
                                        on_message=on_message,
                                        on_error=on_error,
                                        on_close=on_close,
                                        subprotocols=['tr_json2'])
web_socket_app.on_open = on_open

    # Event loop
wst = threading.Thread(target=web_socket_app.run_forever)
wst.start()

try:
    while True:
        time.sleep(1)
except KeyboardInterrupt:
    web_socket_app.close() 

You can the url, <application id>, and <token> from the Refinitiv Data log.

The output looks like this:

1672983386734.png

If this code is unable to establish a websocket connection, the problem could be from network settings.


1672983470882.png (18.6 KiB)
1672983386734.png (46.8 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Hi Jiraponse,

Thanks - comparing the headers between yours and mine its clear that I do not have a token, and only have an application ID.

My understanding is that the application ID is the App Key from the Eikon desktop application, what is the application ID here then?

Thanks.

Wesley

@wesley.ng

Yes, it is an application key.

You can remove token from the headers.

headers = ['User-Agent: Python',application_id]

Please verify the versoin of the RD library that you are using.

I am using the latest one (RD version is 1.0.0).

Hi Jirapongse,

With default company proxy settings:

1673317553659.png

Explicitly setting proxy to None:

1673317645516.png

Is there a way to troubleshoot the network settings?

My version of RD library is 1.0.0b25.

1673317553659.png (7.1 KiB)
1673317645516.png (8.9 KiB)

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.