Python websocket connection to Refinitiv Realtime fails with SSL error

_Tom_
_Tom_ Newcomer
in Refinitiv Data Platform

I am trying to connect to the Refinitiv Realtime Optimized platform in AWS using websockets in python.

I am using the market_price_rdpgw_service_discovery.py example from https://github.com/Refinitiv/websocket-api/tree/master/Applications/Examples/RDP/python#windowslinuxmacos


The HTTP requests for service discovery work and I can setup a token but the websocket connection fails:

Sending Refinitiv Data Platform service discovery request to https://api.refinitiv.com/streaming/pricing/v1/

Refinitiv Data Platform Service discovery succeeded. RECEIVED:
....

Connecting to WebSocket wss://eu-west-1-aws-3-sm.optimized-pricing-api.refinitiv.net:443/WebSocket for session1...
Traceback (most recent call last):
  File "/home/tom/anaconda3/envs/rrt_client/lib/python3.8/site-packages/websocket/_app.py", line 312, in run_forever
    self.sock.connect(
  File "/home/tom/anaconda3/envs/rrt_client/lib/python3.8/site-packages/websocket/_core.py", line 248, in connect
    self.sock, addrs = connect(url, self.sock_opt, proxy_info(**options),
  File "/home/tom/anaconda3/envs/rrt_client/lib/python3.8/site-packages/websocket/_http.py", line 121, in connect
    sock = _ssl_socket(sock, options.sslopt, hostname)
  File "/home/tom/anaconda3/envs/rrt_client/lib/python3.8/site-packages/websocket/_http.py", line 256, in _ssl_socket
    sock = _wrap_sni_socket(sock, sslopt, hostname, check_hostname)
  File "/home/tom/anaconda3/envs/rrt_client/lib/python3.8/site-packages/websocket/_http.py", line 231, in _wrap_sni_socket
    return context.wrap_socket(
  File "/home/tom/anaconda3/envs/rrt_client/lib/python3.8/ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "/home/tom/anaconda3/envs/rrt_client/lib/python3.8/ssl.py", line 1040, in _create
    self.do_handshake()
  File "/home/tom/anaconda3/envs/rrt_client/lib/python3.8/ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:1131)
EOF occurred in violation of protocol (_ssl.c:1131) for session1
WebSocket Closed for session1


I've tried disabling all SSL verification and fixing the version to TSL 1.2 but still get the same error on handshake with the socket.

 def connect(self):
        # Start websocket handshake
        ws_address = "wss://{}/WebSocket".format(self.host)
        print("Connecting to WebSocket " + ws_address + " for " + self.session_name + "...")
        self.web_socket_app = websocket.WebSocketApp(ws_address,
                                                     on_message=self._on_message,
                                                     on_error=self._on_error,
                                                     on_close=self._on_close,
                                                     on_open=self._on_open,
                                                     subprotocols=['tr_json2'])  
        sslopt = {
            'check_hostname': False,
            "verify_mode": ssl.CERT_NONE,
            "cert_reqs": ssl.CERT_NONE,
            "ssl_version": ssl.PROTOCOL_TLSv1_2,
        }
        wst = threading.Thread(target=self.web_socket_app.run_forever, kwargs={'sslopt': sslopt})
        wst.start()


Here are some of the relevant library version:

pip list |grep 'socket\|request\|http\|cert\|url'
certifi              2021.10.8
ndg-httpsclient      0.5.1
requests             2.22.0
requests-toolbelt    0.9.1
urllib3              1.25.11
websocket-client     1.1.0


Has anyone else seen this and been able to resolve?

This is running behind a corporate proxy which I have no control over, but the HTTP requests for service discovery are working which makes me thing the SSL setup is OK in python requests generally, but there is an issue with websockets.

Tagged:

Best Answer

  • Jirapongse
    Jirapongse ✭✭✭✭✭
    Answer ✓

    @_Tom_

    I tested with the mitmdump (Man In The Middle Dump) proxy.

    The code can run properly with the http_proxy environment variable.

    1646044691882.png


Answers

  • umer.nalla
    umer.nalla LSEG

    Hi @_Tom_

    We cannot provide much Websocket API guidance in terms of dealing with corporate proxies as there can be many factors at play and variations depending on your local security implementations.

    However, a few things to try

    Certainly, I am aware of a couple of users where the 2nd option allowed to connect.

    You should speak to your internal security team for further guidance. You can also raise a ticket with the RTO team - who may be able to provide some assistance to your security team.


  • _Tom_
    _Tom_ Newcomer

    I've tried setting http_proxy and https_proxy, it works for the initial http requests but not for the websocket. If I set them incorrectly then I can break the normal requests, setting them correctly goes back to trying the websocket connection and failing:

    Connecting to WebSocket wss://eu-west-1-aws-3-sm.optimized-pricing-api.refinitiv.net:443/WebSocket for session1...
    Traceback (most recent call last):
      File "/home/tom/anaconda3/envs/rrt_client/lib/python3.8/site-packages/websocket/_app.py", line 312, in run_forever
        self.sock.connect(
      File "/home/tom/anaconda3/envs/rrt_client/lib/python3.8/site-packages/websocket/_core.py", line 248, in connect
        self.sock, addrs = connect(url, self.sock_opt, proxy_info(**options),
      File "/home/tom/anaconda3/envs/rrt_client/lib/python3.8/site-packages/websocket/_http.py", line 117, in connect
        sock = _tunnel(sock, hostname, port, auth)
      File "/home/tom/anaconda3/envs/rrt_client/lib/python3.8/site-packages/websocket/_http.py", line 287, in _tunnel
        raise WebSocketProxyException(
    websocket._exceptions.WebSocketProxyException: failed CONNECT via proxy status: 504
    failed CONNECT via proxy status: 504 for session1
    WebSocket Closed for session1

    Basic network connectivity works on port 443 based on telnet:

    telnet eu-west-1-aws-3-sm.optimized-pricing-api.refinitiv.net 443
    Trying 159.43.201.118...
    Connected to eu-west-1-aws-3-sm.optimized-pricing-api.refinitiv.biz.
    Escape character is '^]'.
  • _Tom_
    _Tom_ Newcomer

    For anyone else reading this thread with similar problems it turned out to be network configuration issues. The hosts returned from the service discovery call resolve to internal IPs that we didn't have routing for (e.g. refinitiv.net were CNAMEs for refinitiv.biz internal addresses).

    We did have network connectivity setup for us-east region and just set that group of hosts. The EOF error from the websocket ssl handshake is really because the resolved IP is not a refinitiv RTO server due to the network setup.

  • _Tom_
    _Tom_ Newcomer
    https://community.developers.refinitiv.com/discussion/comment/90917#Comment_90917
    Thanks for checking the proxy setup, I've commented below on the real issue which was company specific network setup. But this will be useful for others.

Categories