Issue retrieving data from datastream python API - possibly a requests 2.22 issue

Joey
Joey Newcomer
in Datastream

Hi I'm using the python API for datastream with no issues locally but struggling to trigger these scripts on my server on a schedule.

The problem I'm facing is that I fly blindly on the server with very little error information - simply the result is just the get_data function retrieves nothing.

This server needs to have the API url whitelisted (as all web urls are otherwise blocked). I have whitelisted the below - Can anyone confirm this is correct?

https://product.datastream.com/DSWSClient/V1/DSService.svc/rest/

https://datastreamprod.int.refinitiv.com/dswsclient/V1/DSService.svc/rest/


Secondly my python installation is different on this server where the requests library is version 2.22 and my local requests library is 2.24. Aligning is very difficult (corporate IT). Are there any known issues utilizing this version of requests through the python API?

Tagged:

Best Answer

  • Gurpreet
    Gurpreet admin
    Answer ✓

    Hello @Joey

    I have been able to get more debugging information by enabling logging and http-debug. You can try and see if this helps troubleshoot the issue:

    # add this code into your python dsws application
    import logging
    import http.client

    http.client.HTTPConnection.debuglevel = 1
    logging.basicConfig()
    logging.getLogger().setLevel(logging.DEBUG)
    requests_log = logging.getLogger("requests.packages.urllib3")
    requests_log.setLevel(logging.DEBUG)
    requests_log.propagate = True

    This generates additional debugging information which you can pipe into a file. Here is what I capture:

    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): product.datastream.com:443
    DEBUG:urllib3.connectionpool:https://product.datastream.com:443 "POST /DSWSClient/V1/DSService.svc/rest/GetToken HTTP/1.1" 200 355

    DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): product.datastream.com:443
    DEBUG:urllib3.connectionpool:https://product.datastream.com:443 "POST /DSWSClient/V1/DSService.svc/rest/GetData HTTP/1.1" 200 360

    send: b'POST /DSWSClient/V1/DSService.svc/rest/GetToken HTTP/1.1\r\nHost: product.datastream.com\r\nUser-Agent: python-requests/2.25.1\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\nContent-Length: 111\r\nContent-Type: application/json\r\n\r\n'
    send: b'{"Password": "****", "Properties": [{"Key": "__AppId", "Value": "PythonLib 1.0.8"}], "UserName": "****"}'
    reply: 'HTTP/1.1 200 OK\r\n'
    header: Content-Type: application/json; charset=utf-8
    header: X-Powered-By: ASP.NET
    header: Date: Thu, 11 Feb 2021 14:37:22 GMT
    header: Content-Length: 355

    send: b'POST /DSWSClient/V1/DSService.svc/rest/GetData HTTP/1.1\r\nHost: product.datastream.com\r\nUser-Agent: python-requests/2.25.1\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\nContent-Length: 584\r\nContent-Type: application/json\r\n\r\n'
    send: b'{"DataRequest": {"DataTypes": [{"Properties": null, "Value": "VO"}, {"Properties": null, "Value": "P"}], "Instrument": {"Properties": null, "Value": "VOD"}, "Date": {"End": "", "Frequency": "", "Kind": 0, "Start": "2017-01-01"}, "Tag": null}, "Properties": {"Key": "Source", "Value": null}, "TokenValue": "fMSwrVbhK****222870"}'
    reply: 'HTTP/1.1 200 OK\r\n'
    header: Content-Type: application/json; charset=utf-8
    header: X-Powered-By: ASP.NET
    header: Date: Thu, 11 Feb 2021 14:37:22 GMT
    header: Content-Length: 360


      Instrument Datatype     Value       Dates
    0        VOD       VO  36773.80  2017-01-01
    1        VOD        P    199.85  2017-01-01


Answers

Categories