error in from_json_to_df in PyDSWS

f.palma
f.palma Newcomer
in Datastream

Hi, I saw there is the same question on the forum, but without any actual solution. Sometimes, when I try to make a request: 

import PyDSWS

ds = PyDSWS.Datastream(username=RefinitivCredentials.username, password=RefinitivCredentials.password)
data = ds.get_data(tickers=tickers, start=start_date, end=end_date, freq=frequency, fields=fields)

I get this:

File "/usr/local/lib/python3.7/site-packages/PyDSWS/datastream.py", line 102, in get_data

df = self.from_json_to_df(response)

File "/usr/local/lib/python3.7/site-packages/PyDSWS/datastream.py", line 41, in from_json_to_df

if response_json['Dates']:

KeyError: 'Dates'"


Moreover, it seems that it does not always happen and I'm not actually able to reproduce it through https://product.datastream.com/DswsClient/Docs/TestRestV1.aspx?



Tagged:

Best Answer

  • Jirapongse
    Jirapongse ✭✭✭✭✭
    Answer ✓

    @f.palma

    Typically, the response will have Dates field like this:

       {
            "AdditionalResponses": [
               ...
            ],
            "DataTypeNames": null,
            "DataTypeValues": [
              ...
            ],
            "Dates": [
                "/Date(1600128000000+0000)/",
                "/Date(1600214400000+0000)/",
                "/Date(1600300800000+0000)/",
                "/Date(1600387200000+0000)/",
                "/Date(1600646400000+0000)/",
                "/Date(1600732800000+0000)/",
                "/Date(1600819200000+0000)/",
               ...
            ],

    The error indicates that the response doesn't have the "Dates" field. We may need to verify the returned response. One way to do that is by using a proxy. You can refer to Use mitmdump to Capture Refinitiv Real-Time - Optimized Content to setup a proxy.

    Then run the mitmdump.exe with the following parameter.

    C:\Program Files (x86)\mitmproxy\bin>mitmdump.exe --flow-detail=3

    Proxy server listening at http://*:8080

    Next, set proxy to the python code.

    import os
    os.environ['HTTP_PROXY']="http://127.0.0.1:8080"
    os.environ['HTTPS_PROXY']="http://127.0.0.1:8080"
    os.environ['REQUESTS_CA_BUNDLE']="c:\\temp\\mitmproxy-ca.pem"

    The certificate files will be created in the config directory (~/.mitmproxy or %USERPROFILE%/.mitmproxy by default). Please change the location of "c:\\temp\\mitmproxy-ca.pem".

    After that, you will see data sent through the proxy.

    image

    Please share the response when the problem occured.

Categories