question

Upvotes
Accepted
f.palma avatar image
1 ●0 ●2 ●2

error in from_json_to_df in PyDSWS

Hi, I saw there is the same question on the forum, but without any actual solution. Sometimes, when I try to make a request: 

import PyDSWS

ds = PyDSWS.Datastream(username=RefinitivCredentials.username, password=RefinitivCredentials.password)
data = ds.get_data(tickers=tickers, start=start_date, end=end_date, freq=frequency, fields=fields)

I get this:

File "/usr/local/lib/python3.7/site-packages/PyDSWS/datastream.py", line 102, in get_data

df = self.from_json_to_df(response)

File "/usr/local/lib/python3.7/site-packages/PyDSWS/datastream.py", line 41, in from_json_to_df

if response_json['Dates']:

KeyError: 'Dates'"


Moreover, it seems that it does not always happen and I'm not actually able to reproduce it through https://product.datastream.com/DswsClient/Docs/TestRestV1.aspx?



datastream-apidsws-apierror
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

wasin.w avatar image   wasin.w

Hello @f.palma

Thank you for your participation in the forum. Is the reply below satisfactory in resolving your query?


If so please can you click the 'Accept' text next to the appropriate reply. This will guide all community members who have a similar question.

Thanks,


AHS

Jirapongse avatar image   Jirapongse

@f.palma

Hi,

Please be informed that a reply has been verified as correct in answering the question, and has been marked as such.

Thanks,

AHS

1 Answer

· Write an Answer
Upvotes
Accepted
Jirapongse avatar image
78.8k ●250 ●52 ●74

@f.palma

Typically, the response will have Dates field like this:

   {
        "AdditionalResponses": [
           ...
        ],
        "DataTypeNames": null,
        "DataTypeValues": [
          ...
        ],
        "Dates": [
            "/Date(1600128000000+0000)/",
            "/Date(1600214400000+0000)/",
            "/Date(1600300800000+0000)/",
            "/Date(1600387200000+0000)/",
            "/Date(1600646400000+0000)/",
            "/Date(1600732800000+0000)/",
            "/Date(1600819200000+0000)/",
           ...
        ],

The error indicates that the response doesn't have the "Dates" field. We may need to verify the returned response. One way to do that is by using a proxy. You can refer to Use mitmdump to Capture Refinitiv Real-Time - Optimized Content to setup a proxy.

Then run the mitmdump.exe with the following parameter.

C:\Program Files (x86)\mitmproxy\bin>mitmdump.exe --flow-detail=3

Proxy server listening at http://*:8080

Next, set proxy to the python code.

import os
os.environ['HTTP_PROXY']="http://127.0.0.1:8080"
os.environ['HTTPS_PROXY']="http://127.0.0.1:8080"
os.environ['REQUESTS_CA_BUNDLE']="c:\\temp\\mitmproxy-ca.pem"

The certificate files will be created in the config directory (~/.mitmproxy or %USERPROFILE%/.mitmproxy by default). Please change the location of "c:\\temp\\mitmproxy-ca.pem".

After that, you will see data sent through the proxy.

Please share the response when the problem occured.


1602844865542.png (33.4 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.