question

Upvotes
mdziel avatar image
7 ●1 ●2 ●3

ReadTimeout on simple requests

I'm just getting started with the Refinitiv Data Library and trying to go through some of the examples contained here:

https://github.com/LSEG-API-Samples/Example.DataLibrary.Python/blob/main/Examples/2-Content/2.04-ESG/EX-2.04.01-ESG.ipynb

However, I get stuck on even the most basic ones like:

response = esg.basic_overview.Definition("MSFT.O").get_data()

I'm running a desktop.workspace session and have Eikon open in the background.

This is the error log:

An error occurred while requesting URL('http://localhost:9060/api/rdp/data/environmental-social-governance/v1/views/basic?universe=MSFT.O').
ReadTimeout('timed out')
Traceback (most recent call last):

  File ~\AppData\Local\anaconda3\lib\site-packages\httpcore\_exceptions.py:8 in map_exceptions
    yield

  File ~\AppData\Local\anaconda3\lib\site-packages\httpcore\backends\sync.py:26 in read
    return self._sock.recv(max_bytes)

TimeoutError: timed out


During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File ~\AppData\Local\anaconda3\lib\site-packages\httpx\_transports\default.py:60 in map_httpcore_exceptions
    yield

  File ~\AppData\Local\anaconda3\lib\site-packages\httpx\_transports\default.py:218 in handle_request
    resp = self._pool.handle_request(req)

  File ~\AppData\Local\anaconda3\lib\site-packages\httpcore\_sync\connection_pool.py:253 in handle_request
    raise exc

  File ~\AppData\Local\anaconda3\lib\site-packages\httpcore\_sync\connection_pool.py:237 in handle_request
    response = connection.handle_request(request)

  File ~\AppData\Local\anaconda3\lib\site-packages\httpcore\_sync\connection.py:90 in handle_request
    return self._connection.handle_request(request)

  File ~\AppData\Local\anaconda3\lib\site-packages\httpcore\_sync\http11.py:105 in handle_request
    raise exc

  File ~\AppData\Local\anaconda3\lib\site-packages\httpcore\_sync\http11.py:84 in handle_request
    ) = self._receive_response_headers(**kwargs)

  File ~\AppData\Local\anaconda3\lib\site-packages\httpcore\_sync\http11.py:148 in _receive_response_headers
    event = self._receive_event(timeout=timeout)

  File ~\AppData\Local\anaconda3\lib\site-packages\httpcore\_sync\http11.py:177 in _receive_event
    data = self._network_stream.read(

  File ~\AppData\Local\anaconda3\lib\site-packages\httpcore\backends\sync.py:24 in read
    with map_exceptions(exc_map):

  File ~\AppData\Local\anaconda3\lib\contextlib.py:153 in __exit__
    self.gen.throw(typ, value, traceback)

  File ~\AppData\Local\anaconda3\lib\site-packages\httpcore\_exceptions.py:12 in map_exceptions
    raise to_exc(exc)

ReadTimeout: timed out


The above exception was the direct cause of the following exception:

Traceback (most recent call last):

  Cell In[33], line 1
    response = esg.basic_overview.Definition("MSFT.O").get_data()

  File ~\AppData\Local\anaconda3\lib\site-packages\refinitiv\data\content\_content_provider_layer.py:58 in get_data
    return super().get_data(session, on_response)

  File ~\AppData\Local\anaconda3\lib\site-packages\refinitiv\data\delivery\_data\_data_provider_layer.py:149 in get_data
    response = get_data(self._data_type, self._provider, session, **self._kwargs)

  File ~\AppData\Local\anaconda3\lib\site-packages\refinitiv\data\delivery\_data\_data_provider_layer.py:51 in get_data
    response = provider.get_data(session, url, auto_retry=auto_retry, **kwargs)

  File ~\AppData\Local\anaconda3\lib\site-packages\refinitiv\data\delivery\_data\_data_provider.py:80 in get_data
    raw_response = self.connection.send(request, session, *args, **kwargs)

  File ~\AppData\Local\anaconda3\lib\site-packages\refinitiv\data\delivery\_data\_connection.py:14 in send
    return session.http_request(request)

  File ~\AppData\Local\anaconda3\lib\site-packages\refinitiv\data\_core\session\_session.py:504 in http_request
    return self._http_service.request(request)

  File ~\AppData\Local\anaconda3\lib\site-packages\refinitiv\data\_core\session\http_service.py:334 in request
    raise error

  File ~\AppData\Local\anaconda3\lib\site-packages\refinitiv\data\_core\session\http_service.py:312 in request
    response = self._client.send(request)

  File ~\AppData\Local\anaconda3\lib\site-packages\refinitiv\data\_core\session\http_service.py:204 in send
    response = client.send(request, follow_redirects=True)

  File ~\AppData\Local\anaconda3\lib\site-packages\httpx\_client.py:901 in send
    response = self._send_handling_auth(

  File ~\AppData\Local\anaconda3\lib\site-packages\httpx\_client.py:929 in _send_handling_auth
    response = self._send_handling_redirects(

  File ~\AppData\Local\anaconda3\lib\site-packages\httpx\_client.py:966 in _send_handling_redirects
    response = self._send_single_request(request)

  File ~\AppData\Local\anaconda3\lib\site-packages\httpx\_client.py:1002 in _send_single_request
    response = transport.handle_request(request)

  File ~\AppData\Local\anaconda3\lib\site-packages\httpx\_transports\default.py:217 in handle_request
    with map_httpcore_exceptions():

  File ~\AppData\Local\anaconda3\lib\contextlib.py:153 in __exit__
    self.gen.throw(typ, value, traceback)

  File ~\AppData\Local\anaconda3\lib\site-packages\httpx\_transports\default.py:77 in map_httpcore_exceptions
    raise mapped_exc(message) from exc

ReadTimeout: timed out

I am however able to get a response for the following query:

response = symbol_conversion.Definition("MSFT.O").get_data()

Thus, I don't think there is anything fundamentally wrong with my configuration.

Please help!

python#technologyrdp-apirefinitiv-data-platformrefinitiv-data-platform-librariesjupyter-notebook
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

raksina.samasiri avatar image   raksina.samasiri

Hi @mdziel ,

Thank you for your participation in the forum. Is the reply below satisfactory in resolving your query?
If so please can you click the 'Accept' text next to the appropriate reply? This will guide all community members who have a similar question.

Thanks,
AHS

mdziel avatar image   mdziel raksina.samasiri
Well, the issue is still not resolved and I have now submitted a support ticket as per @Jirapongse's suggestion.


Best,

Michał

raksina.samasiri avatar image   raksina.samasiri

<AHS>

case 12993231 is still in progress, extend the triage

raksina.samasiri avatar image   raksina.samasiri

<AHS>

case 12993231 is still in progress, extend the triage

3 Answers

· Write an Answer
Upvotes
Jirapongse avatar image
79.2k ●251 ●52 ●74

@mdziel

Thank you for reaching out to us.

It may relate to the HTTP timeout. You can try to increase the timeout value in the RD library configuration file (refinitiv-data.config.json), as shown below. The sample refinitiv-data.config.json is available on GitHub.

{
    "http": {       
        "request-timeout": 60
    },
    "logs": {
...
    }
...
}

Otherwise, you can change the timeout value by using the following code.

rd.get_config()["http.request-timeout"] = 60
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
mdziel avatar image
7 ●1 ●2 ●3

Thank you for your prompt reply!

You may be right, since the timeout occurs after about 20 seconds, which seems to be the default setting for http.request-timeout:


So, I do as you suggest and run:

rd.get_config()["http.request-timeout"] = 60

I then verify that the parameter was indeed changed accordingly:

In [45]: rd.get_config().as_dict()["http.request-timeout"]
Out[45]: 60

Unfortunately, when running the query...

response = esg.basic_overview.Definition("MSFT.O").get_data()

...I still receive a timeout error after about 20 seconds, as if nothing has changed...

What can still be wrong?

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Jirapongse avatar image Jirapongse

@mdziel

You may need to set the timeout before opening the session.

import refinitiv.data as rd
rd.get_config()["http.request-timeout"] = 60
rd.open_session()

Otherwise, please try to use the RD library configuration file (refinitiv-data.config.json) instead.

mdziel avatar image mdziel Jirapongse

Thanks again!

Indeed, now the situation changed in that... I receive the timeout error after about 60 seconds ;-) Of course, I could keep increasing the http.request-timeout value but I wonder if the system is actually supposed to be that slow or could there be something else wrong? Could it be something related to the fact that I'm running Eikon and not the RDP Workspace?

Jirapongse avatar image Jirapongse mdziel

@mdziel

You can enable the debug log to verify what the problem is;

config = rd.get_config()
config.set_param("logs.transports.console.enabled", True)
config.set_param("logs.level", "debug")
config.set_param("http.request-timeout", 60)
rd.open_session()

It could be the problem on the Eikon side. I can't test the code because I don't have permission to acces this ESG service on the desktop session.

Upvotes
mdziel avatar image
7 ●1 ●2 ●3

@Jirapongse, thank you for patiently helping out! I'm adding this as a new answer, because it's too long to fit as a comment.

The debug log has perhaps indeed revealed a valuable clue.

When submitting the successful query:

response = symbol_conversion.Definition("MSFT.O").get_data()

the log I get is:

[2023-10-16T10:58:36.831118+02:00] - [DEBUG] - [sessions.desktop.workspace.1] - [13744] | MainThread
HTTP Request id 7
    url = http://localhost:9060/api/rdp/discovery/search/v1/lookup
    method = POST
    headers = {'Content-Type': 'application/json', 'Authorization': 'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJBcGlWZXJzaW9uIjoiMSIsIkFwcEtleSI6ImEyZThhNDUwYzQyOTRiNDFhMTI2ODc3ZDcyNjFhODhiODQzNTJmNzgiLCJBcHBTY29wZSI6InRyYXBpIiwiTGlicmFyeU5hbWUiOiJSRFAgUHl0aG9uIExpYnJhcnkiLCJMaWJyYXJ5VmVyc2lvbiI6IjEuNC4wIiwiaWF0IjoxNjk3NDQ2NjkzLCJleHAiOjE2OTg2NTYyOTN9.pkR2lNpRoHwnjkgbM5ambAc1LoNG7SEKuedX2E7OFqg', 'x-tr-applicationid': 'MY_APP_KEY'}
    params = None
    cookies = None
    data = None
    json = {'View': <Views.SEARCH_ALL: 'SearchAll'>, 'Terms': 'MSFT.O', 'Scope': '_AllUnique', 'Select': 'DocumentTitle,RIC,IssueISIN,CUSIP,SEDOL,TickerSymbol,IssuerOAPermID,FundClassLipperID'}

and the ensuing response is:

[2023-10-16T10:58:39.572773+02:00] - [DEBUG] - [sessions.desktop.workspace.1] - [13744] | MainThread
HTTP Response id 7
    status_code = 200
    text = {"Matches":{"MSFT.O":{"DocumentTitle":"Microsoft Corp, Ordinary Share, NASDAQ Global Select Consolidated","RIC":"MSFT.O","IssueISIN":"US5949181045","CUSIP":"594918104","TickerSymbol":"MSFT","IssuerOAPermID":"4295907168"}}}


However, when submitting the failed query:

response = esg.basic_overview.Definition("MSFT.O").get_data()

the log I get is:

[2023-10-16T10:58:44.548706+02:00] - [DEBUG] - [sessions.desktop.workspace.1] - [13744] | MainThread
HTTP Request id 8
    url = http://localhost:9060/api/rdp/data/environmental-social-governance/v1/views/basic?universe=MSFT.O
    method = GET
    headers = {'Authorization': 'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJBcGlWZXJzaW9uIjoiMSIsIkFwcEtleSI6ImEyZThhNDUwYzQyOTRiNDFhMTI2ODc3ZDcyNjFhODhiODQzNTJmNzgiLCJBcHBTY29wZSI6InRyYXBpIiwiTGlicmFyeU5hbWUiOiJSRFAgUHl0aG9uIExpYnJhcnkiLCJMaWJyYXJ5VmVyc2lvbiI6IjEuNC4wIiwiaWF0IjoxNjk3NDQ2NjkzLCJleHAiOjE2OTg2NTYyOTN9.pkR2lNpRoHwnjkgbM5ambAc1LoNG7SEKuedX2E7OFqg', 'x-tr-applicationid': 'MY_APP_KEY'}
    params = None
    cookies = None
    data = None
    json = None

Thus it seems that a proper json request is never formed and submitted to begin with and maybe that's why the result is a timeout error?

[2023-10-16T10:59:44.557267+02:00] - [ERROR] - [sessions.desktop.workspace.1] - [13744] | MainThread
An error occurred while requesting URL('http://localhost:9060/api/rdp/data/environmental-social-governance/v1/views/basic?universe=MSFT.O').
    ReadTimeout('timed out')


In any case, does this give you any ideas as to what the root cause might be?

Thanks again!

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Jirapongse avatar image Jirapongse
@mdziel

According to the debug log, it could be the problem in the Eikon process (API Proxy) which didn't send any response back to the library.

You need to contact the Eikon support team via MyRefinitiv to verify what the problem is.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.