Retrieval of Snapshot data with RDP for Python
Merijn
Newcomer
We are new to the RDP library and try to setup a retrieval of snapshot data from Refinitiv.
Currently it looks like my code is able to log on correctly - but the call to get_snapshot gives an error. Can you help us with this?
Here is a short abstract of my code - showing the issue:
import rf_settings as settings
import refinitiv.dataplatform as rdp
session = rdp.open_platform_session(
settings.APP_KEY,
rdp.GrantPassword(
username=settings.RDP_LOGIN,
password=settings.RDP_PASSWORD
)
)
universe = ['EUR=']
fields = ['ASK', 'BID']
response = rdp.Pricing.get_snapshot(universe, fields)
rdp.close_session()
When I run this code I see the following error:
ERROR!!! An error occured while requesting URL('https://api.refinitiv.com/data/pricing/beta3/snapshots?universe=EUR=&fields=ASK,BID').
Exception has occurred: ConnectError
[Errno 11001] getaddrinfo failed
During handling of the above exception, another exception occurred:
The above exception was the direct cause of the following exception:
File "D:\Project\refinitiv\rdp_library_connection.py", line 15, in
response = rdp.Pricing.get_snapshot(universe, fields)
Currently it looks like my code is able to log on correctly - but the call to get_snapshot gives an error. Can you help us with this?
Here is a short abstract of my code - showing the issue:
import rf_settings as settings
import refinitiv.dataplatform as rdp
session = rdp.open_platform_session(
settings.APP_KEY,
rdp.GrantPassword(
username=settings.RDP_LOGIN,
password=settings.RDP_PASSWORD
)
)
universe = ['EUR=']
fields = ['ASK', 'BID']
response = rdp.Pricing.get_snapshot(universe, fields)
rdp.close_session()
When I run this code I see the following error:
ERROR!!! An error occured while requesting URL('https://api.refinitiv.com/data/pricing/beta3/snapshots?universe=EUR=&fields=ASK,BID').
Exception has occurred: ConnectError
[Errno 11001] getaddrinfo failed
During handling of the above exception, another exception occurred:
The above exception was the direct cause of the following exception:
File "D:\Project\refinitiv\rdp_library_connection.py", line 15, in
response = rdp.Pricing.get_snapshot(universe, fields)
0
Best Answer
-
Hi @wasin.w
There were several issues here
- Corporate firewall/network issues - which were addressed internally by client
- wrong credential values being used e.g. My.Refinitiv login details instead of MachineID
- MachineID was not permissioned for RDP Snapshot API - only RTO - so the client was advised to use streaming interface with the Snapshot option
0
Answers
-
Hi @Merijn
your code as far as I can read it looks fine. I tried the following and it worked OK
import refinitiv.dataplatform as rdp
print(rdp.__version__)
rdp.open_platform_session(
APP_KEY,
rdp.GrantPassword(
username = RDP_LOGIN,
password = RDP_PASSWORD
)
)
universe = ['EUR=']
fields = ['ASK', 'BID']
response = rdp.Pricing.get_snapshot(universe, fields)
print(response.data.df)
rdp.close_session()can you confirm which version of the RDP library you are using?
I am using v 1.0.0a8.post1 from refinitiv-dataplatform · PyPI
If you are also using the same version, then please enable logging and attach the output - after removing any credentials etc)
import logging
logging.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', level=logging.DEBUG)0 -
Hi @Merijn
Also, if you are behind a corporate firewall/proxy etc - can you please try and run the code from your personal PC?
One further test, from your corporate PC, are you able to navigate to API Playground (refinitiv.com) in a browser? If so, can you then select the /data/pricing/beta3/snapshots test area and execute one of the GET requests:
0 -
Hi Umer,
Thanks for your quick reply. I ran your code - with logging enabled. I still see the same error.
The version I have installed is 1.0.0a7.post7.
See output from the log created attached.
refinitiv-dataplatform.20210525.13-41-47.txt
I tried connecting to the API Playground. This is not working as well:
I may be able to setup a test from my private pc tomorrow. It is not so easy to avoid the corp network.
0 -
Hi @Merijn
I have discussed your issue with the RDP Lib Dev team and they have requested additional logging information,
Can you please add the following extra two lines to the start of your script and capture the output again, please
import logging
logging.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', level=logging.DEBUG)
import os
os.environ["HTTPX_LOG_LEVEL"] = "TRACE"
import refinitiv.dataplatform as rdp
#....
#open platform session etc...
#....Please remove all 'authorization', b'Bearer sections from any output before uploading (looks like a very long line of alphanumeric output).
Thanks.
0 -
Hi Umer,
I tried with the addtional login, but I don't see any additional output. When I run this code I get the following response - see attached log file created.
import os
import rf_settings as settings
import refinitiv.dataplatform as rdp
import logging
logging.basicConfig(
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', level=logging.DEBUG)
os.environ["HTTPX_LOG_LEVEL"] = "TRACE"
print(rdp.__version__)
rdp.open_platform_session(
settings.APP_KEY,
rdp.GrantPassword(
username=settings.RDP_LOGIN,
password=settings.RDP_PASSWORD
)
)
universe = ['EUR=']
fields = ['ASK', 'BID']
response = rdp.Pricing.get_snapshot(universe, fields)
print(response.data.df)
rdp.close_session()refinitiv-dataplatform.20210526.08-44-51.log.txt
1.0.0a7.post7
2021-05-26 08:44:51,602 - Session session.platform - Thread 1780 | MainThread
server-mode : False
2021-05-26 08:44:51,602 - session.platform - DEBUG - server-mode : False
2021-05-26 08:44:51,605 - Session session.platform - Thread 1780 | MainThread
AuthenticationTokenHandlerThread.authorize()
2021-05-26 08:44:51,605 - session.platform - DEBUG - AuthenticationTokenHandlerThread.authorize()
2021-05-26 08:44:51,606 - Session session.platform - Thread 12992 | AuthenticationTokenHandlerThread
STARTING :: AuthenticationTokenHandlerThread.run()
2021-05-26 08:44:51,606 - session.platform - DEBUG - STARTING :: AuthenticationTokenHandlerThread.run()
2021-05-26 08:44:51,607 - asyncio - DEBUG - Using proactor: IocpProactor
2021-05-26 08:44:51,608 - Session session.platform - Thread 12992 | AuthenticationTokenHandlerThread
Send a token request to https://api.refinitiv.com/auth/oauth2/v1/token
2021-05-26 08:44:51,608 - session.platform - DEBUG - Send a token request to https://api.refinitiv.com/auth/oauth2/v1/token
2021-05-26 08:44:51,608 - Session session.platform - Thread 12992 | AuthenticationTokenHandlerThread
{ 'grant_type' : 'password',
'client_id' : ****************6186,
'username' : vfm.im@devolksbank.nl,
'password' : *****************************************************************,
'scope' : trapi,
'takeExclusiveSignOnControl' : True }
2021-05-26 08:44:51,608 - session.platform - DEBUG - { 'grant_type' : 'password',
'client_id' : ****************6186,
'username' : vfm.im@devolksbank.nl,
'password' : *****************************************************************,
'scope' : trapi,
'takeExclusiveSignOnControl' : True }
2021-05-26 08:44:56,606 - Session session.platform - Thread 1780 | MainThread
RDP connection state = State.Open.
2021-05-26 08:44:56,606 - session.platform - DEBUG - RDP connection state = State.Open.
2021-05-26 08:44:56,607 - Session session.platform - Thread 1780 | MainThread
Send GET request asynchronously to https://api.refinitiv.com/data/pricing/beta3/snapshots?universe=EUR=&fields=ASK,BID
2021-05-26 08:44:56,607 - session.platform - DEBUG - Send GET request asynchronously to https://api.refinitiv.com/data/pricing/beta3/snapshots?universe=EUR=&fields=ASK,BID
2021-05-26 08:44:56,608 - Session session.platform - Thread 1780 | MainThread
Request to https://api.refinitiv.com/data/pricing/beta3/snapshots?universe=EUR=&fields=ASK,BID
headers = {'x-tr-applicationid': 'GE-A-00792776-3-6186'}
params = None
2021-05-26 08:44:56,608 - session.platform - DEBUG - Request to https://api.refinitiv.com/data/pricing/beta3/snapshots?universe=EUR=&fields=ASK,BID
headers = {'x-tr-applicationid': 'GE-A-00792776-3-6186'}
params = None
2021-05-26 08:44:56,617 - Session session.platform - Thread 1780 | MainThread
ERROR!!! An error occured while requesting URL('https://api.refinitiv.com/data/pricing/beta3/snapshots?universe=EUR=&fields=ASK,BID').
2021-05-26 08:44:56,617 - session.platform - ERROR - ERROR!!! An error occured while requesting URL('https://api.refinitiv.com/data/pricing/beta3/snapshots?universe=EUR=&fields=ASK,BID').
Traceback (most recent call last):
File "D:\Program Files\Python39\lib\site-packages\httpx\_exceptions.py", line 326, in map_exceptions
yield
File "D:\Program Files\Python39\lib\site-packages\httpx\_client.py", line 1492, in _send_single_request
(status_code, headers, stream, ext) = await transport.arequest(
File "D:\Program Files\Python39\lib\site-packages\httpx\_transports\default.py", line 169, in arequest
return await self._pool.arequest(
File "D:\Program Files\Python39\lib\site-packages\httpcore\_async\connection_pool.py", line 218, in arequest
response = await connection.arequest(
File "D:\Program Files\Python39\lib\site-packages\httpcore\_async\connection.py", line 93, in arequest
self.socket = await self._open_socket(timeout)
File "D:\Program Files\Python39\lib\site-packages\httpcore\_async\connection.py", line 119, in _open_socket
return await self.backend.open_tcp_stream(
File "D:\Program Files\Python39\lib\site-packages\httpcore\_backends\auto.py", line 44, in open_tcp_stream
return await self.backend.open_tcp_stream(
File "D:\Program Files\Python39\lib\site-packages\httpcore\_backends\asyncio.py", line 270, in open_tcp_stream
return SocketStream(
File "D:\Program Files\Python39\lib\contextlib.py", line 135, in __exit__
self.gen.throw(type, value, traceback)
File "D:\Program Files\Python39\lib\site-packages\httpcore\_exceptions.py", line 12, in map_exceptions
raise to_exc(exc) from None
httpcore.ConnectError: [Errno 11001] getaddrinfo failed
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "d:\Project\refinitiv\rdp_library_connection.py", line 22, in <module>
response = rdp.Pricing.get_snapshot(universe, fields)
File "D:\Program Files\Python39\lib\site-packages\refinitiv\dataplatform\pricing\pricing_.py", line 142, in get_snapshot
result = pricing._get_snapshot(universe=universe,
File "D:\Program Files\Python39\lib\site-packages\refinitiv\dataplatform\pricing\pricing_.py", line 149, in _get_snapshot
return self._snapshot_endpoint.session._loop.run_until_complete(self._get_snapshot_async(universe=universe,
File "D:\Program Files\Python39\lib\site-packages\nest_asyncio.py", line 70, in run_until_complete
return f.result()
File "D:\Program Files\Python39\lib\asyncio\futures.py", line 201, in result
raise self._exception
File "D:\Program Files\Python39\lib\asyncio\tasks.py", line 256, in __step
result = coro.send(None)
File "D:\Program Files\Python39\lib\site-packages\refinitiv\dataplatform\pricing\pricing_.py", line 171, in _get_snapshot_async
endpoint_response = await self._snapshot_endpoint.send_request_async(Endpoint.RequestMethod.GET,
File "D:\Program Files\Python39\lib\site-packages\refinitiv\dataplatform\delivery\data\endpoint.py", line 482, in send_request_async
_response = await self._params.session.http_request_async(_url,
File "D:\Program Files\Python39\lib\site-packages\refinitiv\dataplatform\core\session\platform_session.py", line 464, in http_request_async
return await Session.http_request_async(self, url, method=method, headers=headers,
File "D:\Program Files\Python39\lib\site-packages\refinitiv\dataplatform\core\session\session.py", line 1181, in http_request_async
raise error
File "D:\Program Files\Python39\lib\site-packages\refinitiv\dataplatform\core\session\session.py", line 1178, in http_request_async
request_response = await self._http_session.send(_http_request, **kwargs)
File "D:\Program Files\Python39\lib\site-packages\httpx\_client.py", line 1396, in send
response = await self._send_handling_auth(
File "D:\Program Files\Python39\lib\site-packages\httpx\_client.py", line 1434, in _send_handling_auth
response = await self._send_handling_redirects(
File "D:\Program Files\Python39\lib\site-packages\httpx\_client.py", line 1466, in _send_handling_redirects
response = await self._send_single_request(request, timeout)
File "D:\Program Files\Python39\lib\site-packages\httpx\_client.py", line 1492, in _send_single_request
(status_code, headers, stream, ext) = await transport.arequest(
File "D:\Program Files\Python39\lib\contextlib.py", line 135, in __exit__
self.gen.throw(type, value, traceback)
File "D:\Program Files\Python39\lib\site-packages\httpx\_exceptions.py", line 343, in map_exceptions
raise mapped_exc(message, **kwargs) from exc # type: ignore
httpx.ConnectError: [Errno 11001] getaddrinfo failed
\Program Files\Python39\lib\site-packages\httpx\_client.py:1772: UserWarning: Unclosed <httpx.AsyncClient object at 0x000001C4595C1C40>. See https://www.python-httpx.org/async/#opening-and-closing-clients for details.
[Done] exited with code=1 in 6.399 seconds0 -
Hi @Merijn
Thanks for trying that - but for some reason, the additional trace output is not being generated on your PC. I have asked the RDP Library team for more guidance.
Were you able to try the code on a non-corporate network?
Also, what error message do you see if you try to post the following into your browser on your corporate network?
https://api.refinitiv.com/data/pricing/beta3/snapshots?universe=EUR%3D&fields=ASK%2CBID
Are you able to contact your internal network security and ask them to assist you as well - in terms of firewall / proxy help?
0 -
Hi Umer,
I have not tried setting up RDP outside of the corporate network yet. That will take some more time. I hope we can find another way to test this.
When I try the URL on the server I get the following return message:
0 -
I tried to run the code from my local machine (with Anaconda) - outside the corporate environment. In this case the logon to Refinitiv is failing.
2021-05-26 15:30:03,469 - asyncio - DEBUG - Using selector: SelectSelector
1.0.0a8.post1
2021-05-26 15:30:04,977 - httpx._client - DEBUG - HTTP Request: POST https://api.refinitiv.com/auth/oauth2/v1/token "HTTP/1.1 400 Bad Request"
ERROR!!! Failed to request a new token......
Token requested error b'{"error":"access_denied" ,"error_description":"Invalid username or password." } '
ERROR!!! response 400 while requesting URL('https://api.refinitiv.com/auth/oauth2/v1/token').
ERROR!!! authentication handler raise an exception.
HTTPStatusError('400 Client Error: Bad Request for url: https://api.refinitiv.com/auth/oauth2/v1/token\nFor more information check: https://httpstatuses.com/400')
ERROR!!! CANNOT authorize to RDP authentication endpoint.
Unexpected error 400 Client Error: Bad Request for url: https://api.refinitiv.com/auth/oauth2/v1/token
For more information check: https://httpstatuses.com/400.
(auto-reconnection is disbled)
---------------------------------------------------------------------------
PlatformSessionError Traceback (most recent call last)
<ipython-input-2-7f71bb52b82f> in <module>
15 print(rdp.__version__)
16
---> 17 rdp.open_platform_session(
18 APP_KEY,
19 rdp.GrantPassword(
~\anaconda3\lib\site-packages\refinitiv\dataplatform\factory\core_factory.py in open_platform_session(app_key, grant, deployed_platform_host, deployed_platform_username)
93 close_session()
94 set_default_session(session)
---> 95 session.open()
96 return session
97
~\anaconda3\lib\site-packages\refinitiv\dataplatform\core\session\session.py in open(self)
755 # session is already opened or is opening
756 return self._state
--> 757 self._loop.run_until_complete(self.open_async())
758 return self._state
759
~\anaconda3\lib\site-packages\nest_asyncio.py in run_until_complete(self, future)
96 raise RuntimeError(
97 'Event loop stopped before Future completed.')
---> 98 return f.result()
99 finally:
100 self._thread_id = old_thread_id
~\anaconda3\lib\asyncio\futures.py in result(self)
176 self.__log_traceback = False
177 if self._exception is not None:
--> 178 raise self._exception
179 return self._result
180
~\anaconda3\lib\asyncio\tasks.py in __step(***failed resolving arguments***)
278 # We use the `send` method directly, because coroutines
279 # don't have `__iter__` and `__next__` methods.
--> 280 result = coro.send(None)
281 else:
282 result = coro.throw(exc)
~\anaconda3\lib\site-packages\refinitiv\dataplatform\core\session\platform_session.py in open_async(self)
315
316 # do authentication process with Refinitiv Data Platform (RDP), if it's necessary
--> 317 self._connection.open()
318
319 # the platform session is ready,
~\anaconda3\lib\site-packages\refinitiv\dataplatform\core\session\connection.py in open(self)
184
185 # call authorize for do an authentication mechanism to the RDP platform by another thread
--> 186 self.authorize()
187
188 def authorize(self):
~\anaconda3\lib\site-packages\refinitiv\dataplatform\core\session\connection.py in authorize(self)
230
231 # raise the error
--> 232 raise PlatformSessionError(-1, 'ERROR!!! Authentication handler failed to request a access token.\n'
233 f'{self._authentication_token_handler_thread.last_exception}')
234
PlatformSessionError: Error code -1 | ERROR!!! Authentication handler failed to request a access token.
400 Client Error: Bad Request for url: https://api.refinitiv.com/auth/oauth2/v1/token
For more information check: https://httpstatuses.com/4000
Categories
- All Categories
- 3 Polls
- 6 AHS
- 36 Alpha
- 166 App Studio
- 6 Block Chain
- 4 Bot Platform
- 18 Connected Risk APIs
- 47 Data Fusion
- 34 Data Model Discovery
- 684 Datastream
- 1.4K DSS
- 614 Eikon COM
- 5.2K Eikon Data APIs
- 10 Electronic Trading
- Generic FIX
- 7 Local Bank Node API
- 3 Trading API
- 2.9K Elektron
- 1.4K EMA
- 248 ETA
- 554 WebSocket API
- 37 FX Venues
- 14 FX Market Data
- 1 FX Post Trade
- 1 FX Trading - Matching
- 12 FX Trading – RFQ Maker
- 5 Intelligent Tagging
- 2 Legal One
- 23 Messenger Bot
- 3 Messenger Side by Side
- 9 ONESOURCE
- 7 Indirect Tax
- 60 Open Calais
- 275 Open PermID
- 44 Entity Search
- 2 Org ID
- 1 PAM
- PAM - Logging
- 6 Product Insight
- Project Tracking
- ProView
- ProView Internal
- 22 RDMS
- 1.9K Refinitiv Data Platform
- 641 Refinitiv Data Platform Libraries
- 4 LSEG Due Diligence
- LSEG Due Diligence Portal API
- 4 Refinitiv Due Dilligence Centre
- Rose's Space
- 1.2K Screening
- 18 Qual-ID API
- 13 Screening Deployed
- 23 Screening Online
- 12 World-Check Customer Risk Screener
- 1K World-Check One
- 46 World-Check One Zero Footprint
- 45 Side by Side Integration API
- 2 Test Space
- 3 Thomson One Smart
- 10 TR Knowledge Graph
- 151 Transactions
- 143 REDI API
- 1.8K TREP APIs
- 4 CAT
- 26 DACS Station
- 121 Open DACS
- 1.1K RFA
- 104 UPA
- 192 TREP Infrastructure
- 228 TRKD
- 915 TRTH
- 5 Velocity Analytics
- 9 Wealth Management Web Services
- 89 Workspace SDK
- 11 Element Framework
- 5 Grid
- 18 World-Check Data File
- 1 Yield Book Analytics
- 46 中文论坛
