Retrieving Foreign Exchange Data (FX) using RDP library
Hi - Thanks in advance for any help with this.
I would like to download a subset of historical intraday Foreign Exchange (FXD5) data for all exchanges in this date range: (Aprils of 2010, '13, '16', '19).
Upon purchase, I was given this documentation,
Getting Started with Refinitiv Data Platform
This is the type of data I want to download,
PRICING AND MARKET DATA, Foreign Exchange
Data Dictionary:
Data Guide_Tick_Hist_Matching.pdf specifically the "Matching Prices with Volumes Daily (D5)" data.
I can successfully download this data, file by file, using the Refinitiv API PLAYGROUND but I cannot figure out how to download all files using the RDP Library.
Here's what I've tried,
Generated AppKey (API: EDP API)
Installed refinitiv-dataplatform package in my python3 distribution
Created Jupyter workbook, credentials.ipynb, that defines RDP credentials, i.e.,
RDP_LOGIN = 'myusername@xyz.edu' RDP_PASSWORD = 'myPass' APP_KEY = 'ad890…
- Created another workbook to import RDP API Library Credential file
RDP_LOGIN = 'myusername@xyz.edu' RDP_PASSWORD = 'myPass' APP_KEY = 'ad890……'
- Created RDP OpenPlatform Session to connect to RDP
session = rdp.open_platform_session(
APP_KEY,
rdp.GrantPassword(
username = RDP_LOGIN,
password = RDP_PASSWORD
)
)
# check if session successfully opened
session.get_open_state()
—--return value shows successful creation of session —------
<State.Open: 3>
- Define endpoint
#endpoint = rdp.Endpoint( session,
# "https://api.refinitiv.com/file-store/v1/file-sets
# ?bucket=TICKHISTORY_VBD_UNLIMITED&attributes=venue:FXD5,view:normalised
# &start=2010-03-31T00:00:00Z&end=2010-05-01T00:00:00Z")
base_URL = 'https://api.refinitiv.com'
category_URL = '/file-store'
RDP_version = '/v1'
service_endpoint_URL = '/file-sets'
query_parms = {
'bucket': 'TICKHISTORY_VBD_UNLIMITED',
'attributes': {'venue': 'FXD5', 'view': 'normalised'},
'contentFrom': '2010-04-01T00:00:00Z',
'contentTo':'2010-04-30T00:00:00Z'
}
endpoint_url = base_URL + category_URL + RDP_version + service_endpoint_URL
print("endpoint_URL", endpoint_url)
endpoint = rdp.Endpoint(session, endpoint_url)
print("endpoint", endpoint)
response = endpoint.send_request(method = rdp.Endpoint.RequestMethod.GET, query_parameters = query_parms)
print("Response Status", response.status)
print("data:", response.data.raw)
—-------- Output —-------------
endpoint_URL https://api.refinitiv.com/file-store/v1/file-sets
endpoint <refinitiv.dataplatform.delivery.data.endpoint.Endpoint object at 0x7ff38f090c70>
Response Status {'http_status_code': 200, 'http_reason': 'OK'}
data {'value': []}
Results show successful session established and successful endpoint send_request but GET method returns no data.
Thanks again for any help with this!
Hi @rw74
@zoya faberov and myself have been investigating the file_sets interface and believe it may be what you are after. For example:
response = cfs.file_sets.Definition(
bucket = 'TICKHISTORY_VBD_UNLIMITED',
attributes = {
'venue': 'FXD5',
'view': 'normalised'
},
content_from = "2010-04-01T00:00:00Z",
content_to = "2010-04-30T00:00:00Z",
page_size=100
).get_data()
response.data.df
Give that a try and let us know if that is what you are looking for
Hi @rw74
The Refinitiv Data Library for Python provides an example to retrieve bulk Tick History data. I would suggest you explore that example to see it is works with your workflow.
Thanks @nick.zincone
I was able to make some progress with the example you provided, bulk Tick History, however, this example does not provide enough information for my specific request. I've looked at
help(cfs.buckets.Definition)
for argument definitions but either I am looking in the wrong place or the documentation is incomplete.
In the example you referred me to, the get_data() call is:
response = cfs.buckets.Definition('history').get_data()
How would I incorporate these parameters into this call?
name = "TICKHISTORY_VBD_UNLIMITED"
attributes = [{venue:FXD5, view:normalised}]
contentFrom = "2010-04-01T00:00:00Z"
contentTo = "2010-04-30T00:00:00Z"
Hi @rw74
Can you confirm the help you received from the following command you specified above?
help(cfs.buckets.Definition)
When I performed that command, I did receive some relevant output that would provide the necessary properties to set the values you are interested. That is:
The above output is from version:
1.0.0b10
I've followed up with the development group to provide an example Definition syntax to support your requirements. I will post the details here once I get word.
thanks.
Hi @nick.zincone - Thanks for following up with the syntax on this. I spent quite a lot of time with help(cfs.buckets.Definition) trying many permutations of the parameter values and syntax. E.g. for attributes it says:
attributes : list of str, optional | List of publisher-defined bucket attributes.
What exactly are the publisher-defined bucket attribute strings?
thanks!
Hello @rw74 ,
I fully agree with @nick.zincone that RD library makes CFS -TickHistory retrieval much easier and less effort.
I have run a quick sanity check with RDP Postman starter example
Authentication-> Get Authentication Token
CFS -> CFS Tick History->Get TH File Sets for <VENUE> normalised+by Dates,
entering FXD5 for Venue and with your dates:
https://{
{RDP_HOST}}/file-store/{
{RDP_VERSION}}/file-sets?bucket={
{RDP_TH_Bucket}}&attributes=venue:{
{CFS_VBD_VENUE}},view:normalised&contentFrom=2010-04-01T00:00:00Z&contentTo=2010-04-30T00:00:00Z
I see the results for these dates- FileSets are returned.
If you are able to request with your creds with Postman, another way to check quickly, you could try with Python code included with the article Introduction to Client File Store (CFS) - Python , see Example - Retrieve Tick History File Sets for a Given Venue Limit By Dates
This could help understand where the issue is on your side, having something working and then moving to RD Library integration could also be considered.
This worked for me!
@nick.zincone - The link you posted in your first reply, i.e., bulk Tick History, just needed the above tweak to get the file_sets I needed. The code to download the files was exactly as given in the bulk Tick History example.
Thank you both!