question

Upvotes
Accepted
rw74 avatar image
3 ●0 ●1 ●3

Retrieving Foreign Exchange Data (FX) using RDP library

Hi - Thanks in advance for any help with this.

I would like to download a subset of historical intraday Foreign Exchange (FXD5) data for all exchanges in this date range: (Aprils of 2010, '13, '16', '19).

Upon purchase, I was given this documentation,
Getting Started with Refinitiv Data Platform

This is the type of data I want to download,
PRICING AND MARKET DATA, Foreign Exchange


Data Dictionary:

Data Guide_Tick_Hist_Matching.pdf specifically the "Matching Prices with Volumes Daily (D5)" data.



I can successfully download this data, file by file, using the Refinitiv API PLAYGROUND but I cannot figure out how to download all files using the RDP Library.

Here's what I've tried,

  • Generated AppKey (API: EDP API)

  • Installed refinitiv-dataplatform package in my python3 distribution

  • Created Jupyter workbook, credentials.ipynb, that defines RDP credentials, i.e.,

RDP_LOGIN = 'myusername@xyz.edu'
RDP_PASSWORD = 'myPass'
APP_KEY = 'ad890…


  • Created another workbook to import RDP API Library Credential file
RDP_LOGIN = 'myusername@xyz.edu'
RDP_PASSWORD = 'myPass'
APP_KEY = 'ad890……'


  • Created RDP OpenPlatform Session to connect to RDP
session = rdp.open_platform_session(
    APP_KEY,
    rdp.GrantPassword(
        username = RDP_LOGIN,
        password = RDP_PASSWORD
    )
)

# check if session successfully opened
session.get_open_state()

—--return value shows successful creation of session —------
<State.Open: 3>


  • Define endpoint
#endpoint = rdp.Endpoint( session,
# "https://api.refinitiv.com/file-store/v1/file-sets
# ?bucket=TICKHISTORY_VBD_UNLIMITED&attributes=venue:FXD5,view:normalised
#     &start=2010-03-31T00:00:00Z&end=2010-05-01T00:00:00Z")

base_URL = 'https://api.refinitiv.com'
category_URL = '/file-store'
RDP_version = '/v1'
service_endpoint_URL = '/file-sets'

query_parms = {
    'bucket': 'TICKHISTORY_VBD_UNLIMITED',
    'attributes': {'venue': 'FXD5', 'view': 'normalised'},
    'contentFrom': '2010-04-01T00:00:00Z',
    'contentTo':'2010-04-30T00:00:00Z'
}

endpoint_url = base_URL + category_URL + RDP_version + service_endpoint_URL

print("endpoint_URL", endpoint_url)

endpoint = rdp.Endpoint(session, endpoint_url)

print("endpoint", endpoint)

response = endpoint.send_request(method = rdp.Endpoint.RequestMethod.GET, query_parameters = query_parms)

print("Response Status", response.status)
print("data:", response.data.raw)

—-------- Output —-------------
endpoint_URL https://api.refinitiv.com/file-store/v1/file-sets
endpoint <refinitiv.dataplatform.delivery.data.endpoint.Endpoint object at 0x7ff38f090c70>
Response Status {'http_status_code': 200, 'http_reason': 'OK'}
data {'value': []}


Results show successful session established and successful endpoint send_request but GET method returns no data.

Thanks again for any help with this!

rdp-apirefinitiv-data-platform-librariessessionendpoint
data-guide-tick-hist-matching.pdf (316.1 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

7 Answers

Upvotes
Accepted
nick.zincone avatar image
14k ●61 ●39 ●63

Hi @rw74

@zoya faberov and myself have been investigating the file_sets interface and believe it may be what you are after. For example:

response = cfs.file_sets.Definition(
    bucket = 'TICKHISTORY_VBD_UNLIMITED',
    attributes = {
        'venue': 'FXD5',
        'view': 'normalised'
    },
    content_from = "2010-04-01T00:00:00Z",
    content_to = "2010-04-30T00:00:00Z",    
    page_size=100
).get_data()

response.data.df

Give that a try and let us know if that is what you are looking for

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
nick.zincone avatar image
14k ●61 ●39 ●63

Hi @rw74

The Refinitiv Data Library for Python provides an example to retrieve bulk Tick History data. I would suggest you explore that example to see it is works with your workflow.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
rw74 avatar image
3 ●0 ●1 ●3

Thanks @nick.zincone

I was able to make some progress with the example you provided, bulk Tick History, however, this example does not provide enough information for my specific request. I've looked at

help(cfs.buckets.Definition)

for argument definitions but either I am looking in the wrong place or the documentation is incomplete.


In the example you referred me to, the get_data() call is:

response = cfs.buckets.Definition('history').get_data()


How would I incorporate these parameters into this call?

name = "TICKHISTORY_VBD_UNLIMITED"

attributes = [{venue:FXD5, view:normalised}]

contentFrom = "2010-04-01T00:00:00Z"

contentTo = "2010-04-30T00:00:00Z"
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
nick.zincone avatar image
14k ●61 ●39 ●63

Hi @rw74

Can you confirm the help you received from the following command you specified above?

help(cfs.buckets.Definition)

When I performed that command, I did receive some relevant output that would provide the necessary properties to set the values you are interested. That is:

ahs.png

The above output is from version: 

1.0.0b10

I've followed up with the development group to provide an example Definition syntax to support your requirements. I will post the details here once I get word.

thanks.


ahs.png (60.1 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
rw74 avatar image
3 ●0 ●1 ●3

Hi @nick.zincone - Thanks for following up with the syntax on this. I spent quite a lot of time with help(cfs.buckets.Definition) trying many permutations of the parameter values and syntax. E.g. for attributes it says:

 attributes : list of str, optional
 |          List of publisher-defined bucket attributes.

What exactly are the publisher-defined bucket attribute strings?

thanks!

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
zoya faberov avatar image
32.2k ●40 ●11 ●19

Hello @rw74 ,

I fully agree with @nick.zincone that RD library makes CFS -TickHistory retrieval much easier and less effort.

I have run a quick sanity check with RDP Postman starter example

Authentication-> Get Authentication Token

CFS -> CFS Tick History->Get TH File Sets for <VENUE> normalised+by Dates,

entering FXD5 for Venue and with your dates:

https://{
                {RDP_HOST}}/file-store/{
                {RDP_VERSION}}/file-sets?bucket={
                {RDP_TH_Bucket}}&attributes=venue:{
                {CFS_VBD_VENUE}},view:normalised&contentFrom=2010-04-01T00:00:00Z&contentTo=2010-04-30T00:00:00Z

I see the results for these dates- FileSets are returned.

If you are able to request with your creds with Postman, another way to check quickly, you could try with Python code included with the article Introduction to Client File Store (CFS) - Python , see Example - Retrieve Tick History File Sets for a Given Venue Limit By Dates

This could help understand where the issue is on your side, having something working and then moving to RD Library integration could also be considered.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
rw74 avatar image
3 ●0 ●1 ●3

@nick.zincone @zoya faberov

This worked for me!

@nick.zincone - The link you posted in your first reply, i.e., bulk Tick History, just needed the above tweak to get the file_sets I needed. The code to download the files was exactly as given in the bulk Tick History example.

Thank you both!


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.