How to get FX Rates from DataScope Select Rest API in Python?
Hi,
I am following the Python example here but I am trying to get FX rates. It seems that the POST request is supposed to return a "location" url for me which I then use in a GET request to get the tick history. However, I am not getting this "location" value in my response headers.
I am also unsure about the values required in "Condition" to get historical minute by minute rates. Are there any Python examples for FX rates, for example, to get EUR= rate history each minute for the last month?
requestUrl='https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractRaw'
requestHeaders={
"Prefer":"respond-async",
"Content-Type":"application/json",
"Authorization": "token " + token
}
requestBody={
"ExtractionRequest": {
"@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.IntradayPricingExtractionRequest",
"ContentFieldNames": [
"Bid Price",
"Ask Price"
],
"IdentifierList": {
"@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
"InstrumentIdentifiers": [{
"Identifier": "EUR=",
"IdentifierType": "Ric"
},{
"Identifier": "GBP=",
"IdentifierType": "Ric"
}],
"UseUserPreferencesForValidationOptions":"false"
},
"Condition": {
}
}
}
r2 = requests.post(requestUrl, json=requestBody, headers=requestHeaders)
@mohit.kansal, thanks to your last comment I see we are not using the same end point, and that has an influence on what is returned. Let me explain the different possibilities, using end points Extract, ExtractWithNotes and ExtractRaw.
For all 3 POST calls that follow, the request body contains this:
{
"ExtractionRequest": {
"@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.IntradayPricingExtractionRequest",
"ContentFieldNames": [ "RIC", "Ask Price", "Bid Price", "Trade Date" ],
"IdentifierList": {
"@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
"InstrumentIdentifiers": [
{ "Identifier": "EUR=", "IdentifierType": "Ric" },
{ "Identifier": "GBP=", "IdentifierType": "Ric" }
]
},
"Condition": { "ScalableCurrency": true }
}
}
1) POST to end point Extract:
https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/Extract
1) Result is in JSON format:
{
"@odata.context": "https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#Collection(ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.ExtractionRow)",
"value": [
{
"IdentifierType": "Ric",
"Identifier": "EUR=",
"RIC": "EUR=",
"Ask Price": 1.1968,
"Bid Price": 1.1964,
"Trade Date": "2018-05-04"
},
{
"IdentifierType": "Ric",
"Identifier": "GBP=",
"RIC": "GBP=",
"Ask Price": 1.3568,
"Bid Price": 1.3565,
"Trade Date": "2018-05-04"
}
]
}
2) POST to end point ExtractWithNotes:
https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractWithNotes
2) Result is the same, with an added object (the extraction notes):
{
"@odata.context": "https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.ExtractionResult",
"Contents": [
{
"IdentifierType": "Ric",
"Identifier": "EUR=",
"RIC": "EUR=",
"Ask Price": 1.1971,
"Bid Price": 1.1967,
"Trade Date": "2018-05-04"
},
{
"IdentifierType": "Ric",
"Identifier": "GBP=",
"RIC": "GBP=",
"Ask Price": 1.3571,
"Bid Price": 1.3566,
"Trade Date": "2018-05-04"
}
],
"Notes": [
"Extraction Services Version 12.0.38730 (37673302648a), Built Apr 24 2018 19:25:15\r\nProcessing started at 04052018 02:07:34 PM.\r\nUser ID: 33314\r\nExtraction ID: 302678907\r\nSchedule: _OnD_0x0629f1fb29db2f96 (ID = 0x0629f1fb9edb2f96)\r\nInput List (2 items): _OnD_0x0629f1fb29db2f96 (ID = 0629f1fb481b2f96) Created: 04052018 02:07:30 PM Last Modified: 04052018 02:07:31 PM\r\nSchedule Time: 04052018 02:07:32 PM\r\nReport Template (10 fields): _OnD_0x0629f1fb29db2f96 (ID = 0x0629f1fb2ebb2f96) Created: 04052018 02:07:30 PM Last Modified: 04052018 02:07:30 PM\r\nNo prices needed currency scaling.\r\nReal-time data was snapped at 04052018 02:07:34 PM, it was scheduled to snap at 04052018 02:07:32 PM.\r\nNo embargo required for this report.\r\nProcessing completed successfully at 04052018 02:07:34 PM, taking 0.405 Secs.\r\nExtraction finished at 04052018 12:07:34 PM UTC, with servers: x08I11, QSHA02 (0.1 secs), QSHC17 (0.0 secs)\r\nUsage Summary for User 33314, Client 11122, Template Type Intraday Pricing\r\nBase Usage\r\n Instrument Instrument Terms Price\r\n Count Type Subtype Source Source\r\n------- ----------------------------------- ---------------------------- -------------- ----------------------------------------\r\n 2 Money Market N/A N/A\r\n-------\r\n 2 Total instruments charged.\r\n 0 Instruments with no reported data.\r\n=======\r\n 2 Instruments in the input list.\r\nNo TRPS complex usage to report -- 2 Instruments in the input list had no reported data.\r\nWriting RIC maintenance report.\r\n",
"Identifier,IdentType,Source,RIC,RecordDate,MaintType,OldValue,NewValue,Factor,FactorType\r\n"
]
}
3a) POST to end point ExtractRaw:
https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractRaw
3a) Result is different, and implies a different workflow:
{
"@odata.context": "https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#RawExtractionResults/$entity",
"JobId": "0x0629f211ff3b2f96",
"Notes": [
"Extraction Services Version 12.0.38730 (37673302648a), Built Apr 24 2018 19:25:15\r\nProcessing started at 04052018 02:09:06 PM.\r\nUser ID: 33314\r\nExtraction ID: 302678940\r\nSchedule: _OnD_0x0629f211ff3b2f96 (ID = 0x0629f212781b2f96)\r\nInput List (2 items): _OnD_0x0629f211ff3b2f96 (ID = 0629f21233db2f96) Created: 04052018 02:09:04 PM Last Modified: 04052018 02:09:05 PM\r\nSchedule Time: 04052018 02:09:05 PM\r\nReport Template (4 fields): _OnD_0x0629f211ff3b2f96 (ID = 0x0629f21211bb2f96) Created: 04052018 02:09:04 PM Last Modified: 04052018 02:09:04 PM\r\nNo prices needed currency scaling.\r\nReal-time data was snapped at 04052018 02:09:07 PM, it was scheduled to snap at 04052018 02:09:05 PM.\r\nNo embargo required for this report.\r\nProcessing completed successfully at 04052018 02:09:07 PM, taking 0.525 Secs.\r\nExtraction finished at 04052018 12:09:07 PM UTC, with servers: x08I17, QSHA02 (0.1 secs), QSHC18 (0.0 secs)\r\nUsage Summary for User 33314, Client 11122, Template Type Intraday Pricing\r\nBase Usage\r\n Instrument Instrument Terms Price\r\n Count Type Subtype Source Source\r\n------- ----------------------------------- ---------------------------- -------------- ----------------------------------------\r\n 2 Money Market N/A N/A\r\n-------\r\n 2 Total instruments charged.\r\n 0 Instruments with no reported data.\r\n=======\r\n 2 Instruments in the input list.\r\nNo TRPS complex usage to report -- 2 Instruments in the input list had no reported data.\r\nWriting RIC maintenance report.\r\n",
"Identifier,IdentType,Source,RIC,RecordDate,MaintType,OldValue,NewValue,Factor,FactorType\r\n"
]
}
The parameter JobId is required for the next call.
3b) GET call, using the JobId, to endpoint:
https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/RawExtractionResults('0x0629f211ff3b2f96')/$value
3b) Result is CSV formatted, and compressed (using gzip). CSV contents:
RIC,Ask Price,Bid Price,Trade Date EUR=,1.1973,1.1969,2018/05/04 GBP=,1.3572,1.3568,2018/05/04
As you can see, the end point you choose defines the data format, the presence or not of extraction notes (we recommend you use them), and the workflow. You can therefore choose what is best suited to your use case.
One additional point must be taken into account
In the above simple examples, there are only 2 instruments, the extraction is very quick and practically always completes in less than 30 seconds. All responses have HTTP status 200 OK. If you have more instruments, the extraction might take longer. In that case you will receive a 202 Accepted response (it means the extraction has not yet completed), with an empty body. Among the response headers there will be an item called Location, whose value is an URL. This URL must be polled using a GET, at a reasonable interval (like once a minute), until you receive a 200 OK. This mechanism is described in the Async key mechanisms page.
I hope this helps clarify.
@mohit.kansal, to come back to your original query which was for Python, please note that our sample "TRTH_OnDemand_IntradayBars" was updated on 23 May 2018 to better manage the Async delivery. You might find this of interest.
@mohit.kansal, on these 3 endpoints, please also note that we no longer recommend usage of the Extract endpoint, it might become obsolete in the future. This point has been added to the DSS FAQ.
From your query I am guessing you have the TRTH product (not DSS), but are attempting to use a DSS call. This is fairly common, as both products use the same servers.
An IntradyPricingExtractionRequest is a DSS call; if you have TRTH, you are not permissioned for this call, and it will return an error. This can easily be checked by looking at the HTTP status of the response. As I see your condition parameter is empty, that will also generate an error (and HTTP status 400). Can you tell us what HTTP status you receive ?
On the topic of the workflow of a query, and possible responses (like HTTP 202 and others), see TRTH REST API Tutorial 3 which describes it for On Demand requests.
For 1 minute values the easiest should be to use a Tick History Intraday Summary request. This is described in TRTH REST API Tutorial 6.
The easiest way to debug things is to first run API calls from Postman, which requires no programming, and displays the response body, headers, and HTTP status. Once you know that the calls deliver what you require, it is easy to program them in Python. Postman usage is described in TRTH REST API Tutorials Introduction (it contains a link to a collection of Postman calls that you can experiment with), and some additional tips are here.
And by the way the TRTH_OnDemand_IntradayBars Python sample available in the downloads should cover your requirement. It illustrates the conditions you need to set (just change the SummaryInterval to OneMinute if that is the interval you want).
I am using the TRTH_OnDemand_IntradayBars Python sample but it seems that FX instruments behave a bit differently from what I am seeing. The code above returns 200 status, but does not return a "location" url as mentioned in the sample.
If I do include parameters for "Condition" such as "ReportDateRangeType": "Range", I then receive a 400 status with a "Malformed Request Payload" error which says the property could not be parsed correctly even though they are exactly as written in the code sample.
Right now I am on a DSS trial which includes the Tick History product. Are there any examples for FX instruments in particular?
Thanks Christiaan.
Ok, so you have access to both DSS and TRTH, that clarifies things.
An IntradyPricingExtractionRequest is a DSS call which returns a single "real-time" (or delayed, depending on your account's permissions) snapshot of the price today. It will not deliver historical minute by minute rates. For more info on this call see the DSS REST API Tutorial 3. The parameters for "Condition" such as "ReportDateRangeType": "Range" are not appropriate for this API call, hence the 400 status. How to understand API calls is described here for DSS users (these tips are very similar to those I sent you for TRTH).
A 200 status means the request was accepted; in this case the data is in the body of the response, there is no location URL in the headers. The query workflow mechanisms and possible responses (like HTTP 202 and others) described in the TRTH tutorial I mentioned are also valid in DSS, so studying the TRTH REST API Tutorial 3 will help you understand it. This help page is also very useful in this context.
To illustrate I just ran an IntradyPricingExtractionRequest in Postman, here is the request:
{
"ExtractionRequest": {
"@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.IntradayPricingExtractionRequest",
"ContentFieldNames": [ "RIC", "Ask Price", "Bid Price", "Trade Date" ],
"IdentifierList": {
"@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
"InstrumentIdentifiers": [
{ "Identifier": "EUR=", "IdentifierType": "Ric" },
{ "Identifier": "GBP=", "IdentifierType": "Ric" }
]
},
"Condition": { "ScalableCurrency": true }
}
}
And the response (200 OK):
{
"@odata.context": "https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#Collection(ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.ExtractionRow)",
"value": [
{
"IdentifierType": "Ric",
"Identifier": "EUR=",
"RIC": "EUR=",
"Ask Price": 1.2096,
"Bid Price": 1.2093,
"Trade Date": "2018-04-30"
},
{
"IdentifierType": "Ric",
"Identifier": "GBP=",
"RIC": "GBP=",
"Ask Price": 1.377,
"Bid Price": 1.3765,
"Trade Date": "2018-04-30"
}
]
}
It delivers a snapshot of the latest price. If I run it again, I get different prices.
Considering your requirement description (historical intervalised prices), the original TRTH_OnDemand_IntradayBars Python sample available in the TRTH downloads should cover your needs.
@Christiaan Meihsl
Thanks Christiaan, that clarifies things a bit. However, I just ran the same request from Postman and have gotten a response which tells me that "2 Instruments in the input list had no reported data". Do you know why this might be?
{
"@odata.context": "https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#RawExtractionResults/$entity",
"JobId": "0x0629034157fb3026",
"Notes": [
"Extraction Services Version 12.0.38730 (37673302648a), Built Apr 24 2018 19:25:15\r\nProcessing started at 05/01/2018 11:35:31.\r\nUser ID: 9017723\r\nExtraction ID: 302254635\r\nSchedule: _OnD_0x0629034157fb3026 (ID = 0x06290342161b3026)\r\nInput List (2 items): _OnD_0x0629034157fb3026 (ID = 0629034187bb3026) Created: 05/01/2018 11:35:28 Last Modified: 05/01/2018 11:35:30\r\nSchedule Time: 05/01/2018 11:35:30\r\nReport Template (4 fields): _OnD_0x0629034157fb3026 (ID = 0x062903415bdb3026) Created: 05/01/2018 11:35:28 Last Modified: 05/01/2018 11:35:28\r\nNo prices needed currency scaling.\r\nReal-time data was snapped at 05/01/2018 11:35:31, it was scheduled to snap at 05/01/2018 11:35:30.\r\nNo embargo required for this report.\r\nProcessing completed successfully at 05/01/2018 11:35:31, taking 0.142 Secs.\r\nExtraction finished at 05/01/2018 14:35:31 UTC, with servers: x08I18, QSHA02 (0.0 secs), QSHC12 (0.0 secs)\r\nUsage Summary for User 9017723, Client 108123, Template Type Intraday Pricing\r\nBase Usage\r\n Instrument Instrument Terms Price\r\n Count Type Subtype Source Source\r\n------- ----------------------------------- ---------------------------- -------------- ----------------------------------------\r\n 2 Money Market N/A N/A\r\n-------\r\n 2 Total instruments charged.\r\n 0 Instruments with no reported data.\r\n=======\r\n 2 Instruments in the input list.\r\nNo TRPS complex usage to report -- 2 Instruments in the input list had no reported data.\r\n"
]
}
@mohit.kansal, re "I just ran the same request from Postman", you mean the one I posted in my 2nd answer ? To clear any doubt could you post the endpoint & request body you used ?
I ran my request now, received data for EUR= & GBP=. Extraction notes are similar to yours, including the end part:
2 Money Market N/A N/A\r\n-------\r\n 2 Total instruments charged.\r\n 0 Instruments with no reported data.\r\n=======\r\n 2 Instruments in the input list.\r\nNo TRPS complex usage to report -- 2 Instruments in the input list had no reported data.
I cannot explain why it says "instruments had no data".
@Christiaan Meihsl no, I didn't receive any data unfortunately
@Christiaan Meihsl
Got it. This has been super helpful. Thanks for being so thorough Christiaan- it's much appreciated.