Hi, how can I get RAW-TAS and RAW-MD data in trth v2
Our old code can extract this from trth v1. How to get this in v2.
Example of RAW-TAS:
#RIC,Date[G],Time[G],GMT Offset,Type,MSG/FID Number,Message Type,FID Name,FID Value,FID Enum String,PE Code,Template Number,RTL,Sequence Number,Source RIC,Number of FIDs
VOD.L,22-AUG-2017,02:55:20.377,+1,Raw-TAS, 84,CORRECTION_PC,,,,5625,0,35296,17201,VOD.L,112
Example of RAW-MD
#RIC,Date[G],Time[G],GMT Offset,Type,MSG/FID Number,Message Type,FID Name,FID Value,FID Enum String,PE Code,Template Number,RTL,Sequence Number,Source RIC,Number of FIDs
VOD.L,15-AUG-2017,02:55:00.555,+1,Raw-MD , 84,CORRECTION_PC,,,,249,0,48432,36772,VOD.LO,98
These two TRTH v1 types have another name in TRTH v2.
To request Raw-TAS using On-demand extraction, you have to use TickHistoryRawExtractionRequest and then set Domain code to "MarketPrice".
POST https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractRaw
Prefer: respond-async
Content-Type: application/json
Authorization: Token <Your Token>
{
"ExtractionRequest": {
"@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.TickHistoryRawExtractionRequest",
"IdentifierList": {
"@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
"InstrumentIdentifiers": [{
"Identifier": "TRI.N",
"IdentifierType": "Ric"
}]
},
"Condition": {
"MessageTimeStampIn": "GmtUtc",
"ReportDateRangeType": "Range",
"QueryStartDate": "2016-09-29T12:00:00.000Z",
"QueryEndDate": "2016-09-29T12:10:00.000Z",
"ExtractBy": "Ric",
"SortBy": "SingleByRic",
"DomainCode": "MarketPrice",
"DisplaySourceRIC": true
}
}
}
To request Raw-MD using On-demand extraction, you have to use TickHistoryMarketDepthExtractionRequest and then set view to "LegacyLevel2"
POST https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractRaw
Prefer: respond-async
Content-Type: application/json
Authorization: Token <Your Token>
{
"ExtractionRequest": {
"@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.TickHistoryMarketDepthExtractionRequest",
"IdentifierList": {
"@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
"InstrumentIdentifiers": [{
"Identifier": "TRI.N",
"IdentifierType": "Ric"
}]
},
"Condition": {
"View": "LegacyLevel2",
"MessageTimeStampIn": "GmtUtc",
"ReportDateRangeType": "Range",
"QueryStartDate": "2016-09-29T00:00:00.000Z",
"QueryEndDate": "2016-09-29T12:00:00.000Z",
"DisplaySourceRIC": true
}
}
}
ok thanks @Warat B
btw why don't I get SourceRIC in TickHistoryRawExtractionRequest even if I set "DisplaySourceRIC": true
@muneish.adya, you get it, but only as the 1st element of the 1st line of an update:
CARR.PA,Market Price,2016-09-29T12:00:00.942730204Z,Raw,UPDATE,UNSPECIFIED,,,,8132,,2800,13
The RIC is not included in the next lines (those with the individual updates by FID), because they are part of the same update:
,,,FID,4981,,TRTN,0.89, ,,,FID,4982,,TRTN_3MT,5.85, etc.
In other words, an update has a kind of "header" line that includes the RIC, timestamp, and number of changed FIDs, followed by all the individual changes, one line by FID.
In V1 the RIC was in all lines. V2 removed it to avoid duplication.
Beg to disagree , if I do a TickHistoryRawExtractionRequest with displaySourceRic=true, I never get a column called SourceRIC in the output. Also, in TimeAndSales request is there a view which can give me the raw data like in MarketDepth?
@muneish.adya, maybe we are not talking about the same thing ?
I am referring to what is described in REST API Tutorial 8, i.e. a TickHistoryRawExtractionRequest. If you run the same query you should see the RIC as the first column of the first line of an update, like in this case where the RIC is CARR.PA:
CARR.PA,Market Price,2016-09-29T12:00:00.950557616Z,Raw,UPDATE,UNSPECIFIED,,,,8132,,2800,13
,,,FID,4981,,TRTN,0.89,
,,,FID,4982,,TRTN_3MT,5.85,I retested it now to be sure.
That said, for a TickHistoryRawExtractionRequest the value of DisplaySourceRIC has no influence on the output.
So can I assume that I can add a SourceRIC column from my self and simply put the value from RIC column?
in version 1, in RAW-TAS, we get SourceRIC column from trth, so how come its missing in version 2.
@muneish.adya, the output of call TickHistoryRawExtractionRequest is predefined, i.e. TRTH output fields are not configurable.
Obviously, once you have retrieved the result from the server, you can programmatically add anything you want to it (like a SourceRIC column), but this treatment is done on your own machine independently from TRTH. And yes, you can use the RIC column for that.
In V1 the RIC was in all lines, but repeating the RIC in every single line is not very efficient in terms of processing and storage. V2 removed it to avoid duplication.
Please read my question carefully. My question is NOT why RIC is not present on all lines. My question is WHY SOURCERIC IS MISSING COMPLETELY in v2 RAW data? Is there a way to get it in v2 RAW data? What is a safe replacement for the column SOURCERIC if reuters will not provide it on v2.
Unfortunately all your replies so far have been about why RIC value not present on all rows. I get the sense behind it and thanks for the extra information. The question I have is different though.
The reason I am asking is because we display Source ric in our final trade and quote tables that we prepare from the raw data. Some downstream systems "might" have a dependency on it. Now that its missing in v2 apparently whats the solution?
Sample data from v1 just to clear things:
Value,FID Enum String,PE Code,Template Number,RTL,Sequence Number,Source RIC,Number of FIDs
VOD.L,22-AUG-2017,02:55:20.377,+1,Raw-TAS, 84,CORRECTION_PC,,,,5625,0,35296,17201,VOD.L,112
@muneish.adya, I submitted a query to the data support on the 8th, and will keep you informed when they answer. That said, the output of this query has a fixed format, there is no way of configuring it to add the field you want.
@muneish.adya, to your query: why don't I get SourceRIC in TickHistoryRawExtractionRequest even if I set "DisplaySourceRIC": true
This was submitted to the data support team. Here is their answer:
In certain cases the RIC of the item being displayed can be different from the requested RIC; this is the case for RIC aliases. It is by design that TRTH v2 replaces the TRTH v1 “Source RIC” column with other similar columns when "display Source RIC" is set to true.
This column is an optional field, it will be delivered only if:
- "DisplaySourceRIC": true
- and the requested RIC is an alias RIC.
Various scenarios:
1: The request is for raw Market Price data of a normal RIC
<RIC> is always equal to 'Source RIC’, therefore the ‘Source RIC’ is not delivered.
2: The request is for raw Market Price data of an alias RIC
<RIC> is not equal to ‘Source RIC’, therefore the ‘Alias Underlying RIC’ is delivered (instead of ‘Source RIC’).
3: The request is for raw Market Depth data of a normal RIC
<RIC> may or may not be equal to ‘Source RIC’, therefore the ‘Market Depth RIC’ is delivered (instead of ‘Source RIC’).
4: The request is for raw Market Depth data of an alias RIC
<RIC> is not equal to ‘Source RIC’, therefore the‘Alias Underlying RIC’ and the ‘Market Depth RIC’ are delivered (instead of ‘Source RIC’).
In other words, TRTH v2 provides more explicit information of ‘Source RIC’ than TRTH v1.
I tried this out. Here is an example for use case 3. Request:
{
"ExtractionRequest": {
"@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.TickHistoryMarketDepthExtractionRequest",
"IdentifierList": {
"@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
"InstrumentIdentifiers": [{
"Identifier": "CARR.PA",
"IdentifierType": "Ric"
}]
},
"Condition": {
"View": "LegacyLevel2",
"MessageTimeStampIn": "GmtUtc",
"ReportDateRangeType": "Range",
"QueryStartDate": "2016-09-29T12:00:00.000Z",
"QueryEndDate": "2016-09-29T12:10:00.000Z",
"DisplaySourceRIC": true
}
}
}
Result, containing the 'Market Depth RIC' (0CARR.PA):
#RIC,Domain,Date-Time,Type,MsgClass/FID number,UpdateType/Action,FID Name,FID Value,FID Enum String,PE Code,Template Number,Key/Msg Sequence Number,Market Depth RIC,Number of FIDs CARR.PA,Market Price,2016-09-29T12:00:02.512691123Z,Legacy Level 2,UPDATE,UNSPECIFIED,,,,6508,,31440,0CARR.PA,3 ,,,FID,17,,ACTIV_DATE,2016-09-29, ,,,FID,5,,TIMACT,12:00:00.000000000, ,,,FID,4148,,TIMACT_MS,43202452, CARR.PA,Market Price,2016-09-29T12:00:03.172723827Z,Legacy Level 2,UPDATE,UNSPECIFIED,,,,6508,,31456,0CARR.PA,6
Note: this request was for a LegacyLevel2 view. For this particular instrument the Market Depth RIC is not returned if you request a RawMarketByPrice or RawMarketByOrder view.
Conclusion:
If the source RIC is the same as the RIC it will not be delivered, you can use the RIC field. If the source RIC is different from the RIC then it will be delivered, in field(s) ‘Alias Underlying RIC’ and/or ‘Market Depth RIC’.