Can extract only part of =ECI data even though entire data can be extracted via GUI of same TRTH ID

Hi Christiaan, how can I edit this to public?

Hi Christiaan,

Thank you for your support on this issue.

We tried REST API extraction several times and still unable to extract all available data rows for USCONC=ECI RIC.

We can still able to extract only several raws of historical records.

Can you please look at the following TRTH ID and two Job IDs and advise what is wrong?

TRTH ID: xxxxxxx

password: xxxxxxxx

JSON　（Specified Prefer: respond-async in Request header）

{"ExtractionRequest":{
  "@odata.type":"#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.TickHistoryTimeAndSalesExtractionRequest",
  "ContentFieldNames":[
    "Reference Change - Change Type","Reference Change - New Value","Reference Change - Old Value",
    "Trade - Actual","Trade - Forecast","Trade - Forecast High","Trade - Forecast Low",
    "Trade - Number of Forecasts","Trade - Prior","Trade - Qualifiers","Trade - Revised"
  ],
  "IdentifierList":{
    "@odata.type":"#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
    "InstrumentIdentifiers":[
      {"Identifier":"USCPF=ECI","IdentifierType":"Ric"},
      {"Identifier":"USCONC=ECI","IdentifierType":"Ric"}
    ],
    "ValidationOptions":{"AllowOpenAccessInstruments":true}
  },
  "Condition":{
    "MessageTimeStampIn":"LocalExchangeTime",
    "ApplyCorrectionsAndCancellations":"false",
    "ReportDateRangeType":"Range",
    "QueryStartDate":"2015-05-01T00:00:00.000Z",
    "QueryEndDate":"2018-06-01T00:00:00.000Z",
    "DisplaySourceRIC":"true"
  }
 }
}

Resopnse 201：Accepted

Responded JOBID => 0x063a8e835b1b2f86

https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/RawExtractionResults('0x063a8e835b1b2f86')/$value

responded data was as follows:

#RIC,Alias Underlying RIC,Domain,Date-Time,Type,Qualifiers,Actual,Prior,Revised,Forecast,Frcst High,Frcst Low,No. Frcts,Change Type,Old Value,New Value
USCONC=ECI,,Market Price,2015-05-12T14:00:57.840610750Z,Trade,Apr[CTBTR_1LL];[GV3_TEXT],,,,,,,,,,
USCONC=ECI,,Market Price,2015-05-12T14:00:59.163783520Z,Trade,May[CTBTR_1LL];[GV3_TEXT],,95.2,,,,,,,,
USCONC=ECI,,Market Price,2015-05-15T12:15:51.968994250Z,Trade,May[CTBTR_1LL];[GV3_TEXT],,,,97,98.1,95,5,,,
USCONC=ECI,,Market Price,2015-05-15T18:11:42.735467740Z,Trade,May[CTBTR_1LL];[GV3_TEXT],,,,95.2,,91,11,,,
USCONC=ECI,,Market Price,2015-05-18T09:54:05.613603120Z,Trade,May[CTBTR_1LL];[GV3_TEXT],,,,96.1,,,12,,,<br>

Even though we didnt specify Prefer: respond-async, extracted values were identical to above.

Job ID extraction: 0x063a8d06f81b2f96

Regards,

Hi Christiaan,

Thank you. Let me respond back to your questions/requests.

Please find the result of request as follows:

JSON Request:

{"ExtractionRequest":{
  "@odata.type":"#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.TickHistoryTimeAndSalesExtractionRequest",
  "ContentFieldNames":["Reference Change - Change Type","Reference Change - New Value","Reference Change - Old Value","Trade - Actual","Trade - Forecast","Trade - Forecast High","Trade - Forecast Low","Trade - Number of Forecasts","Trade - Prior","Trade - Qualifiers","Trade - Revised"],
  "IdentifierList":{
    "@odata.type":"#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
    "InstrumentIdentifiers":[
      {"Identifier":"USCPF=ECI","IdentifierType":"Ric"},
      {"Identifier":"USCONC=ECI","IdentifierType":"Ric"}
    ],
    "ValidationOptions":{"AllowOpenAccessInstruments":true}
  },
  "Condition":{
    "MessageTimeStampIn":"LocalExchangeTime",
    "ApplyCorrectionsAndCancellations":"false",
    "ReportDateRangeType":"Range",
    "QueryStartDate":"2015-05-01T00:00:00.000Z",
    "QueryEndDate":"2018-06-01T00:00:00.000Z",
    "DisplaySourceRIC":"true"
  }
}}

ClientSessionId and CorrelationId returned by above request.

ClientSessionId:f9a189ff-3054-4ea2-829c-f148914aa465

CorrelationId:dccd3c2c-9855-4d7b-aaf9-7d66b89d2546

JSON Response:

@{"@odata.context"=>"https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#RawExtractionResults/$entity",
"JobId"=>"0x063b2ee5ab2b2f86", "Notes"=>["Extraction
Services Version 12.0.39050 (262c8b8409ca), Built Jun 12 2018 20:14:54\nUser
ID: xxxxxxx\nExtraction ID: 2000000032748502\nSchedule: 0x063b2ee5ab2b2f86 (ID
= 0x0000000000000000)\nInput List (2 items):  (ID = 0x063b2ee5ab2b2f86)
Created: 06/27/2018 10:32:26 Last Modified: 06/27/2018 10:32:26\nReport
Template (11 fields): _OnD_0x063b2ee5ab2b2f86 (ID = 0x063b2ee5b00b2f86)
Created: 06/27/2018 10:28:47 Last Modified: 06/27/2018 10:28:47\nSchedule
dispatched via message queue (0x063b2ee5ab2b2f86), Data source identifier
(F5BBEEBBCD9045D4822338B255021ED8)\nSchedule Time: 06/27/2018
10:28:50\nProcessing started at 06/27/2018 10:28:50\nProcessing completed
successfully at 06/27/2018 10:32:26\nExtraction finished at 06/27/2018 01:32:26
UTC, with servers: tm14n03, TRTH (193.115 secs)\nInstrument
<RIC,USCPF=ECI> expanded to 1 RIC: USCPF=ECI.\nInstrument
<RIC,USCONC=ECI> expanded to 1 RIC: USCONC=ECI.\nTotal instruments after
instrument expansion = 2\nRange Query from 2015-04-30T15:00:00.000 to 2018-05-31T15:00:00.000
(UTC)\nQuota Message: INFO: Tick History Cash Quota Count Before Extraction:
47031; Instruments Approved for Extraction: 0; Tick History Cash Quota Count
After Extraction: 47031, 9406.2% of Limit; Tick History Cash Quota Limit:
500\nQuota Message: ERROR: The RIC 'USCONC=ECI' in the request would exceed
your quota limits. Adjust your input list to continue.\nQuota Message: ERROR:
The RIC 'USCPF=ECI' in the request would exceed your quota limits. Adjust your
input list to continue.\nQuota Message: WARNING: Tick History Cash Quota has
been reached or exceeded\nQuota Message: Note: Quota has exceeded, however, it
is not being enforced at this time but you can still make your extractions and
instruments are still being counted. Please contact your Account Manager for
questions. \nManifest: #RIC,Domain,Start,End,Status,Count\nManifest:
USCONC=ECI,Market
Price,2015-05-12T14:00:57.840610750Z,2018-05-29T15:00:46.257032016Z,Active,776\nManifest:
USCPF=ECI,Market Price,2015-05-08T12:11:11.160864350Z,2018-05-10T12:30:02.537586691Z,Active,709\n"]}

All data rows fetched by this JobID (we havent gotten any SessionId abd CorrelationId for this request)

#RIC Alias Underlying RIC Domain Date-Time Type Qualifiers Actual Prior Revised Forecast   Frcst High Frcst Low No. Frcts Change Type Old Value New Value
USCONC=ECI Market Price 2015-05-12T14:00:57.840610750Z Trade Apr[CTBTR_1LL];[GV3_TEXT]
USCONC=ECI Market Price 2015-05-12T14:00:59.163783520Z Trade May[CTBTR_1LL];[GV3_TEXT] 95.2
USCONC=ECI Market Price 2015-05-15T12:15:51.968994250Z Trade May[CTBTR_1LL];[GV3_TEXT] 97 98.1 95 5
USCONC=ECI Market Price 2015-05-15T18:11:42.735467740Z Trade May[CTBTR_1LL];[GV3_TEXT] 95.2 91 11
USCONC=ECI Market Price 2015-05-18T09:54:05.613603120Z Trade May[CTBTR_1LL];[GV3_TEXT] 96.1 12

1. Do we receive data for both instruments, or only one ? -> As you see above, we only got 5 rows as above for USCONC=ECI.
2. How many lines of data do you receive for each instrument ? -> Only 5 rows.
3. Please post the extraction notes received with the JobId.-> Pls see notes file as above. As you see, we receive an error that 'WARNING: Tick History Cash Quota has been reached or exceeded\nQuota Message: Note: Quota has exceeded, however, it is not being enforced at this time but you can still make your extractions and instruments are still being counted.', however this is an internal ID for test and I was told before that internal TRTH IDs will not be affected by Quota even though we receive an error in notes file. The message also says it is not being enforced at this time, and we can still extract full records of USCONC=ECI etc via TRTH GUI of the same user ID. Therefore I dont think this quota message would be an issue.

Thank you and regards,

Hiroko

Hi Christiaan,

Thank you.

Although our tech here was able to extract
full data of =ECI via Postman, there are some points we need to confirm.

We tried the followings:

Used Firefox: the first 669 bytes were
received (same as previous 5 rows)

Used Chrome: the first 669 bytes were retrieved
as CSV, and rests were in binary.

Used Edge: never completed downloads

It seems that a tool which is used .NET Framework
like PostMan can extract correctly compressed gunzip files, but we cannot
extract gunzpi files by other HTTP client.
Have you heard of any anomalies of TRTH gunzip files? It seems that only compress/extraction through
.NET Framework is successful.

Would you try the followings?

• 1)Please Return response data that can be expanded with major browsers
  like Chrome, Firefox, Edge

OR

2) If gzip is not
specified by Accept-Encoding, please return response data in plaintext
if possibile

Regards,