DSS REST API - Timeout during extraction request

grzegorz.zajac

Hi Team,

My client from Baader Bank AG is facing timeouts when utilizing Immediate extractions via DSS Rest API and would like to know if his request should be changed and/or can be optimized. Hopefully you can have a look and comment:

"

What the current flow of application is looking like:

1. We are collecting the array of instruments (ISINs and MICs) from our data, for which i need to resolve the prices.

2. I am querying datascope.refinitiv.com/RestApi/v1/Search/HistoricalBulkSearch API for bulk ric resolution. To do this, i am mapping MIC to MEC using fixed mapping table on my side. RIC resolution works like a charm, thank you for the hint, btw.

3. After we have a bundle of RICs, for which i need prices extracted, i am creating instrument list using datascope.refinitiv.com/RestApi/v1/Extractions/InstrumentLists

4. I am filling the created instrument list with collected instruments.

5. After all instruments are in the list, i am creating immediate extraction request. Here are the example request details:

curl --location 'https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractRaw' \

--header 'Prefer: respond-async' \

--header 'Content-Type: application/json' \

--header 'Authorization: Token _HTE3~~~~1v9U' \

--data-raw '{

"ExtractionRequest": {

"@odata.type":

"#DataScope.Select.Api.Extractions.ExtractionRequests.TickHistoryIntradaySummariesExtractionRequest",

"ContentFieldNames": [

"High Ask",

"High Bid",

"Low Ask",

"Low Bid",

"No. Asks",

"No. Bids",

"No. Trades"

],

"IdentifierList": {

"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentListIdentifierList",

"InstrumentListId": "0x08adcfb6f0f934d2"

},

"Condition": {

"MessageTimeStampIn": "GmtUtc",

"ReportDateRangeType": "Range",

"QueryStartDate": "2023-10-26T13:00:00.000-00:00",

"QueryEndDate": "2023-10-26T14:00:00.000-00:00",

"SummaryInterval": "OneMinute",

"TimebarPersistence": true,

"DisplaySourceRIC": true

}

}

}'

Here where the problem starts. Even if I'm using list of 2 instruments, and small query window, this request takes 60 - 100 seconds to get the response with extraction id. Sometimes it takes 120+ seconds. (But, if there are errors in request body, response is received immediately). Because of that, we often getting timeouts and need to retry.

Is this expected behavior?... Looks strange for async request. Maybe some recommendations/workarounds possible? "

Jirapongse

@grzegorz.zajac

Thank you for reaching out to us.

I think it is timeout in the application, not in Tick History REST API.

According to the information, it is an expected behavior.

As far as I know, the API doesn't guarantee that the extraction must be completed within the finite time. There are many factors that may impact extraction, such as current active requests, server load, number of items, number of fields, and number of extracted data.

After sending the request, the application must check the HTTP status of the returned response.

If the HTTP status of the returned resposne is 202, it means that the extraction is in-progress and the API will provide the polling URL in the response's header (Location). The client needs to use this URL to poll for the data.

If the HTTP status of the returned resposne is 200, it means that the extraction is completed and the data is available to download.

For more information, please refer to the REST API Tutorial 6: On Demand intraday bars extraction tutorial.

However, if the application sends a new extraction request while the previous extraction request is in-progress, the extraction requests may overwhelm the server and all extraction requests may be delayed.