question

Upvotes
Accepted
1 0 1 1

DSS REST API - Timeout during extraction request

Hi Team,

My client from Baader Bank AG is facing timeouts when utilizing Immediate extractions via DSS Rest API and would like to know if his request should be changed and/or can be optimized. Hopefully you can have a look and comment:

"

What the current flow of application is looking like:

1. We are collecting the array of instruments (ISINs and MICs) from our data, for which i need to resolve the prices.

2. I am querying datascope.refinitiv.com/RestApi/v1/Search/HistoricalBulkSearch API for bulk ric resolution. To do this, i am mapping MIC to MEC using fixed mapping table on my side. RIC resolution works like a charm, thank you for the hint, btw.

3. After we have a bundle of RICs, for which i need prices extracted, i am creating instrument list using datascope.refinitiv.com/RestApi/v1/Extractions/InstrumentLists

4. I am filling the created instrument list with collected instruments.

5. After all instruments are in the list, i am creating immediate extraction request. Here are the example request details:

curl --location 'https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractRaw' \

--header 'Prefer: respond-async' \

--header 'Content-Type: application/json' \

--header 'Authorization: Token _HTE3~~~~1v9U' \

--data-raw '{

"ExtractionRequest": {

"@odata.type":

"#DataScope.Select.Api.Extractions.ExtractionRequests.TickHistoryIntradaySummariesExtractionRequest",

"ContentFieldNames": [

"High Ask",

"High Bid",

"Low Ask",

"Low Bid",

"No. Asks",

"No. Bids",

"No. Trades"

],

"IdentifierList": {

"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentListIdentifierList",

"InstrumentListId": "0x08adcfb6f0f934d2"

},

"Condition": {

"MessageTimeStampIn": "GmtUtc",

"ReportDateRangeType": "Range",

"QueryStartDate": "2023-10-26T13:00:00.000-00:00",

"QueryEndDate": "2023-10-26T14:00:00.000-00:00",

"SummaryInterval": "OneMinute",

"TimebarPersistence": true,

"DisplaySourceRIC": true

}

}

}'

Here where the problem starts. Even if I'm using list of 2 instruments, and small query window, this request takes 60 - 100 seconds to get the response with extraction id. Sometimes it takes 120+ seconds. (But, if there are errors in request body, response is received immediately). Because of that, we often getting timeouts and need to retry.

Is this expected behavior?... Looks strange for async request. Maybe some recommendations/workarounds possible? "

#technologydss-rest-api#producton-demand-extraction
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

@grzegorz.zajac

Hi,

Thank you for your participation in the forum.

Is the reply below satisfactory in answering your question?

If yes please click the 'Accept' text next to the reply. This will guide all community members who have a similar question.

Otherwise please post again offering further insight into your question.

Thanks,

AHS

@grzegorz.zajac

Hi,

Please be informed that a reply has been verified as correct in answering the question, and marked as such.

Thanks,

AHS

1 Answer

· Write an Answer
Upvote
Accepted
79.2k 251 52 74

@grzegorz.zajac

Thank you for reaching out to us.

I think it is timeout in the application, not in Tick History REST API.

According to the information, it is an expected behavior.

As far as I know, the API doesn't guarantee that the extraction must be completed within the finite time. There are many factors that may impact extraction, such as current active requests, server load, number of items, number of fields, and number of extracted data.

After sending the request, the application must check the HTTP status of the returned response.

If the HTTP status of the returned resposne is 202, it means that the extraction is in-progress and the API will provide the polling URL in the response's header (Location). The client needs to use this URL to poll for the data.

If the HTTP status of the returned resposne is 200, it means that the extraction is completed and the data is available to download.

For more information, please refer to the REST API Tutorial 6: On Demand intraday bars extraction tutorial.

However, if the application sends a new extraction request while the previous extraction request is in-progress, the extraction requests may overwhelm the server and all extraction requests may be delayed.


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.