DataScope Select Search API failing when maxpagesize variable is set

hamda.muse

Hi,

I've noticed when using the following payload and headers:

payload = json.dumps({
  "SearchRequest": {
    "ExchangeCodes": [
      "CMX"
    ],
    "IdentifierType": "RICRoot",
    "Identifier": "GC",
    "AssetStatus": "Active",
    "FuturesAndOptionsType": "FuturesOnOptions"
  }
})
headers = {
  'Prefer': 'respond-async;odata.maxpagesize=100000',
  'Content-Type': 'application/json'
}

url = "https://selectapi.datascope.refinitiv.com/RestApi/v1/Search/FuturesAndOptionsSearch"

requests.request("POST", url, headers=headers, data = payload, auth=HTTPBasicAuth(username, password))

The request times out with the following error:

 raise ProxyError(e, request=request)
requests.exceptions.ProxyError: HTTPSConnectionPool(host='selectapi.datascope.refinitiv.com', port=443): Max retries exceeded with url: /RestApi/v1/Search/FuturesAndOptionsSearch (Caused by ProxyError('Cannot connect to proxy.', RemoteDisconnected('Remote end closed connection without response')))

The request works after removing 'odata.maxpagesize=100000' from the 'Prefer' header, but this truncates the results for RIC roots with many underlying RICs.

Also, when changing the "FuturesAndOptionsType" to "Futures" for the same RIC root the request is completed successfully, even with 'odata.maxpagesize=100000' included in the header.

How do we get the full results for the payload above?

Gurpreet

Hello @hamda.muse,

You are specifying a very large page size due to which the request is timing out before it gets fulfilled. I would recommend that you specify a smaller page size and use the @odata.nextlink property in response message to iterate and get the remainder data.

For more information on how to use this server driven paging feature, please read this online help for DSS.