For a deeper look into our DataScope Select REST API, look into:

Overview |  Quickstart |  Documentation |  Downloads |  Tutorials

question

Upvotes
Accepted
Siarhei avatar image
1 ●0 ●1 ●4

How to get all items from result of search request

Hi,

Could you please help me to get a collection of all ValidatedInstrument are returned by search request.

My code:

SearchContext.Preferences.MaxPageSize = null;
EquitySearchRequest request = EquitySearchRequest.Create(
	null, //Sub types
	null, //Asset categories
	null, //Company name
	null, //Currency codes
	null, //Description
	null, //Domiciles codes
	null, //Fair value indicator
	new[] { "NYS" }, //Exchange codes
	null, //GICs codes
	null, //File codes
	null, //Org identifier
	null, //Asset status
	null, //Ticker
	IdentifierType.Ric, //Identifier types
	null, //Identifier
	IdentifierType.Ric);
IDssEnumerable<ValidatedInstrument> results = SearchContext.EquitySearch(request);
var list = new List<ValidatedInstrument>();
list.AddRange(results); // adds 250 items
while (results.HasNextPage)
{
	results = SearchContext.EquitySearch(results.NextLink); // throws exception
	list.AddRange(results);
}

Exception details:

ThomsonReuters.Dss.Api.ValidationException was unhandled by user code
  Context=NullObjectSpace
  HResult=-2146233088
  Message=Value cannot be null.
ParameterName=SearchRequest
  Method=POST
  RequestCorrelationId=251d15f9-9f45-4ea9-9e59-20846da21109
  RetryCount=0
  ServerAppVersion=11.2.662.64
  Source=ThomsonReuters.Dss.RestApi.Client
  StatusCode=400
  Url=https://hosted.datascopeapi.reuters.com/RestApi/v1/Search/EquitySearch?$skiptoken='????'
  StackTrace:
       at ThomsonReuters.Dss.Api.Core.HttpOData.Http.HttpClientExceptionDecorator.<ParseResponseError>d__1b.MoveNext() in d:\BuildAgent\work\70c2df2f37febbb5\src\ServiceLayer\ServiceLayer.Client\Core\HttpOData\Http\HttpClientExceptionDecorator.cs:line 284
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at ThomsonReuters.Dss.Api.Core.HttpOData.Http.HttpClientExceptionDecorator.<HandleExceptions>d__11.MoveNext() in d:\BuildAgent\work\70c2df2f37febbb5\src\ServiceLayer\ServiceLayer.Client\Core\HttpOData\Http\HttpClientExceptionDecorator.cs:line 169
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at ThomsonReuters.Dss.Api.Core.HttpOData.Http.HttpClientExceptionDecorator.<HandleExceptions>d__11.MoveNext() in d:\BuildAgent\work\70c2df2f37febbb5\src\ServiceLayer\ServiceLayer.Client\Core\HttpOData\Http\HttpClientExceptionDecorator.cs:line 201
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at ThomsonReuters.Dss.Api.Core.HttpOData.Http.HttpAsyncJobStrategy.<InvokeAsync>d__0.MoveNext() in d:\BuildAgent\work\70c2df2f37febbb5\src\ServiceLayer\ServiceLayer.Client\Core\HttpOData\Http\HttpAsyncJobStrategy.cs:line 33
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at ThomsonReuters.Dss.Api.Core.HttpOData.HttpODataClient.<InternalExecuteAsync>d__87.MoveNext() in d:\BuildAgent\work\70c2df2f37febbb5\src\ServiceLayer\ServiceLayer.Client\Core\HttpOData\HttpODataClient.cs:line 552
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at ThomsonReuters.Dss.Api.Core.HttpOData.HttpODataClient.<ExecuteAsync>d__7d`1.MoveNext() in d:\BuildAgent\work\70c2df2f37febbb5\src\ServiceLayer\ServiceLayer.Client\Core\HttpOData\HttpODataClient.cs:line 525
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at ThomsonReuters.Dss.Api.Core.OperationsBaseForHttpODataClient.<CallOnHttpODataClient>d__10b`1.MoveNext() in d:\BuildAgent\work\70c2df2f37febbb5\src\ServiceLayer\ServiceLayer.Client\Core\OperationsBaseForHttpODataClient.cs:line 0
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at ThomsonReuters.Dss.Api.Core.Helper.<ReAuthenticateIfNecessaryAsync>d__6`1.MoveNext() in d:\BuildAgent\work\70c2df2f37febbb5\src\ServiceLayer\ServiceLayer.Client\Core\Helper.cs:line 0
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at ThomsonReuters.Dss.Api.Core.OperationsBaseForHttpODataClient.<ExecuteAsync>d__a0`1.MoveNext() in d:\BuildAgent\work\70c2df2f37febbb5\src\ServiceLayer\ServiceLayer.Client\Core\OperationsBaseForHttpODataClient.cs:line 0
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at ThomsonReuters.Dss.Core.RestApi.Threading.Tasks.AsyncHelper.RunSync[TResult](Func`1 func) in d:\BuildAgent\work\70c2df2f37febbb5\src\Core\Threading\Tasks\AsyncHelper.cs:line 25
       at ThomsonReuters.Dss.Api.Core.OperationsBase.InvokeSynchronously[T](Func`1 action) in d:\BuildAgent\work\70c2df2f37febbb5\src\ServiceLayer\ServiceLayer.Client\Core\OperationsBase.cs:line 397
       at ThomsonReuters.Dss.Api.DssDataServiceContext.<>c__DisplayClassf`1.<Execute>b__e() in d:\BuildAgent\work\70c2df2f37febbb5\src\ServiceLayer\ServiceLayer.Client\Core\DssDataServiceContext.cs:line 278
       at ThomsonReuters.Dss.Api.Core.Helper.ReAuthenticateIfNecessary[T](IDssDataServiceContext context, Func`1 method) in d:\BuildAgent\work\70c2df2f37febbb5\src\ServiceLayer\ServiceLayer.Client\Core\Helper.cs:line 40
       at ThomsonReuters.Dss.Api.Search.SearchContext.EquitySearch(ResourceLink`1 nextOrDeltaLink) in d:\BuildAgent\work\70c2df2f37febbb5\src\Api\RestApi.Client\SearchClientApi.generated.cs:line 423

I'm not sure that my code is correct, because it wtitten in different style as, for example, to collect instrument lists:

internal static List<InstrumentList> GetInstrumentLists(this ExtractionsContext context)
{
	var result = new List<InstrumentList>();
	var list = context.InstrumentListOperations.GetAll(); // get the first page of instrument lists (default page size is 250)
	do
	{
		result.AddRange(list);
		list = (list.HasNextPage ? context.InstrumentListOperations.GetAll(list.NextLink) : null); // goto next page in loop to retrieve all instrument lists
	}
	while (list != null);
	return result;
}

because I can't find any *Operations method in the SearchContext.

RestAPI client v.11.2.662.0.

Thanks!

dss-rest-apidatascope-selectdssworld-checkc#search
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

2 Answers

Upvote
Accepted
warat.boonyanit avatar image
4.4k ●10 ●6 ●9

The pagination isn’t working correctly with the EquitySearch endpoint.

The current behavior is that if you want to request the next page, you also need to include the original request body in the post request, or else you’ll get back a 400 error. However, even if you send the search request in the body, then the next page will return the original 250 instruments + the next 250 instruments, giving you duplicates if you’re storing each page.

The temporary workaround for this is to increase the max page size such that all instruments are returned within a single page.

This issue should be fixed in v12.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Siarhei avatar image Siarhei

Thanks Warat! I already tested my code without use pagination and looks like it works fine when I set, for example:

SearchContext.Preferences.MaxPageSize = 100000;

I got over 87,000 instruments in some cases and it surpassed my expectations (50,000 returned in the DSS site GUI).

Christiaan Meihsl avatar image Christiaan Meihsl

Note: V12 was released on 23 April 2018.

Upvotes
veerapath.rungruengrayubkul avatar image
11.3k ●25 ●9 ●14

Hi @Siarhei,

I have found the same exception. It seems that the EquitySearchRequest is required for requesting next page. However, I cannot find any interfaces to pass both EquitySearchRequest and nextLink parameters too.

Please contact DSS support team via the Raise A Case,selecting product "Reuters DataScope Select".

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.