Entity Search Results Limited to Maximum of 2000?

Dear Support,

I am trying to iterate through all the quotes on the ASX using the following query:

https://api.thomsonreuters.com/permid/search?access-token={token}&entityType=quote&q=mic:XASX&start=1&num=100

using batches of 100 quotes and incrementing the start value by 100 for each batch.

The results I am getting looks like:

{
    "result": {
        "quotes": {
            "entityType": "quotes",
            "total": 2000,
            "start": 1,
            "num": 100,
            "entities": [
                {
...

The issue is that the "total" field appears to be limited to "2000" quotes irrespective of how many quotes are on the exchange (in my case around 2400).

When I use the web interface, I can see the same limitation as well:

Am I missing something?
Regards,

David

Component react.asset.discussionTagsAsset had an error.

Resolver not registered: react.asset.discussionTagsAsset

Accepted answers

RaphaelS

As the bulk files have been dropped by now, are there any plans to lift/eliminate the 2k limit? I guess then the API will be truly open and not quasi closed as it is right now.

All comments

david.snowsill

Hi Jirapongse,

This is a show stopper for us.
We need to make a decision whether we continue to utilise PermId in the next few weeks or move across to an alternative identifier.

Thanks,

David

Jirapongse

I have contacted the PermId team to verify the problem.

david.snowsill

Hi Jirapongse,

Have you had any feedback on this issue?

Thanks,

David

Jirapongse

@david.snowsill

They are looking into the issue.

tsafi.tsur.1

Hi David,

We do have limitation on the amount of search results, as the search purpose is to search for a specific entity by name or other id.

If you want to iterate over all of ours quotes, I recommend you download a a quote bulk file containing all of our quotes. We publish the files on a weekly basis. and provide Atom feed for consuming the incremental updates.

Notes

1.In order to reach the download page, you should login to the site.

2. We provide bulk files in two formats (TTL & NTRIPLES)

3. You can find additional info in our dev portal and user guide

Please Let me know whether additional info is needed.

Thanks

Tsafi

david.snowsill

Hi Tsafi,

Your API also allows searching by a single exchange (as per my example) which is what we are doing and why we took the approach that we have to use the API.

To rebuild everything using the Bulk files will involve us rewriting everything we have done so far. Can your limit be lifted?

Why is there a limit of the search API when your API forces a paging mechanism to download anyway?

Regards,
David

david.snowsill

Hi Tsafi,

Any thoughts on my questions?

Regards,

David

david.snowsill

Hi Tsafi,
As the Atom feed is currently broken (see other threads in forum) and unlikely to be fixed soon, would it be possible to increase the maximum number of entities from the Entity Search feed as a short term workaround until the Atom feed is fixed?
Regards,
david

RaphaelS

As the bulk files have been dropped by now, are there any plans to lift/eliminate the 2k limit? I guess then the API will be truly open and not quasi closed as it is right now.

Jirapongse

According to the response:

We do have limitation on the amount of search results, as the search purpose is to search for a specific entity by name or other id.

We don't have a plan to lift this limit.

EXPLORE OUR SITES