question

Upvotes
david.snowsill avatar image
28 ●8 ●8 ●14

Entity Search Results Limited to Maximum of 2000?

Dear Support,

I am trying to iterate through all the quotes on the ASX using the following query:

https://api.thomsonreuters.com/permid/search?access-token={token}&entityType=quote&q=mic:XASX&start=1&num=100

using batches of 100 quotes and incrementing the start value by 100 for each batch.

The results I am getting looks like:

{
    "result": {
        "quotes": {
            "entityType": "quotes",
            "total": 2000,
            "start": 1,
            "num": 100,
            "entities": [
                {
...

The issue is that the "total" field appears to be limited to "2000" quotes irrespective of how many quotes are on the exchange (in my case around 2400).

When I use the web interface, I can see the same limitation as well:

Am I missing something?
Regards,

David

apipermid-api
capture.png (14.0 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

david.snowsill avatar image david.snowsill

Hi Jirapongse,

This is a show stopper for us.
We need to make a decision whether we continue to utilise PermId in the next few weeks or move across to an alternative identifier.

Thanks,

David

Jirapongse avatar image Jirapongse david.snowsill

I have contacted the PermId team to verify the problem.

david.snowsill avatar image david.snowsill Jirapongse

Hi Jirapongse,

Have you had any feedback on this issue?

Thanks,

David

Show more comments

3 Answers

Upvotes
tsafi.tsur.1 avatar image
161 ●1 ●0 ●1

Hi David,

We do have limitation on the amount of search results, as the search purpose is to search for a specific entity by name or other id.

If you want to iterate over all of ours quotes, I recommend you download a a quote bulk file containing all of our quotes. We publish the files on a weekly basis. and provide Atom feed for consuming the incremental updates.

Notes

1.In order to reach the download page, you should login to the site.

2. We provide bulk files in two formats (TTL & NTRIPLES)

3. You can find additional info in our dev portal and user guide

Please Let me know whether additional info is needed.

Thanks

Tsafi

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
david.snowsill avatar image
28 ●8 ●8 ●14

Hi Tsafi,

Your API also allows searching by a single exchange (as per my example) which is what we are doing and why we took the approach that we have to use the API.

To rebuild everything using the Bulk files will involve us rewriting everything we have done so far. Can your limit be lifted?

Why is there a limit of the search API when your API forces a paging mechanism to download anyway?


Regards,
David

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

david.snowsill avatar image david.snowsill

Hi Tsafi,

Any thoughts on my questions?

Regards,

David

Upvotes
david.snowsill avatar image
28 ●8 ●8 ●14

Hi Tsafi,
As the Atom feed is currently broken (see other threads in forum) and unlikely to be fixed soon, would it be possible to increase the maximum number of entities from the Entity Search feed as a short term workaround until the Atom feed is fixed?
Regards,
david

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.