Entity Search Results Limited to Maximum of 2000?
Dear Support,
I am trying to iterate through all the quotes on the ASX using the following query:
https://api.thomsonreuters.com/permid/search?access-token={token}&entityType=quote&q=mic:XASX&start=1&num=100
using batches of 100 quotes and incrementing the start value by 100 for each batch.
The results I am getting looks like:
{
"result": {
"quotes": {
"entityType": "quotes",
"total": 2000,
"start": 1,
"num": 100,
"entities": [
{
...
The issue is that the "total" field appears to be limited to "2000" quotes irrespective of how many quotes are on the exchange (in my case around 2400).
When I use the web interface, I can see the same limitation as well:
Am I missing something?
Regards,
David
Hi Jirapongse,
This is a show stopper for us.
We need to make a decision whether we continue to utilise PermId in the next few weeks or move across to an alternative identifier.
Thanks,
David
Hi David,
We do have limitation on the amount of search results, as the search purpose is to search for a specific entity by name or other id.
If you want to iterate over all of ours quotes, I recommend you download a a quote bulk file containing all of our quotes. We publish the files on a weekly basis. and provide Atom feed for consuming the incremental updates.
Notes
1.In order to reach the download page, you should login to the site.
2. We provide bulk files in two formats (TTL & NTRIPLES)
3. You can find additional info in our dev portal and user guide
Please Let me know whether additional info is needed.
Thanks
Tsafi
Hi Tsafi,
Your API also allows searching by a single exchange (as per my example) which is what we are doing and why we took the approach that we have to use the API.
To rebuild everything using the Bulk files will involve us rewriting everything we have done so far. Can your limit be lifted?
Why is there a limit of the search API when your API forces a paging mechanism to download anyway?
Regards,
David
Hi Tsafi,
As the Atom feed is currently broken (see other threads in forum) and unlikely to be fixed soon, would it be possible to increase the maximum number of entities from the Entity Search feed as a short term workaround until the Atom feed is fixed?
Regards,
david