Pagination

Elif

I am using LSEG library in Python.

I was able to identify first 10,000 observation of any data set I want to obtain (e.g. VC investments, M&A deals). However, I cannot do pagination meaning going through data coming after 10,000th row. 

1. I tried to use skip + top, it gives skip + top cannot exceed 10,000 error. 
2. I tried filtering by using PermID:

 

“permid_filter = f"(OAPermID ge {start_permid} and OAPermID lt {start_permid + chunk_size})"  But, it was unable to fetch any data.

 How can I view and get entries after the top 10,000 observation from LSEG database? Could you help me with that or put me in a contact with someone who can help?

Jirapongse

@Elif

Thank you for reaching out to us.

I assume that you are using the Search API.

ld.discovery.search

The purpose of the Search API is searching for required entries. The number of entries in the response per request is limited to 10,000 entries and the (top + skip) value must not exceed 10,000.

You can use the Navigators to categorize and summarize the results and then use the results from the Navigators to customize the filter. For more information regarding the Navigators, please refer to the Building Search into your Application Workflow article.

To dump the entire database, you may need to contact your LSEG account team or sales team directly to find the products that suit your requirements.