Error while attempting to download bulk news data

Xim_O

I have a python script to down load news data using this function

response = news.headlines.Definition(ric, date_from=start, date_to=end, count=999999).get_data()

I am attempting to down load bulk data for a list of 600 RICs by passing one RIC at a time into the function above. About 20 RICS in I get the following error.

What is the cause of this and how do I solve for it?

---------------------------------------------------------------------------LDError Traceback (most recent call last)Cell In[3], line 29 26 pbar2.set_postfix({"status": f"querying {year}"}) 27 pbar2.refresh()---> 29 response = news.headlines.Definition(ric, date_from=start, date_to=end, count=999999).get_data() 31 count = 0 33 # Handle both response formatsFile c:\Users\mwu\miniconda3\envs\prodigal_env\Lib\site-packages\lseg\data\content\news\headlines\_definition.py:104, in Definition.get_data(self, session, on_page_response) 72 """ 73 Returns a response from the API to the library 74 (...) 101 >>> response = definition.get_data() 102 """ 103 self._kwargs["on_page_response"] = on_page_response--> 104 return super().get_data(session)File c:\Users\mwu\miniconda3\envs\prodigal_env\Lib\site-packages\lseg\data\content\news\_news_data_provider_layer.py:13, in NewsDataProviderLayer.get_data(self, session) 11 def get_data(self, session: Optional["Session"] = None): 12 session = get_valid_session(session)---> 13 response = super().get_data(session) 14 return response... 29 error = LDError(code=error_code, message=error_message) 31 error.response = response---> 32 raise errorLDError: Error code 429 | Too many requests, please try again later.

Jirapongse

@Xim_O

Thank you for reaching out to us.

The 429 error indicates that the application exceeds the API limitations.

This news endpoint is typically not intended for bulk downloads. For alternative solutions to support this use case, please contact your LSEG account team or sales representative directly.

wasin.w V3

Hello @Xim_O

Please refer to the Usage and Limits Guideline document.

Xim_O

https://community.developers.refinitiv.com/discussion/comment/127688#Comment_127688

Is there a way to workout how close on the day you are to the data limit?

wasin.w V3

Hello @Xim_O

Unfortunately, the API does not have this functionality. Please contact your LSEG Representative or Account Manager to help you with the data monitoring.