rate limit when requesting history via the LSEG Data Library

davidk

I gather that when requesting historical pricing, every stock is being requested individually, even when requesting one date, is that correct? E.g. if I just ask for Russell 3000 stocks since the start of June, I'm making 3,000 separate requests? That at least seems to be what is happening from debug logs.

I ask partly because I believe the rate limit may be far too low if that is the case. I believe I hit the daily rate limit today, on the debug log I am getting the following error:

2025-06-11 18:18:45 httpcore.http11 DEBUG> receive_response_headers.complete return_value=(b'HTTP/1.1', 429, b'Too Many Requests', [(b'Server', b'nginx/1.17.2'), (b'Date', b'Wed, 11 Jun 2025 22:18:45 GMT'), (b'Content-Type', b'text/html; charset=utf-8'), (b'Content-Length', b'42'), (b'Connection', b'keep-alive'), (b'Access-Control-Allow-Origin', b'*'), (b'X-Request-Id', b'4417aee8-4046-43f0-892b-f38f5f1d2355'), (b'RateLimit-Remaining', b'0'), (b'VolumeLimit-Remaining', b'4967298'), (b'QueueLimit-Remaining', b'100'), (b'RateLimit-Policy', b'5;w=1, 10000;w=86400'), (b'VolumeLimit-Policy', b'50000;w=60, 5000000;w=86400'), (b'RateLimit-Resource', b'*'), (b'VolumeLimit-Resource', b'*'), (b'ETag', b'W/"2a-UpTsLJ74nYuiLgNgEwlQMxGqwrE"')])

As I understand it the "Rate Limit" in this case is the number of requests per day. From https://developers.lseg.com/en/api-catalog/lseg-data-platform/lseg-data-library-for-python/documentation it says we can do 10,000 requests per day or 5 per second, which is in keeping with the Rate Limit policy. However, if I am burning 3,000 requests just getting 1 day of the Russell 3K for 1 data type, that could cause me problems (for example, if I also want to get TOPIX, and to get minutely bars as well as daily data for several days). How do we get around this? Other data fields seem to allow multiple securities per request which make this more manageable.

Jirapongse

@davidk

Thank you for reaching out to us.

If you encounter a 429 error, it likely indicates that your current usage has exceeded the allowed limits.

I understand that you're requesting both interday and intraday historical data. Please note that the historical endpoints support only one RIC per request. Given this requirement, you may need to consult your LSEG account representative or sales team directly for further assistance.

For historical data, we have another API (Tick History) which may be more suitable for your requirements. Please discuss this with your LSEG account team or Sales team.

davidk

https://community.developers.refinitiv.com/discussion/comment/126268#Comment_126268

Hi Jirapongse— understood on the bar data.For the daily data, if I'm just getting pricing data like open, close, volume, etc., it appears I can use a single request to the get_data API, e.g.

ld.get_data(['MSFT.O', 'AAPL.O'], ['TR.OpenPrice.date', 'TR.Volume.date', 'TR.OpenPrice', 'TR.Volume'], parameters={'SDate':'2025-06-01', 'EDate':'2025-06-06'}, header_type=lseg.ld.content._header_type.HeaderType.NAME)

then it seems to use just one request. Is there any reason not to do this?

Jirapongse

The library uses different endpoints to get historical data.

For TR.xxx fields, it gets data from the fundamental and reference endpoint which supports multiple RICs in a request message.

For real-time fields (such as BID, and ASK), it uses the historical pricing API to retrieve historical data. This historical pricing API supports only one RIC in a request message.

Sorry for the confusion. You didn't share the code in the question so I understand that you may use the historical pricing API to get historical data.