Refinitiv Data Library for Python: Call limits

Hello,

I'm using the Refinitiv data library for python. In particular, I'm using the following functions and have been getting some errors for too many requests.

For rd.discovery.search() and rd.get_history(), I've been getting:

RDError: Error code 429 | Too many requests, please try again later.

For rd.get_data(), I've been getting:

RDError: Error code -1 | Too many requests, please try again later.

I have found a lot of threads quoting the Eikon limits, but I don't think the above functions involves sourcing data from Eikon.

What are all the limits for these functions (daily requests/volume limits, per second limits, and per request limits?)

Thank you!

Find more posts tagged with

error429

refinitiv-data-platform-libraries

error-1

#technology

Accepted answers

aramyan.h

Hi Ryan,

That depends on the fields. The library uses different underlying services for TR (e.g TR.Revenue) and non-TR fields(e.g. B_YLD_1). If you have only TR or only non-TR fields (which I believe is your case) that will be considered as 1 request. If you have both TR and non-TR fields on a single request, then that's 2 separate requests.

Best regards,

Haykaz

All comments

aramyan.h

Hi @Ryan Lo ,

Both Eikon and RD Libraries share same underlying services and proxy settings in regards to the limits, that is why the Limits on Eikon you are referring to, which can be found also here (Documentation | Devportal (lseg.com)) are applicable to RD Library functions as well.

The particular limit you have above is received when you reach 10000 daily request limit.

Best regards,

Haykaz

Ryan Lo

Hi @aramyan.h ,

I've reviewed the code and I don't think we've reached the 10,000 daily request limit. The file only added up to around 65MB after 45 minutes so I don't think we've reached any volume-related limits either.

Using rd.get_history(), we were pulling data for a total of 6,458 RICs, separating the requests out into loops of 100 RICs and 3 fields. We've only submitted around 100 requests before we got the following:

RDError: Error code 429 | Too many requests, please try again later.

The API was also only returning a response around every 30 seconds, and this was working consistently for around 100 requests for about 45 minutes, so I would assume this is a daily limit that we've ran into.

The code is follows:

fields_TS = ['B_YLD_1','OAS_BID','AST_SWPSPD']

df1 = []
counter = 0
RICS_per_loop_TS = 100

for k in range(0, len(RICs), RICS_per_loop_TS):
    batch = RICs[k:k+RICS_per_loop_TS]
    counter = counter+1

    # loop for the same set of RICs but for all of the fields in fields_TS
    for i in range(3):
        df0 = pd.DataFrame
        df0 = rd.get_history(
            instr = batch,
            start = "2016-07-23",
            end   = "2024-07-23",
            fields     = fields_TS[i],
            interval   = '1'
            )
        time.sleep(0.25)
        df1.append(df0)

I have removed all the irrelevant bits of code, but essentially in every loop we pull data for 100 RICs, and we have 3 separate requests for each field.

I would really appreciate it if you could help us out!

Thank you,

Ryan

aramyan.h

Hi @Ryan Lo ,

Under the hood the library sends separate requests for each RIC which counts towards the limit, so 100 requests for 100 rics that adds up to the 10000 limit.

You can verify the request if you enable the logs as below:

rd.get_config().set_param(
    param=f"logs.transports.console.enabled", value=True
)
session = rd.open_session()
session.set_log_level("DEBUG")

Best regards,

Haykaz

Ryan Lo

Hi @aramyan.h

Thank you for the above.

If I include all 3 fields within one call to the get_history method, would that triple the requests, or would that just triple the data points within those 100 requests?

Kind regards,

Ryan

aramyan.h

Hi Ryan,

Best regards,

Haykaz

Ryan Lo

Hi Haykaz @aramyan.h,

Thanks for the above.

What about the get_data method? Do they use the same principles as above, where one request with 100 RICs (say with multiple TR fields, and no non-TR fields) would count as 100 requests? And does this fall under the same 10,000 daily limit, or is there a separate 10,000 daily limit for each method?

Kind regards,

Ryan

aramyan.h

Hi @Ryan Lo ,

For rd.get_data with no TR fields, one request with many RICs will send single request, however please note you will not be able to use paramaters param with non-TR fields, meaning you can't get historical data with that.

In fact, you can check the requests if you enable the logs as below:

import refinitiv.data as rd
rd.get_config().set_param(
    param=f"logs.transports.console.enabled", value=True
)
session = rd.open_session()
session.set_log_level("DEBUG")

As for the 10,000 limit, it is counted across all methods

Hope this helps.

Best regards,

Haykaz

EXPLORE OUR SITES