question

Upvotes
Accepted
18 2 3 10

"Too many requests" when trying to download historic bond yields

Hello - I need to download historic yields for bonds -- and the size of the download would be too big for Excel. When I increase the size of the download, I get errors, usually this one: "RDError: Error code 429 | Too many requests, please try again later." I've also just had timeouts or it returns empty data frames. Below is my code. I have tried to break the downloads into more loops to shrink the size, but that doesn't help and it seems that the download sizes are about what I could get throug the Excel add-in instead of the Python data api anyway. How can I fix my code?

import refinitiv.data as rd
import pandas as pd
from datetime import date
from dateutil.relativedelta import relativedelta
import os

type = ["GOVT","CORP"]
mths_back = (2024-2016)*12
enddate = date.today()
startdate = enddate - relativedelta(months=mths_back)
interval_on_lookback = '1W'
fields_TS = ['B_YLD_1','OAS_BID','AST_SWPSPD']

rd.open_session()
df_RIC = rd.discovery.search(
view = rd.discovery.Views.GOV_CORP_INSTRUMENTS,
top = 10000,
filter =  f"((DbType eq '{type[1]}') and (RCSBondGradeLeaf eq 'Investment Grade' or RCSBondGradeLeaf eq 'High Yield') and IsConvertible eq false and MaturityDate ge {startdate} and MaturityDate le {enddate} and (((SPsRatingsScope((RatingType eq 'S&P') and (CurrentRatingRank ge 15) and (CurrentRatingRank le 24))) or (RatingsScope((RatingType eq 'MDY') and (CurrentRatingRank ge 8) and (CurrentRatingRank le 17))) or (RatingsScope((RatingType eq 'FTC') and (CurrentRatingRank ge 9) and (CurrentRatingRank le 18)))) and (not (OfferingTypeDescription xeq 'Private placement-144A') and not (OfferingTypeDescription xeq 'Private placement-144A with reg rights') and not (OfferingTypeDescription xeq 'Private placement-144A no reg rights') and not (OfferingTypeDescription xeq 'Private placement-144A - Exchange Offer') and not (OfferingTypeDescription xeq 'Private place-144A Exch Offer with reg') and not (OfferingTypeDescription xeq 'Regulation S/Private placement-144A') and not (OfferingTypeDescription xeq 'Private placement-144A Exch Offer no reg') and not (OfferingTypeDescription xeq 'Private placement-144A - Section 3(c)(7)')) and RCSCurrency in ('C:6' 'C:3' 'C:4')))",
    select = "RIC",
order_by = "RIC"
    )
RICs=df_RIC['RIC'].to_list()


RICs = [x for x in RICs if isinstance(x, str)]
RICs = list(set(RICs))
RICs = sorted(RICs)

df1 =[]
for i in range(len(fields_TS)):
df0 = rd.get_history(
            RICs,
            start = startdate,
            end   = enddate,
            fields     = fields_TS[i],
            interval   = interval_on_lookback
    )
df0['interval'] = interval_on_lookback
df0['field'] = fields_TS[i]
df1 = df1.append(df0)
rd.close_session()
df1
eikon-data-apipythonworkspace-data-api#technologypython apitime-seriesbondssearchprice-historyyield
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
Accepted
84.6k 287 53 77

@LRE42

Thank you for reaching out to us.

It could be the same problem in this discussion.

Although you are using a batch with the get_history method to get historical real-time fields, internally it will send one request per item.

For example, if the batch size is 100, a call to the get_history method will send 100 requests.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
18 2 3 10

Hello -- thanks a lot for the reply. This is helpful (I think my colleague did post the same question too).

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.