"Too many requests" when trying to download historic bond yields

Hello - I need to download historic yields for bonds -- and the size of the download would be too big for Excel. When I increase the size of the download, I get errors, usually this one: "RDError: Error code 429 | Too many requests, please try again later." I've also just had timeouts or it returns empty data frames. Below is my code. I have tried to break the downloads into more loops to shrink the size, but that doesn't help and it seems that the download sizes are about what I could get throug the Excel add-in instead of the Python data api anyway. How can I fix my code?
import refinitiv.data as rd
import pandas as pd
from datetime import date
from dateutil.relativedelta import relativedelta
import os
type = ["GOVT","CORP"]
mths_back = (2024-2016)*12
enddate = date.today()
startdate = enddate - relativedelta(months=mths_back)
interval_on_lookback = '1W'
fields_TS = ['B_YLD_1','OAS_BID','AST_SWPSPD']
rd.open_session()
df_RIC = rd.discovery.search(
view = rd.discovery.Views.GOV_CORP_INSTRUMENTS,
top = 10000,
filter = f"((DbType eq '{type[1]}') and (RCSBondGradeLeaf eq 'Investment Grade' or RCSBondGradeLeaf eq 'High Yield') and IsConvertible eq false and MaturityDate ge {startdate} and MaturityDate le {enddate} and (((SPsRatingsScope((RatingType eq 'S&P') and (CurrentRatingRank ge 15) and (CurrentRatingRank le 24))) or (RatingsScope((RatingType eq 'MDY') and (CurrentRatingRank ge 8) and (CurrentRatingRank le 17))) or (RatingsScope((RatingType eq 'FTC') and (CurrentRatingRank ge 9) and (CurrentRatingRank le 18)))) and (not (OfferingTypeDescription xeq 'Private placement-144A') and not (OfferingTypeDescription xeq 'Private placement-144A with reg rights') and not (OfferingTypeDescription xeq 'Private placement-144A no reg rights') and not (OfferingTypeDescription xeq 'Private placement-144A - Exchange Offer') and not (OfferingTypeDescription xeq 'Private place-144A Exch Offer with reg') and not (OfferingTypeDescription xeq 'Regulation S/Private placement-144A') and not (OfferingTypeDescription xeq 'Private placement-144A Exch Offer no reg') and not (OfferingTypeDescription xeq 'Private placement-144A - Section 3(c)(7)')) and RCSCurrency in ('C:6' 'C:3' 'C:4')))",
select = "RIC",
order_by = "RIC"
)
RICs=df_RIC['RIC'].to_list()
RICs = [x for x in RICs if isinstance(x, str)]
RICs = list(set(RICs))
RICs = sorted(RICs)
df1 =[]
for i in range(len(fields_TS)):
df0 = rd.get_history(
RICs,
start = startdate,
end = enddate,
fields = fields_TS[i],
interval = interval_on_lookback
)
df0['interval'] = interval_on_lookback
df0['field'] = fields_TS[i]
df1 = df1.append(df0)
rd.close_session()
df1
Best Answer
-
Thank you for reaching out to us.
It could be the same problem in this discussion.
Although you are using a batch with the get_history method to get historical real-time fields, internally it will send one request per item.
For example, if the batch size is 100, a call to the get_history method will send 100 requests.
0
Answers
-
Hello -- thanks a lot for the reply. This is helpful (I think my colleague did post the same question too).
0
Categories
- All Categories
- 3 Polls
- 6 AHS
- 36 Alpha
- 166 App Studio
- 6 Block Chain
- 4 Bot Platform
- 18 Connected Risk APIs
- 47 Data Fusion
- 34 Data Model Discovery
- 684 Datastream
- 1.4K DSS
- 614 Eikon COM
- 5.2K Eikon Data APIs
- 10 Electronic Trading
- Generic FIX
- 7 Local Bank Node API
- 3 Trading API
- 2.9K Elektron
- 1.4K EMA
- 248 ETA
- 552 WebSocket API
- 37 FX Venues
- 14 FX Market Data
- 1 FX Post Trade
- 1 FX Trading - Matching
- 12 FX Trading – RFQ Maker
- 5 Intelligent Tagging
- 2 Legal One
- 23 Messenger Bot
- 3 Messenger Side by Side
- 9 ONESOURCE
- 7 Indirect Tax
- 60 Open Calais
- 275 Open PermID
- 44 Entity Search
- 2 Org ID
- 1 PAM
- PAM - Logging
- 6 Product Insight
- Project Tracking
- ProView
- ProView Internal
- 22 RDMS
- 1.9K Refinitiv Data Platform
- 640 Refinitiv Data Platform Libraries
- 4 LSEG Due Diligence
- LSEG Due Diligence Portal API
- 4 Refinitiv Due Dilligence Centre
- Rose's Space
- 1.2K Screening
- 18 Qual-ID API
- 13 Screening Deployed
- 23 Screening Online
- 12 World-Check Customer Risk Screener
- 1K World-Check One
- 46 World-Check One Zero Footprint
- 45 Side by Side Integration API
- 2 Test Space
- 3 Thomson One Smart
- 10 TR Knowledge Graph
- 151 Transactions
- 143 REDI API
- 1.8K TREP APIs
- 4 CAT
- 26 DACS Station
- 121 Open DACS
- 1.1K RFA
- 104 UPA
- 191 TREP Infrastructure
- 228 TRKD
- 915 TRTH
- 5 Velocity Analytics
- 9 Wealth Management Web Services
- 90 Workspace SDK
- 11 Element Framework
- 5 Grid
- 18 World-Check Data File
- 1 Yield Book Analytics
- 46 中文论坛