Refinitiv API errors

I tried running this code (attached below). The code is inconsistent in giving results. It worked in the very first attempt for 12 companies but then it throws an error when I ran it for more than 2 companies and sometimes it runs in 3rd/4th attempts. In this inconsistency problem, we can not just say that this code will run for x companies in y attempts (y<=5). Is this a server overload problem or inherent issues with this API?

Code

rd.open_session()

# Define a function to chunk a list into smaller lists of a specific chunk size

def chunk_list(first, chunk_size):

for i in range(0, len(first), chunk_size):

yield first[i:i + chunk_size]

# Set the chunk size

chunk_size = 1000

# Define a list of RICs

RIC_list = ['GE', 'ROK','TT', 'EMR', 'DHR'] #, 'SCHN.PA', 'ABBN.S', 'SIEGn.DE', 'DHR','HON.O','JCI'] #, 'ZOMT.NS', 'INAR.NS', 'TATE.NS', 'COMU.NS', 'CENA.NS',

fields1 = [

'TR.FundClassId','TR.FundPortfolioName','TR.FundParentNameHistory', 'TR.FundPortfolioName.investorid',

'TR.FundOwnershipTurnover(Sdate=2024-06-30)',

'TR.FundAdjShrsHeld(Scale=6, Sdate=2024-06-30)','TR.FundAdjShrsHeld(Scale=6, Sdate=2024-03-31)','TR.FundAdjShrsHeld(Scale=6, Sdate=2023-12-31)','TR.FundAdjShrsHeld(Scale=6, Sdate=2023-09-30)',

'TR.FundAdjShrsHeld(Scale=6, Sdate=2023-06-30)','TR.FundAdjShrsHeld(Scale=6, Sdate=2023-03-31)', 'TR.FundAdjShrsHeld(Scale=6, Sdate=2022-12-31)','TR.FundAdjShrsHeld(Scale=6, Sdate=2022-09-30)',

'TR.FundAdjShrsHeld(Scale=6, Sdate=2022-06-30)','TR.FundAdjShrsHeld(Scale=6, Sdate=2022-03-31)', 'TR.FundAdjShrsHeld(Scale=6, Sdate=2021-12-31)','TR.FundAdjShrsHeld(Scale=6, Sdate=2021-09-30)',

'TR.FundHoldingsDate(Sdate=2024-06-30)','TR.FundHoldingsDate(Sdate=2024-03-31)','TR.FundHoldingsDate(Sdate=2023-12-31)','TR.FundHoldingsDate(Sdate=2023-09-30)',

'TR.FundHoldingsDate(Sdate=2023-06-30)','TR.FundHoldingsDate(Sdate=2023-03-31)','TR.FundHoldingsDate(Sdate=2022-12-31)','TR.FundHoldingsDate(Sdate=2022-09-30)',

'TR.FundHoldingsDate(Sdate=2022-06-30)','TR.FundHoldingsDate(Sdate=2022-03-31)','TR.FundHoldingsDate(Sdate=2022-12-31)','TR.FundHoldingsDate(Sdate=2022-09-30)',

'TR.FundHoldingsDate(Sdate=2022-06-31)', 'TR.FundTotalEquityAssets','TR.FundInvtStyleCode','TR.FundInvestorType','TR.FundAddrCountry','TR.FundParentType','TR.FdShrhldrActivistType', 'TR.FundInvtOrientation','TR.FundEarliestHoldDate','TR.FundAggPE12MonthsFwd','TR.FundContactFirstName','TR.FundContactMiddleInit','TR.FundContactLastName','TR.FundestorContactTitle',

'TR.FdConPrimaryFunction','TR.FundContactEmailAddr','TR.FundContactTelCntry','TR.FundContactTelAreaCode','TR.FundContactTelExt','TR.FundContactTelNumber','TR.FundAggRevenue','TR.FundAggReturnOnEquity','TR.FundAggReturnOnAssets'

]

parameters = {}

df_combined = None

retry_max = 5

retry_count = 1

# Define a function to get data with retry logic

def get_data_with_retry(chunk, fields, parameters):

global retry_count

while retry_count <= retry_max:

try:

df = rd.get_data(chunk, fields=fields, parameters=parameters)

return df

except RDError as e:

print(f"RDError code: {e.code}")

print(f"RDError message: {e.message}")

print(f"Attempt {retry_count} failed. Retrying...")

time.sleep(2**retry_count) # Exponential backoff

retry_count += 1

raise Exception("Maximum retry attempts reached.")

# Try to retrieve data in chunks and handle errors

try:

for idx, chunk in enumerate(chunk_list(RIC_list, chunk_size)):

print('i:', idx , 'c:', chunk)

df = get_data_with_retry(chunk, fields1, parameters)

if df_combined is None:

df_combined = df

else:

df_combined = df_combined.append(df)

print(f'{len(chunk)} chunks completed')

retry_count = 1 # Reset retry count for the next chunk

except Exception as e:

print(f"An error occurred: {e}")

Error:

Any help to understand this error and make code consistently give results would be much appreciated.

Find more posts tagged with

refinitiv-data-platform

#technology

Accepted answers

Jirapongse

@atul.arya

Thank you for reaching out to us.

There are two types of timed out.

1. The client timed out

The client timed out can be configured via the configuration file (refinitiv-data.config.json) or the code.

{ 
  "http": {
    "request-timeout": 60
  },
...
}

The code is:

config = rd.get_config()
config.set_param("http.request-timeout", 60)
rd.open_session()

2. The server timed out

The server timed out can't be modified. Therefore, the number of items, fields, or date range should be reduced.

According to the error, it should be the client timed out. Please try to increase the request-timeout value in the RD configurations.

All comments

Jirapongse

@atul.arya

Thank you for reaching out to us.

There are two types of timed out.

1. The client timed out

The client timed out can be configured via the configuration file (refinitiv-data.config.json) or the code.

{ 
  "http": {
    "request-timeout": 60
  },
...
}

The code is:

config = rd.get_config()
config.set_param("http.request-timeout", 60)
rd.open_session()

2. The server timed out

The server timed out can't be modified. Therefore, the number of items, fields, or date range should be reduced.

According to the error, it should be the client timed out. Please try to increase the request-timeout value in the RD configurations.

atul.arya

Hi @Jirapongse

Thank you for your response.

I have tested the code with your above-mentioned suggestion. This helped resolve most of the errors we were facing earlier. However, we still encounter some errors infrequently. While the solution works well for smaller lists of companies, it does not scale to the level we were aiming for.

We appreciate your assistance.

Jirapongse

Yes, it wil help the client timed out. However, it will not solve the server timed out if the application is still requesting a lot of data. Please try to reduce the number of items, fields, or requested date range in each request.

As far as I know, the API is not designed for applications to request a lot of data in one request. You may contact your LSEG account team or sales directly to discuss alternative methods.

EXPLORE OUR SITES