Continuously getting backend error and internal server error.

Hi, I have around 6000 RICS and i want to fetch the earnings data for those RICS on daily basis. I'm using eikon. But continuously I'm getting backend error and internal server error. and even if I ran it on the for all 6000 RICS I'm getting the data for only in between 3 to 3.5K. I'm tried giving the time sleep of 2 sec or 1 sec but that also not works. How can I solve this issue. Is Eikon is capable of fetching the data for all the RICS or is there any another solution for this. Because I need to fetch the data for all on daily basis (each business day)

Find more posts tagged with

support

developer-community

helpdesk

#product

Accepted answers

aramyan.h

Hi @vishnu01 ,

This caused because of the service overload. You may try to make the request in chunks (e.g 1000 RICs in chunk) wrapped by try except statement so your code will retry in case of the error. There is an example in this thread which you may find useful.

Best regards,

Haykaz

All comments

aramyan.h

Hi @vishnu01 ,

Best regards,

Haykaz

vishnu01

Hi @aramyan.h

Thank you for the response , I'm attempting to get the market cap and I'm using the code below to do so. I get an HTTP timeout exception if I increase the chunk size , and backend error if I provide one RIC at a time.

I frequently receive the backend error even when I train on small list of RIC's.

Our goal is to setup the regular tasks to collect the data like market cap, earnings, merge and acquisitions, estimates and numerous others factors.

So how can I get the data without the errors ?

chunk_size = 50

merged_df = pd.DataFrame()

try:

for i in range(0, len(Final_df), chunk_size):

rics_chunk = Final_df['ric'].iloc[i:i + chunk_size].tolist()

df, err = ek.get_data(rics_chunk, fields=['TR.MarketCapLocalCurn', 'TR.MarketCapLocalCurn.date'],

parameters={'SDate': '2010-01-01', 'EDate': '2023-11-07', 'Frq': 'D'})

merged_chunk = pd.merge(df, Final_df.iloc[i:i + chunk_size], left_on='Instrument', right_on='ric', how='left')

merged_df = pd.concat([merged_df, merged_chunk])

break

except Exception as e:

print(f"Error occurred: {e}")

aramyan.h

Hi @vishnu01 ,

have you looked at the thread I have shared? there is an example how to do that. I have tried to midify your code, you may try it and see if works:

chunk_size = 50

merged_df = pd.DataFrame()

for i in range(0, len(Final_df), chunk_size):
    retries = 0
    rics_chunk = Final_df['ric'].iloc[i:i + chunk_size].tolist()
    while retries < 3:
        try:
            df, err = ek.get_data(rics_chunk, fields=['TR.MarketCapLocalCurn', 'TR.MarketCapLocalCurn.date'],
            parameters={'SDate': '2010-01-01', 'EDate': '2023-11-07', 'Frq': 'D'})
            merged_chunk = pd.merge(df, Final_df.iloc[i:i + chunk_size], left_on='Instrument', right_on='ric', how='left')
            merged_df = pd.concat([merged_df, merged_chunk])
        except Exception as e:
            print(f"Error occurred: {e}")
            retries +=1
            continue
        break

Best regards,

Haykaz

vishnu01

Hi @aramyan.h,

When I attempted to use the code you provided above, Eikon gave me two errors. I tried reducing the chunk size all the way to 10, but the problem persisted.

Error code 408 | HTTP TimeoutException and "code":504,"message":"A remote server did not reply in a timely fashion.

aramyan.h

Hi @vishnu01 ,

I think your issue is coming from the line merged_chunk = pd.merge(df, Final_df.iloc[i:i + chunk_size], left_on='Instrument', right_on='ric', how='left'), not the API.

I have commented out that line and have tested with chunk 100 and it works fine:

Also I have indented the break that should have been inside while (updated in the code above)

Best regards,

Haykaz

EXPLORE OUR SITES