What is the best way to get a list of fields for a list of instruments for a historical date rang...

jokari69

...e with get_data?

Let's say I want to get around 500 fields for around 4000 instruments (stocks) for about 20 year (SDate = 2020-01-01). Given the multiple limits I cannot just use

get_date(instrument, fields, {'SDate': '2000-01-01', 'EDate': today_as_string}) 

as this will give all kinds of errors / limits breaching.

Currently I will chunk the instruments 50 at a time, select 1 field and query for the full date range.

dfs = []
for field in fields:
    for instruments_chunk in split_list(instruments, max_items_in_list=50): 
        df = ek.get_data(instruments_chunk, [field + 'date', field], {'SDate': '2000-01-01', 'EDate': today_as_string})
        df.rename(columns={df.columns[1]: 'date'}, inplace=True)
        df['date'] = pd.to_datetime(df['date']).dt.date.astype('datetime64')
        dfs.append(df)
final_df = pd.concat(dfs)

Although this works it is very slow and will take several days to complete.

Is there a better solution to this kind of problem?

Gurpreet

@jokari69,

Eikon data api is not geared towards this kind of use. Refinitiv offers Datascope and Tickhistory products which can provide very granular data for a wide range of instruments. Please contact your Refinitiv account manager for getting access to these products.

jokari69

https://community.developers.refinitiv.com/discussion/comment/86455#Comment_86455

Understood. Thank you for your detailed answer.