Data Retrieve Limit with Get_data in Code book (refinitiv.dataplatform.eikon package)

Hello,
I want retrieve data from Eikon API for 'TR.CommonName,TR.F.TotRevBizActiv etc fields by importing instruments from the excel file and the instruments will be more than 1lakh OApermids,

will code book able to retrieve data for 1 Lakh instruments for various fields? if yes please share sample code to retrieve the data, if no please suggest some alternatives to retrieve the data.

Regards,

Basava Reddy E.

Find more posts tagged with

python

eikon

workspace

api-limits

eikon-data-api

refinitiv-dataplatform-eikon

workspace-data-api

eikon-for-office

Accepted answers

chavalit-jintamalit

Hi @Basavareddy.E

This is a sample code:

import pandas as pd
from pydash.arrays import chunk
import time

df = pd.read_excel('Transpose.xlsx')
orgid_list = df['Orgid'].tolist()
orgid_list = list(map(str, orgid_list))
chunklist = chunk(orgid_list, 2000)

fields = ['TR.CommonName','TR.F.TotRevBizActiv']
df_all = pd.DataFrame()
count = 0

for singlelist in chunklist:
    print(count)
    count = count + 1
    df_data,err = ek.get_data(singlelist, fields)
    df_all = df_all.append(df_data)
    df_all.reset_index(drop=True, inplace=True)
    time.sleep(1)

df_all.to_excel('output.xlsx')

The code read data from an excel and convert the OrgID integer into String.

Then divide them into 2000 instruments per chunk and request the data.

After it receives the data, it appends the result to df_all variable.

Finally, export the data from df_all into output.xlsx file

All comments

jason.ramchandani01

Hi @Basavareddy.E - thanks for your question. For limits regarding the Eikon Data APIs please see the guidance here. So for a list of 100,000 RICs you would need to break that down into the max number of datapoints per API call (10,000) by the number of fields you wanted (say 2 fields) = 5000 instruments per call (or any variants thereof). So you could then chunk your large list of RICs into say chunks of 5000 RICs. You could do this using the pydash library (pip install pydash) and then the following code:

from pydash.arrays import chunk
chunklist = chunk(YOUR_LARGE_LIST, 5000)
chunklist

Once you have your chunks you can loop through them presenting each to the API.

Desktop products are not generally designed for this sort of bulk extraction - so you should consider our DataScope Select Product which is designed for these sorts of volumes.

Regarding code samples - CodeBook already has samples of using get_data function built-in under Examples-->Eikon Data API-->Eikon Data API - get_data.ipynb (see screenshot below)

I hope this can help.