Fewer Observations when downloading data again

Antonio_v

Hello everyone,

I am a PhD student and I've been using Refinitiv to download firm-level data for my research. I'm currently facing a problem. I downloaded my main dataset around September 2024. In March 2025, I needed to add another variable, so I re-run the same script to download the data through the Python API. The resulting dataset had around 500 observations less than the first one I downloaded. I got the same result using both the Eikon API and Datastream API. Today, I tried to download the dataset again, and I got another 100 observations less. I guess it might be important to specify that my university provides me access only to the Workspace for Student. I tried to contact the Helpdesk, but they suggested to post my query here given the scale of the data request.

The following is the main line used to retrieve the data through the Python Eikon API:
data_y, err = ek.get_data(instruments=[permID_list[x]], fields=['TR.F.ToTRevenue(SDate=2023-03-31,EDate=-13,Period=FY0,Frq=FY,Curn=EUR)', 'TR.F.LaborRelExpnTot(SDate=2023-03-31,EDate=-13,Period=FY0,Frq=FY,Curn=EUR)', 'TR.F.COGSTot(SDate=2023-03-31,EDate=-13,Period=FY0,Frq=FY,Curn=EUR)', 'TR.F.TotAssets(SDate=2023-03-31,EDate=-13,Period=FY0,Frq=FY,Curn=EUR)', 'TR.F.TotLiab(SDate=2023-03-31,EDate=-13,Period=FY0,Frq=FY,Curn=EUR)', 'TR.F.TotFixedAssetsNet(SDate=2023-03-31,EDate=-13,Period=FY0,Frq=FY,Curn=EUR)', 'TR.F.SGATot(SDate=2023-03-31,EDate=-13,Period=FY0,Frq=FY,Curn=EUR)', 'TR.F.PPEGrossTot(SDate=2023-03-31,EDate=-13,Period=FY0,Frq=FY,Curn=EUR)', 'TR.F.PPENetTot(SDate=2023-03-31,EDate=-13,Period=FY0,Frq=FY,Curn=EUR)', 'TR.F.InvstLT(SDate=2023-03-31,EDate=-13,Period=FY0,Frq=FY,Curn=EUR)','TR.F.CashCashEquivTot(SDate=2023-03-31,EDate=-13,Period=FY0,Frq=FY,Curn=EUR)', 'TR.F.GrossProfIndPropTot(SDate=2023-03-31,EDate=-13,Period=FY0,Frq=FY,Curn=EUR)', 'TR.F.OpProfBefNonRecurIncExpn(SDate=2023-03-31,EDate=-13,Period=FY0,Frq=FY,Curn=EUR)', 'TR.F.IntrExpn(SDate=2023-03-31,EDate=-13,Period=FY0,Frq=FY,Curn=EUR)', 'TR.F.DebtTot(SDate=2023-03-31,EDate=-13,Period=FY0,Frq=FY,Curn=EUR)', 'TR.F.InvntTot(SDate=2023-03-31,EDate=-13,Period=FY0,Frq=FY)', 'TR.NACEClassification', 'TR.NAICSSectorAllCode', 'TR.NAICSIndustryGroupAllCode', 'TR.TRBCIndustry', 'TR.HeadquartersCountry', 'TR.RegistrationCountry', 'TR.OrgFoundedYear'])
In other words, for each of the permID codes specified in a separate csv file ("permID_list"), I retrieve the following data items: Total Revenue, Total Labour Expenditure, Total COGS, Total Assets, Total Liabilities, Total Fixed Assets, Total SGA, Total PPE Gross, Total PPE Net, Investment Long Term, Total Cash and Cash Equivalents, Total Gross Profits, Total Operating Profits, Interest Expenditure, Total Debt, Total Inventories, NACE classification, NAICS code (2-digit), NAICS code (4-digit), TRBC Industry, Country of Headquarters, Country of Registration, Organisation Founding Year. All the time series variables are requested annually, for a time span covering from 2010 to 2022. 

Is there any way I could recover the data from September 2024? Having as many observations as possible is really important for my research and the completion of my thesis.

Thank you for any help you could provide!

Gurpreet

Hello @Antonio_v

Can you please provide the list of PermIDs that you are using in this query. Also please note that we recommend using the LSEG Data Library which is the strategic API.

Furthermore, you can separate the date into another object into the API call. For e.g.

 ek.get_data(instruments=[...], fields=[...], parameters={'SDate':'2020-10-01', 'EDate':'2020-10-05', 'Period': 'FQ0'…})

Antonio_v

https://community.developers.refinitiv.com/discussion/comment/126542#Comment_126542

Hi @Gurpreet

Thanks for your reply! I attach here the file with the list of PermIDs. Would using the LSEG Data Library solve my issue? I will try today to check

list_permID.xlsx

Gurpreet

Hello @Antonio_v

Your instrument list is quite large. Combined with the number of fields being requested, you are certainly running into the limits of data that can be retrieved from the desktop.

Please see the usage and limits guideline document and try to keep the requested data within the limit.