Hi, I need to scrape data on aound 70,000 companies. Since I cannot manually get 10 year data for...

Hi, I need to scrape data on aound 70,000 companies. Since I cannot manually get 10 year data for all these companies via the refinitiv query portal, any recommendations of how I should d o this?

Find more posts tagged with

stata

datastream-api

dsws-api

Accepted answers

Alex Putkov.1

@adhish.sawant

What Refinitiv products or services does your university subscribe to? If it subscribes to Refinitiv Eikon, then you can use Eikon Data APIs to retrieve the data you're interested in. See the Quick Start Guide for Eikon Data APIs to get started and the tutorial titled "Fundamentals API - Company Tearsheet Estimate" to learn how to construct data retrieval expressions and how to go about metadata discovery (finding field names and parameters) for use with Eikon Data APIs.

All comments

Gurpreet

Hi @adhish.sawant,

You can pace out your requests and try to get data from DSWS. Here is guide about DSWS limits. There is no limit on history or frequency, but It is limited to 10 million datapoints per month.

If this does not work for you, then a bulk feed product like Refinitiv Tick History is a better option. You can talk to your Refinitiv account manager about getting access to RTH.

adhish.sawant

Hi @gurpreet.bal ,
Nice to hear from you!
Will DSWS require me to do some coding?

Gurpreet

Yes, DSWS and RTH will require some coding. There is a helper library provided for Python. You can start with a quick start and then do step by step tutorials [here][1].

[1]: https://developers.refinitiv.com/en/api-catalog/eikon/datastream-web-service/Quickstart

adhish.sawant

@gurpreet.bal there is a possibility that my uni does not have access to DSWS. I have reduced my final sample to around 10,000 companies. I need their company fundamentals and ESG scores for the past 1o years. In the absence of DSWS do you have any alternative methods for a single user access system?

jonathan.legrand

hi @adhish.sawant ,

If you can find the data you're looking for in the Data Item Browser (article about it linked below), then you can use the Eikon Data API (page about it linked below)

Data Item Browser:
https://developers.refinitiv.com/en/article-catalog/article/how-to-discover-available-fields-for-data-grid-service-on-jet-ap

Eikon Data API:

https://developers.refinitiv.com/en/search-results?q=Eikon+Data+API

Gurpreet

Hi @adhish.sawant ,

The first step is to find out what Refinitiv products you do have access to, and then we can advise on the usage.

adhish.sawant

we only have single-user access on a dedicated PC according to my librarian

Gurpreet

You should be able to perform your query using the single user access - ask for a block of time.

jonathan.legrand

Hi @adhish.sawant ,

Would you mind precessing what you have single-user access to? The Eikon Data API? The Refinitiv Data Platform API? The Datastream API?
If your request is not related to our programmatic APIs, I'm afraid that this forum is not appropriate for your questions...

I suggest you have a look at this landing page that showcase articles related to ESG:

https://developers.refinitiv.com/en/use-cases-catalog/sustainable-investing

Alex Putkov.1

@adhish.sawant

adhish.sawant

Hi team,

My uni just confirmed, I have access to the Eikon Data API but not to DatastreamDSWS. Please advice based on this information. Thanks

Alex.Putkov12

@adhish.sawant

Thank you for confirming. Please see my previous post on this thread, and follow the links provided in my post.

EXPLORE OUR SITES