question

Upvotes
Accepted
16 4 7 10

COMPANY SEARCH RDP Python

Actually I wanted to look for getting all kind of companies with ISINs, Revenue etc .

Can I do that with Python rdp? I know that Excel box has this corporate search engine but I can export only 10000 at a time .. I know that I should be using some universe with rdp.search

but do not know which one to use and what to put in Filter. Also how can I get the totality (I know there is GroupSize, but can it be arbitrarily "big" ?) . Many thanks in advance. Grigorios


#technologyrdp-api#productrdp search
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Hello @grigorios.mamalis

Thank you for your participation in the forum. Is the reply below satisfactory in resolving your query?


If so please can you click the 'Accept' text next to the appropriate reply? This will guide all community members who have a similar question.

Thanks,


AHS

Please be informed that a reply has been verified as correct in answering the question, and has been marked as such.

Thanks,
AHS

Upvotes
Accepted
17.1k 80 39 63

Hi @grigorios.mamalis

Based on your category filter of "Company/Issuers" and organizations that are listed vs unlisted, and you have no other filtering requirements, you will need to apply some advanced filtering and navigation techniques to work with limits of the server. For example, these are the number of hits you are working with:
1696347130302.png

For example, you may want to consider narrowing down based on the organization type, i.e.

1696347357646.png

There are many other factors and categories to consider. You're biggest challenge is the amount of data you want to extract - assuming you want to see listed and unlisted "company/issuers".

I would suggest you refer to dealing with Limits and some techniques you can apply.


1696347130302.png (4.8 KiB)
1696347357646.png (18.3 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
78.7k 248 52 74

@grigorios.mamalis

Thank you for reaching out to us.

I checked and found that the search endpoint on RDP also limits 10000 entries in the response.

However, you can use the navigators property to verify how to use additional fields in the filter to reduce the number entries in the response. For example, if I set the ExchangeName field in the navigators property, the output looks like this:

"Navigators": {
    "ExchangeName": {
      "Buckets": [
        {
          "Label": "CME:Index and Options Market",
          "Count": 18103
        },
        {
          "Label": "Chicago Mercantile Exchange",
          "Count": 3118
        },
        {
          "Label": "INTL FCStone Inc",
          "Count": 2550
        },

You will see the estimated number of entries in each exchange. For more information, you can refer to the Build queries easily using Refinitiv's Search API with Python and Jupyter article.

Morever, you can use the Advanaced Search App to create an initial query for you, as mentioned in the Find content and functionality using Refinitiv Data Library with Eikon Advanced Search article.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
16 4 7 10

capturecodebk-corporatesearch.gifHi @Jirapongse,


many thanks for the fedback. I tried "Corporates/Issuers" rather and got more than 10K rows in Advanced Search. However, the Python code from this gave very few rows in CODEBK. See below. Any thoughts? Grigorios



import refinitiv.data as rd

from refinitiv.data.content import fundamental_and_reference

import datetime



df = rd.discovery.search(

filter = "( SearchAllCategoryv2 eq 'Companies/Issuers')",

select = "CommonName,Gics,RCSOriginalAccountsCurrencyLeaf,PrimaryRIC,Orgid,BondsCount,CdsCount,EquitiesCount,FundsCount,FuturesCount,LoanCount,MortgagesCount,OptionsCount,WarrantsCount,OAPermID,OwnershipExists,OrganisationStatus,MktCapCompanyUsd,RCSFilingCountryLeaf,RCSTRBC2012Leaf,UltimateParentOrganisationOrgid,UltimateParentCompanyOAPermID,RatingX1XRatingRank,BusinessEntity,PI,SearchAllCategoryv3,SearchAllCategoryv2,SearchAllCategory,DTSubjectName,UltimateParentOrganisationName,DTSimpleType,RCSOrganisationSubTypeLeaf,LEI,RCSCountryHeadquartersLeaf"

)

df.shape


==========================

(10,29)


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

@grigorios.mamalis

Use the top parameter. The default value of the top parameter is 10.

  top: int, optional
        the maximum number of documents to retrieve. Must be non-negative.
        default: 10
Upvotes
16 4 7 10

@Jirapongse Thanks for the hint. Now to the more difficult:

a. how do I get the full monty of the listed/unlisted companies? Is that by adding filters? Which ones and how (example)?

b. how do I get revenues/EVIC when appropriate ?.

Thanks


import refinitiv.data as rd

rd.open_session()

df = rd.discovery.search(

filter = "( SearchAllCategoryv2 eq 'Companies/Issuers')",

select = "CommonName, ISIN, RIC, Gics,RCSOriginalAccountsCurrencyLeaf,PrimaryRIC,Orgid,OrganisationStatus,MktCapCompanyUsd,UltimateParentOrganisationOrgid,UltimateParentCompanyOAPermID,RatingX1XRatingRank,BusinessEntity,PI,SearchAllCategoryv3,SearchAllCategoryv2,SearchAllCategory,DTSubjectName,UltimateParentOrganisationName,DTSimpleType,RCSOrganisationSubTypeLeaf,LEI,RCSCountryHeadquartersLeaf",

top = 10000

)

df.head()

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
16 4 7 10

Hi @Zincone, many thanks for the help. I am going through that. Quick question: how can I generalize the below query for a list of corporates. Thank you. Grigorios

lst = ["IBM", "Goldman Sachs", "Deutsche Bank"]


import refinitiv.data as rd

from refinitiv.data.content import fundamental_and_reference, search as srch


rd.discovery.search(

view = rd.discovery.Views.ORGANISATIONS,

query = "Name(IBM)",

top = 10,

filter = "( SearchAllCategoryv2 eq 'Companies/Issuers' and ((RCSCountryHeadquarters xeq 'G:6J')))",

select = "CommonName,Gics,RCSOriginalAccountsCurrencyLeaf,PrimaryRIC,Orgid,BondsCount,CdsCount,EquitiesCount,FundsCount,FuturesCount,LoanCount,MortgagesCount,OptionsCount,WarrantsCount,OAPermID,OwnershipExists,OrganisationStatus,MktCapCompanyUsd,RCSFilingCountryLeaf,RCSTRBC2012Leaf,UltimateParentOrganisationOrgid,UltimateParentCompanyOAPermID,RatingX1XRatingRank,BusinessEntity,PI,SearchAllCategoryv3,SearchAllCategoryv2,SearchAllCategory,DTSubjectName,UltimateParentOrganisationName,DTSimpleType,RCSOrganisationSubTypeLeaf,RCSCountryHeadquartersLeaf,RCSCountryHeadquarters"

)



icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
16 4 7 10

Hi @nick.zincone ,


I think I found an answer to my question:

lst = ["IBM", "UBS", "Credit Suisse" ]

query = " or ".join(lst)

discovery.search(

view = rd.discovery.Views.ORGANISATIONS,

query = mystr,

etc

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
17.1k 80 39 63

Hi @grigorios.mamalis

Yes, you will be able to pull down data related to your above query. However, if you are not expecting close to 6000 hits, you will need to apply some additional filtering. For example, when I query for only "Credit Suisse", I get back around 1840 hits. What I would do at this point is determine if all hits are relevant. You will begin to realize there is quite a bit of data associated with your query and that there may be data you are not interested in. For example, looking at some of the boolean conditions you can filter on:

1696520078280.png

It may be the case you are only interested in publicly traded hits (IsPublic) or whether the data is active (IsActive), etc. There are so many categories and conditions that you can discover that will help you narrow down your result set. For example, you may only be interested in the status of the organisations that are listed. Here is a current breakdown for my query:
1696520203997.png

This goes back to my reference to "Limits" and that once you begin to realize the granularity of data you will receive, you will undoubtedly need to filter our unwanted hits. The article referenced above: Build queries easily using Refinitiv's Search API with Python and Jupyter will provide tools for you to discover properties and conditions to help you filter your search.


1696520078280.png (32.9 KiB)
1696520203997.png (3.4 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
16 4 7 10

Hi @zincone, many thanks for all the help!

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.