How to download data for all the mines in the world (all Countries) using python call?

I want to download data for all the mines in the world. I am using a python call:

mines = rd.discovery.search_templates["Mines"]
df = mines.search(commodity="Gold", region="Australia")

but i want to ask for all commodities in all countries. is this possible?

Find more posts tagged with

api

#content

#product

Accepted answers

Jirapongse

@jeremiemae.celajes

You can use the mines.search() method but the output will be limited to 10,000 entries, as mentioned by my colleague.

Therefore, you may need to use the search.Definition() directly with an appropriate filter. For example, I sent two requests. The first request gets all coal mines and the second request gets all mines except coal mines.

The request for all coal mines looks like this:

coal = search.Definition(
    view=search.Views.PHYSICAL_ASSETS,
    select="RIC,RCSRegionLeaf,RCSCommodityTypeLeaf,DTSubjectName,PhysicalAssetStatus,Latitude,Longitude", 
    filter="RCSAssetTypeLeaf eq 'Mine' and RCSCommodityTypeLeaf eq 'Coal'",
    top=10000
).get_data()
coal.data.df

The output is:

The request for all mines except coal mines looks like this:

others = search.Definition(
    view=search.Views.PHYSICAL_ASSETS,
    select="RIC,RCSRegionLeaf,RCSCommodityTypeLeaf,DTSubjectName,PhysicalAssetStatus,Latitude,Longitude", 
    filter="RCSAssetTypeLeaf eq 'Mine' and not(RCSCommodityTypeLeaf eq 'Coal')",
    top=10000
).get_data()
others.data.df

Totally, you will get 10,253 entries.

All comments

jeremiemae.celajes

Any assistance is greatly appreciated

jason.ramchandani01

@jeremiemae.celajes So the technology underlying the search templates is the RDP Search API -which has a result per call limit of 10,000. Thus were you to include all facilities in all regions for all commodities you might have a result set greater than 10,000. So for such larger queries you can try to use the Search API directly and this article and this article can help you with such queries. There are many samples in the accompanying notebooks. I think you would still need to narrow the search down by larger geographic regions. I hope this is of use.